Remove the notion of "sys*"

- This idea originates from very early in the project and was, at the
  time, a very easy way to categorise things.
- Now, it doesn't really make much sense - it is fairly arbitary, often
  occuring when there is a change in kernel, but not from builder-hex0
  to fiwix, and sysb is in reality completely unnecessary.
- In short, the sys* stuff is a bit of a mess that makes the project
  more difficult to understand.
- This puts everything down into one folder and has a manifest file that
  is used to generate the build scripts on the fly rather than using
  coded scripts.
- This is created in the "seed" stage.

stage0-posix -- (calls) --> seed -- (generates) --> main steps

Alongside this change there are a variety of other smaller fixups to the
general structure of the live-bootstrap rootfs.

- Creating a rootfs has become much simpler and is defined as code in
  go.sh. The new structure, for an about-to-be booted system, is

/
-- /steps (direct copy of steps/)
-- /distfiles (direct copy of distfiles/)
-- all files from seed/*
-- all files from seed/stage0-posix/*

- There is no longer such a thing as /usr/include/musl, this didn't
  really make any sense, as musl is the final libc used. Rather, to
  separate musl and mes, we have /usr/include/mes, which is much easier
  to work with.
- This also makes mes easier to blow away later.
- A few things that weren't properly in packages have been changed;
  checksum-transcriber, simple-patch, kexec-fiwix have all been given
  fully qualified package names.
- Highly breaking change, scripts now exist in their package directory
  but NOT WITH THE packagename.sh. Rather, they use pass1.sh, pass2.sh,
  etc. This avoids manual definition of passes.
  - Ditto with patches; default directory is patches, but then any patch
    series specific to a pass are named patches-passX.
This commit is contained in:
fosslinux 2023-11-07 10:51:23 +11:00
parent 0907cfd073
commit 6ed2e09f3a
546 changed files with 700 additions and 1299 deletions

View file

@ -0,0 +1,18 @@
/*
SPDX-FileCopyrightText: 2023 Richard Masters <grick23@gmail.com>
SPDX-License-Identifier: MIT
*/
#define CONFIG_USE_DEFAULT_CONFIG 0
#define CONFIG_HAVE_OWN_OFLAGS 0
#define CONFIG_HAVE_OWN_ERRNO 1
#define CONFIG_HAVE_OWN_ASSERT 0
#define CONFIG_BLOCK_DEV_CACHE_SIZE 16
typedef long long unsigned int uint64_t;
typedef long long int int64_t;
#define fseeko fseek
#define ftello ftell
#define PRIu32 "u"
#define PRId32 "d"
#define PRIx32 "x"
#define PRIu16 "hu"

View file

@ -0,0 +1,177 @@
# SPDX-FileCopyrightText: 2023 Richard Masters <grick23@gmail.com>
# SPDX-License-Identifier: MIT
/usr/bin/blood-elf
/usr/bin/catm
/usr/bin/chmod
/usr/bin/get_machine
/usr/bin/hex2
/usr/bin/kaem
/usr/bin/match
/usr/bin/M1
/usr/bin/M2-Mesoplanet
/usr/bin/M2-Planet
/usr/bin/mkdir
/usr/bin/sha256sum
/usr/bin/unbz2
/usr/bin/ungz
/usr/bin/untar
/usr/bin/cp
/usr/bin/replace
/usr/bin/rm
/usr/bin/checksum-transcriber
/usr/bin/tcc
/usr/bin/tcc-0.9.26
/usr/bin/kexec-linux
/usr/lib/mes/libc.a
/usr/lib/mes/libgetopt.a
/usr/lib/mes/crt1.o
/usr/lib/mes/crti.o
/usr/lib/mes/crtn.o
/usr/lib/mes/tcc/libtcc1.a
/usr/include/mes/alloca.h
/usr/include/mes/argz.h
/usr/include/mes/ar.h
/usr/include/mes/assert.h
/usr/include/mes/ctype.h
/usr/include/mes/dirent.h
/usr/include/mes/dirstream.h
/usr/include/mes/dlfcn.h
/usr/include/mes/endian.h
/usr/include/mes/errno.h
/usr/include/mes/fcntl.h
/usr/include/mes/features.h
/usr/include/mes/float.h
/usr/include/mes/getopt.h
/usr/include/mes/grp.h
/usr/include/mes/inttypes.h
/usr/include/mes/libgen.h
/usr/include/mes/limits.h
/usr/include/mes/locale.h
/usr/include/mes/math.h
/usr/include/mes/memory.h
/usr/include/mes/pwd.h
/usr/include/mes/setjmp.h
/usr/include/mes/signal.h
/usr/include/mes/stdarg.h
/usr/include/mes/stdbool.h
/usr/include/mes/stddef.h
/usr/include/mes/stdint.h
/usr/include/mes/stdio.h
/usr/include/mes/stdlib.h
/usr/include/mes/stdnoreturn.h
/usr/include/mes/string.h
/usr/include/mes/strings.h
/usr/include/mes/termio.h
/usr/include/mes/time.h
/usr/include/mes/unistd.h
/usr/include/mes/arch/kernel-stat.h
/usr/include/mes/arch/syscall.h
/usr/include/mes/linux/syscall.h
/usr/include/mes/linux/x86/syscall.h
/usr/include/mes/mes/builtins.h
/usr/include/mes/mes/cc.h
/usr/include/mes/mes/config.h
/usr/include/mes/mes/constants.h
/usr/include/mes/mes/lib.h
/usr/include/mes/mes/lib-cc.h
/usr/include/mes/mes/lib-mini.h
/usr/include/mes/mes/mes.h
/usr/include/mes/mes/symbols.h
/usr/include/mes/sys/cdefs.h
/usr/include/mes/sys/dir.h
/usr/include/mes/sys/file.h
/usr/include/mes/sys/ioctl.h
/usr/include/mes/sys/mman.h
/usr/include/mes/sys/param.h
/usr/include/mes/sys/resource.h
/usr/include/mes/sys/select.h
/usr/include/mes/sys/stat.h
/usr/include/mes/sys/timeb.h
/usr/include/mes/sys/time.h
/usr/include/mes/sys/times.h
/usr/include/mes/sys/types.h
/usr/include/mes/sys/ucontext.h
/usr/include/mes/sys/user.h
/usr/include/mes/sys/wait.h
/init
/kaem.run
/external/distfiles/make-3.82.tar.bz2
/external/distfiles/patch-2.5.9.tar.gz
/external/distfiles/gzip-1.2.4.tar.gz
/external/distfiles/tar-1.12.tar.gz
/external/distfiles/sed-4.0.9.tar.gz
/external/distfiles/bzip2-1.0.8.tar.gz
/external/distfiles/coreutils-5.0.tar.bz2
/external/distfiles/heirloom-devtools-070527.tar.bz2
/external/distfiles/bash-2.05b.tar.gz
/external/distfiles/flex-2.5.11.tar.gz
/external/distfiles/tcc-0.9.27.tar.bz2
/external/distfiles/musl-1.1.24.tar.gz
/external/distfiles/tcc-0.9.27.tar.bz2
/external/distfiles/musl-1.1.24.tar.gz
/external/distfiles/tcc-0.9.27.tar.bz2
/external/distfiles/sed-4.0.9.tar.gz
/external/distfiles/bzip2-1.0.8.tar.gz
/external/distfiles/m4-1.4.7.tar.gz
/external/distfiles/flex-2.6.4.tar.gz
/external/distfiles/bison-3.4.1.tar.gz
/external/distfiles/bison-3.4.1.tar.gz
/external/distfiles/bison-3.4.1.tar.gz
/external/distfiles/grep-2.4.tar.gz
/external/distfiles/diffutils-2.7.tar.gz
/external/distfiles/coreutils-5.0.tar.bz2
/external/distfiles/coreutils-6.10.tar.gz
/external/distfiles/gawk-3.0.4.tar.gz
/external/distfiles/perl-5.000.tar.gz
/external/distfiles/perl-5.003.tar.gz
/external/distfiles/perl5.004_05.tar.gz
/external/distfiles/perl5.005_03.tar.gz
/external/distfiles/perl-5.6.2.tar.gz
/external/distfiles/autoconf-2.52.tar.bz2
/external/distfiles/automake-1.6.3.tar.bz2
/external/distfiles/automake-1.6.3.tar.bz2
/external/distfiles/autoconf-2.53.tar.bz2
/external/distfiles/automake-1.7.tar.bz2
/external/distfiles/autoconf-2.54.tar.bz2
/external/distfiles/autoconf-2.55.tar.bz2
/external/distfiles/automake-1.7.8.tar.bz2
/external/distfiles/autoconf-2.57.tar.bz2
/external/distfiles/autoconf-2.59.tar.bz2
/external/distfiles/automake-1.8.5.tar.bz2
/external/distfiles/help2man-1.36.4.tar.gz
/external/distfiles/autoconf-2.61.tar.bz2
/external/distfiles/automake-1.9.6.tar.bz2
/external/distfiles/automake-1.10.3.tar.bz2
/external/distfiles/autoconf-2.64.tar.bz2
/external/distfiles/automake-1.11.2.tar.bz2
/external/distfiles/autoconf-2.69.tar.gz
/external/distfiles/libtool-2.2.4.tar.bz2
/external/distfiles/automake-1.15.1.tar.gz
/external/distfiles/binutils-2.30.tar.bz2
/external/distfiles/musl-1.1.24.tar.gz
/external/distfiles/tcc-0.9.27.tar.bz2
/external/distfiles/gcc-core-4.0.4.tar.bz2
/external/distfiles/automake-1.16.3.tar.gz
/external/distfiles/findutils-4.2.33.tar.gz
/external/distfiles/gnulib-8e128e.tar.gz
/external/distfiles/musl-1.2.4.tar.gz
/external/distfiles/gcc-core-4.0.4.tar.bz2
/external/distfiles/automake-1.16.3.tar.gz
/external/distfiles/util-linux-2.19.1.tar.gz
/external/distfiles/e2fsprogs-1.45.7.tar.gz
/external/distfiles/CaseFolding.txt
/external/distfiles/DerivedAge.txt
/external/distfiles/DerivedCombiningClass.txt
/external/distfiles/DerivedCoreProperties.txt
/external/distfiles/NormalizationCorrections.txt
/external/distfiles/NormalizationTest.txt
/external/distfiles/UnicodeData.txt
/external/distfiles/v10.0.1.tar.gz
/external/distfiles/kbd-1.15.tar.gz
/external/distfiles/make-3.82.tar.bz2
/external/distfiles/ed-1.4.tar.gz
/external/distfiles/bc-1.07.1.tar.gz
/external/distfiles/v2.0.22.tar.gz
/external/distfiles/linux-4.9.10.tar.gz
/external/distfiles/deblob-4.9
/external/distfiles/curl-7.88.1.tar.bz2

View file

@ -0,0 +1,295 @@
/*
* SPDX-FileCopyrightText: Copyright (c) 2015 Grzegorz Kostka (kostka.grzegorz@gmail.com)
* SPDX-License-Identifier: BSD-3-Clause
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* - Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* - Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* - The name of the author may not be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
* Modifications:
* SPDX-FileCopyrightText: 2023 Richard Masters <grick23@gmail.com>
* SPDX-License-Identifier: MIT
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <stdbool.h>
#include <inttypes.h>
#include <time.h>
#include <unistd.h>
#include <sys/time.h>
#include <ext4.h>
#include <ext4_mkfs.h>
#include "../blockdev/linux/file_dev.h"
#include "../blockdev/windows/file_windows.h"
#define BLOCK_SIZE 1024
#define FILENAME_LENGTH 256
#define INITRD_MB 1152
const char *input_name = NULL;
/**@brief Block device handle.*/
static struct ext4_blockdev *bd;
/**@brief Block cache handle.*/
static struct ext4_bcache *bc;
static struct ext4_fs fs;
static struct ext4_mkfs_info info = {
.block_size = BLOCK_SIZE,
.journal = false,
.inode_size = 128,
};
#define MKDEV(major, minor) (((major) << 8) | (minor))
static bool open_filedev(void)
{
file_dev_name_set(input_name);
bd = file_dev_get();
if (!bd) {
puts("open_filedev: fail");
return false;
}
return true;
}
bool lwext4_mount(struct ext4_blockdev *bdev, struct ext4_bcache *bcache)
{
int r;
bc = bcache;
bd = bdev;
if (!bd) {
puts("lwext4_mount: no block device");
return false;
}
/* ext4_dmask_set(DEBUG_ALL); */
r = ext4_device_register(bd, "ext4_fs");
if (r != EOK) {
printf("ext4_device_register: rc = %d\n", r);
return false;
}
r = ext4_mount("ext4_fs", "/mp/", false);
if (r != EOK) {
printf("ext4_mount: rc = %d\n", r);
return false;
}
r = ext4_recover("/mp/");
if (r != EOK && r != ENOTSUP) {
printf("ext4_recover: rc = %d\n", r);
return false;
}
/* ext4_cache_write_back("/mp/", 1); */
return true;
}
bool lwext4_umount(void)
{
int r;
/* ext4_cache_write_back("/mp/", 0); */
r = ext4_umount("/mp/");
if (r != EOK) {
printf("ext4_umount: fail %d", r);
return false;
}
return true;
}
bool copy_file(char *src_path, char *dest_path)
{
printf("copy_file: %s\n", src_path);
ext4_file dest_file;
FILE *src_file = fopen(src_path, "rb");
if (!src_file) {
printf("fopen '%s' error.\n", src_path);
return EXIT_FAILURE;
}
fseek(src_file, 0, SEEK_END);
int src_len = ftell(src_file);
char * src_mem = malloc(src_len);
int err;
fseek(src_file, 0, SEEK_SET);
if (src_len > 0) {
int read_len = fread(src_mem, src_len, 1, src_file);
fclose(src_file);
if (read_len < 1) {
printf("src fread error file: '%s' read count: %d\n", src_path, read_len);
}
}
err = ext4_fopen(&dest_file, dest_path, "wb");
if (err != EOK) {
printf("ext4_open error: %d\n", err);
return EXIT_FAILURE;
}
if (src_len > 0) {
err = ext4_fwrite(&dest_file, src_mem, src_len, 0);
if (err != EOK) {
printf("ext4_fwrite error: %d\n", err);
return EXIT_FAILURE;
}
}
err = ext4_fclose(&dest_file);
if (err != EOK) {
printf("ext4_fclose error: %d\n", err);
return EXIT_FAILURE;
}
free(src_mem);
}
bool copy_file_list(char *file_list_path)
{
char src_filename[FILENAME_LENGTH];
char dst_filename[FILENAME_LENGTH];
FILE *file_list = fopen(file_list_path, "r");
while(fgets(src_filename, FILENAME_LENGTH, file_list)) {
/* Skip comments */
if (src_filename[0] == '#') {
continue;
}
src_filename[strlen(src_filename) - 1] = 0; /* strip newline */
strcpy(dst_filename, "/mp");
strcat(dst_filename, src_filename);
copy_file(src_filename, dst_filename);
}
fclose(file_list);
}
int main(int argc, char **argv)
{
int err;
char zeros[BLOCK_SIZE];
unsigned int next_file_address;
next_file_address = *((unsigned int *) 0x7F8D);
printf("Starting fiwix.ext2 at addr 0x%08x\n", next_file_address);
/* Create zeroed out disk image file */
input_name = "/boot/fiwix.ext2";
memset(zeros, 0, BLOCK_SIZE);
FILE *ext2file = fopen(input_name, "w");
int b;
for (b=0; b < (BLOCK_SIZE * INITRD_MB); b++)
fwrite(zeros, BLOCK_SIZE, 1, ext2file);
fclose(ext2file);
if (!open_filedev()) {
printf("open_filedev error\n");
return EXIT_FAILURE;
}
/* ext4_dmask_set(DEBUG_ALL); */
err = ext4_mkfs(&fs, bd, &info, F_SET_EXT2_V0);
if (err != EOK) {
printf("ext4_mkfs error: %d\n", err);
return EXIT_FAILURE;
}
memset(&info, 0, sizeof(struct ext4_mkfs_info));
err = ext4_mkfs_read_info(bd, &info);
if (err != EOK) {
printf("ext4_mkfs_read_info error: %d\n", err);
return EXIT_FAILURE;
}
printf("Created filesystem with parameters:\n");
printf("Size: %"PRIu64"\n", info.len);
printf("Block size: %"PRIu32"\n", info.block_size);
printf("Blocks per group: %"PRIu32"\n", info.blocks_per_group);
printf("Inodes per group: %"PRIu32"\n", info.inodes_per_group);
printf("Inode size: %"PRIu32"\n", info.inode_size);
printf("Inodes: %"PRIu32"\n", info.inodes);
printf("Journal blocks: %"PRIu32"\n", info.journal_blocks);
printf("Features ro_compat: 0x%x\n", info.feat_ro_compat);
printf("Features compat: 0x%x\n", info.feat_compat);
printf("Features incompat: 0x%x\n", info.feat_incompat);
printf("BG desc reserve: %"PRIu32"\n", info.bg_desc_reserve_blocks);
printf("Descriptor size: %"PRIu32"\n",info.dsc_size);
printf("Label: %s\n", info.label);
if (!lwext4_mount(bd, bc))
return EXIT_FAILURE;
puts("ext4_dir_mk /mp/dev");
err = ext4_dir_mk("/mp/dev");
if (err != EOK) {
printf("ext4_dir_mk error: %d\n", err);
}
puts("ext4_dir_mk /mp/tmp");
err = ext4_dir_mk("/mp/tmp");
if (err != EOK) {
printf("ext4_dir_mk error: %d\n", err);
}
puts("ext4_mknod /mp/dev/console");
err = ext4_mknod("/mp/dev/console", EXT4_DE_CHRDEV, MKDEV(5, 1));
if (err != EOK) {
printf("ext4_mknod error: %d\n", err);
return EXIT_FAILURE;
}
puts("ext4_mknod /mp/dev/ram0");
err = ext4_mknod("/mp/dev/ram0", EXT4_DE_BLKDEV, MKDEV(1, 0));
if (err != EOK) {
printf("ext4_mknod error: %d\n", err);
return EXIT_FAILURE;
}
puts("ext4_mknod /mp/dev/ram1");
err = ext4_mknod("/mp/dev/ram1", EXT4_DE_BLKDEV, MKDEV(1, 1));
if (err != EOK) {
printf("ext4_mknod error: %d\n", err);
return EXIT_FAILURE;
}
copy_file_list("/steps/lwext4-1.0.0-lb1/files/fiwix-file-list.txt");
if (!lwext4_umount())
return EXIT_FAILURE;
puts("Fiwix ext2 initrd created successfully.");
return EXIT_SUCCESS;
}

View file

@ -0,0 +1 @@
a1c3fc496659b55e3600c06c76c374b76da5f19d153f44dbf5796fe7a0d8283d /usr/bin/make_fiwix_initrd

View file

@ -0,0 +1,61 @@
#!/bin/sh
# SPDX-FileCopyrightText: 2023 Richard Masters <grick23@gmail.com>
# SPDX-License-Identifier: MIT
set -ex
mkdir build src
cd src
ungz --file ${DISTFILES}/${pkg}.tar.gz --output ${pkg}.tar
cd ..
cd build
untar --file ../src/${pkg}.tar
cd ${pkg}
mkdir -p build_generic/include/generated
catm build_generic/include/generated/ext4_config.h ../../config/ext4_config.h
cd blockdev/linux
tcc -m32 -march=i386 -std=c89 -I../../include -I../../build_generic/include -I../../../tcc/tcc-0.9.27/include -DVERSION="\"1.0\"" -c file_dev.c
cd ../../src
alias cc="tcc -m32 -march=i386 -std=c89 -I../include -I../build_generic/include -I../../tcc/tcc-0.9.27/include"
cc -c ext4.c
cc -c ext4_balloc.c
cc -c ext4_bcache.c
cc -c ext4_bitmap.c
cc -c ext4_block_group.c
cc -c ext4_blockdev.c
cc -c ext4_crc32.c
cc -c ext4_debug.c
cc -c ext4_dir.c
cc -c ext4_dir_idx.c
cc -c ext4_extent.c
cc -c ext4_fs.c
cc -c ext4_hash.c
cc -c ext4_ialloc.c
cc -c ext4_inode.c
cc -c ext4_journal.c
cc -c ext4_mbr.c
cc -c ext4_mkfs.c
cc -c ext4_super.c
cc -c ext4_trans.c
cc -c ext4_xattr.c
cp ../../../files/make_fiwix_initrd.c make_fiwix_initrd.c
tcc -m32 -march=i386 -std=c89 -I../include -I../build_generic/include -I../../tcc/tcc-0.9.27/include -DVERSION="\"1.0\"" -c make_fiwix_initrd.c
tcc -m32 -o ${BINDIR}/make_fiwix_initrd ext4.o ext4_balloc.o ext4_bcache.o ext4_bitmap.o ext4_block_group.o ext4_blockdev.o ext4_crc32.o ext4_debug.o ext4_dir.o ext4_dir_idx.o ext4_extent.o ext4_fs.o ext4_hash.o ext4_ialloc.o ext4_inode.o ext4_journal.o ext4_mbr.o ext4_mkfs.o ext4_super.o ext4_trans.o ext4_xattr.o ../blockdev/linux/file_dev.o make_fiwix_initrd.o
cd ../../..
# Checksums
if match x${UPDATE_CHECKSUMS} xTrue; then
sha256sum -o ${pkg}.checksums \
/usr/bin/make_fiwix_initrd
cp ${pkg}.checksums ${SRCDIR}
else
sha256sum -c ${pkg}.checksums
fi

View file

@ -0,0 +1 @@
https://github.com/rick-masters/lwext4/releases/download/v1.0.0-lb1/lwext4-1.0.0-lb1.tar.gz a90526665123d788fc23d14354468d22cc2e3e9e43a6c44ea452fbbec12b8451