Remove the notion of "sys*"

- This idea originates from very early in the project and was, at the time, a very easy way to categorise things. - Now, it doesn't really make much sense - it is fairly arbitary, often occuring when there is a change in kernel, but not from builder-hex0 to fiwix, and sysb is in reality completely unnecessary. - In short, the sys* stuff is a bit of a mess that makes the project more difficult to understand. - This puts everything down into one folder and has a manifest file that is used to generate the build scripts on the fly rather than using coded scripts. - This is created in the "seed" stage. stage0-posix -- (calls) --> seed -- (generates) --> main steps Alongside this change there are a variety of other smaller fixups to the general structure of the live-bootstrap rootfs. - Creating a rootfs has become much simpler and is defined as code in go.sh. The new structure, for an about-to-be booted system, is / -- /steps (direct copy of steps/) -- /distfiles (direct copy of distfiles/) -- all files from seed/* -- all files from seed/stage0-posix/* - There is no longer such a thing as /usr/include/musl, this didn't really make any sense, as musl is the final libc used. Rather, to separate musl and mes, we have /usr/include/mes, which is much easier to work with. - This also makes mes easier to blow away later. - A few things that weren't properly in packages have been changed; checksum-transcriber, simple-patch, kexec-fiwix have all been given fully qualified package names. - Highly breaking change, scripts now exist in their package directory but NOT WITH THE packagename.sh. Rather, they use pass1.sh, pass2.sh, etc. This avoids manual definition of passes. - Ditto with patches; default directory is patches, but then any patch series specific to a pass are named patches-passX.
2026-03-04 10:25:25 +01:00 · 2023-11-07 10:51:23 +11:00 · 2023-11-07 10:51:23 +11:00 · 6ed2e09f3a
commit 6ed2e09f3a
parent 0907cfd073
546 changed files with 700 additions and 1299 deletions
--- a/steps/python-3.11.1/files/Setup.local
+++ b/steps/python-3.11.1/files/Setup.local
@ -0,0 +1,179 @@
+# SPDX-FileCopyrightText: 2023 Andrius Štikonas <andrius@stikonas.eu>
+# SPDX-License-Identifier: PSF-2.0
+
+# Based on Modules/Setup file
+
+*static*
+
+# Modules that should always be present (POSIX and Windows):
+
+_asyncio _asynciomodule.c
+_bisect _bisectmodule.c
+_contextvars _contextvarsmodule.c
+_csv _csv.c
+_datetime _datetimemodule.c
+_decimal _decimal/_decimal.c
+_heapq _heapqmodule.c
+_json _json.c
+_lsprof _lsprof.c rotatingtree.c
+_multiprocessing -I$(srcdir)/Modules/_multiprocessing _multiprocessing/multiprocessing.c _multiprocessing/semaphore.c
+_opcode _opcode.c
+_pickle _pickle.c
+_queue _queuemodule.c
+_random _randommodule.c
+_socket socketmodule.c
+_statistics _statisticsmodule.c
+_struct _struct.c
+_typing _typingmodule.c
+_zoneinfo _zoneinfo.c
+array arraymodule.c
+audioop audioop.c
+binascii binascii.c
+cmath cmathmodule.c
+math mathmodule.c
+mmap mmapmodule.c
+select selectmodule.c
+
+# XML
+_elementtree _elementtree.c
+pyexpat pyexpat.c
+
+# hashing builtins
+_blake2 _blake2/blake2module.c _blake2/blake2b_impl.c _blake2/blake2s_impl.c
+_md5 md5module.c
+_sha1 sha1module.c
+_sha256 sha256module.c
+_sha512 sha512module.c
+_sha3 _sha3/sha3module.c
+
+# text encodings and unicode
+_codecs_cn cjkcodecs/_codecs_cn.c
+_codecs_hk cjkcodecs/_codecs_hk.c
+_codecs_iso2022 cjkcodecs/_codecs_iso2022.c
+_codecs_jp cjkcodecs/_codecs_jp.c
+_codecs_kr cjkcodecs/_codecs_kr.c
+_codecs_tw cjkcodecs/_codecs_tw.c
+_multibytecodec cjkcodecs/multibytecodec.c
+unicodedata unicodedata.c
+
+# Modules with some UNIX dependencies
+
+_posixsubprocess _posixsubprocess.c
+_posixshmem -I$(srcdir)/Modules/_multiprocessing _multiprocessing/posixshmem.c -lrt
+fcntl fcntlmodule.c
+grp grpmodule.c
+ossaudiodev ossaudiodev.c
+resource resource.c
+spwd spwdmodule.c
+syslog syslogmodule.c
+termios termios.c
+
+# Modules with UNIX dependencies that require external libraries
+
+#_crypt _cryptmodule.c -lcrypt
+#nis nismodule.c -I/usr/include/tirpc -lnsl -ltirpc
+
+# Modules that require external libraries.
+
+#_bz2 _bz2module.c -lbz2
+#_ctypes _ctypes/_ctypes.c _ctypes/callbacks.c _ctypes/callproc.c _ctypes/stgdict.c _ctypes/cfield.c -ldl -lffi -DHAVE_FFI_PREP_CIF_VAR -DHAVE_FFI_PREP_CLOSURE_LOC -DHAVE_FFI_CLOSURE_ALLOC
+# The _dbm module supports NDBM, GDBM with compat module, and Berkeley DB.
+#_dbm _dbmmodule.c -lgdbm_compat -DUSE_GDBM_COMPAT
+#_gdbm _gdbmmodule.c -lgdbm
+_lzma _lzmamodule.c -llzma
+#_uuid _uuidmodule.c -luuid
+zlib  zlibmodule.c -lz
+
+# The readline module also supports libeditline (-leditline).
+# Some systems may require -ltermcap or -ltermlib.
+#readline readline.c -lreadline -ltermcap
+
+# OpenSSL bindings
+#_ssl _ssl.c $(OPENSSL_INCLUDES) $(OPENSSL_LDFLAGS) $(OPENSSL_LIBS)
+#_hashlib _hashopenssl.c $(OPENSSL_INCLUDES) $(OPENSSL_LDFLAGS) -lcrypto
+
+# To statically link OpenSSL:
+_ssl _ssl.c $(OPENSSL_INCLUDES) $(OPENSSL_LDFLAGS) \
+    -l:libssl.a -Wl,--exclude-libs,libssl.a \
+    -l:libcrypto.a -Wl,--exclude-libs,libcrypto.a
+_hashlib _hashopenssl.c $(OPENSSL_INCLUDES) $(OPENSSL_LDFLAGS) \
+    -l:libcrypto.a -Wl,--exclude-libs,libcrypto.a
+
+# The _tkinter module.
+#
+# The command for _tkinter is long and site specific.  Please
+# uncomment and/or edit those parts as indicated.  If you don't have a
+# specific extension (e.g. Tix or BLT), leave the corresponding line
+# commented out.  (Leave the trailing backslashes in!  If you
+# experience strange errors, you may want to join all uncommented
+# lines and remove the backslashes -- the backslash interpretation is
+# done by the shell's "read" command and it may not be implemented on
+# every system.
+
+# *** Always uncomment this (leave the leading underscore in!):
+#_tkinter _tkinter.c tkappinit.c -DWITH_APPINIT $(TCLTK_INCLUDES) $(TCLTK_LIBS) \
+# *** Uncomment and edit to reflect where your Tcl/Tk libraries are:
+#	-L/usr/local/lib \
+# *** Uncomment and edit to reflect where your Tcl/Tk headers are:
+#	-I/usr/local/include \
+# *** Uncomment and edit to reflect where your X11 header files are:
+#	-I/usr/X11R6/include \
+# *** Or uncomment this for Solaris:
+#	-I/usr/openwin/include \
+# *** Uncomment and edit for Tix extension only:
+#	-DWITH_TIX -ltix8.1.8.2 \
+# *** Uncomment and edit for BLT extension only:
+#	-DWITH_BLT -I/usr/local/blt/blt8.0-unoff/include -lBLT8.0 \
+# *** Uncomment and edit for PIL (TkImaging) extension only:
+#     (See http://www.pythonware.com/products/pil/ for more info)
+#	-DWITH_PIL -I../Extensions/Imaging/libImaging  tkImaging.c \
+# *** Uncomment and edit for TOGL extension only:
+#	-DWITH_TOGL togl.c \
+# *** Uncomment and edit to reflect where your X11 libraries are:
+#	-L/usr/X11R6/lib \
+# *** Or uncomment this for Solaris:
+#	-L/usr/openwin/lib \
+# *** Uncomment these for TOGL extension only:
+#	-lGL -lGLU -lXext -lXmu \
+# *** Uncomment for AIX:
+#	-lld \
+# *** Always uncomment this; X11 libraries to link with:
+#	-lX11
+
+# Some system have -lcurses 
+#_curses -lncurses -lncursesw -ltermcap _cursesmodule.c
+#_curses_panel -lpanel -lncurses _curses_panel.c
+
+# macOS specific module, needs SystemConfiguration and CoreFoundation framework
+# _scproxy _scproxy.c
+
+# Examples
+
+#xx xxmodule.c
+#xxlimited xxlimited.c
+#xxlimited_35 xxlimited_35.c
+xxsubtype xxsubtype.c  # Required for the test suite to pass!
+
+# Testing
+
+#_xxsubinterpreters _xxsubinterpretersmodule.c
+#_xxtestfuzz _xxtestfuzz/_xxtestfuzz.c _xxtestfuzz/fuzzer.c
+#_testbuffer _testbuffer.c
+#_testinternalcapi _testinternalcapi.c
+
+# Some testing modules MUST be built as shared libraries.
+
+#*shared*
+#_ctypes_test _ctypes/_ctypes_test.c
+#_testcapi _testcapimodule.c
+#_testimportmultiple _testimportmultiple.c
+#_testmultiphase _testmultiphase.c
+
+# ---
+# Uncommenting the following line tells makesetup that all following modules
+# are not built (see above for more detail).
+#
+#*disabled*
+#
+# _sqlite3 _tkinter _curses pyexpat
+# _codecs_jp _codecs_kr _codecs_tw unicodedata
--- a/steps/python-3.11.1/pass1.sh
+++ b/steps/python-3.11.1/pass1.sh
@ -0,0 +1,102 @@
+# SPDX-FileCopyrightText: 2022 fosslinux <fosslinux@aussies.space>
+#
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+src_prepare() {
+    default
+
+    # Delete generated files that won't be regenerated
+    rm Lib/pydoc_data/topics.py \
+        Misc/stable_abi.toml
+
+    # Regenerate ssl_data for ssl module
+    rm Modules/_ssl_data_300.h Modules/_ssl_data.h
+    python Tools/ssl/make_ssl_data.py ../openssl-1.1.1l Modules/_ssl_data_111.h
+    sed -i 's#$(srcdir)/Modules/_ssl_data.h ##' Makefile.pre.in
+    sed -i 's#$(srcdir)/Modules/_ssl_data_300.h ##' Makefile.pre.in
+
+    # Regenerate encodings
+    grep generated -r . -l | grep encodings | xargs rm
+    mkdir Tools/unicode/in Tools/unicode/out
+    mv ../CP437.TXT Tools/unicode/in/
+    pushd Tools/unicode
+    python gencodec.py in/ ../../Lib/encodings/
+    popd
+
+    # Regenerate stringprep
+    rm Lib/stringprep.py
+    mv ../rfc3454.txt .
+    python Tools/unicode/mkstringprep.py > Lib/stringprep.py
+
+    # Regenerate unicode
+    rm Modules/unicodedata_db.h Modules/unicodename_db.h Objects/unicodetype_db.h
+    mkdir -p Tools/unicode/data
+    mv ../*.txt ../*.zip Tools/unicode/data/
+    python Tools/unicode/makeunicodedata.py
+
+    # Regenerate Lib/re/_casefix.py
+    rm Lib/re/_casefix.py
+    python Tools/scripts/generate_re_casefix.py Lib/re/_casefix.py
+
+    # Regenerate Programs/test_frozenmain.h
+    rm Programs/test_frozenmain.h
+    python Programs/freeze_test_frozenmain.py Programs/test_frozenmain.h
+
+    # Create dummy Python/stdlib_module_names.h
+    echo 'static const char* _Py_stdlib_module_names[] = {};' > Python/stdlib_module_names.h
+
+    # Regenerate autoconf
+    autoreconf-2.71 -fi
+}
+
+src_configure() {
+    mv Setup.local Modules
+    MACHDEP=linux ac_sys_system=Linux \
+    CPPFLAGS="-U__DATE__ -U__TIME__" \
+    PKG_CONFIG_PATH="${LIBDIR}/pkgconfig/" \
+    LDFLAGS="-static" \
+        ./configure \
+        --build=i386-unknown-linux-musl \
+        --host=i386-unknown-linux-musl \
+        --prefix="${PREFIX}" \
+        --libdir="${LIBDIR}" \
+        --with-system-ffi \
+        --disable-shared
+}
+
+src_compile() {
+    # Regenerations
+    # We have to choose the order ourselves because the Makefile is extremely lax about the order
+    # First of all, do everything that doesn't use any C
+    rm Modules/_blake2/blake2s_impl.c
+    make "${MAKEJOBS}" regen-opcode \
+        regen-opcode-targets \
+        regen-typeslots \
+        regen-token \
+        regen-ast \
+        regen-keyword \
+        regen-sre \
+        clinic \
+        regen-pegen-metaparser \
+        regen-pegen \
+        regen-global-objects
+
+    # Do the freeze regen process
+    make "${MAKEJOBS}" regen-frozen
+    make "${MAKEJOBS}" regen-deepfreeze
+    make "${MAKEJOBS}" regen-global-objects
+
+    make "${MAKEJOBS}" CPPFLAGS="-U__DATE__ -U__TIME__"
+
+    # Regen Python/stdlib_module_names.h (you must have an existing build first)
+    make "${MAKEJOBS}" regen-stdlib-module-names
+
+    # Now rebuild with proper stdlib_module_names.h
+    make "${MAKEJOBS}" CPPFLAGS="-U__DATE__ -U__TIME__"
+}
+
+src_install() {
+    default
+    ln --symbolic --relative "${DESTDIR}${LIBDIR}/python3.11/lib-dynload" "${DESTDIR}${PREFIX}/lib/python3.11/lib-dynload"
+    ln --symbolic --relative "${DESTDIR}${PREFIX}/bin/python3.11" "${DESTDIR}${PREFIX}/bin/python"
+}
--- a/steps/python-3.11.1/patches/empty-date.patch
+++ b/steps/python-3.11.1/patches/empty-date.patch
@ -0,0 +1,20 @@
+SPDX-FileCopyrightText: 2022 fosslinux <fosslinux@aussies.space>
+
+SPDX-License-Identifier: PSF-2.0
+
+When defined __DATE__ is in the format MMM DD YYYY. xx/xx/xx does
+not comply with this so the parser in Lib/platform.py fails on it.
+
+Upstream PR: https://github.com/python/cpython/pull/100389
+
+--- Modules/getbuildinfo.c	2022-12-21 13:41:18.120573458 +1100
+++ Modules/getbuildinfo.c	2022-12-21 13:41:30.399716652 +1100
+@@ -8,7 +8,7 @@
+ #ifdef __DATE__
+ #define DATE __DATE__
+ #else
+-#define DATE "xx/xx/xx"
+#define DATE "xxx xx xxxx"
+ #endif
+ #endif
+ 
--- a/steps/python-3.11.1/patches/multiarch.patch
+++ b/steps/python-3.11.1/patches/multiarch.patch
@ -0,0 +1,19 @@
+SPDX-FileCopyrightText: 2022 fosslinux <fosslinux@aussies.space>
+
+SPDX-License-Identifier: PSF-2.0
+
+On x86_64 system our GCC has multiarch a little broken for Python's
+build system which doesn't really bother me so just pretend that
+there is no multiarch.
+
+--- configure.ac.bak	2022-12-21 19:35:44.560977616 +1100
+++ configure.ac	2022-12-21 19:36:00.735143246 +1100
+@@ -1096,7 +1096,7 @@
+ AS_CASE([$ac_sys_system],
+   [Darwin*], [MULTIARCH=""],
+   [FreeBSD*], [MULTIARCH=""],
+-  [MULTIARCH=$($CC --print-multiarch 2>/dev/null)]
+  [MULTIARCH=""]
+ )
+ AC_SUBST([MULTIARCH])
+ AC_MSG_RESULT([$MULTIARCH])
--- a/steps/python-3.11.1/sources
+++ b/steps/python-3.11.1/sources
@ -0,0 +1,24 @@
+https://www.python.org/ftp/python/3.11.1/Python-3.11.1.tar.xz 85879192f2cffd56cb16c092905949ebf3e5e394b7f764723529637901dfb58f
+http://ftp.unicode.org/Public/3.2-Update/UnicodeData-3.2.0.txt 5e444028b6e76d96f9dc509609c5e3222bf609056f35e5fcde7e6fb8a58cd446
+http://ftp.unicode.org/Public/3.2-Update/CompositionExclusions-3.2.0.txt 1d3a450d0f39902710df4972ac4a60ec31fbcb54ffd4d53cd812fc1200c732cb
+http://ftp.unicode.org/Public/3.2-Update/EastAsianWidth-3.2.0.txt ce19f35ffca911bf492aab6c0d3f6af3d1932f35d2064cf2fe14e10be29534cb
+http://ftp.unicode.org/Public/3.2-Update/DerivedCoreProperties-3.2.0.txt 787419dde91701018d7ad4f47432eaa55af14e3fe3fe140a11e4bbf3db18bb4c
+http://ftp.unicode.org/Public/3.2-Update/DerivedNormalizationProps-3.2.0.txt bab49295e5f9064213762447224ccd83cea0cced0db5dcfc96f9c8a935ef67ee
+http://ftp.unicode.org/Public/3.2-Update/LineBreak-3.2.0.txt d693ef2a603d07e20b769ef8ba29afca39765588a03e3196294e5be8638ca735
+http://ftp.unicode.org/Public/3.2-Update/SpecialCasing-3.2.0.txt 1f7913b74dddff55ee566f6220aa9e465bae6f27709fc21d353b04adb8572b37
+http://ftp.unicode.org/Public/3.2-Update/CaseFolding-3.2.0.txt 370f3d1e79a52791c42065946711f4eddb6d9820726afd0e436a3c50360475a9
+http://ftp.unicode.org/Public/3.2-Update/Unihan-3.2.0.zip 0582b888c4ebab6e3ce8d340c74788f1a68ca662713a1065b9a007f24bb4fe46
+http://ftp.unicode.org/Public/14.0.0/ucd/UnicodeData.txt 36018e68657fdcb3485f636630ffe8c8532e01c977703d2803f5b89d6c5feafb UnicodeData-14.0.0.txt
+http://ftp.unicode.org/Public/14.0.0/ucd/CompositionExclusions.txt 3360762fc3295cea54ab251c31df621d05ba4b94d46c60eaac29aa16d70ad1e0 CompositionExclusions-14.0.0.txt
+http://ftp.unicode.org/Public/14.0.0/ucd/EastAsianWidth.txt f901ac011aa32a09224d6555da71e2532c59c1d3381322829de0e3b880507250 EastAsianWidth-14.0.0.txt
+http://ftp.unicode.org/Public/14.0.0/ucd/DerivedCoreProperties.txt e3eddd7d469cd1b0feed7528defad1a1cc7c6a9ceb0ae4446a6d10921ed2e7bc DerivedCoreProperties-14.0.0.txt
+http://ftp.unicode.org/Public/14.0.0/ucd/DerivedNormalizationProps.txt b2c444c20730b097787fdf50bd7d6dd3fc5256ab8084f5b35b11c8776eca674c DerivedNormalizationProps-14.0.0.txt
+http://ftp.unicode.org/Public/14.0.0/ucd/LineBreak.txt 9e06e9f35c6959fb91dcc7993f90d58523c3079bc62c6b25f828b4cdebc5d70c LineBreak-14.0.0.txt
+http://ftp.unicode.org/Public/14.0.0/ucd/NameAliases.txt 14b3b677d33f95c51423dce6eef4a6a28b4b160451ecedee4b91edb6745cf4a3 NameAliases-14.0.0.txt
+http://ftp.unicode.org/Public/14.0.0/ucd/NamedSequences.txt db5745688affcdc0c3927a1ee0667018a96a7b24513f866d5235e98fef6c2436 NamedSequences-14.0.0.txt
+http://ftp.unicode.org/Public/14.0.0/ucd/SpecialCasing.txt c667b45908fd269af25fd55d2fc5bbc157fb1b77675936e25c513ce32e080334 SpecialCasing-14.0.0.txt
+http://ftp.unicode.org/Public/14.0.0/ucd/CaseFolding.txt a566cd48687b2cd897e02501118b2413c14ae86d318f9abbbba97feb84189f0f CaseFolding-14.0.0.txt
+http://ftp.unicode.org/Public/14.0.0/ucd/Unihan.zip 2ae4519b2b82cd4d15379c17e57bfb12c33c0f54da4977de03b2b04bcf11852d Unihan-14.0.0.zip
+http://ftp.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/PC/CP437.TXT 6bad4dabcdf5940227c7d81fab130dcb18a77850b5d79de28b5dc4e047b0aaac
+https://www.ietf.org/rfc/rfc3454.txt eb722fa698fb7e8823b835d9fd263e4cdb8f1c7b0d234edf7f0e3bd2ccbb2c79
+http://mirror.math.princeton.edu/pub/openssl/source/old/1.1.1/openssl-1.1.1l.tar.gz 0b7a3e5e59c34827fe0c3a74b7ec8baef302b98fa80088d7f9153aa16fa76bd1