Commit graph

103 commits

Author SHA1 Message Date
vxtls
c124c08096 refactor(rootfs): [MAY BRAKE]stage0 resume updater to deterministic internal entrypoint and split bootstrap mode handlers 2026-03-07 16:29:31 -05:00
vxtls
adcd73b017 fix(resume): preserve stage0-image incremental continuation by remapping init entry after script regeneration 2026-03-07 16:15:18 -05:00
vxtls
b18cba40b3 fix(resume): regenerate and realign guix script entrypoint; suffix duplicate guix package names 2026-03-07 15:31:17 -05:00
vxtls
076a70976a fix(resume): regenerate step scripts on stage0-image boot so manifest and break-after updates take effect 2026-03-07 15:11:26 -05:00
vxtls
666d9792c4 fix(rootfs): support fresh qemu break-after by injecting manifest jump before init image build 2026-03-06 16:28:50 -05:00
vxtls
c921403c06 fix(rootfs): reuse stage0 work-image flow for fresh qemu break-after and drop forced ext4 mount type 2026-03-06 16:02:39 -05:00
vxtls
c55f989e3c fix(rootfs): restrict break-after to stage0-image resume path and restore clean first-run bootstrap 2026-03-06 15:54:07 -05:00
vxtls
a1f3068047 fix(rootfs): allow break-after checkpoints on first qemu kernel-bootstrap run 2026-03-06 15:24:38 -05:00
vxtls
65314bc926 fix(network): gate dhcp on get_network state across boots 2026-03-06 09:27:41 -05:00
vxtls
c90d9da159 fix(boot): add minimal early mounts for /dev and /proc in stage0 resume path 2026-03-04 18:55:04 -05:00
vxtls
d5fc082dab fix(stage0-image): preserve runtime /steps/env while syncing step sources 2026-03-04 17:23:48 -05:00
vxtls
886d5a5237 fix(stage0-image): sync steps trees in-place to preserve generated resume scripts 2026-03-04 17:18:05 -05:00
vxtls
74ff570f76 fix(stage0-image): always refresh steps trees into stage0-work image before boot 2026-03-04 17:06:42 -05:00
vxtls
82124dfd91 refactor(stage0-breakpoint): use deterministic steps-guix refresh policy for break vs resume modes 2026-03-04 09:45:45 -05:00
vxtls
9d1870c931 refactor(stage0-image): remove python-side resume network patching logic 2026-03-03 19:23:38 -05:00
vxtls
f11bbe6733 refactor(resume-init): move network bring-up into generated jump init and keep checkpoint compatibility patch 2026-03-03 19:18:51 -05:00
vxtls
84400964ed fix(stage0-resume): wrap jump-resume init with dhcpcd bring-up before continuing scripts 2026-03-03 19:10:38 -05:00
vxtls
b733cf2526 fix(stage0-resume): avoid false /init guix handoff check and preserve steps-guix resume scripts 2026-03-03 18:56:09 -05:00
vxtls
7253e3fb33 feat(qemu-ci): scope internal breakpoints as steps:NAME or steps-guix:NAME with default steps-guix recopy 2026-03-03 13:14:27 -05:00
vxtls
85c389044d fix(kernel-bootstrap): unify external raw container flow and remove default second disk 2026-03-03 08:33:19 -05:00
vxtls
beb9fb12f9 refactor+docs(payload.img, payload.img discovery): split offline distfiles at improve: import_payload so main image is minimal and payload.img is primary carrier, detect payload.img automaticly using magic number 2026-03-01 21:05:29 -05:00
vxtls
f30c20b7be fix(kernel-bootstrap): restore kexec-fiwix baseline and move post-fiwix distfiles into raw payload import path
The instability was not caused by kexec-fiwix logic itself but by oversized early-stage payload pressure in kernel-bootstrap mode.
When too many distfiles are injected before the Fiwix handoff, the early memory/file-table assumptions become fragile and KVM can fail during transition.
This change restores kexec-fiwix.c to the known baseline (matching commit 984b8322...) and fixes the actual failure mode by moving non-early distfiles out of the initial image.

What changed:
- Keep only bootstrap-required distfiles in early init image.
- Generate a separate raw payload image (LBPAYLD1 format) for the remaining distfiles.
- Attach payload image as an extra disk in QEMU/bare-metal kernel-bootstrap flow.
- Add a dedicated C89/tcc-compatible importer (payload-import) that scans payload disks and copies files into /external/distfiles after jump: fiwix.
- Insert improve: import_payload immediately after jump: fiwix so the full distfile set is restored before heavy build steps continue.
- Add PAYLOAD_REQUIRED config gating so this behavior is active only in kernel-bootstrap paths that need it.

Why this design:
- Preserves minimal early environment assumptions (no dependency on full shell utilities for the copy operation itself).
- Avoids adding filesystem-construction toolchain dependencies for the payload container by using a simple length-prefixed raw format.
- Keeps bare-metal and QEMU behavior aligned: both can carry full build artifacts without overloading the early handoff stage.
- Leaves kexec-fiwix behavior deterministic and auditable by reverting to a known-good baseline implementation.
2026-03-01 13:45:16 -05:00
vxtls
548f45ec1d rootfs: refresh stage0-image mirrors in bootstrap.cfg at runtime
When booting with --stage0-image, mirror ports can change between runs
(e.g. file:// -> transient SimpleMirror port), but the reused image kept
stale MIRRORS/MIRRORS_LEN values in /steps/bootstrap.cfg.

Update stage0-work image preparation to patch bootstrap.cfg on each run:
- rewrite MIRRORS and MIRRORS_LEN from current CLI mirrors
- keep existing --build-guix-also handoff checks/sync behavior

This fixes guest downloads trying old 10.0.2.2:<stale-port> endpoints
during steps-guix builds.
2026-02-21 13:49:49 -05:00
vxtls
6fcb8f4d4a bootstrap: add stage0 image reuse for guix and simplify guix handoff flow
- add --stage0-image in rootfs.py for qemu to boot an existing kernel-bootstrap image
- when --stage0-image is combined with --build-guix-also, update BUILD_GUIX_ALSO in image config and sync /steps-guix into the image
- require stage0 /init to contain guix handoff marker instead of patching /init implicitly
- add run_steps_guix_if_requested() to make_bootable-generated /init so rebooted stage0 images can enter steps-guix directly
- run /steps-guix/0.sh with bash in after.sh
- make script-generator start mode convention-based: /steps stays kaem-first, alternate roots (e.g. /steps-guix) start in bash
- remove redundant explicit /steps config-root argument from seed/preseeded/reconfigure script-generator calls
2026-02-18 14:35:17 -05:00
vxtls
21ddab36c3 script-generator: support explicit manifest/config roots and wire all callers 2026-02-18 10:14:17 -05:00
Samuel Tyler
bbccded731
Use a "real" copyright line 2025-10-17 18:53:10 +11:00
Andrius Štikonas
3d1d1e751f
Merge pull request #528 from fosslinux/mirror-docs
Document mirrors a bit better
2025-09-08 01:04:45 +01:00
fosslinux
00e39e13d0 Force people to include a mirror 2025-07-17 17:47:43 +10:00
Alexandre Gomes Gaigalas
6e5e42307b Allow --qemu and file:// --mirrors to be used together
When using file:// mirrors, rootfs.py will spawn a local HTTP
server at 127.0.0.1.

In combination with --qemu, this poses a problem: downloads
on the host machine will work, but downloads on the guest
machine will not reach the local server.

This commit introduces a change to rewrite the address to
10.0.2.2 inside the guest, only when QEMU=True is set in
the configuration, allowing this combination to build
past the get_network improvement step.
2025-06-11 22:28:28 -03:00
Andrius Štikonas
f3553639f9 Improve support for pre-python 3.12 f-strings. 2025-02-04 23:55:16 +00:00
fosslinux
7d50a224c6 Support file:// mirrors
Spawns a simple HTTP server to host the mirror.
Useful for testing mirror support or CI in chroot or bwrap modes.
2025-02-02 10:02:32 +11:00
fosslinux
df1c9e9aba Support mirrors within 2025-02-02 10:02:32 +11:00
fosslinux
4c08ba4de3 Integrate configurator with rootfs.py 2024-05-28 21:57:36 +10:00
Gábor Stefanik
c1c6b278e6 Use qemu's GUI support in interactive mode
Previously, there was no way to access the additional consoles in
qemu mode, due to qemu running in -nographic mode with a serial
console. This is perfectly fine in non-interactive mode, but we can
make interactive mode more usable by using a graphical console like
on bare metal.

Also drop -no-reboot from interactive mode, since rebooting (e.g.
to load a newly compiled kernel) makes sense in this case.
2024-02-25 02:39:43 +01:00
Gábor Stefanik
32138d13b9 Clean pre-Bash build artifacts before building the Linux kernel
All of these are archived, in a compressed form, in base.tar.bz2.
2024-02-17 15:32:02 +01:00
fosslinux
2182076773 Use a constant umask 2024-01-26 10:29:57 +11:00
rick-masters
82235e5bd0 Account for int and NoneType in string concatenations. 2024-01-10 18:03:50 +00:00
fosslinux
01a4f90378
Merge pull request #388 from Googulator/kernel-bootstrap-jobs
Restore multicore build support with kernel-bootstrap
2024-01-10 07:16:10 +00:00
fosslinux
31753cccb5
Merge pull request #386 from Googulator/debug-trap
Support interactive issue resolution prompts
2024-01-10 07:15:33 +00:00
Gábor Stefanik
50d1d68f99 Restore multicore build support with kernel-bootstrap
This was removed as part of the simplify refactor, severely slowing
down qemu and bare-metal builds. Restoring it brings us back to the
same build times that we saw before the refactor.
2024-01-08 12:56:15 +01:00
Gábor Stefanik
a0f13c0ed2 Support interactive issue resolution prompts
This adds a new flag, -i / --interactive, which enables opening a
Bash prompt whenever something goes wrong in the bootstrap. This is
highly useful when developing or debugging live-bootstrap, but it
needs to be off by default, for use in automated processes.

In the future, asking for variables at runtime could (and perhaps
should) also be gated behind this flag.
2024-01-08 12:54:23 +01:00
Gábor Stefanik
d5008f93a5 Support swap files in qemu or bare-metal mode
This can help alleviate memory pressure when bootstrapping
with high core counts.
2024-01-02 04:32:08 +01:00
Gábor Stefanik
bb067d2856 Fix "Bootstrapping" message being printed too late in bwrap runs
This should be printed at the beginning, but instead is only shown
at the end, due to output buffering - force an explicit flush.
2023-12-28 07:25:35 +01:00
Gábor Stefanik
81e3123262 Rename tmpdir to target and always preserve it, dropping --preserve
There is nothing temporary about our "tmpdir" - its sole purpose is to
contain the final product of the bootstrap process. Thus, removing it
at the end of bootstrap amounts to doing the entire process for nothing.

To remedy this, --tmpdir is renamed --target, keeping the -t short form,
and defaulting to "target" instead of "tmp" to make its purpose clearer.
The --preserve option is removed, as the target is now always preserved.
2023-12-26 16:34:04 +01:00
Gábor Stefanik
30fb367e7a Simplify bwrap setup code and support more than 3 passes 2023-12-26 13:01:24 +11:00
Gábor Stefanik
2c6ac94974 Replace --clearenv with env - to support older bwrap
This makes bwrap work on Debian 10.
2023-12-26 13:01:24 +11:00
Gábor Stefanik
f66f9efa2c Update bwrap bootstrap code to match the simplify refactor 2023-12-26 13:01:24 +11:00
Gábor Stefanik
be1333ee8b Make tmpdir a method parameter of prepare()
Fixes pylint errors.
2023-12-22 13:08:20 +01:00
Gábor Stefanik
55d3c36e09 Move kernel-bootstrap source image out of the directory it's generated from
This way, an incomplete version of the image itself won't get included
in srcfs anymore.
2023-12-22 13:07:45 +01:00
Gábor Stefanik
8d193df823 Fix printed image paths for bare-metal bootstrap 2023-12-22 13:06:59 +01:00