feat: rewrite image builds in python - #5494
Draft
tedohayer wants to merge 11 commits into
Draft
Conversation
Replaces the substring-derived variant matrix and ~200 lines of inline bash in .github/workflows/build.yml (version resolution, tag dedup, label assembly) with one stdlib-only Python file plus golden tests. Fixes SHA_HEAD_SHORT always being empty in the old build-args file. Wires build.yml's matrix and the "Define base variables" / "Pull Images and find versions" / "Prepare build args file" / "Apply Labels" / "Generate tags" steps through build.py, via a new compute-matrix job (GitHub requires the matrix in a separate preceding job) and a "Resolve build config" step. The Containerfile and build_files/ are untouched; just_scripts/ and the Justfile are next.
…bugs Adds rechunk/sbom/test subcommands to build.py and rewires build.yml's Build Image, Run Rechunker, Generate SBOM, and Run goss tests steps to use them. Build Image now reads its target/build-args from resolve's already-computed resolved.json, so the "Prepare build args file" step is gone entirely. Rewires the Justfile's build/run-container recipes onto build.py, fixing just_scripts/build-image.sh's real bugs: it requested Fedora 43 against a repo pinned to 44, used a KERNEL_FLAVOR that matched no real akmods tag, and never passed NVIDIA_BASE. Verified end-to-end: `just build` now pulls the correct ogc-44-... akmods tag where it previously requested a nonexistent bazzite-43-... one. get-defaults.sh, list-images.sh, and cleanup-images.sh are kept rather than deleted as originally planned -- get-defaults.sh is still needed by the (out-of-scope) ISO build scripts, and the other two are independent housekeeping utilities. list-images.sh/cleanup-images.sh get a one-line fix for the new localhost/<image>:build tag scheme (dropping the stale -build suffix in their filter), plus a pre-existing --format bug in cleanup-images.sh's image-id extraction. build-image.sh and run-image.sh are deleted -- fully superseded.
Adds push/sign/sbom-attach subcommands to build.py and rewires build.yml's Push to GHCR, Sign container image, Upload SBOM, and Sign SBOM OCI Artifact steps to use them. `sign` is one command backing both cosign call sites, since they were doing the identical invocation against different refs. Keeps the double-push in cmd_push with a comment explaining why (podman-container-tools/podman#27796 makes a single push's digest unstable, and signing/SBOM/attestation all key off it), and keeps the nick-fields/retry wrapper in YAML around the whole `./build.py push` call so a transient registry failure retries the full push+tag sequence atomically, as before. Install Cosign/ORAS, the registry logins, and the actions/attest step stay in YAML -- tool installs and a marketplace action wrapping arbitrary shell aren't worth wrapping.
The previous commit accidentally picked up compiled bytecode from a local pytest run.
build_iso.yml, clean.yml, and changelog.py each carried their own hand-maintained copy of the 10-image list; clean.yml's had drifted the worst, still listing 12 retired budgie/asus/surface variants alongside the current 10. build_iso.yml and clean.yml now compute their image list from `./build.py images` in a preceding step -- build_iso.yml needed a separate compute-matrix job (same reason as build.yml's: GitHub Actions has to know a job's matrix before scheduling it), clean.yml just needed a step output since it's not a matrix. changelog.py imports IMAGES from build.py directly, since it's already stdlib-only Python living in the same repo. Note: clean.yml's package list now only tracks currently-published images. If ghcr.io still has old packages under retired names (the budgie/asus/surface variants), those no longer get pruned by this job and would need a one-off manual cleanup.
Containerfile:66 bind-mounts the firmware submodule (a real host directory) into the build container. On any SELinux-enforcing host -- Bazzite included -- this fails with "Permission denied", because the confined container_t domain the build runs under has no read access to files labeled user_home_t, which is what a normal checkout gets. Every other --mount=type=bind in this file mounts from a previous build stage (from=ctx/akmods/etc.), which is already buildah-managed storage correctly labeled container_file_t, so this is the only affected line in the whole Containerfile. GitHub's runners don't run SELinux in enforcing mode, so CI never surfaces this -- it only shows up building locally on an SELinux host. Adding relabel=shared makes buildah relabel the mount to the shared container type itself, durably, on every build -- no per-run flags, no manual chcon needed. Verified end-to-end: reset firmware/ to its natural user_home_t label, then a plain `just build` completed clean from that state with no workarounds. relabel is an SELinux-specific mechanism (buildah checks whether SELinux is enabled before attempting to relabel anything), so this is a no-op on non-SELinux hosts like GitHub's ubuntu-26.04 runners.
Ports unstable's "Start switch to chunkah" commit (a831c86) into build.py: cosign-verifies quay.io/coreos/chunkah:latest before use, carries Env/Cmd/containers.bootc over via a podman-inspect config file, strips stale ostree/rpm-ostree/buildah bookkeeping labels that shouldn't survive into the published image, and drops the old buildah-unshare squash step entirely -- chunkah prunes /sysroot, /run, and /tmp itself. No longer needs --privileged. build.yml's "Run Rechunker" step already called ./build.py rechunk, so the workflow only needed two changes: rename it to "Run Chunkah" to match, and move Install Cosign earlier (unconditional, no PR guard) since chunkah verification now has to happen on every build, not just ones that reach the signing steps. Verified via --dry-run against a simulated resolve->labels->rechunk chain: argument order and label handling match the ported bash exactly (labels file entries first, then the five stale-label strips appended after, --max-layers as a separate token not --max-layers=N).
tedohayer
requested review from
EyeCantCU,
HikariKnight,
KyleGospo and
noelmiller
as code owners
August 20, 2026 17:01
tedohayer
changed the title
tedohayer
changed the title
Not up to standards ⛔
|
tedohayer
marked this pull request as draft
August 20, 2026 17:57
KyleGospo
force-pushed
the
unstable
branch
5 times, most recently
from
August 27, 2026 06:59
f0fa44e to
fa4c66a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the ~200 lines of inline bash spread across
.github/workflows/build.ymlandjust_scripts/with a single stdlib-only Python file (build.py, zero runtime dependencies) that both CI and a contributor's laptop call, so local builds and CI builds resolve identically.build.py— the 10-image variant table (single source of truth, replacing four separately hand-maintained copies of the same list acrossbuild.yml,build_iso.yml,clean.yml, andchangelog.py), pure/unit-tested resolution logic for versions, tags, dedup, and labels (test_build.py, 45 golden tests), and one subcommand per CI step:doctor,images,matrix,resolve,build,shell,rechunk,sbom,test,push,sign,sbom-attach.build.yml— every step that used to be inline bash now calls the matchingbuild.pysubcommand. The matrix moved into a separatecompute-matrixjob since GitHub Actions needs it known before scheduling.Justfile—just build/just runnow route throughbuild.pyinstead of the oldjust_scripts/build-image.sh/run-image.shbuild_iso.yml/clean.yml/changelog.py— now read the image list frombuild.py's table instead of their own copies.clean.yml's had drifted the worst, still listing 12 retired budgie/asus/surface variants.Containerfile— one-line fix (relabel=sharedon the firmware submodule's bind mount) for a SELinux denial. I needed this to build locally on Bazzite 44.Test plan
python3 -m pytest test_build.py— 45/45 passingruff check build.py test_build.py— cleanjust just-check— cleanyaml.safe_loadchangelog.pyimportsIMAGESfrombuild.pyand still runs (--help)bazzite(kinoite) via unmodifiedjust build— succeeded,bootc container lintpassed 13/13./build.py --dry-run rechunkreproduces the exact chunkah invocation from the ported bash