Skip to content

Commit 21a77f7

Browse files
ajlennonajlennon
authored
ci: bootstrap layer gate and repair Jaguar baseline (#51)
* bsp: fix existing Jaguar U-Boot patches Pin the focused LmP v96 backport that corrects stale patch context for the handheld and Phasora images. Assisted-by: Codex * ci: enforce layer adoption regression gate * ci: fix layer gate expressions * ci: bound layer gate build concurrency * ci: use current Actions runtimes * ci: emit layer matrix without jq * ci: route Yocto jobs to capable runner * ci: strengthen layer adoption evidence * ci: preserve protected layer adoption tuples * ci: test protected layer adoption matrix * ci: protect exact product feature tuples * ci: exercise signing in layer adoption gate * ci: use valid job-level key path * ci: bound layer gate disk usage * ci: consolidate layer adoption regression on ai-tools * ci: run container steps with bash * ci: keep kas overlay inside worktree * ci: quote layer state normalisation safely * ci: replay captured command failures * ci: initialize protected layer submodules * ci: initialize protected layer worktrees * ci: share kas worktree preparation * ci: track current bitbake dependency graph * ci: diagnose selected environment mismatches * ci: use inode-aware disk thresholds * ci: bind local kas changes to adoption gate * ci: validate kernel signing identity * ci: force test signing paths * ci: canonicalise layer adoption evidence * ci: keep adoption gate on controlled runner * ci: key baseline cache by capture schema * ci: source warnings from cooker logs * ci: normalise encoded checkout provenance * ci: audit immutable baseline repair Build both sides with the exact two-file Jaguar U-Boot backport while validating the old and new submodule pins, repair branch, ancestry and changed-file set. Assisted-by: Codex * ci: include packaging baseline repair Extend the exact LmP v96 backport contract to cover board-scripts empty-directory QA failures while preserving full baseline and candidate builds. Assisted-by: Codex * ci: include Handheld DTS baseline repair Extend the exact LmP v96 backport contract to the undefined Handheld audio-mute pinmux while preserving complete symmetric builds. Assisted-by: Codex * ci: include Handheld DTS baseline repair Extend the exact LmP v96 backport contract to the current EVK base and inherited regulator required by the pinned kernel. Assisted-by: Codex * ci: retain repaired empty package baseline Extend the exact LmP v96 backport pin so products that request board-scripts can complete rootfs even when their payload is intentionally empty. Assisted-by: Codex * ci: include remaining DTS baseline repairs Extend the exact LmP v96 backport contract to the INST and Phasora EVK-base corrections found by the full product gate. Assisted-by: Codex * ci: make kernel warnings sstate-independent Force kernel_configcheck on both sides before collecting cooker warnings so shared sstate cannot create false warning deltas. Assisted-by: Codex * ci: include Phasora SPDX baseline repair Extend the exact LmP v96 repair contract to the custom loader license text required by create-spdx. Assisted-by: Codex * ci: match renamed license repair surface Track Git's canonical renamed path in the exact baseline repair file set. Assisted-by: Codex * ci: treat removed warnings as improvements Keep the gate fail-closed for candidate-added warnings while avoiding false failures when shared sstate suppresses a pre-existing baseline warning. Assisted-by: Codex <codex@openai.com> * ci: shard layer adoption tuples Run every protected tuple as an independent fail-fast-disabled matrix shard while preserving the single required Layer Adoption Gate aggregator. Keep the shared local driver full-matrix by default. Signed-off-by: Alex Lennon <ajlennon@dynamicdevices.co.uk> Assisted-by: Codex * ci: materialize LFS-backed KAS configs Resolve protected Git LFS KAS inputs in the shared regression driver for both baseline and candidate worktrees, preserving identical local and CI preparation and failing closed if smudging does not occur. Signed-off-by: Alex Lennon <ajlennon@dynamicdevices.co.uk> Assisted-by: Codex * ci: retire Jaguar inst and Phasora tuples Record the product-owner retirement and remove the corresponding image and mfgtool shards from the protected build matrix.\n\nAssisted-by: Codex <noreply@openai.com> * ci: focus adoption gate on Jaguar screen Run the current integration phase against the screen image and recovery tuple, while documenting that other active products remain deferred rather than deprecated.\n\nAssisted-by: Codex <noreply@openai.com>
1 parent dda5440 commit 21a77f7

31 files changed

Lines changed: 2187 additions & 218 deletions

‎.gitattributes‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,6 @@
55
*.bin filter=lfs diff=lfs merge=lfs -text
66
*.itb filter=lfs diff=lfs merge=lfs -text
77
*mfgtool* filter=lfs diff=lfs merge=lfs -text
8+
# Build entry points are source, even when their names contain "mfgtool".
9+
scripts/kas-build-mfgtools.sh -filter -diff -merge text eol=lf
810
fitImage-* filter=lfs diff=lfs merge=lfs -text

‎.github/actionlint.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
self-hosted-runner:
2+
labels:
3+
- yocto

‎.github/workflows/kas-build-ci.yml‎

Lines changed: 0 additions & 203 deletions
This file was deleted.

‎.github/workflows/layer-adoption-gate.yml‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
name: Layer Adoption Gate
2+
3+
on:
4+
pull_request:
5+
branches: [main, develop]
6+
push:
7+
branches: [main, develop]
8+
workflow_dispatch:
9+
inputs:
10+
base_sha:
11+
description: Baseline commit to compare
12+
required: true
13+
14+
concurrency:
15+
group: layer-adoption-${{ github.event.pull_request.number || github.ref }}
16+
cancel-in-progress: true
17+
18+
defaults:
19+
run:
20+
shell: bash
21+
22+
jobs:
23+
detect:
24+
name: Detect material layer change
25+
runs-on: [self-hosted, Linux, X64, yocto, ai-tools]
26+
outputs:
27+
material: ${{ steps.detect.outputs.material }}
28+
base_sha: ${{ steps.base.outputs.sha }}
29+
tuple_ids: ${{ steps.detect.outputs.tuple_ids }}
30+
steps:
31+
- uses: actions/checkout@v7
32+
with:
33+
fetch-depth: 0
34+
- id: base
35+
name: Resolve immutable baseline
36+
env:
37+
PR_BASE: ${{ github.event.pull_request.base.sha }}
38+
PUSH_BASE: ${{ github.event.before }}
39+
INPUT_BASE: ${{ inputs.base_sha }}
40+
run: |
41+
sha="${PR_BASE:-${INPUT_BASE:-${PUSH_BASE:-}}}"
42+
if [ -z "$sha" ] || printf '%s' "$sha" | grep -Eq '^0+$'; then
43+
sha=$(git rev-parse HEAD^)
44+
fi
45+
git cat-file -e "$sha^{commit}"
46+
echo "sha=$sha" >> "$GITHUB_OUTPUT"
47+
- id: detect
48+
name: Validate repository and require an adoption contract
49+
run: |
50+
python3 -m unittest discover -s scripts/validation/tests -p 'test_*.py'
51+
find scripts -type f -name '*.sh' -print0 | xargs -0 -r -n1 bash -n
52+
python3 -m json.tool ci/layer-adoption-contract.json >/dev/null
53+
python3 -m json.tool ci/layer-adoption-tuples.json >/dev/null
54+
# Mail-format patch payloads legitimately contain the conventional
55+
# "-- " separator. Check repository sources without rewriting the
56+
# third-party patches that BitBake applies.
57+
git diff --check '${{ steps.base.outputs.sha }}...${{ github.sha }}' -- . ':(exclude)**/*.patch'
58+
python3 scripts/validation/detect-layer-adoption.py \
59+
--base '${{ steps.base.outputs.sha }}' \
60+
--head '${{ github.sha }}' \
61+
--github-output "$GITHUB_OUTPUT"
62+
tuple_ids=$(python3 -c 'import json; print(json.dumps([entry["id"] for entry in json.load(open("ci/layer-adoption-tuples.json"))["tuples"]], separators=(",", ":")))')
63+
echo "tuple_ids=$tuple_ids" >> "$GITHUB_OUTPUT"
64+
regression:
65+
name: Existing product regression (${{ matrix.tuple_id }})
66+
needs: detect
67+
if: needs.detect.outputs.material == 'true'
68+
strategy:
69+
fail-fast: false
70+
matrix:
71+
tuple_id: ${{ fromJSON(needs.detect.outputs.tuple_ids) }}
72+
# Each tuple is an independent shard so one failure cannot hide later
73+
# product failures. The final Layer Adoption Gate remains the single
74+
# branch-protection contract, and local driver runs still cover all tuples.
75+
runs-on: [self-hosted, Linux, X64, yocto, ai-tools]
76+
container:
77+
image: ghcr.io/siemens/kas/kas@sha256:d989add57fc441fe9e27bb2dd6ed98c5597b44c807928e35a72dc1cfbdda9abe
78+
# Match the dedicated ai-tools runner account so BitBake's root-user
79+
# sanity check remains active and bind-mounted cache files stay writable.
80+
options: --privileged --platform linux/amd64 --user 1002:1002 -v /home/ghrunner/yocto-layer-adoption:/var/cache/layer-adoption
81+
env:
82+
LAYER_ADOPTION_CACHE: /var/cache/layer-adoption
83+
steps:
84+
- name: Reset job-owned workspace
85+
run: |
86+
workspace=$(realpath -m -- "$GITHUB_WORKSPACE")
87+
test -n "$workspace"
88+
test "$workspace" != /
89+
for name in candidate baseline evidence; do
90+
target=$(realpath -m -- "$workspace/$name")
91+
test "$(dirname "$target")" = "$workspace"
92+
rm -rf -- "$target"
93+
done
94+
- uses: actions/checkout@v7
95+
with:
96+
fetch-depth: 0
97+
path: candidate
98+
- name: Create baseline worktree
99+
working-directory: candidate
100+
run: git worktree add ../baseline '${{ needs.detect.outputs.base_sha }}'
101+
- name: Require safe build capacity
102+
run: |
103+
available_kib=$(df -Pk "$GITHUB_WORKSPACE" | awk 'NR == 2 {print $4}')
104+
minimum_kib=$((150 * 1024 * 1024))
105+
if [ "$available_kib" -lt "$minimum_kib" ]; then
106+
echo "ERROR: layer-adoption build requires at least 150 GiB free; found $((available_kib / 1024 / 1024)) GiB" >&2
107+
exit 1
108+
fi
109+
- name: Prepare persistent test-only signing identity
110+
run: |
111+
set -euo pipefail
112+
keys="$LAYER_ADOPTION_CACHE/test-keys"
113+
if ! candidate/scripts/validation/generate-layer-adoption-test-keys.sh \
114+
--check "$keys"; then
115+
test "$keys" = /var/cache/layer-adoption/test-keys
116+
rm -rf -- "$keys"
117+
rm -rf -- "$LAYER_ADOPTION_CACHE/baselines"
118+
temporary=$(mktemp -d "$LAYER_ADOPTION_CACHE/.test-keys.XXXXXX")
119+
candidate/scripts/validation/generate-layer-adoption-test-keys.sh "$temporary"
120+
mv "$temporary" "$keys"
121+
fi
122+
- name: Build and compare every protected tuple
123+
run: |
124+
python3 candidate/scripts/validation/run-layer-adoption-regression.py \
125+
--baseline baseline \
126+
--candidate candidate \
127+
--evidence evidence \
128+
--cache "$LAYER_ADOPTION_CACHE" \
129+
--test-keys "$LAYER_ADOPTION_CACHE/test-keys" \
130+
--tuple-id '${{ matrix.tuple_id }}'
131+
- name: Preserve comparison evidence
132+
if: always()
133+
uses: actions/upload-artifact@v7
134+
with:
135+
name: layer-adoption-evidence-${{ matrix.tuple_id }}
136+
path: evidence
137+
retention-days: 14
138+
- name: Remove job-owned build trees
139+
if: always()
140+
run: |
141+
workspace=$(realpath -m -- "$GITHUB_WORKSPACE")
142+
test -n "$workspace"
143+
test "$workspace" != /
144+
for name in candidate/build baseline/build evidence; do
145+
target=$(realpath -m -- "$workspace/$name")
146+
case "$target" in
147+
"$workspace/candidate/build"|"$workspace/baseline/build"|"$workspace/evidence") ;;
148+
*) echo "ERROR: refusing unsafe cleanup target: $target" >&2; exit 1 ;;
149+
esac
150+
rm -rf -- "$target"
151+
done
152+
153+
required:
154+
name: Layer Adoption Gate
155+
needs: [detect, regression]
156+
if: always()
157+
runs-on: [self-hosted, Linux, X64, yocto, ai-tools]
158+
steps:
159+
- name: Enforce gate result
160+
env:
161+
DETECT: ${{ needs.detect.result }}
162+
MATERIAL: ${{ needs.detect.outputs.material }}
163+
REGRESSION: ${{ needs.regression.result }}
164+
run: |
165+
test "$DETECT" = success
166+
if [ "$MATERIAL" = true ]; then
167+
test "$REGRESSION" = success
168+
else
169+
test "$REGRESSION" = skipped
170+
fi

0 commit comments

Comments
 (0)