Skip to content

Commit 5dfb9dc

Browse files
committed
[docs] Specify mechanical release evidence assembly
Signed-off-by: Viwat Vchirawongkwin <viwat.v@chula.ac.th> (cherry picked from commit 70d8189143b065638eccdcd30ab9ae19b237508a)
1 parent 44f28c2 commit 5dfb9dc

3 files changed

Lines changed: 103 additions & 15 deletions

File tree

‎docs/specifications/firmware/TDD.md‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -613,11 +613,13 @@ negative tests land while release qualification remains pending. The
613613
maintainer then:
614614

615615
1. runs the engineering baseline on the two exact owned profiles;
616-
2. commits the canonical, redacted raw evidence under
617-
`docs/validation/firmware/oi1/`;
618-
3. derives every profile threshold mechanically with the frozen formulas;
619-
4. commits the populated policy and its evidence SHA-256; and
620-
5. builds the final tagged candidate and reruns verify-mode HIL on both exact
616+
2. runs `assemble-oi1-baseline` against the immutable staged inputs and the
617+
two bench fragments so the tool creates the canonical, redacted evidence
618+
under `docs/validation/firmware/oi1/`, derives every threshold with the
619+
frozen formulas, and atomically updates the policy with its evidence
620+
SHA-256;
621+
3. reviews and commits those mechanically assembled files; and
622+
4. builds the final tagged candidate and reruns verify-mode HIL on both exact
621623
profiles.
622624

623625
A policy has exactly two threshold-bearing profile entries and one deferred
@@ -632,6 +634,10 @@ matching baseline-evidence digest, and immutable build measurements in
632634
`PYBLE_HIL_RECORDS_V2`. Its runtime observation is pending. Finalization may
633635
fill observations, operator fields, and derived checks only; it must prove the
634636
policy and build portions remain byte/semantically equal to the candidate.
637+
The `assemble-hil-report` helper accepts only bounded per-profile mutable
638+
evidence, copies candidate-frozen fields from the pending report, derives the
639+
footprint/reliability pass from validated observations, and emits the completed
640+
V2 report atomically before finalization.
635641

636642
The validator recomputes image/headroom arithmetic, sample counts, heap
637643
minima, latency maximum, goodput from recorded durations, threshold

‎docs/specifications/firmware/browser-flashing.md‎

Lines changed: 52 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
11
# PyBLE ESP32-Family Browser Flashing and Release Bundle
22

3-
Status: **FROZEN v1.26** · Owner: project maintainer · Frozen:
4-
2026-07-30 (`[docs]`; pre-v1 two-profile release eligibility and explicit
3+
Status: **FROZEN v1.27** · Owner: project maintainer · Frozen:
4+
2026-07-31 (`[docs]`; pre-v1 two-profile release eligibility and explicit
55
C3 deferral; evidence-derived resource policy and exact HIL V2 records;
6-
pre-policy two-root baseline-input staging;
6+
pre-policy two-root baseline-input staging and mechanical baseline/policy
7+
assembly;
78
exact-profile manifest, C3 silicon-revision,
89
license-audit safety, candidate-finalization, and candidate pin-state
910
amendments, plus the real-tool license-evidence and exact license-catalog
@@ -18,7 +19,8 @@ markers, generated-header inputs, and direct-object reconciliation, plus
1819
component-owned linked outputs, lexical exact-path validation, nested-build
1920
logical paths, shell-free compiler/linker command receipts, executable
2021
version-matched recovery-command syntax, and the canonical pre-v1 same-origin
21-
publication channel with an optional byte-identical mirror, on the same date)
22+
publication channel with an optional byte-identical mirror, plus bounded
23+
completed-HIL report assembly, on the same date)
2224

2325
This document is the source of truth for the initial browser-provisioning
2426
release bundle. It refines
@@ -201,6 +203,30 @@ policy, baseline-evidence file, release tag, license evidence, or HIL approval.
201203
Its output is measurement input only: it is not a release candidate and MUST
202204
NOT be accepted by the website or public release validator.
203205

206+
After both exact-profile baseline runs succeed, the release tool MUST provide
207+
an `assemble-oi1-baseline` operation. It accepts the immutable staged baseline
208+
input tree above, exactly two canonical single-profile fragments emitted by
209+
the OI-1 bench, the clean canonical PyBLE proof checkout, and one explicit UTC
210+
`created_at` value. It MUST derive `source_commit` from that checkout's exact
211+
`HEAD` and `firmware_version` from its `versions.lock`; operator-supplied
212+
substitutes for either identity are forbidden. Before writing, it MUST require
213+
the proof checkout to be clean, require exactly one fragment for each profile
214+
regardless of input order, validate every fragment and observation field,
215+
bind each fragment's firmware hash, manifest hash, and build measurements to
216+
the corresponding staged bytes, and mechanically derive all nine thresholds
217+
with the frozen formulas.
218+
219+
The operation MUST canonicalize and create the baseline evidence at
220+
`docs/validation/firmware/oi1/<HEAD>.json`, compute the digest of those exact
221+
bytes, and atomically update `firmware/qualification/oi1-gates.json` with the
222+
exact frozen policy shape and derived thresholds. The baseline path is
223+
no-replace: an existing different file is fatal; an existing byte-identical
224+
file is an idempotent input. The policy update MUST be an atomic same-directory
225+
replacement, and both complete byte payloads MUST pass the production
226+
baseline/policy validator before either destination is changed. This operation
227+
is evidence assembly only; it does not approve a release or mutate staged
228+
measurement inputs.
229+
204230
The protected candidate site's build-selected SHA-256 of `release.json` is the
205231
root identity of the candidate exercised during HIL. The completed HIL
206232
evidence MUST record that exact lowercase 64-hex digest. Public finalization
@@ -1246,6 +1272,28 @@ In a candidate every value is `pending`. In a public report every value is
12461272
`passed`; `footprint_reliability` MUST be set by the validator only after the
12471273
V2 observations pass, not accepted as independent operator testimony.
12481274

1275+
The release tool MUST provide an `assemble-hil-report` operation so completing
1276+
this contract never requires hand-editing `HIL_REPORT.md`. It accepts one
1277+
immutable pending candidate, exactly two JSON completion fragments (one per
1278+
profile, in either input order), the canonical qualification checkout, and one
1279+
new no-replace output path. A completion fragment contains only the mutable
1280+
profile ID, physical board descriptions/capacities, UTC test time,
1281+
operator/sign-off and environment strings, the six operator-demonstration
1282+
checks other than `footprint_reliability`, one completed `oi1_observation`, and
1283+
the redacted console log. It MUST NOT accept status, release identity, artifact
1284+
hashes, policy, or build measurements from an operator fragment.
1285+
1286+
The assembler MUST validate the pending candidate first, compute the exact
1287+
candidate `release.json` SHA-256 itself, copy every candidate-frozen field from
1288+
the embedded pending records, require all six supplied checks to be `passed`,
1289+
validate the observation and every profile threshold, and only then insert
1290+
`footprint_reliability: passed` and `status: passed`. It MUST render exactly one
1291+
canonical `PYBLE_HIL_RECORDS_V2` marker, validate the completed payload against
1292+
the candidate bytes and committed policy, write the output atomically, and
1293+
prove the candidate and completion-fragment inputs did not change during the
1294+
operation. It never mutates the candidate and does not perform public bundle
1295+
promotion; `finalize-public` remains the only promotion step.
1296+
12491297
`oi1_observation` is JSON `null` in a pending candidate. In a completed report
12501298
it is an object with exactly:
12511299

‎tests/firmware_tests/hil/README.md‎

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -109,12 +109,26 @@ For the owned N16R8 S3, change the identity arguments to:
109109

110110
Baseline output is deliberately one **profile fragment**, not a release
111111
approval and not a malformed one-profile substitute for the frozen
112-
two-profile baseline envelope. Retain both successful fragments and assemble
113-
them, in policy order, with the common source commit, firmware version, and
114-
UTC timestamp before committing the baseline evidence. The command prints the
115-
mechanically derived profile thresholds for review; those numbers do not
116-
become policy until the retained two-profile baseline and its digest are
117-
committed.
112+
two-profile baseline envelope. Retain both successful fragments, then assemble
113+
the envelope and policy without hand-editing JSON:
114+
115+
```sh
116+
python3 ../../../firmware/scripts/release_bundle.py \
117+
assemble-oi1-baseline \
118+
<private>/oi1-inputs \
119+
<private>/esp32-4mb-baseline-profile.json \
120+
<private>/esp32-s3-n16r8-baseline-profile.json \
121+
--repo-root ../../.. \
122+
--created-at 2026-08-01T00:00:00Z
123+
```
124+
125+
The helper obtains the common source commit and firmware version from the
126+
clean proof checkout, binds both fragments to the staged bytes, creates the
127+
canonical commit-scoped evidence file, and atomically updates
128+
`firmware/qualification/oi1-gates.json` with mechanically derived thresholds.
129+
Review and commit both generated files together. The individual bench command
130+
also prints its derived thresholds for review; neither fragment nor assembled
131+
baseline is release approval.
118132

119133
After that policy exists, run the exact final candidate in verify mode:
120134

@@ -146,6 +160,26 @@ release finalizer. It does not edit a policy, release bundle, or HIL report.
146160
Any workload, integrity, threshold, or physical-power-cycle failure exits
147161
non-zero without writing a successful observation.
148162

163+
After all other protected-candidate checks pass, put each verify observation
164+
and that profile's bounded operator metadata/checks into one completion JSON
165+
fragment, then create the completed report without editing Markdown:
166+
167+
```sh
168+
python3 ../../../firmware/scripts/release_bundle.py \
169+
assemble-hil-report \
170+
<private>/candidate-v0.4.2 \
171+
<private>/esp32-4mb-hil-completion.json \
172+
<private>/esp32-s3-n16r8-hil-completion.json \
173+
<private>/completed-HIL_REPORT.md \
174+
--qualification-repo-root ../../..
175+
```
176+
177+
The completion fragment has only mutable board/operator/environment fields,
178+
six operator-demonstration checks (all `passed`), `oi1_observation`, and the
179+
redacted console log. The helper computes the selected candidate digest,
180+
copies every frozen identity/policy/build field, and derives the
181+
`footprint_reliability` pass only after validating the observation.
182+
149183
The C3 profile is intentionally refused by the CLI. A source build or license
150184
audit for `esp32-c3-4mb` is not HIL evidence and cannot enable that profile.
151185

0 commit comments

Comments
 (0)