Skip to content

Commit 9b73def

Browse files
committed
Review of #321: NOT YET DUE for final-look only and only on the derived readout; unresolvable stamp falls back to last-in-file
- NOT YET DUE now applies to final-look alone, and only when the Current stage line is gate-check's own derived readout. Hand-written and scaffold lines read as unreadable, so the row stays owed. Every other obligation keeps master's verdict. - _ob_newest_stamp: if any VALID AT stamp does not resolve (squash, rebase, shallow clone), use the last stamp in the file as master did. Before, the older resolvable stamp won and the report read STALE. - CHANGELOG: cite #155's field report as the motivation and say the checker is proven by fixtures only. Fixtures: valid-at T3b; final-look T9-T11 (each fails on the previous head). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
1 parent 8c0b176 commit 9b73def

4 files changed

Lines changed: 57 additions & 31 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,8 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are.
1717

1818
## Unreleased
1919

20-
- fix(obligations): **Each `PROOF-OF-LOOK` line is checked against the `VALID AT` stamp of its own section, and the report's newest stamp is chosen by commit time ([#155](https://github.com/MendixMau/mxcli-project-toolkit/issues/155)).** A review file that keeps dated addenda was still FAULTing: every proof line was compared with the single newest stamp, so the first section's honest screenshots read "older than VALID AT". The field project's only way back to green was to re-shoot every page and delete its older proof lines. A citation now belongs to the nearest stamp above it (one above every stamp belongs to the headline's), on pretty-printed and one-line HTML alike. "Newest" was last-in-file, so an addendum written above the original stamp went STALE; it is now the stamp with the newest commit time. A stale shot inside a section still FAULTs. Fixture `test-obligation-valid-at.sh` T4-T8 (3 fail on master). Field run 2026-09-26, requirements-driven build — MendixMau
21-
- fix(obligations): **The final LOOK is held to the per-module LOOK's proof rules, and project-wide obligations are not owed before their stage.** Found in an independent review of #303. `final-look` (from-stage 6) read PENDING on a project still at Stage P, because the from-stage column was only compared with `--adopt`. A project or cluster row whose from-stage is above the register's `## Current stage` now reads `NOT YET DUE` and is not counted as pending. Module rows are unchanged, and an unreadable stage line still means owed. `module-review.md` § "The final LOOK" requires `PROOF-OF-LOOK` lines, but the check ran only for `look`; it now runs for `final-look` too. An unstamped final-look report could never go stale and so passed forever; it is now FAULT ("NO VALID AT"). Fixture `test-obligation-final-look.sh` T4-T10 (4 fail on master). — MendixMau
20+
- fix(obligations): **Each `PROOF-OF-LOOK` line is checked against the `VALID AT` stamp of its own section, and the report's newest stamp is chosen by commit time ([#155](https://github.com/MendixMau/mxcli-project-toolkit/issues/155)).** A review file that keeps dated addenda was still FAULTing: every proof line was compared with the single newest stamp, so the first section's honest screenshots read "older than VALID AT". The field project's only way back to green was to re-shoot every page and delete its older proof lines. A citation now belongs to the nearest stamp above it (one above every stamp belongs to the headline's), on pretty-printed and one-line HTML alike. "Newest" was last-in-file, so an addendum written above the original stamp went STALE; it is now the stamp with the newest commit time. If any stamp no longer resolves (squash, rebase, shallow clone), the last stamp in the file is used, as before, and staleness reads as unmeasured. A stale shot inside a section still FAULTs. Motivated by the field report in #155 (a requirements-driven build, 2026-09-26). The checker is proven by fixtures only (`test-obligation-valid-at.sh` T3b, T4-T8; 3 fail on master); no field run against a real report yet — MendixMau
21+
- fix(obligations): **The final LOOK is held to the per-module LOOK's proof rules, and is not owed before Stage 6.** Found in an independent review of #303. `final-look` (from-stage 6) read PENDING on a project still at Stage P, because the from-stage column was only compared with `--adopt`. It now reads `NOT YET DUE` while gate-check's own derived `## Current stage` readout (the line carrying "(derived by gate-check") is below 6. A hand-written or scaffold stage line counts as unreadable, so the row stays owed; hand-written lines disagreed with the gates in 8 of 12 field projects. Every other obligation keeps its verdict. `module-review.md` § "The final LOOK" requires `PROOF-OF-LOOK` lines, but the check ran only for `look`; it now runs for `final-look` too. An unstamped final-look report could never go stale and so passed forever; it is now FAULT ("NO VALID AT"). Fixture `test-obligation-final-look.sh` T4-T11 (3 fail on master). — MendixMau
2222
- learn(brd-to-build-plan): **No `LOOK` row kind; the paragraph next to the Kind list says why ([#68](https://github.com/MendixMau/mxcli-project-toolkit/issues/68), [#188](https://github.com/MendixMau/mxcli-project-toolkit/issues/188) item 6).** A look is a `HARNESS` row naming the LOOK pass. Whether it happened is read from the `look` and `fidelity` obligations and the look-ledger, never from a State cell a person can set by hand. A second record could say "looked at" while the gate says nobody did. — MendixMau
2323
- learn(testing-shape, test-result-audit): **three test-strength rules, each with a denominator: refusals probed on the server, every undeclared 4xx a failure, one clean-database final run** ([#172](https://github.com/MendixMau/mxcli-project-toolkit/issues/172)). A field comparison of two builds against one frozen spec found the toolkit build checked permission refusals only in the client (a hidden menu item), logged unexpected 400/422 responses as information, and reached 20/20 by stitching three runs. `testing-shape.md` §7 gains three report lines (`N of N restricted pairs probed server-side`, `undeclared 4xx/5xx: 0 of M requests — each fails its step`, `one clean-DB run <id>: N of N scenarios incl. timers, not stitched`); `test-result-audit.md` gains check 2e, which carries the reasons and the field evidence and asks for those numbers. Both baseline skills stay inside the 80,000-word baseline budget (79,997): the reasons live in the on-demand audit skill, and two redundant sentences in `testing-shape.md` §4's verdict paragraphs were folded (the empty-set corollary is stated in the Trace row, `harness-architecture.md` and `report-schema.md`). The backing inbox note is trimmed to its unpromoted "Keep" items. — MendixMau
2424
- learn(learned-detection-gaps, module-review): **two UI defects every model gate passes: a lazy-loaded combo box filter hides its preset, and a theme that pins row heights by position lets the next row cover an inserted widget** ([#299](https://github.com/MendixMau/mxcli-project-toolkit/issues/299)). Both passed check, mxbuild, lint and the scripted checks on a 127-module production app and were found only in the running app. Two register rows with the first oracle and the pre-build check (pluggable properties that change *when* data loads; grep the theme for fixed `height` on positional row classes before an insert), and two LOOK lines in `module-review.md` §4c with denominators (`N of N filters show their preset on first load`, `N of N widgets inserted into an existing container fully visible`). — jopterhorst

‎bin/lib/obligation-check.sh‎

Lines changed: 26 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
# FAULT a mark exists but cannot be read as evidence (no denominator on a pass that owes one)
2020
# WAIVED the register says this one is out of scope, with a reason
2121
# ADOPTED the project joined the pipeline above this obligation's stage
22-
# NOT YET DUE a project/cluster row whose from-stage the register's Current stage has not reached
22+
# NOT YET DUE final-look only: gate-check's derived Current stage has not reached its from-stage
2323
# N/A this project is not running the pipeline at all (a-la-carte; existing-app-assurance.md)
2424

2525
set -uo pipefail
@@ -167,18 +167,23 @@ _ob_rank() { case "$1" in P|p) echo 0 ;; [0-7]) echo $(( $1 + 1 )) ;; *) echo 99
167167
# _ob_current_stage <register> → prints P or 0-7 from the register's "## Current stage" line, or
168168
# nothing when it cannot be read (no section, or a hand-written line that names no stage).
169169
#
170-
# Why: the from-stage column was read only against --adopt, so a project/cluster row was owed
171-
# from the day PROJECT.md existed. final-look (from-stage 6, the whole-app LOOK run LAST) read
172-
# PENDING on a project still at Stage P — an obligation reported before it can be performed is
173-
# noise, and noise teaches people to skip the obligation lines (review of #303, 2026-10-13).
174-
# The readout is gate-check's own ("**Stage N — Title**, …", or "**All gates passed**"); the
175-
# scaffold line init-project.sh writes ("**Stage P — Kickoff**") parses the same way.
176-
# Unreadable → empty → the caller keeps the old behaviour (owed): "could not tell the stage"
177-
# must never read as "not due yet".
170+
# Why: the from-stage column was read only against --adopt, so final-look (from-stage 6, the
171+
# whole-app LOOK run LAST) read PENDING on a project still at Stage P (review of #303).
172+
#
173+
# ONLY gate-check's own derived readout counts — a line containing "(derived by gate-check".
174+
# A hand-written line, and the scaffold line init-project.sh writes ("**Stage P — Kickoff**"),
175+
# read as unreadable → empty → owed. Field evidence (review of #321): hand-written lines
176+
# disagreed with the gates in 8 of 12 projects and gate-check never overwrites them; the
177+
# scaffold line often never advances because projects query `gate-check.sh . 5` directly.
178+
# "Could not tell the stage" must never read as "not due yet".
178179
_ob_current_stage() {
179180
local reg="$1" line
180181
[ -f "$reg" ] || return 0
181182
line="$(awk '/^## Current stage/{f=1;next} f && /^## /{exit} f && /^\*\*/{print;exit}' "$reg" 2>/dev/null | tr -d '\r')"
183+
case "$line" in
184+
*"(derived by gate-check"*) ;;
185+
*) return 0 ;;
186+
esac
182187
case "$line" in
183188
"**All gates passed"*) printf '7'; return 0 ;;
184189
esac
@@ -270,22 +275,25 @@ _ob_has_denominator() {
270275
# "Newest" is by COMMIT TIME, not by position (#155). A report kept as one dated file with an
271276
# addendum per build row carries several stamps; whether the author appends addenda at the
272277
# bottom or prepends them under the headline, the stamp that says the most recent thing is the
273-
# one whose commit is newest. Unresolvable stamps are skipped; with none resolvable, the last
274-
# stamp in the file is returned (and the callers then treat it as unmeasurable).
278+
# one whose commit is newest. If ANY stamp does not resolve here (history rewritten by a squash
279+
# or rebase, a shallow clone), the order of the stamps cannot be measured, so the last stamp in
280+
# the file is returned, as before: picking an older resolvable stamp instead would report STALE
281+
# on a report this checker cannot actually date (review of #321).
275282
_ob_valid_at_stamps() {
276283
tr -d '\r' < "$1" 2>/dev/null \
277284
| grep -oiE 'VALID[[:space:]]+AT:?[[:space:]]*[0-9a-f]{7,40}' \
278285
| grep -oiE '[0-9a-f]{7,40}$'
279286
}
280287
_ob_newest_stamp() {
281-
local root="$1" hit="$2" h t best="" best_t=-1 last=""
288+
local root="$1" hit="$2" h t best="" best_t=-1 last="" unresolved=0
282289
while IFS= read -r h; do
283290
[ -n "$h" ] || continue
284291
last="$h"
285292
t="$(git -C "$root" show -s --format=%ct "${h}^{commit}" 2>/dev/null || true)"
286-
[ -n "$t" ] || continue
293+
if [ -z "$t" ]; then unresolved=1; continue; fi
287294
if [ "$t" -ge "$best_t" ]; then best="$h"; best_t="$t"; fi
288295
done <<< "$(_ob_valid_at_stamps "$hit")"
296+
[ "$unresolved" = 1 ] && best=""
289297
printf '%s' "${best:-$last}"
290298
}
291299
_ob_stale() {
@@ -452,10 +460,11 @@ mxtk_obligations_report() {
452460
continue
453461
fi
454462

455-
# Not yet due: a project/cluster row whose stage the register has not reached. Module rows
456-
# are not touched — they are owed by a module being OPENED FOR WORK (_ob_modules), which is
457-
# its own "due" test, and their verdicts stay exactly as before.
458-
if [ "$scope" != "module" ] && [ "$(_ob_rank "$from_stage")" -gt "$current_rank" ]; then
463+
# Not yet due: final-look ONLY, the one row that by definition runs last. Every other row
464+
# keeps its verdict whatever the readout says — an early stale FAIL holds the derived
465+
# readout back while the build goes on, and a general rule would hide owed passes behind it
466+
# (review of #321).
467+
if [ "$ob" = "final-look" ] && [ "$(_ob_rank "$from_stage")" -gt "$current_rank" ]; then
459468
printf 'Obligation %-10s NOT YET DUE — owed from stage %s; the register'"'"'s Current stage is %s (%s)\n' \
460469
"$ob" "$from_stage" "$current" "$skill"
461470
continue

‎tests/wave2/test-obligation-final-look.sh‎

Lines changed: 25 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,9 @@
1212
# Review of #303 (2026-10-13), T4-T10: the final LOOK owes the same PROOF-OF-LOOK lines as the
1313
# per-module LOOK (module-review.md § "The final LOOK" requires them; only ob=look was checked);
1414
# an unstamped final report is FAULT (with no VALID AT it could never go stale, so it would pass
15-
# forever); and the row is NOT YET DUE until the register's Current stage reaches its from-stage
16-
# (6) — it read PENDING from the day PROJECT.md existed. Module rows keep their verdicts.
15+
# forever); and final-look (ONLY) is NOT YET DUE until gate-check's DERIVED Current stage
16+
# reaches 6. A hand-written or scaffold stage line is unreadable → owed; every other row keeps
17+
# its verdict (review of #321: hand-written lines disagree with the gates in 8 of 12 projects).
1718
# Subject: obligation-check.sh. Producer of the report: module-review.md § "The final LOOK".
1819
set -uo pipefail
1920
ARG="${1:?usage: test-obligation-final-look.sh /path/to/bin/gate-check.sh (or lib/obligation-check.sh)}"
@@ -84,26 +85,38 @@ printf '# PROJECT.md\n\n## Current stage\n\n**Stage 5 — Build**, in progress (
8485
l="$(line)"
8586
case "$l" in *"NOT YET DUE"*) ok "final-look not owed before stage 6" ;; *) bad "expected NOT YET DUE, got: $l" ;; esac
8687

87-
echo "== T9: register at Stage 6, or all gates passed -> owed (PENDING) =="
88-
printf '# PROJECT.md\n\n## Current stage\n\n**Stage 6 — Test**, in progress.\n' > "$P/PROJECT.md"
88+
D=' (derived by gate-check on 2026-10-13; re-run it rather than editing this line).'
89+
echo "== T9: derived Stage 6, or all gates passed -> owed (PENDING) =="
90+
printf '# PROJECT.md\n\n## Current stage\n\n**Stage 6 — Test**, in progress%s\n' "$D" > "$P/PROJECT.md"
8991
l="$(line)"
9092
case "$l" in *PENDING*) ok "owed at stage 6" ;; *) bad "expected PENDING at stage 6, got: $l" ;; esac
91-
printf '# PROJECT.md\n\n## Current stage\n\n**All gates passed** — through Stage 7 (Cutover)\n' > "$P/PROJECT.md"
93+
printf '# PROJECT.md\n\n## Current stage\n\n**All gates passed** — through Stage 7 (Cutover) — gates passed: P, 0%s\n' "$D" > "$P/PROJECT.md"
9294
l="$(line)"
9395
case "$l" in *PENDING*) ok "owed when all gates passed" ;; *) bad "expected PENDING after all gates, got: $l" ;; esac
9496
printf '# PROJECT.md\n\n## Current stage\n\nWe are somewhere in the build.\n' > "$P/PROJECT.md"
9597
l="$(line)"
9698
case "$l" in *PENDING*) ok "unreadable stage keeps the old verdict (owed)" ;; *) bad "unreadable stage changed the verdict: $l" ;; esac
9799

98-
echo "== T10: module rows keep their verdicts at Stage P =="
99-
mod() { "$OBCHECK" "$P" 2>&1 | grep -E '^Obligation (look|sweep|journeys|fidelity|dispatch) ' | sed -E 's/ +/ /g'; }
100-
rm -f "$P/PROJECT.md"; before="$(mod)"
100+
echo "== T10: a hand-written stage line is not honoured -> owed =="
101+
printf '# PROJECT.md\n\n## Current stage\n\n**Stage 5 — Build**, in progress.\n' > "$P/PROJECT.md"
102+
l="$(line)"
103+
case "$l" in *PENDING*) ok "hand-written Stage 5 line ignored, final-look owed" ;; *) bad "hand-written line honoured: $l" ;; esac
104+
printf '# PROJECT.md\n\n## Current stage\n\n**Stage 5 verdict: COMPLETE**\n' > "$P/PROJECT.md"
105+
l="$(line)"
106+
case "$l" in *PENDING*) ok "free-form hand-written line ignored, final-look owed" ;; *) bad "free-form line honoured: $l" ;; esac
107+
108+
echo "== T11: scaffold line with a module open -> final-look owed; module and other rows unchanged =="
109+
others() { "$OBCHECK" "$P" 2>&1 | grep -E '^Obligation ' | grep -vE '^Obligation final-look ' | sed -E 's/ +/ /g'; }
110+
rm -f "$P/PROJECT.md"; before="$(others)"
101111
printf '# PROJECT.md\n\n## Current stage\n\n**Stage P — Kickoff**, in progress.\n' > "$P/PROJECT.md"
102-
after="$(mod)"
103-
[ -n "$before" ] && [ "$before" = "$after" ] && ok "module verdicts identical with and without a Stage P register" \
104-
|| bad "module verdicts changed: before=[$before] after=[$after]"
112+
after="$(others)"
113+
l="$(line)"
114+
case "$l" in *PENDING*) ok "scaffold Stage P line ignored, final-look owed" ;; *) bad "scaffold line honoured: $l" ;; esac
115+
[ -n "$before" ] && [ "$before" = "$after" ] && ok "every other obligation line identical with and without the register" \
116+
|| bad "other verdicts changed: before=[$before] after=[$after]"
117+
printf '# PROJECT.md\n\n## Current stage\n\n**Stage 3 — Architecture**, in progress%s\n' "$D" > "$P/PROJECT.md"
105118
l="$("$OBCHECK" "$P" 2>&1 | grep -E '^Obligation skeleton ')"
106-
case "$l" in *"NOT YET DUE"*) ok "project row from stage 5 not due at Stage P" ;; *) bad "skeleton at Stage P: $l" ;; esac
119+
case "$l" in *PENDING*) ok "skeleton (from 5) still owed under a derived Stage 3 readout" ;; *) bad "skeleton hidden: $l" ;; esac
107120

108121
echo "pass=$PASS fail=$FAIL"
109122
[ "$FAIL" -eq 0 ]

‎tests/wave2/test-obligation-valid-at.sh‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,10 @@ echo "== T3: newest stamp sits past byte 8000 -> still seen =="
5050
{ printf '<p>ModuleX: 1 of 1 pages reviewed</p>\nVALID AT: %s\n' "$OLD"; head -c 9000 /dev/zero | tr '\0' 'x'; printf '\nVALID AT: %s\n' "$NEW"; } > "$REP"
5151
[ "$(stale)" -eq 0 ] && ok "stamp past 8000 bytes seen" || bad "stamp past 8000 bytes ignored (STALE)"
5252

53+
echo "== T3b: one stamp unresolvable here (squash/rebase/shallow) -> last-in-file, unmeasurable, not STALE =="
54+
printf '<p>ModuleX: 1 of 1 pages reviewed</p>\nVALID AT: %s\n<h2>Addendum</h2>\nVALID AT: deadbee1\n' "$OLD" > "$REP"
55+
[ "$(stale)" -eq 0 ] && ok "unresolvable stamp falls back to last-in-file (unmeasured)" || bad "older resolvable stamp picked over an unresolvable one (STALE)"
56+
5357
# ── T4-T8: dated report (owes PROOF-OF-LOOK), commits and shot mtimes at fixed times ──
5458
Q="$WORK/q"; mkdir -p "$Q/architecture/modules/ModuleX" "$Q/mdlsource" "$Q/design/ui-reviews/shots"
5559
git -C "$Q" init -q . 2>/dev/null

0 commit comments

Comments
 (0)