You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit ba461e5
Browse filesBrowse the repository at this point in the historyBrowse files
report: honour aborted findings; setup kind in the schema; drop dead credentials branch
- report-normalize.js: a findings file with `aborted` (preflight refusal or
a signal) yields a report with top-level `aborted`, every kept journey
row flagged notAResult/abortedRun, and the journey-runner instrument a
fault — an aborted run never reads as results. --selftest cases; and
test-journey-preflight.sh runs the real runner's aborted file through
the real normalizer (fails on master's normalizer).
- skills/report-schema.md: kind enum gains `setup`; `aborted` documented.
- config.js: drop the `credentials().source === 'template default'`
branch no template emits; the Demo2025! placeholder rule stays.
Refs #286
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
Copy file name to clipboardExpand all lines: CHANGELOG.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,7 +17,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are.
17
17
18
18
## Unreleased
19
19
20
-
- fix(journeys): **the #302 preflight no longer refuses a password the project set in `project.config.js`; only the template placeholder is refused.** As shipped, #302 refused every run whose `TEST_PASS`/`APP_PASS` was not in the environment unless `ALLOW_CONFIG_CREDENTIALS=1` was set. But `project.config.template.js` tells every project to put its real demo password in `DEFAULT_PASS`, nothing in the toolkit loads an env file before `verify-module.sh` runs the journeys, and `sync-project.sh` refreshes unedited runner/config copies. So after the next sync every journeys rung would have faulted with rc 2. Now `config.js` reports `passSource` (`env` / `project.config.js` / `template placeholder`) and only the placeholder (`Demo2025!`, or a `credentials()` reporting `source: 'template default'`) is refused. The env still wins, and `ALLOW_CONFIG_CREDENTIALS=1` still overrides. **Verdict change to know about from #302:** a refused admin credential (HTTP 401/403) used to leave every data rung INVALID with exit 1; the suite now aborts before the first journey with exit 2. A SIGINT/SIGTERM now writes the findings slot as `aborted: "signal SIGINT"` instead of leaving the previous run's file there. `report-normalize.js` keeps `rung: 'setup'` rows as kind `setup` instead of folding them into `ui`. Fixtures: `test-journey-preflight.sh` 74 cases (9 new, all failing on the #302 code: project-set password runs, placeholder refuses, env wins, SIGINT writes aborted findings end to end) and a `report-normalize.js --selftest` case. Refs [#286](https://github.com/MendixMau/mxcli-project-toolkit/issues/286) — MendixMau
20
+
- fix(journeys): **the #302 preflight no longer refuses a password the project set in `project.config.js`; only the template placeholder is refused.** As shipped, #302 refused every run whose `TEST_PASS`/`APP_PASS` was not in the environment unless `ALLOW_CONFIG_CREDENTIALS=1` was set. But `project.config.template.js` tells every project to put its real demo password in `DEFAULT_PASS`, nothing in the toolkit loads an env file before `verify-module.sh` runs the journeys, and `sync-project.sh` refreshes unedited runner/config copies. So after the next sync every journeys rung would have faulted with rc 2. Now `config.js` reports `passSource` (`env` / `project.config.js` / `template placeholder`) and only the template's placeholder (`Demo2025!`) is refused. The env still wins, and `ALLOW_CONFIG_CREDENTIALS=1` still overrides. **Verdict change to know about from #302:** a refused admin credential (HTTP 401/403) used to leave every data rung INVALID with exit 1; the suite now aborts before the first journey with exit 2. A SIGINT/SIGTERM now writes the findings slot as `aborted: "signal SIGINT"` instead of leaving the previous run's file there. `report-normalize.js` keeps `rung: 'setup'` rows as kind `setup` instead of folding them into `ui` (`report-schema.md` enum updated). It also honours `aborted` in a findings file: the report gets a top-level `aborted`, every kept journey row is flagged `notAResult`, and the journey-runner instrument is a fault, so an aborted run never reads as results. Fixtures: `test-journey-preflight.sh` 74 cases (9 new, all failing on the #302 code: project-set password runs, placeholder refuses, env wins, SIGINT writes aborted findings end to end) and `report-normalize.js --selftest` cases (setup kind, aborted run). Refs [#286](https://github.com/MendixMau/mxcli-project-toolkit/issues/286) — MendixMau
21
21
- fix(e2e): **`helpers.js` reads OQL rows by SELECT alias, never by position.** `mxcli oql --direct --json` prints keys alphabetically, leaves out a column that is null in every row, and over a view entity adds `ID` beside the alias. `Object.values(r)` therefore shifted columns, and `oqlScalar` returned a view row's ID instead of its count; a string value holding `]` or `}` truncated the brace-counting slice, so a real answer read as "no measurement" ([#154](https://github.com/MendixMau/mxcli-project-toolkit/issues/154), card-disbursement build). Rows are now keyed by the parsed SELECT aliases in SELECT order with explicit nulls, `oqlScalar` reads the first alias (null rather than a guess when the list has no alias), new `oqlRows`, string-aware JSON slicing; the admin preflight uses the same parser. **Behaviour change to check in your project:** `oqlScalar` now returns the **first SELECT alias**, where it used to return the alphabetically-first key. A multi-column seed, scope capture or baseline query whose value flows into `{{vars}}` may now resolve to a different column. Put the column you mean first, or make the query single-column. Golden: the released mxcli v0.26.0 binary's verbatim stdout/stderr for four queries against a stub admin API (`tests/wave2/fixtures/oql-json/`); fixture `tests/wave2/test-oql-rows.sh` (15 cases, 0/15 on the old helper). No live-app field run yet — MendixMau
22
22
- fix(e2e): **a negative-only span claim is judged against a liveness reference, and an app that dies mid-suite reads APP-DOWN, not FAIL.** A `mustNotFire`-only step (Cancel on a reset confirmation) captures zero spans, which also happens when Jaeger is down, so it was INVALID on every run and its control mutant was unsupported ([#150](https://github.com/MendixMau/mxcli-project-toolkit/issues/150)). Rung 2 now waits the full window and requires a declared `spans.liveness` span or any service span since the journey began (`otel.js liveness`; the window starts at the journey's own start with a 2 s clock-skew margin); no evidence stays INVALID. An `otel.js` older than the runner, i.e. a hand-edited copy kept by sync, reads INVALID naming the stale file instead of throwing a TypeError. The `trace-negative` mutant injects the forbidden name after that check. A `spans` block declaring neither `ordered` nor `mustNotFire` is now a loud INVALID instead of a silent skip; the array shape `[{mf,status}]` and `{}` are named as such. `examples/validate-journeys.js` accepts a negative-only claim, which it used to reject as "NEVER checked" (contradicting the runner). It also checks the new `spans.liveness` field (only read for a negative-only claim) and a `mustNotFire` that is not a valid regex. `skills/journey-examples.md` documents the liveness rule. Fixture `tests/wave2/test-validate-journeys-spans.sh` (12 cases, 8 fail on the old validator). The shipped `project-tests/e2e/example.journey.json` is rewritten into the shape the runner reads. Its `name`/`role`, `value` seed, `page`/`expectPage`/`expectText`/`click` steps, array-shaped `spans` and SQL-style `<module>$<entity>` queries had drifted from it: the validator reported 3 problems on it, now 0, and all 7 control mutants are supported on it. Between journeys the runner probes `baseUrl` (3 tries); down before a journey → `APP-DOWN` INVALID, not walked; down after one → its FAIL rows are flagged ([#285](https://github.com/MendixMau/mxcli-project-toolkit/issues/285) harness half; a watch rebuild took a runtime down mid-suite on 2026-10-12). `APP_PROBE=0` disables it. Fixture `tests/wave2/test-journey-liveness.sh` (36 cases over the captured Jaeger trace and a real local HTTP server); main-loop wiring not fixtured (needs a browser), no field run yet — MendixMau
23
23
- learn(testing-shape, test-result-audit): **three test-strength rules, each with a denominator: refusals probed on the server, every undeclared 4xx a failure, one clean-database final run** ([#172](https://github.com/MendixMau/mxcli-project-toolkit/issues/172)). A field comparison of two builds against one frozen spec found the toolkit build checked permission refusals only in the client (a hidden menu item), logged unexpected 400/422 responses as information, and reached 20/20 by stitching three runs. `testing-shape.md` §7 gains three report lines (`N of N restricted pairs probed server-side`, `undeclared 4xx/5xx: 0 of M requests — each fails its step`, `one clean-DB run <id>: N of N scenarios incl. timers, not stitched`); `test-result-audit.md` gains check 2e, which carries the reasons and the field evidence and asks for those numbers. Both baseline skills stay inside the 80,000-word baseline budget (79,997): the reasons live in the on-demand audit skill, and two redundant sentences in `testing-shape.md` §4's verdict paragraphs were folded (the empty-set corollary is stated in the Trace row, `harness-architecture.md` and `report-schema.md`). The backing inbox note is trimmed to its unpromoted "Keep" items. — MendixMau
@@ -266,7 +265,7 @@ case "$(line CRED)" in *'"abort":2,"calls":0'*'SETUP credentials: TEST_PASS is
266
265
echo"== T2b': only the template placeholder is refused (#302 follow-up) =="
267
266
[ "$(line CRED_PROJ)"='{"abort":null,"calls":1,"cred":false}' ] && ok "password set in project.config.js → no refusal, probe runs"|| bad "project-set password refused (breaks every project after sync)""$(line CRED_PROJ)"
268
267
case"$(line CRED_TPL)"in*'"abort":2'*'placeholder password'*DEFAULT_PASS*) ok "template placeholder → refused, names DEFAULT_PASS as the fix" ;; *) bad "placeholder not refused""$(line CRED_TPL)" ;; esac
269
-
[ "$(line PS)"='["env","env","project.config.js","template placeholder","template placeholder"]' ] && ok "passwordSource: env wins; project value is the project's; template value / 'template default' is the placeholder"|| bad "passwordSource wrong""$(line PS)"
268
+
[ "$(line PS)"='["env","env","project.config.js","template placeholder"]' ] && ok "passwordSource: env wins; project value is the project's; the template value is the placeholder"|| bad "passwordSource wrong""$(line PS)"
270
269
[ -n"$TPL_PASS" ] && [ "$(line TPL_LISTED)"=true ] && ok "the shipped template's DEFAULT_PASS is the listed placeholder"|| bad "template DEFAULT_PASS not in TEMPLATE_PASSWORDS (got '$TPL_PASS')""$(line TPL_LISTED)"
&& ok "report-normalize marks the aborted run: report.aborted, rows flagged notAResult, instrument fault" \
367
+
|| bad "aborted run normalised as results""$(tail -3 "$WORK/rn.out"; node -e 'const r=require(process.argv[1]);console.log(r.aborted)'"$WORK/docs/report.json"2>&1)"
368
+
else
369
+
echo" skip — report-normalize.js not beside $SUT"
370
+
fi
360
371
361
372
echo"== T10: real runner — missing env var, every journey affected → one SETUP line, no probe =="
0 commit comments