Skip to content

Commit cc56d9b

Browse files
committed
journeys: #302 follow-up — accept a project-set password, SIGINT findings, setup kind
- config.js reports passSource (env / project.config.js / template placeholder); the preflight refuses only the template placeholder (Demo2025!, or credentials() source 'template default'). As shipped, #302 refused every non-env password, though the template tells projects to put their real one in DEFAULT_PASS and nothing loads an env file before verify-module — every journeys rung would FAULT after a sync. An admin object without passSource keeps the #302 rule. - SIGINT/SIGTERM write the findings slot as aborted ("signal SIGINT") instead of leaving the previous run's file there. - report-normalize.js: rung 'setup' rows keep kind 'setup' (were 'ui'). - CHANGELOG names ALLOW_CONFIG_CREDENTIALS and the 403 verdict change (data rungs INVALID exit 1 -> suite aborts exit 2). test-journey-preflight.sh 74/74 (9 new cases fail on master: project-set password runs, placeholder refuses, env wins, SIGINT end to end); report-normalize --selftest new setup-kind case. Refs #286 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
1 parent 7f73f8d commit cc56d9b

6 files changed

Lines changed: 133 additions & 24 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are.
1717

1818
## Unreleased
1919

20+
- fix(journeys): **the #302 preflight no longer refuses a password the project set in `project.config.js`; only the template placeholder is refused.** As shipped, #302 refused every run whose `TEST_PASS`/`APP_PASS` was not in the environment unless `ALLOW_CONFIG_CREDENTIALS=1` was set. But `project.config.template.js` tells every project to put its real demo password in `DEFAULT_PASS`, nothing in the toolkit loads an env file before `verify-module.sh` runs the journeys, and `sync-project.sh` refreshes unedited runner/config copies. So after the next sync every journeys rung would have faulted with rc 2. Now `config.js` reports `passSource` (`env` / `project.config.js` / `template placeholder`) and only the placeholder (`Demo2025!`, or a `credentials()` reporting `source: 'template default'`) is refused. The env still wins, and `ALLOW_CONFIG_CREDENTIALS=1` still overrides. **Verdict change to know about from #302:** a refused admin credential (HTTP 401/403) used to leave every data rung INVALID with exit 1; the suite now aborts before the first journey with exit 2. A SIGINT/SIGTERM now writes the findings slot as `aborted: "signal SIGINT"` instead of leaving the previous run's file there. `report-normalize.js` keeps `rung: 'setup'` rows as kind `setup` instead of folding them into `ui`. Fixtures: `test-journey-preflight.sh` 74 cases (9 new, all failing on the #302 code: project-set password runs, placeholder refuses, env wins, SIGINT writes aborted findings end to end) and a `report-normalize.js --selftest` case. Refs [#286](https://github.com/MendixMau/mxcli-project-toolkit/issues/286) — MendixMau
2021
- fix(e2e): **`helpers.js` reads OQL rows by SELECT alias, never by position.** `mxcli oql --direct --json` prints keys alphabetically, leaves out a column that is null in every row, and over a view entity adds `ID` beside the alias. `Object.values(r)` therefore shifted columns, and `oqlScalar` returned a view row's ID instead of its count; a string value holding `]` or `}` truncated the brace-counting slice, so a real answer read as "no measurement" ([#154](https://github.com/MendixMau/mxcli-project-toolkit/issues/154), card-disbursement build). Rows are now keyed by the parsed SELECT aliases in SELECT order with explicit nulls, `oqlScalar` reads the first alias (null rather than a guess when the list has no alias), new `oqlRows`, string-aware JSON slicing; the admin preflight uses the same parser. **Behaviour change to check in your project:** `oqlScalar` now returns the **first SELECT alias**, where it used to return the alphabetically-first key. A multi-column seed, scope capture or baseline query whose value flows into `{{vars}}` may now resolve to a different column. Put the column you mean first, or make the query single-column. Golden: the released mxcli v0.26.0 binary's verbatim stdout/stderr for four queries against a stub admin API (`tests/wave2/fixtures/oql-json/`); fixture `tests/wave2/test-oql-rows.sh` (15 cases, 0/15 on the old helper). No live-app field run yet — MendixMau
2122
- fix(e2e): **a negative-only span claim is judged against a liveness reference, and an app that dies mid-suite reads APP-DOWN, not FAIL.** A `mustNotFire`-only step (Cancel on a reset confirmation) captures zero spans, which also happens when Jaeger is down, so it was INVALID on every run and its control mutant was unsupported ([#150](https://github.com/MendixMau/mxcli-project-toolkit/issues/150)). Rung 2 now waits the full window and requires a declared `spans.liveness` span or any service span since the journey began (`otel.js liveness`; the window starts at the journey's own start with a 2 s clock-skew margin); no evidence stays INVALID. An `otel.js` older than the runner, i.e. a hand-edited copy kept by sync, reads INVALID naming the stale file instead of throwing a TypeError. The `trace-negative` mutant injects the forbidden name after that check. A `spans` block declaring neither `ordered` nor `mustNotFire` is now a loud INVALID instead of a silent skip; the array shape `[{mf,status}]` and `{}` are named as such. `examples/validate-journeys.js` accepts a negative-only claim, which it used to reject as "NEVER checked" (contradicting the runner). It also checks the new `spans.liveness` field (only read for a negative-only claim) and a `mustNotFire` that is not a valid regex. `skills/journey-examples.md` documents the liveness rule. Fixture `tests/wave2/test-validate-journeys-spans.sh` (12 cases, 8 fail on the old validator). The shipped `project-tests/e2e/example.journey.json` is rewritten into the shape the runner reads. Its `name`/`role`, `value` seed, `page`/`expectPage`/`expectText`/`click` steps, array-shaped `spans` and SQL-style `<module>$<entity>` queries had drifted from it: the validator reported 3 problems on it, now 0, and all 7 control mutants are supported on it. Between journeys the runner probes `baseUrl` (3 tries); down before a journey → `APP-DOWN` INVALID, not walked; down after one → its FAIL rows are flagged ([#285](https://github.com/MendixMau/mxcli-project-toolkit/issues/285) harness half; a watch rebuild took a runtime down mid-suite on 2026-10-12). `APP_PROBE=0` disables it. Fixture `tests/wave2/test-journey-liveness.sh` (36 cases over the captured Jaeger trace and a real local HTTP server); main-loop wiring not fixtured (needs a browser), no field run yet — MendixMau
2223
- learn(testing-shape, test-result-audit): **three test-strength rules, each with a denominator: refusals probed on the server, every undeclared 4xx a failure, one clean-database final run** ([#172](https://github.com/MendixMau/mxcli-project-toolkit/issues/172)). A field comparison of two builds against one frozen spec found the toolkit build checked permission refusals only in the client (a hidden menu item), logged unexpected 400/422 responses as information, and reached 20/20 by stitching three runs. `testing-shape.md` §7 gains three report lines (`N of N restricted pairs probed server-side`, `undeclared 4xx/5xx: 0 of M requests — each fails its step`, `one clean-DB run <id>: N of N scenarios incl. timers, not stitched`); `test-result-audit.md` gains check 2e, which carries the reasons and the field evidence and asks for those numbers. Both baseline skills stay inside the 80,000-word baseline budget (79,997): the reasons live in the on-demand audit skill, and two redundant sentences in `testing-shape.md` §4's verdict paragraphs were folded (the empty-set corollary is stated in the Trace row, `harness-architecture.md` and `report-schema.md`). The backing inbox note is trimmed to its unpromoted "Keep" items. — MendixMau

‎project-tests/e2e/config.js‎

Lines changed: 23 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,22 @@ const PORT = STACK.port;
8181
// and page-audit.js cannot end up running as two different identities.
8282
const CREDS = PROJ.credentials();
8383

84+
// Where the test password came from, never the password itself (#278, #302 follow-up).
85+
// env — TEST_PASS (or its APP_PASS alias)
86+
// project.config.js — a password the project set there (the template tells every
87+
// project to put its real demo password in DEFAULT_PASS)
88+
// template placeholder — the value the shipped template carries, i.e. nobody set one
89+
// Only the last is refused by journey-runner's preflight. Refusing every non-env password
90+
// (as #302 first did) broke every project whose real password lives in project.config.js,
91+
// because nothing in the toolkit loads an env file before verify-module runs the journeys.
92+
// A credentials() that reports `source: 'template default'` is honoured as well.
93+
const TEMPLATE_PASSWORDS = ['Demo2025!']; // project.config.template.js DEFAULT_PASS
94+
function passwordSource({ env = process.env, creds = CREDS } = {}) {
95+
if (env.TEST_PASS || env.APP_PASS) return 'env';
96+
if (creds.source === 'template default' || TEMPLATE_PASSWORDS.includes(creds.pass)) return 'template placeholder';
97+
return 'project.config.js';
98+
}
99+
84100
// ── M2EE admin credential: where it came from, never what it is ──────────────
85101
// Every data rung, seed and outcome query goes through `mxcli oql --direct` with this
86102
// token. It used to be `M2EE_ADMIN_PASS || '1'`: a literal default that silently differed
@@ -156,11 +172,13 @@ const cfg = {
156172
// is wrong: the login page shows the same generic message either way.
157173
user: CREDS.user,
158174
pass: CREDS.pass,
159-
// Whether the test password came from the environment (TEST_PASS, or its APP_PASS alias)
160-
// rather than project.config.js's default. journey-runner's preflight refuses to log in on a
161-
// default (#278): a runner started without the project's env-loading step submitted the
162-
// template password, then the admin fallback, and locked two demo accounts for ~5 min each.
175+
// Whether the test password came from the environment (TEST_PASS, or its APP_PASS alias),
176+
// and passSource (see passwordSource above). journey-runner's preflight refuses to log in on
177+
// the TEMPLATE PLACEHOLDER only (#278): a runner started without the project's env-loading
178+
// step submitted the template password, then the admin fallback, and locked two demo
179+
// accounts for ~5 min each. A password the project set in project.config.js is accepted.
163180
passFromEnv: !!(process.env.TEST_PASS || process.env.APP_PASS),
181+
passSource: passwordSource(),
164182
// Last-resort identity so a credential problem degrades to a reported finding
165183
// instead of a zero-coverage run. Set FALLBACK_USER='' to disable.
166184
fallbackUser: CREDS.fallbackUser,
@@ -213,4 +231,4 @@ function resolveTarget() {
213231
return t;
214232
}
215233

216-
module.exports = { cfg, NAV, TARGETS, resolveTarget, resolveAdmin };
234+
module.exports = { cfg, NAV, TARGETS, resolveTarget, resolveAdmin, passwordSource, TEMPLATE_PASSWORDS };

‎project-tests/e2e/journey-runner.js‎

Lines changed: 43 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1078,17 +1078,26 @@ function preflight(journeys, { env = process.env, probe = runAdminProbe, admin =
10781078
rows.push({ rung: 'setup', name: `preflight: ${name}`, verdict: 'INVALID', detail, requirement: '' });
10791079
};
10801080

1081-
// 0. login credential (#278). Unset TEST_PASS means the run would submit project.config's
1082-
// default and then the admin fallback. Each rejected submit counts toward Mendix's
1083-
// 3-attempt lockout, and in the field two demo accounts were locked for ~5 min each. So no
1084-
// login is attempted at all. The override exists for a project whose config default IS its
1085-
// real demo password.
1086-
if (!admin.passFromEnv && env.ALLOW_CONFIG_CREDENTIALS !== '1') {
1081+
// 0. login credential (#278). A run that would submit the TEMPLATE PLACEHOLDER password
1082+
// (nobody set TEST_PASS, nobody edited project.config.js's DEFAULT_PASS) then tries the admin
1083+
// fallback; each rejected submit counts toward Mendix's 3-attempt lockout, and in the field
1084+
// two demo accounts were locked for ~5 min each. So no login is attempted at all.
1085+
// A password the PROJECT set in project.config.js is accepted: the template tells every
1086+
// project to put its real demo password there, and nothing in the toolkit loads an env file
1087+
// before verify-module runs the journeys — refusing it (#302 as first shipped) faulted every
1088+
// journeys rung after the next sync. ALLOW_CONFIG_CREDENTIALS=1 overrides when the
1089+
// placeholder really is the demo password. An admin object without passSource (older
1090+
// config.js) keeps the #302 rule: refuse unless the password came from the env.
1091+
const placeholder = admin.passSource !== undefined
1092+
? admin.passSource === 'template placeholder'
1093+
: !admin.passFromEnv;
1094+
if (placeholder && env.ALLOW_CONFIG_CREDENTIALS !== '1') {
10871095
setup('credentials', `TEST_PASS is unset — the run would log in as "${admin.user}" with ` +
1088-
"project.config.js's default password and then try the fallback account, and each rejected " +
1089-
'attempt counts toward the account lockout. Load the project env first (export TEST_USER / ' +
1096+
"project.config.template.js's placeholder password and then try the fallback account, and each " +
1097+
'rejected attempt counts toward the account lockout. Put the real demo password in ' +
1098+
"project.config.js's DEFAULT_PASS, or load the project env first (export TEST_USER / " +
10901099
'TEST_PASS, e.g. `set -a; . ./.env; set +a`), or set ALLOW_CONFIG_CREDENTIALS=1 if the ' +
1091-
'config default is the real demo password. Suite ABORTED before any login');
1100+
'placeholder is the real demo password. Suite ABORTED before any login');
10921101
return { lines, rows, skip, abort: 2 };
10931102
}
10941103

@@ -1355,8 +1364,29 @@ function armRelease(page, li) {
13551364
if (!ok) console.log(' ⚠ logout NOT confirmed — this run\'s session may still be open on the runtime ' +
13561365
'(trial licences cap live sessions; a restart clears them)');
13571366
};
1367+
}
1368+
1369+
// An interrupted run (SIGINT/SIGTERM) writes its findings slot as aborted, with whatever it
1370+
// measured so far. Exiting without writing left the PREVIOUS run's file in the slot, reading
1371+
// as this run — the same stale-slot defect the preflight abort already guards against.
1372+
function writeAbortedFindings(reason, extra = {}) {
1373+
try {
1374+
fs.mkdirSync(cfg.artifactsDir, { recursive: true });
1375+
fs.writeFileSync(findingsFile(), JSON.stringify({
1376+
capturedAt: new Date().toISOString(), aborted: reason,
1377+
configuredUser: cfg.user, actualUser: extra.actualUser ?? null, usedFallback: !!extra.usedFallback,
1378+
baseUrl: cfg.baseUrl, positiveControl: POSITIVE_CONTROL, results, walks,
1379+
}, null, 2));
1380+
} catch (e) { console.error(` ⚠ could not write the aborted findings file: ${e.message}`); }
1381+
}
1382+
let SIGNAL_CTX = {};
1383+
function armSignals() {
13581384
for (const sig of ['SIGINT', 'SIGTERM']) {
1359-
process.once(sig, () => { RELEASE().finally(() => process.exit(2)); });
1385+
process.once(sig, () => {
1386+
console.log(`\n ABORTED by ${sig} — findings written as aborted; this run is not a result.`);
1387+
writeAbortedFindings(`signal ${sig}`, SIGNAL_CTX);
1388+
RELEASE().finally(() => process.exit(2));
1389+
});
13601390
}
13611391
}
13621392

@@ -1383,20 +1413,17 @@ const findingsFile = () => path.join(cfg.artifactsDir,
13831413
console.log(` ABORTED at preflight — 0 of ${journeys.length} journeys walked. Fix the SETUP line(s) above.`);
13841414
// Written, not skipped: an aborted run must not leave the previous run's file in the slot
13851415
// reading as this one.
1386-
fs.mkdirSync(cfg.artifactsDir, { recursive: true });
1387-
fs.writeFileSync(findingsFile(), JSON.stringify({
1388-
capturedAt: new Date().toISOString(), aborted: 'preflight',
1389-
configuredUser: cfg.user, actualUser: null, usedFallback: false,
1390-
baseUrl: cfg.baseUrl, positiveControl: POSITIVE_CONTROL, results, walks,
1391-
}, null, 2));
1416+
writeAbortedFindings('preflight');
13921417
console.log(` findings → ${path.relative(cfg.root, findingsFile())}`);
13931418
process.exit(pf.abort);
13941419
}
13951420

1421+
armSignals();
13961422
const { browser, page } = await H.launchBrowser();
13971423

13981424
const li = await H.login(page);
13991425
armRelease(page, li);
1426+
SIGNAL_CTX = { actualUser: li.user, usedFallback: li.usedFallback };
14001427

14011428
// Landing check: once per run, right after login, before any click. One run signs in as
14021429
// one role, so "N of N roles" is the count over the per-role runs.

‎project-tests/e2e/report-normalize.js‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,9 @@ const INPUTS = {
118118
wireframeDir: 'design/wireframes',
119119
};
120120

121-
const KINDS = new Set(['ui', 'trace', 'data', 'outcome', 'control',
121+
// 'setup': journey-runner's preflight / precondition / APP-DOWN rows (rung 'setup'). They are
122+
// not UI checks; before 2026-10-13 they fell back to 'ui' and read as a screen finding.
123+
const KINDS = new Set(['ui', 'trace', 'data', 'outcome', 'control', 'setup',
122124
'build', 'lint', 'coverage', 'conformance', 'stack']);
123125

124126
// ── Verdict maps — one table per source dialect, nothing inline ──────────────
@@ -2600,6 +2602,13 @@ function selftest() {
26002602
t('and emits no product checks from it',
26012603
jCtl.checks.filter(c => c.instrument === 'journey-runner').length, 0);
26022604

2605+
// A setup row (preflight, precondition, APP-DOWN) keeps its own kind, never 'ui'.
2606+
const jSetup = normalize({ ...base, journeyFindings: { status: 'ok', path: 'journey-findings.json',
2607+
data: { capturedAt: '2026-01-01T00:00:00Z', positiveControl: false,
2608+
results: [{ rung: 'setup', name: 'preflight: admin', verdict: 'INVALID', detail: 'x', requirement: '' }] } } });
2609+
t('a setup row is kind "setup", not "ui"',
2610+
jSetup.checks.filter(c => c.id === 'journey/_/preflight-admin').map(c => c.kind), ['setup']);
2611+
26032612
// ---- 3b. the control instrument is per-rung, and a copied real run is not one ----
26042613
const ctlRow = (rung, verdict) => ({
26052614
rung: 'control', verdict, detail: `d-${rung}`,

‎skills/journey-proof.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ each one was knowable before the first click.
8686

8787
| Line | Meaning | Do |
8888
|---|---|---|
89-
| `SETUP credentials: TEST_PASS is unset` | aborted, exit 2, **no login attempted** | load the project env (`TEST_USER`/`TEST_PASS`). A run without it submitted the config default and then the admin fallback, and locked two demo accounts for ~5 min each (#278). `ALLOW_CONFIG_CREDENTIALS=1` only when the config default is the real password |
89+
| `SETUP credentials: TEST_PASS is unset … placeholder password` | aborted, exit 2, **no login attempted** | the run would submit the **template placeholder** (nobody set `TEST_PASS` and `project.config.js`'s `DEFAULT_PASS` is still the template value), then the admin fallback. That locked two demo accounts for ~5 min each (#278). Put the real demo password in `DEFAULT_PASS`, or export `TEST_PASS`. A password the project set in `project.config.js` is accepted. `ALLOW_CONFIG_CREDENTIALS=1` only when the placeholder really is the demo password |
9090
| `SETUP admin: … refused the credential (HTTP 403); credential from <source>` | suite **aborted**, exit 2, nothing walked | fix the credential at the named source. Resolution order: `M2EE_ADMIN_PASS` env → this app's `.mxcli/run-local.json` (`mxcli run --local`) → `.docker/.env` → a labelled guess. The value is never printed |
9191
| `SETUP admin: admin API unreachable at host:port` | aborted, exit 2 | app down, or set `M2EE_ADMIN_PORT`. `ALLOW_NO_DATA_RUNG=1` runs anyway with every data rung INVALID |
9292
| `SETUP env: missing MOCK_X (needed by J1, J3)` | **one** line, those journeys not walked | export the vars a journey names in `requiresEnv` |

0 commit comments

Comments
 (0)