Skip to content

Commit eb7cc80

Browse files
authored
inbox: five field notes from an existing-app legacy-widget rebuild (from #309 by @AlvinnYeo) (#450)
Lands #309 by @AlvinnYeo. Independent review: APPROVE. # PR 450 (lands #309 by @AlvinnYeo) review r1 (delta self-review) Verdict: APPROVE - w1 must-fix 1 done: every #244 status now says closed 2026-10-14 by #322, upstream root cause remains (gh issue view 244 confirms). No STOP-table wording left in the note. - w1 must-fix 2 done: body drops the stale byte-identical claim. - Inbox lane only (contrib/inbox + one CHANGELOG line, credit " — AlvinnYeo"); nothing loaded or routed; zero behaviour risk. - Optional word limits not enforced (inbox notes, maintainer call).
1 parent 300a80e commit eb7cc80

6 files changed

Lines changed: 393 additions & 0 deletions

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
1919

2020
## Unreleased
2121

22+
- process(inbox): **five field notes from an existing-app legacy-widget rebuild** — exec.sh keeps an MDL write when mxbuild crashes on it and the next exec runs on "Baseline NOT measured"; a signal stop of `run --test-endpoint` leaves AfterStartupMicroflow on `MxTest.RegisterEndpoint` (second field case for #244); Unit-row delete refused as ORPHAN, rename-then-drop works; a durable-rebuild kit with a "restate from DESCRIBE" STOP row (DESCRIBE omits module-role and layout-row visibility: 97 role locks in 16 documents, none shown); learned entries for File Uploader image mode (CE0463), native-mapped `radiobuttons`/`image`, tab-page `CaptionParams`/`Visible:` silently dropped, BUG-39 on widget XPath, and two-hop data-source paths. Three further notes (restore-mpr usage, build-plan-status flag order, another project's mxcli test blocking exec.sh) were dropped because master already fixes them. Mendix 11.14.0 Beta, mxcli v0.25.0, toolkit c1efcb3 — AlvinnYeo
2223
- fix(build-plan-status): --refresh keeps an existing build-plan.json current, so a day of building no longer reads as 0 built — MendixMau
2324
- learn(unit-test): the unit-test skill names the per-flow `Unit flows:` list next to `Unit modules:`, so whether a flow gets a test reads the same in both places (follow-up to #435) — MendixMau
2425
- learn(load-test): **new on-demand skill for load testing a page without correlating Mendix's client protocol by hand: Playwright records the real `/xas/` calls, a probe maps each opaque `runtimeOperation` to its microflow through Jaeger, and the chosen call is replayed with the session's own cookie and CSRF token.** Ten field traps (HTTP 200 on a failed action, caught REST errors under a green microflow span, the wrong call hammered, a recording that still held the previous page, a tracing check that failed open, a harness that throttled itself, harness vs app limits, requests vs users, sampled trace counts, empty data) and a four-line report that says what the run does not prove. Routed ondemand for lead, test and review; no instrument ships yet and the roadmap's "no load testing" line is reworded. Field evidence: a warehouse demo app, Mendix 11, 16 runs 2026-09-09 to 2026-10-08, bursts of 1000 and 2000 concurrent calls on one session — MendixMau
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Durable rebuild of legacy-widget pages: a four-part kit, and "restate from DESCRIBE" as a STOP item
2+
3+
**From:** a field project (anonymised) — an existing Mendix app (old widgets throughout) changed with mxcli + this toolkit
4+
**Date:** 2026-10-13
5+
**Kind:** process
6+
**Field evidence:** 79 non-writable widgets on 40 live documents (DESCRIBE `-- NOT re-executable` count)
7+
and 97 role-locked widgets in 16 documents (BSON scan; DESCRIBE showed none). 34 pages and snippets
8+
were restated whole; the read-back afterwards found 0 non-writable widgets and 0 role locks on them.
9+
Two drafted scripts would have silently dropped role or row visibility had they run. Mendix 11.14.0
10+
Beta (Classic client, Production security), mxcli v0.25.0, toolkit c1efcb3.
11+
**Proposed target:** `skills/existing-app-change.md` (a "legacy-widget pages" section), a STOP row in
12+
`skills/learned-mdl-preflight.md`, possibly a `project-bin/` scan once field-proven as a file
13+
14+
**Currency check:** 2026-10-16 against master d1cbd41: partly stale. STOP row 21 in `skills/learned-mdl-preflight.md` now covers restating a document another script also edits. Still open: the legacy-widget section in `skills/existing-app-change.md` and the scan for role locks that DESCRIBE does not show.
15+
---
16+
17+
## What is already covered
18+
19+
- `bug-logs/mxcli-bugs.md` BUG-115 addendum: "Grep a generated script for `NOT re-executable` before
20+
executing it" (for the File Manager widget).
21+
- `bug-logs/mxcli-bugs.md` (the `hasRole()` entry): no MDL syntax writes `ConditionalVisibilitySettings.ModuleRoles`.
22+
- `skills/learned-mdl-cannot-express.md` (master bda5bed, #277): "A widget visible to one role and not
23+
another — nothing", with the execute-grant recipe for action buttons.
24+
25+
## What is not covered (findings)
26+
27+
1. **DESCRIBE does not print module-role visibility.** One portal page had 19 role-locked widgets; DESCRIBE
28+
printed none of them (nor a hidden "show page" action behind 9 image widgets). A page restated from that
29+
DESCRIBE builds clean and shows every widget to every role.
30+
2. **A drafted restatement said "the snippet has no visibility rules".** It restated a side-menu snippet
31+
from DESCRIBE (navigation-list items have no names, so one item cannot be dropped alone). A BSON audit
32+
found 13 role-locked items. Running the draft would have exposed all of them, with a clean build.
33+
3. **DESCRIBE does not print layout-grid row visibility.** Two form pages had nine detail sections shown
34+
by type through row visibility; a snippet had 56 row-level "show only when it has a value" rules. None
35+
appear in DESCRIBE. The drafters re-wrote them as container `Visible:` (which DESCRIBE does print and
36+
mxcli does write); an audit confirmed every rule was carried over.
37+
4. **Tab-page `Visible:` and `CaptionParams` are accepted and not written** (see the learned-entries note).
38+
Found only by reading the unit back after exec.
39+
40+
So "copy the page from DESCRIBE, edit, `create or replace`" is silently lossy on any app built in Studio Pro.
41+
42+
## Proposed STOP row (learned-mdl-preflight.md)
43+
44+
| # | Operation | Why STOP | Do instead |
45+
|---|---|---|---|
46+
| new | Restate an existing page, snippet or layout from `describe` output (`create or replace`, or `alter … replace` of a container) | DESCRIBE omits module-role visibility, layout-grid row/column visibility and legacy-widget internals; check, exec and mxbuild all pass while the rules vanish | Run the visibility scan (below) on the unit first. Zero role locks and zero row rules: go. Otherwise rewrite each rule as container `Visible:` or flow access, or leave the page to `alter page` / Studio Pro |
47+
48+
Probe that falsifies it: in Studio Pro, set one widget visible to one module role and one layout row
49+
visible on an attribute; `describe page`; grep the output for the role name and the attribute. Today
50+
(v0.25.0): 0 hits for both.
51+
52+
## The kit (each part was done by hand in the field; none is a toolkit file yet)
53+
54+
1. **BSON visibility scan, per document.** Decode each page/snippet/layout unit under `mprcontents/`
55+
(the toolkit's `project-bin/mxunit_bson.py` codec is enough) and count (a) any
56+
`ConditionalVisibilitySettings` whose `ModuleRoles` list is non-empty, (b) any
57+
`ConditionalVisibilitySettings` on a `Forms$LayoutGridRow` or `Forms$LayoutGridColumn`. Report per
58+
document: widget name, kind, roles or attribute. The field scans were inline one-offs, not a saved
59+
script; a shipped version needs its own fixture.
60+
2. **Non-writable count, per document.** `describe` every page, snippet and layout in the app's own
61+
modules (Marketplace modules excluded) and count `-- NOT re-executable` lines, grouped by the BSON
62+
type in the comment. Field result: `Forms$DropDown` 29, `Forms$ReferenceSelector` 27,
63+
`Forms$DataGrid` 16, sign-in widgets 4, `Forms$TemplateGrid` 1, `Forms$ImageUploader` 1,
64+
`Forms$SidebarToggleButton` 1.
65+
3. **Rebuild brief per document.** Inventory first (DESCRIBE plus the scan), then each widget marked
66+
KEEP / CUT / MOVED / NEW against the wireframe, then the restatement. Replacements used:
67+
drop-down → `combobox` on the enumeration; reference selector → `combobox` on the association (a
68+
microflow data source when the filter needs the context object); old data grid → Data grid 2;
69+
template grid → `listview`; image uploader → File Uploader in files mode plus a `dynamicimage`;
70+
role-only visibility → a per-viewer flag from a data-source microflow, or flow access (Mendix hides a
71+
button whose flow the user cannot run). The restating script becomes the document's single owner;
72+
older `alter page` blocks on it are marked superseded (compare #269).
73+
4. **Post-exec read-back.** After each exec: DESCRIBE again (0 `NOT re-executable`), rerun the scan
74+
(0 role locks unless intended), and grep the unit for every `Visible:` / `CaptionParams` the script
75+
wrote. Item 4 above was caught only this way.
76+
77+
Cost, for scale: six drafting agents in parallel; one gated exec per document (about 5–10 minutes each
78+
with mxbuild and lint), serial because there is one writer to the `.mpr`.
79+
80+
Upstream asks (mxcli), for the maintainer to file or link: print module-role and row/column visibility in
81+
DESCRIBE (or at least a `-- role-based visibility not shown (N roles)` comment), accept it on write, give
82+
navigation-list items stable names, and a legacy-widget migration (`DataGrid` → Data grid 2,
83+
`ReferenceSelector`/`DropDown` → combobox) behind a dry run.
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# exec.sh keeps an MDL write when mxbuild crashes after it, and the next exec runs on an unloadable model
2+
3+
**From:** a field project (anonymised) — an existing Mendix app being changed with mxcli + this toolkit
4+
**Date:** 2026-10-13
5+
**Kind:** bug
6+
**Field evidence:** the project's BUILD-LOG shows two "gate could not run" rows 46 s apart: the
7+
script that made the model unloadable, then the next script, which exec.sh accepted and ran on top
8+
of it. Mendix 11.14.0 Beta (Classic client), mxcli v0.25.0, toolkit commit c1efcb3 (`project-bin/exec.sh`
9+
is byte-identical on master 0c62f6f, blob a03adec).
10+
**Proposed target:** `project-bin/exec.sh` (gate branch "mxbuild failed to run" and the pre-flight
11+
"Baseline NOT measured" branch); one line in `skills/iterative-build-loop.md`
12+
13+
---
14+
15+
## What happened (finding, reproduced once in the field)
16+
17+
A page script set a data view's data source to a two-hop association path in its full form
18+
(`$Parent/MyModule.A_B/MyModule.B/MyModule.B_C/MyModule.C`). `mxcli check` passed. After the
19+
exec, the mxbuild loader itself crashed instead of reporting CE errors:
20+
21+
```
22+
An error occurred when trying to set the 'DestinationEntity' property of a Entity ref step in a Page.
23+
```
24+
25+
exec.sh printed:
26+
27+
```
28+
✗ mxbuild failed to run (exit 1) — gate could not verify the model.
29+
→ Snapshot preserved. Open in SP to verify manually before proceeding.
30+
```
31+
32+
and kept the write. It did exit 1 and cleared the verification stamp (good), but the agent driving
33+
the build ran the next script. exec.sh's pre-flight printed `⚠ Baseline NOT measured: mxbuild exited 1
34+
without checking the model.`, carried on, applied the next script, and hit the same crash.
35+
36+
BUILD-LOG rows, verbatim except the script names:
37+
38+
```
39+
| 2026-10-13T12:26:18 | `25g-PageA-rebuild.mdl` | fail | ❌ gate failed | 5 error(s): CE1613 — script rolled back · approval: auto (PROJECT.md) |
40+
| 2026-10-13T12:28:46 | `25g-PageA-rebuild.mdl` | fail | ❌ gate could not run | mxbuild exit 1 · approval: auto (PROJECT.md) |
41+
| 2026-10-13T12:29:32 | `25h-SnippetB-rebuild.mdl` | fail | ❌ gate could not run | mxbuild exit 1 · approval: auto (PROJECT.md) |
42+
```
43+
44+
Recovery: `./bin/restore-mpr.sh <full path to the snapshot taken before 25g>` then
45+
`./bin/verify-model.sh` (0 errors). The 25h row had to be re-run afterwards.
46+
47+
## Why it matters
48+
49+
- The header comment of exec.sh says an unverified gate is rolled back **for `--patch` only**. That is
50+
true and deliberate, but for MDL the "kept" case was designed for "mxbuild could not start"
51+
(wrong version, no JDK). Here mxbuild started, ran on the baseline cleanly a minute earlier, and
52+
crashed only on the written model. That is a property of the write, not of the machine.
53+
- The next exec has no way to tell the two apart: "Baseline NOT measured" is a warning, not a refusal.
54+
So in an unattended chain the damage compounds: every following script is written onto a model
55+
nothing can load, and each one gets its own snapshot of the broken state.
56+
57+
## Proposed fix (hypothesis, not built)
58+
59+
1. **Restore when mxbuild ran clean on the baseline but cannot run after the write.** exec.sh already
60+
has both facts: the pre-flight measured a baseline (`BASE_KNOWN=1`) and the post-exec run exits
61+
non-zero with no errors file. In that case, call `restore_snapshot` exactly as the `--patch` path
62+
does, and log `↩ rolled back: mxbuild crashed on the written model`. Keep today's behaviour when the
63+
baseline itself was not measured (that is the real "machine cannot build" case).
64+
2. **Refuse a write when the baseline cannot be measured** and the last BUILD-LOG row for this model
65+
is `gate could not run`, with one STOP line naming `bin/restore-mpr.sh` and the snapshot to restore.
66+
`ALLOW_UNVERIFIED=1` stays the explicit override.
67+
3. Reword the kept-write message: "Open in SP to verify" is not something an agent can do. Something
68+
like `STOP: the model may not load. Restore with ./bin/restore-mpr.sh <snapshot path>` is actionable.
69+
70+
Probe that would falsify the claim (for the retest fixture): a fake mxbuild that exits 0 on the
71+
baseline and exits 1 with no errors file after the exec; today exec.sh exits 1 and leaves the
72+
written model in place; after the fix the model on disk equals the snapshot.
73+
74+
Related: `--patch` already restores on an unverified gate (header lines 25–29); PR #304 touches the
75+
same file (per-project lock) but not this branch.
Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Learned entries from a legacy-widget rebuild: File Uploader image mode, native-mapped keywords, tab pages, XPath comparands, multi-hop data sources
2+
3+
**From:** a field project (anonymised)
4+
**Date:** 2026-10-13
5+
**Kind:** learning
6+
**Field evidence:** each entry below was hit while restating 34 pages of an existing app, through
7+
`bin/exec.sh` (mxbuild delta gate) on mxcli v0.25.0, Mendix 11.14.0 Beta, Classic (Dojo) client,
8+
toolkit c1efcb3. Error texts are verbatim; module/entity names are genericised.
9+
**Proposed target:** per entry below (`skills/learned-file-upload-widget.md`,
10+
`skills/learned-mdl-cannot-express.md`, `skills/learned-page-patterns.md`, `bug-logs/mxcli-bugs.md`)
11+
12+
---
13+
14+
## 1. File Uploader: image mode fails the build; files mode against a `System.Image` subtype works
15+
16+
Target: `skills/learned-file-upload-widget.md` §7 ("Image mode … was not tested").
17+
Status: files mode is ALREADY covered there (PR #139); this adds image mode and the image-preview pattern.
18+
19+
```sql
20+
fileuploader upPictures (
21+
uploadMode: 'images',
22+
associatedImages: association $currentObject/MyModule.Picture_Asset,
23+
readOnlyMode: true,
24+
enableCustomButtons: false
25+
)
26+
```
27+
28+
`mxcli check` passed. mxbuild: `CE0463 The definition of this widget has changed.` on
29+
`File uploader upPictures`; the gate rolled back (rollback verified by DESCRIBE). The same widget with
30+
`uploadMode: 'files'` builds clean (finding).
31+
32+
What built clean instead (finding, mxbuild 0 errors): uploads in **files** mode against an entity that
33+
extends `System.Image` (which extends `FileDocument`), with a separate `dynamicimage` preview beside it;
34+
read-only picture lists as a `listview` whose template holds a `dynamicimage`.
35+
36+
Also stale: `skills/learned-mdl-cannot-express.md` row 1 still says "A file upload control — nothing …
37+
⛔ not buildable", which contradicts `learned-file-upload-widget.md`. Suggest pointing that row at the
38+
File Uploader skill.
39+
40+
Related (finding): the installed FileDropper widget's definition lists `dataFileEntity` and
41+
`dataContextAssociation` as known properties with no mapping; `check` accepts them, exec does not write
42+
them, and mxbuild then fails. Upstream ask: refuse unmapped definition properties with MDL-WIDGET07.
43+
44+
## 2. `radiobuttons` and `image` can resolve to NATIVE mobile widgets on a web page
45+
46+
Target: `skills/learned-page-patterns.md` (or the widgets section of `learned-mdl-cannot-express.md`).
47+
Status: NEW (no toolkit mention).
48+
49+
In this project `DESCRIBE WIDGET TYPE radiobuttons` resolves to `com.mendix.widget.native.radiobuttons`,
50+
and `check` warns MDL-WIDGET10 that the label is dropped. The project's `image.def.json` maps `image` to
51+
`com.mendix.widget.native.image.Image`; a probe wrote `image` with no properties. DESCRIBE prints existing
52+
**web** radio groups with the same `radiobuttons` keyword, so re-running DESCRIBE output writes a native
53+
widget into a web page (finding for the resolution and the warning; the radiobuttons case was not taken
54+
through mxbuild).
55+
56+
Rule used: Boolean → `checkbox`, enumeration → `combobox`, images → `dynamicimage`. Before relying on a
57+
keyword in an existing app, run `DESCRIBE WIDGET TYPE <keyword>` and check the id is a `web` widget.
58+
59+
## 3. Tab pages: `CaptionParams` and `Visible:` pass every gate and are not written
60+
61+
Target: `bug-logs/mxcli-bugs.md` as `## BUG-DRAFT-tabpage-captionparams-visible-dropped:`, plus a row in
62+
`learned-mdl-cannot-express.md` next to the existing "count badge on a tab" row.
63+
Status: NEW. (The existing row covers `Badge:`, which `check` refuses; these two are accepted.)
64+
65+
```sql
66+
tabpage tpDue (Caption: 'Due today or earlier · {1}', CaptionParams: ({1} = DueCount)) { … }
67+
tabpage tpOther (Caption: 'Unassigned · {1}', Visible: $currentObject/ShowOtherTab) { … }
68+
```
69+
70+
`check`, exec and mxbuild all passed, with no MDL-WIDGET07. The stored caption is literal text (the tab
71+
reads `· {1}`), and the tab page has no visibility setting: `ShowOtherTab` occurs 0 times in the unit.
72+
Found by a BSON read-back after exec (finding).
73+
74+
Rule: no dynamic captions and no visibility on tab pages; put the count in a line inside the tab, and
75+
put anything conditional in a container with `Visible:` (container visibility is written correctly).
76+
77+
## 4. DESCRIBE omits module-role visibility and layout-grid row visibility
78+
79+
Target and detail: see the inbox note "Durable rebuild of legacy-widget pages" (same date). Listed
80+
here so triage sees it among the learned entries. Status: PARTLY (role visibility is known to be
81+
unwritable; that DESCRIBE also hides it is not recorded).
82+
83+
## 5. Page XPath with a `$currentObject/<association>` comparand → CE0161 (BUG-39 on widgets)
84+
85+
Target: `bug-logs/mxcli-bugs.md` BUG-39 (add a confirmation), `learned-page-patterns.md`.
86+
Status: PARTLY. BUG-39 records it for microflow retrieves only.
87+
88+
```sql
89+
combobox cbLevel2 (
90+
Attribute: Helper_Level2,
91+
DataSource: database from "MyModule"."Level2"
92+
where [MyModule.Level2_Level1 = $currentObject/MyModule.Helper_Level1]
93+
)
94+
```
95+
96+
`check --references` passed; mxbuild: 3 × `CE0161 Error(s) in XPath constraint`, one per cascading combo
97+
box; rolled back (finding). Studio Pro accepts the same constraint.
98+
99+
Workaround that built: each combo box takes its options from a microflow data source that retrieves the
100+
parent into a variable and filters on it.
101+
102+
## 6. Two-hop association data source: neither path form is writable → use a microflow data source
103+
104+
Target: `learned-page-patterns.md` (next to the BUG-04 multi-hop ContentParams section), and
105+
`bug-logs/mxcli-bugs.md` as `## BUG-DRAFT-two-hop-datasource-path:`.
106+
Status: NEW. BUG-20 (cross-module traversal writing a null `DestinationEntityId`) was fixed in v0.17.0;
107+
this is a same-module, two-hop path on v0.25.0.
108+
109+
Short form, exactly as DESCRIBE prints it:
110+
111+
```sql
112+
DataSource: $Parent/MyModule.A_B/MyModule.B_C
113+
```
114+
115+
mxcli stored the last segment as an entity:
116+
`CE1613 The selected entity 'MyModule.B_C' no longer exists.` (5 errors); rolled back.
117+
118+
Full form:
119+
120+
```sql
121+
DataSource: $Parent/MyModule.A_B/MyModule.B/MyModule.B_C/MyModule.C
122+
```
123+
124+
`check` passed; the mxbuild loader crashed:
125+
`An error occurred when trying to set the 'DestinationEntity' property of a Entity ref step in a Page.`
126+
(This crash is what exposed the exec.sh gap in the separate inbox note.)
127+
128+
So DESCRIBE output containing such a path is not re-executable. Rule: no multi-hop association path as a
129+
data-view source; use a microflow data source that retrieves over each association in turn (built clean).
130+
131+
## Not included here
132+
133+
Other cases from the same run (Data grid 2 custom-content column losing its Attribute; XPath through a
134+
reverse association needing read access mxbuild does not check; names `check --references` does not
135+
validate) are upstream mxcli material and are left for a separate note.

0 commit comments

Comments
 (0)