Skip to content

Commit 6ac355e

Browse files
akoclaude
andcommitted
fix(modelsdk): skip virtual System domain model in MOVE reference sweeps
MOVE ENTITY / MOVE ENUMERATION cross-module printed spurious warnings on the modelsdk engine: Warning: Could not update OQL queries: load domain model 00000000-0000-0000-0000-000000000002: open .../00000000-...002.mxunit: no such file or directory Both reference-update sweeps iterate ListDomainModels(), which injects the virtual System-module domain model (ID ...002) so platform entities resolve. Each sweep then re-loads every listed DM from disk via loadDomainModelGen, but the System module isn't stored in mprcontents, so its .mxunit read fails. Legacy never hit this: it walks in-memory domain-model objects. Skip the System DM at the top of each loop — it never holds user enum refs or user view-entity OQL, so skipping is correct and removes the disk-read failure regardless of iteration order. - association_move_write.go: UpdateEnumerationRefsInAllDomainModels - move_view_write.go: UpdateOqlQueriesForMovedEntity - move_refs_system_test.go: regression test (reproduces the error pre-fix) - fix-issue.md: symptom table entry Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 1ce7010 commit 6ac355e

4 files changed

Lines changed: 50 additions & 1 deletion

File tree

‎.claude/skills/fix-issue.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,7 @@ to the symptom table below, so the next similar issue costs fewer reads.
101101
| `mxcli oql` against a docker-deployed app returns **0 rows** for tables that clearly have data (false negative — tester concluded "saving is broken") | NOT a database-config problem (that's a red herring — proven: aligning the runtime to Postgres with data still returned 0). The OQL preview servlet (`/dev/preview_execute_oql`) is registered by `com.mendix.basis.livepreview.LivePreviewRegistry` → `AppContainer.addDevelopmentServlet`, which **only mounts the servlet when the JVM system property `mendix.running.locally.by.studiopro=true` is set** (Studio Pro sets it; a deployed `bin/start` PAD doesn't). Without it the endpoint returns HTTP 200 `{"result":-5,"message":"Action not found"}`, which mxcli silently parsed as empty data. (Found by decompiling `com.mendix.mxruntime.jar` / `com.mendix.appcontainer.jar`.) | `cmd/mxcli/docker/templates/docker-compose.yml` (the `command`) + `cmd/mxcli/docker/oql.go` (`oqlDevError`) | Add `-J -Dmendix.running.locally.by.studiopro=true` to the start command (each JVM opt needs its own `-J`), next to the existing `-Dmendix.live-preview=enabled` — this mounts the dev servlets and `mxcli oql` returns live data (verified end-to-end; DTAP mode is NOT required, DB config is irrelevant — the preview runs in-process against the runtime DataStore). Separately, make `oqlDevError` surface `{"result":<0,"message":…}` (not just `{"error":…}`) so the failure isn't swallowed as 0 rows. **Related startup bug found:** the generated `etc/Default` inherits the project's ports, which are `0` when unset → the runtime rejects port 0 and won't start; `patchRuntimePorts` (patch.go) forces `runtime.http { port = 8080 }` / `admin { port = 8090 }`. Verified with a full docker build/run/seed cycle |
102102
| `GRANT VIEW ON PAGE Module.Page TO Module.Role` prints success but the page's allowed roles appear to stay empty: `mxcli lint` still reports `CE0557`/`MPR007` ("used as home page … but has no allowed roles") across a freshly rebuilt catalog, while entity-access grants on the same roles verify fine (`describe entity` shows them). Reported on **modelsdk** engine (default), Mendix 11.11 | NOT a write bug — the grant **does** persist (`AllowedModuleRoles: [3, "Module.Role"]` on disk; `mx check` = 0 errors; Studio Pro would load clean). The regression was on the **read** path: modelsdk `pageFromGen` didn't populate `Page.AllowedRoles`, and `mxcli lint`'s CE0557 rule counts `len(pg.AllowedRoles)` from `ListPages()` — so it read every page as having zero roles and false-fired. `SHOW ACCESS ON PAGE` / `SHOW SECURITY MATRIX` under-reported the same way. The entity-vs-page asymmetry the reporter saw = different read paths (entity `describe` worked; page `lint` didn't), not different writes | `mdl/backend/modelsdk/page.go` → `pageFromGen` | Already fixed by `88a2d50b1` (issue #722): populate `out.AllowedRoles` from the gen `AllowedRolesQualifiedNames()`, mirroring `microflowFromGen`. Guard for the reporter's exact end-to-end flow: `TestUpdateAllowedRoles_PagePersistsForLintRead` (`mdl/backend/modelsdk/page_roles_test.go`) writes a page role then reopens fresh and asserts the lint read path (`ListPages`→`AllowedRoles`) sees it. **Diagnosis tip**: when a write "reports success but the value stays empty" *only in a read/lint/show command*, dump the raw unit (`mxcli bson dump`) — if the bytes are there, it's a read-adapter (`*FromGen`) gap, not a write bug; confirm by reverting the suspected read commit and re-running |
103103
| `MOVE ENTITY X to OtherModule` where a **view entity's OQL** joins through `X`: passes `mxcli check`, but `mx check` on **Mendix 10.x** reports `CE0174 "…path '…/DmTest.Customer' does not resolve to an entity"` at the view entity — **only on the modelsdk engine**, only on 10.x (11.x is clean; legacy is clean) | Mendix <11 stores a view entity's OQL in **two** places: the top-level `DomainModels$ViewEntitySourceDocument` unit AND inline on the entity's `DomainModels$OqlViewEntitySource` (see `entityToGen`, gated `major < 11`). modelsdk's `UpdateOqlQueriesForMovedEntity` rewrote only the source document, leaving the inline copy pointing at the old module. 11.x doesn't write the inline copy (OQL lives only on the source doc), so it was never affected — hence 10.x-only. Legacy updates the inline copy incidentally by re-serializing the domain model | `mdl/backend/modelsdk/move_view_write.go` → `UpdateOqlQueriesForMovedEntity` | After the ViewEntitySourceDocument sweep, also iterate every domain model (`ListDomainModels` → `loadDomainModelGen`), find each entity whose `Source()` is `*genDm.OqlViewEntitySource` with an `Oql()` containing the old qualified name, `SetOql(strings.ReplaceAll(...))`, and `persistDM`. Mirrors the `UpdateEnumerationRefsInAllDomainModels` walk. **Diagnosis tip**: when a value updates in one storage location but a stale copy elsewhere still breaks `mx check`, dump raw `mprcontents` and grep for the value across `$Type`s — Mendix often duplicates a field (OQL, captions) and a version gate (`major < 11`) can make one copy version-conditional. Repro: `mdl-examples/bug-tests/move-entity-oql-view-inline-ref.mdl` |
104+
| `MOVE ENTITY` / `MOVE ENUMERATION` cross-module prints `Warning: Could not update OQL queries` / `Could not update enumeration references: load domain model 00000000-0000-0000-0000-000000000002: open …/mprcontents/00/00/00000000-…002.mxunit: no such file or directory` — **modelsdk engine only**. The move itself succeeds and (today) user modules are still updated, because `ListDomainModels` appends the failing System DM **last** — but the warning is spurious and the ordering dependence is fragile | Both reference-update sweeps iterate `ListDomainModels()`, which **injects the virtual System-module domain model** (ID `…002`, `buildSystemDomainModel`) so platform entities resolve. Each sweep then re-loads every listed DM from disk via `loadDomainModelGen(info.ID)` — but the System module isn't stored in `mprcontents`, so its `.mxunit` read fails. Legacy never hit this: it walks in-memory `*domainmodel.DomainModel` objects and never re-reads from disk | `mdl/backend/modelsdk/association_move_write.go` → `UpdateEnumerationRefsInAllDomainModels`; `mdl/backend/modelsdk/move_view_write.go` → `UpdateOqlQueriesForMovedEntity` | Skip the System DM at the top of each DM loop: `if string(info.ID) == meta.SystemDomainModelID { continue }` (import `modelsdk/meta`). It never holds user enum refs or user view-entity OQL, so skipping is correct and removes the disk-read failure regardless of iteration order. **Diagnosis tip**: a "no such file / not found" on the fixed ID `…002` (or module `…001`) means code is trying to load the *virtual* System unit from disk — filter it out, don't try to read it. Regression: `TestUpdateRefsForMovedEntity_SkipsSystemDomainModel` (`move_refs_system_test.go`) |
104105
| A page whose widget has a **conditional visibility/editability** setting fails to **load** on **Mendix 11.12** — `StorageLoadException: Conditional editability settings … has an invalid value '' for property Attribute. The text ' ' is not a valid AttributeIdentifier` (the `' '` is `\x03\x00\x00\x00` = an int32 list marker; sometimes the value is the object's own `$Type` string). **Both engines**, ≤ 11.11 tolerated it. Reproduces only in the full doctype gate, not minimally — a red herring: it looked like `mx update-widgets` (harness step) was the cause, but the corruption is on disk from mxcli. My pymongo scans missed it because the value is a non-string type / the reader mis-parses | mxcli serialized `Forms$ConditionalVisibilitySettings` / `ConditionalEditabilitySettings` `Attribute` as **BSON null** when there's no attribute-based condition. Studio Pro writes the **empty string `""`** (Attribute is a BY_NAME `AttributeIdentifier`). 11.12's stricter streaming reader rejects the null and mis-reads the following bytes. Found by comparing a Studio-Pro-authored settings node (`str ""`) against mxcli's (`NoneType`) | legacy: `sdk/mpr/writer_widgets.go` (`serializeConditionalVisibility`/`serializeConditionalEditability`); modelsdk: `mdl/backend/modelsdk/widget_write.go` (RegisterTypeDefaults) + new `codec.TypeDefaults.EmptyStringFields` | Emit `Attribute: ""` not null. Legacy: change the `bson.D` value. Modelsdk: move `"Attribute"` from `NullFields` to the new `EmptyStringFields` (encoder emits `""`). **Diagnosis pattern**: when mx reports a corrupt value that pymongo *can't see on disk*, and the value looks like a `$Type` string or `\x03\x00\x00\x00`, suspect a wrong BSON *type* for a field (null where a string is required) that the streaming reader mis-aligns — compare field types against a Studio-Pro-authored node, don't just compare key presence. Repro: `mdl-examples/bug-tests/conditional-settings-attribute-empty-1112.mdl` |
105106
| A CLOSE PAGE microflow/nanoflow activity (`close page;`) fails `mx check` on **Mendix 11.12** with `CE0117 "Error(s) in expression"` at the Close page activity — **legacy engine only** (modelsdk passes; ≤ 11.6 tolerated it). Any annotation (`@caption`/`@color`/`@position`) on the activity reliably exposes it | The legacy `sdk/mpr` writer serialized the `Microflows$CloseFormAction` page-count under the field name `NumberOfPagesToClose`, but the metamodel storage name is `NumberOfPages` (the codec/modelsdk already used it — found by diffing the two engines' BSON: `legacy=NumberOfPagesToClose, modelsdk=NumberOfPages`). 11.12 doesn't recognise the old name, so the real `NumberOfPages` field is absent → defaults to an empty expression → CE0117 | `sdk/mpr/writer_microflow_actions.go` (`*microflows.ClosePageAction` case) + `sdk/mpr/parser_microflow_actions.go` | Write `NumberOfPages` (not `NumberOfPagesToClose`); the reader accepts both (`NumberOfPages` first, legacy `NumberOfPagesToClose` fallback) for round-trip fidelity on existing projects. **Diagnosis pattern**: a CE that says "invalid/empty expression" on an activity with no visible expression usually means a *renamed/missing* field defaulting to empty — diff the passing engine's BSON against the failing one to spot the field-name drift. Repro: `mdl-examples/bug-tests/close-page-numberofpages-1112.mdl` |
106107
| On **Mendix 11.12**, `mx check` fails to load projects on the **legacy** engine for several doctypes (`01-domain-model`, `10-odata`, `14-project-settings`, `22-published-rest`) with `System.InvalidOperationException: Expected '$ID' as the first property of a storage object, but got 'X'` at `StreamingBsonUnitReader`. The bad first-key varies run-to-run (`ThemeModuleOrder`, `EnableRspackBundler`, `Oql`, `MarkAsUsed`, `PopupResizable`…) — the tell-tale of random map-key order. modelsdk passes; ≤ 11.11 tolerates any order | Several legacy `sdk/mpr` writers preserve round-trip fidelity by carrying parsed subtrees as Go maps (`ProjectSettings.RawParts`, ref-marking `raw`, rename `raw`, domain-model `raw`) and marshalling them back. `bson.Marshal` emits **map keys in random order**, so `$ID` only lands first by luck. 11.12 rejects any storage object whose first key isn't `$ID`. The earlier `$ID`-first fix (bson.D reorder in writer_domainmodel/security) missed every map-passthrough site | `sdk/mpr/writer_settings.go` (+ `writer_refs`, `writer_rename`, `writer_domainmodel`, `writer_odata`, `writer_rest`, `writer_customblob`, `writer_security`, `writer_modules`, `writer_microflow`, … — all unit-boundary marshals) via new `sdk/mpr/writer_order.go` | Wrap every unit-boundary `bson.Marshal(doc)` in `marshalUnitIDFirst(doc)` = `bson.Marshal(bsonutil.HoistStorageID(doc))`. **Use the non-sorting `HoistStorageID`, NOT `OrderStorageValue`**: `OrderStorageValue` *sorts* the non-`$ID` keys, which corrupts template-derived pluggable-widget/datagrid page trees (mx then aborts with `got 'LabelTemplate'`, regressing `29-datagrid`/`33-alter-page`). `HoistStorageID` only lifts `$ID`/`$Type` to the front and preserves every other key's order → no-op on already-correct writers, fixes only the map-passthrough ones. **Do NOT normalize `writer_pages.go` or the page/workflow mutators** — their output is already `$ID`-first and they embed delicate widget maps the hoist would sort-corrupt. Verify on 11.12 with the doctype gate; regression-check `29`/`33` specifically. (Left unfixed: `03-page` LabelTemplate — a separate pre-existing issue where on-disk bytes read `$ID`-first yet mx still objects) |

‎mdl/backend/modelsdk/association_move_write.go‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import (
1010
"github.com/mendixlabs/mxcli/modelsdk/codec"
1111
"github.com/mendixlabs/mxcli/modelsdk/element"
1212
genDm "github.com/mendixlabs/mxcli/modelsdk/gen/domainmodels"
13+
"github.com/mendixlabs/mxcli/modelsdk/meta"
1314
"github.com/mendixlabs/mxcli/sdk/domainmodel"
1415
)
1516

@@ -130,6 +131,13 @@ func (b *Backend) UpdateEnumerationRefsInAllDomainModels(oldQualifiedName, newQu
130131
return fmt.Errorf("UpdateEnumerationRefsInAllDomainModels: list domain models: %w", err)
131132
}
132133
for _, info := range dms {
134+
// The System module's domain model is virtual (not stored in mprcontents),
135+
// so it can't be re-loaded from disk — and it never references a user
136+
// enumeration. Skip it; otherwise loadDomainModelGen fails with a spurious
137+
// "no such file or directory" warning.
138+
if string(info.ID) == meta.SystemDomainModelID {
139+
continue
140+
}
133141
gdm, err := b.loadDomainModelGen(info.ID)
134142
if err != nil {
135143
return err
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
// SPDX-License-Identifier: Apache-2.0
2+
3+
package modelsdkbackend
4+
5+
import "testing"
6+
7+
// TestUpdateRefsForMovedEntity_SkipsSystemDomainModel is a regression test for
8+
// the MOVE ENTITY / MOVE ENUMERATION reference-update warnings ("Could not
9+
// update OQL queries" / "Could not update enumeration references": load domain
10+
// model 00000000-0000-0000-0000-000000000002: ... no such file or directory).
11+
//
12+
// ListDomainModels injects the virtual System-module domain model (its unit is
13+
// not stored in mprcontents). The reference-update sweeps re-load every listed
14+
// domain model from disk, so they must skip the System DM instead of failing on
15+
// its missing unit. Both methods should return cleanly even when nothing matches.
16+
func TestUpdateRefsForMovedEntity_SkipsSystemDomainModel(t *testing.T) {
17+
proj := copyFixture(t)
18+
19+
b := New()
20+
if err := b.Connect(proj); err != nil {
21+
t.Fatalf("connect: %v", err)
22+
}
23+
t.Cleanup(func() { _ = b.Disconnect() })
24+
25+
// Names that match nothing in the fixture: the sweep still walks every domain
26+
// model (including the virtual System one) and must not error on it.
27+
if err := b.UpdateEnumerationRefsInAllDomainModels("NoSuch.Enum", "Other.Enum"); err != nil {
28+
t.Fatalf("UpdateEnumerationRefsInAllDomainModels errored on System DM: %v", err)
29+
}
30+
if _, err := b.UpdateOqlQueriesForMovedEntity("NoSuch.Entity", "Other.Entity"); err != nil {
31+
t.Fatalf("UpdateOqlQueriesForMovedEntity errored on System DM: %v", err)
32+
}
33+
}

‎mdl/backend/modelsdk/move_view_write.go‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import (
1010
"github.com/mendixlabs/mxcli/modelsdk/codec"
1111
"github.com/mendixlabs/mxcli/modelsdk/element"
1212
genDm "github.com/mendixlabs/mxcli/modelsdk/gen/domainmodels"
13+
"github.com/mendixlabs/mxcli/modelsdk/meta"
1314
mmpr "github.com/mendixlabs/mxcli/modelsdk/mpr"
1415
"github.com/mendixlabs/mxcli/modelsdk/mprread"
1516
)
@@ -37,7 +38,6 @@ func (b *Backend) MoveConstant(c *model.Constant) error {
3738
return b.writer.MoveUnit(string(c.ID), string(c.ContainerID))
3839
}
3940

40-
4141
// UpdateOqlQueriesForMovedEntity rewrites every ViewEntitySourceDocument whose
4242
// OQL text references oldQualifiedName to newQualifiedName. Used after a MOVE
4343
// ENTITY so view sources joining through the moved entity still resolve (CE0174).
@@ -77,6 +77,13 @@ func (b *Backend) UpdateOqlQueriesForMovedEntity(oldQualifiedName, newQualifiedN
7777
return updated, fmt.Errorf("UpdateOqlQueriesForMovedEntity: list domain models: %w", err)
7878
}
7979
for _, info := range dms {
80+
// The System module's domain model is virtual (not stored in mprcontents),
81+
// so it can't be re-loaded from disk — and it holds no view entities with
82+
// inline OQL. Skip it; otherwise loadDomainModelGen fails with a spurious
83+
// "no such file or directory" warning.
84+
if string(info.ID) == meta.SystemDomainModelID {
85+
continue
86+
}
8087
gdm, err := b.loadDomainModelGen(info.ID)
8188
if err != nil {
8289
return updated, err

0 commit comments

Comments
 (0)