Repository navigation
Commit 8847e09
feat: the package upgrade plan — compare the installed and incoming versions, an install dry run, os package install --dry-run (#30983)
<!-- claude-projects-attribution: {"github_login":"os-elon-musk"} -->
_Requested by **Elon** · [project
thread](https://claude.ai/code/project/chan_01WedDzV26nniwxv7jC88B9D?thread=cmsg_01WedDzV26nniwxv7jC88B9DGzmgyjCZ4eKJ8i6w2aNSj5)_
domain:server#1 · os-elon-musk · session_016WV4cyPqWLpc3AxaZrs98n
Refs #30884 (PR 1 of 2: widening only. PR 2 carries `Fixes #30884` and
the Q3 refusal.)
Clause-②: yes (widening) — new exports from `@objectstack/metadata-core`
(`comparePackageVersions` and its types, `typeChangeCategory`,
`isEmptyPackageUpgradePlan`, `upgradePlanSeedDatasets`,
`seedModeOverwritesMatchedRows`), a new `planUpgrade` method on
objectql's `manifest` service, a new `dryRun` key on the `POST
/api/v1/marketplace/install-local` body, a new `--dry-run` flag on `os
package install`, and new `os diff` entries (declared and suspected
renames, seed datasets). Nothing a request did before answers
differently.
Metadata-protocol: none
Claim: 6686492425 (re-posted with this branch) · Branch:
`claude/project-thread-89x8tb` · Scope note: 6685166393
## 维护者速读
- 改了什么:升级软件包前可以先看「会变什么」。平台比较已安装版本和新版本,列出增删的对象和字段、字段类型变化(带漂移等级)、用
`renamedFrom` 声明的改名、没声明却像改名的「疑似改名」,以及种子数据集的增删改和各自的模式。入口有三个:安装接口带
`dryRun: true` 只返回预览、什么都不写;`os package install --dry-run` 打印预览;`os diff`
改用同一个比较。
- 为什么改:今天升级时字段改名会变成一个新的空列,旧数据留在旧列里看不到;改类型也是静默的(卡 #30884,史诗 cloud#3652
Q3)。预览还读当前环境:每个受影响的列有多少行数据,以及 upsert 种子重放会覆盖多少行客户改过的数据。
- 风险与回滚:小。这一版只加不收,以前能装的包照样能装;Q3 的默认拒绝在 PR 2。注意一点:不认识 `dryRun`
的旧运行时会直接安装,CLI 会认出来并报错退出,但那次安装已经发生。回滚即撤 PR。
-
先看:`packages/metadata-core/src/package-upgrade-plan.ts`(比较本身),`packages/objectql/src/package-upgrade-environment.ts`(行数和「被覆盖的客户编辑」怎么数),install-local
的 2a′ 步(干跑在哪一步回答、为什么在任何写入之前)。
- 两分钟验证:`pnpm --filter @objectstack/objectql exec vitest run
src/package-upgrade-environment.test.ts`,预期 7 条全绿,其中「a seeded upsert
dataset with one row edited reports that row」断言三行种子里只有客户改过的 `won` 被报出。
- 需要你做的:无,席位落地。
## What changes
**`@objectstack/metadata-core`: the comparison**
(`package-upgrade-plan.ts`, new)
`comparePackageVersions(installed, incoming)` takes two flat package
bodies (`objects`, `data`) and returns:
- objects added and removed, by name;
- fields added and removed within an object both versions declare, each
with `required` and `stored`;
- type changes, each with `typeChangeCategory`'s drift word (`safe` when
the installed field had no stored column, `destructive` when the new one
has none, `needs_confirm` when both are stored — the column keeps its
old type under the additive sync);
- declared renames: a new key whose `renamedFrom` names an installed key
the new version no longer declares. The pair is listed as a rename, not
as a removal and an addition. A declared rename whose type changes is
also a type change, judged on its own (scope note 6685166393), carrying
`renamedFrom`;
- suspected renames: within one object, a stored field removed while
stored fields are added with nothing linking them. One finding per
removed field, every unlinked added stored field a candidate. Both keys
stay listed as removed and added;
- seed datasets added, removed and changed (`records`, `mode`,
`columns`, `env`), each with its mode (the spec default `ignore` when
none is declared), keyed by object plus a non-default match key and
locales.
It is pure and decides nothing. The drift vocabulary is restated as
`UpgradeDriftCategory`, because this package cannot depend on a driver;
a type-level pin in the CLI's tests holds it equal to driver-sql's
`DriftCategory`. Also exported: `upgradePlanSeedDatasets` (so the
environment half pairs datasets by the plan's own keys) and
`seedModeOverwritesMatchedRows` (`upsert`, `update`).
**`@objectstack/objectql`: the environment half**
(`package-upgrade-environment.ts`, new; `planUpgrade` on the `manifest`
service in `plugin.ts`)
`planUpgrade(installed, incoming)` folds each side with core's
`resolveStackCollection`, runs the comparison, and adds two read-only
readings, with the system context, across every organization:
- rows holding a value (`$ne: null`) in each stored column the upgrade
removes, renames (declared or suspected) or retypes (not a `safe`
retype), and the rows of each object it removes;
- for each `upsert` or `update` dataset both versions declare, the rows
the replay matches by `externalId`, and of those the rows a customer
edited that the replay overwrites: some column the replay writes holds a
value other than the installed seed value and other than the incoming
one. Values are compared as the replay compares them (temporal storage
form, blank-as-null form). Credential, file, reference, computed and
expression columns, `applyOn: 'create'` columns and the match key are
never compared. Up to 20 edited keys are named.
A reading that fails is `null` and is logged, never `0`.
**`@objectstack/cloud-connection`: the dry run**
(`marketplace-install-local-plugin.ts`)
- Step 0′: `dryRun` must be a boolean (`400 INVALID_REQUEST` otherwise).
A dry run of an `unmanaged` install is refused `400`, since that branch
writes and a copy follows no upgrade.
- Before the customization-pair branch: a dry run of a pair is refused
`400`.
- Step 2a′: after every refusal that says the package cannot install at
all, and before the D5e ceremony (2b), the access gate (2c) and the
first write (3), `dryRun: true` answers `200` with `{ dryRun,
manifestId, version, installedVersion, plan, environment, message }`.
The installed side is the ledger entry, converted forward as the
incoming one is. A `manifest` service with no `planUpgrade` answers `503
SERVICE_UNAVAILABLE`, never an empty plan. A ledger entry that does not
parse is reported (`installedRecordUnreadable`).
**`@objectstack/cli`**
- `os diff` builds its object and field entries from
`comparePackageVersions`. Every shape it printed before prints byte for
byte as before (pinned before the move). New: a `renamedFrom` rename
prints as one `modified` entry (`renamed from X, declared by
renamedFrom`) instead of a removal plus an addition (`renamedFrom` has
not shipped yet: changeset `30880-field-renamed-from` is pending); a
suspected rename prints a `modified` entry on the removed key beside its
removal and addition; seed datasets print under `data`, after every
other category, with their mode.
- `os package install --dry-run` sends `dryRun: true` and prints the
plan, each finding with the environment's reading beside it. A runtime
that answers without `dryRun` (one that predates the key, and so
installed) is printed as a failure, exit 1.
## Why
Card #30884 (epic objectstack-ai/cloud#3652, Phase 2 "change preview
before install"; rulings Q3 and D3). This PR is the preview. The refusal
ruled in Q3 is PR 2.
Re-measured on `616311c1`, the card's "Today" items 1–6 still hold
(`diff.ts:162`, `:223-224`; `migrate/plan.ts:62`, `:137-139`;
`schema-drift.ts:6-9`, `:135`; install-local `:85-97`, `:454`, `:3456`,
ledger `:1319`, `install.ts:174`, `:307`;
`upgrade-impact-report.ts:4-17`; `seed.zod.ts:193` and the
`18.seed-mode-default-create-only` migration). Item 7 no longer holds:
`renamedFrom` landed with #30934 (`field.zod.ts:2610`, `object.zod.ts`'s
`refuseDishonestRenamedFrom`).
## Tests
The card's Acceptance bullets this PR carries, each with its test. The
other three (the 409 with its token, the declared-rename refusal, the
ledger code) belong to PR 2.
| Acceptance bullet | Test |
|---|---|
| a field removed and one added of the same type in one object, no
`renamedFrom` → one suspected rename | `metadata-core`
`package-upgrade-plan.test.ts` › renames › "a field removed and one
added of the same type in one object, no renamedFrom → one suspected
rename" |
| the same pair with `renamedFrom: ['old']` → one declared rename and no
suspected rename | same file › renames › "the same pair with
renamedFrom: ['company'] on the new field → one declared rename and no
suspected rename" |
| a type change carries its drift category | same file › type changes ›
"a type change carries its drift category" |
| an `upsert` dataset change and an `ignore` dataset change are both
listed with their modes | same file › seed datasets › "an upsert dataset
change and an ignore dataset change are both listed with their modes" |
| `os diff` output is unchanged for its existing fixtures | `cli`
`diff-upgrade-plan.test.ts` › "every shape, in order, with each entry's
key order" (pinned on `main` before the move, then held) |
| `dryRun: true` writes nothing (ledger file and registry unchanged) |
`cloud-connection` `marketplace-install-local-upgrade-plan.test.ts` › "a
re-install: the installed ledger body against the incoming one;
registry, schema and ledger untouched" (ledger bytes compared file by
file; `register` and `syncSchemas` never called) |
| environment half: a seeded `upsert` dataset, one row edited, reports
that row | `objectql` `package-upgrade-environment.test.ts` › "a seeded
upsert dataset with one row edited reports that row" (real ObjectQL
engine; of three seeded rows only the edited `won` is named) |
| a declared rename whose type changes is also a type change (scope note
6685166393) | `package-upgrade-plan.test.ts` › "a declared rename whose
type changes is ALSO a type change, judged on its own";
`diff-upgrade-plan.test.ts` › "a declared rename that changes type
prints the type change too" |
New tests: 42 (`metadata-core` 17, `objectql` 7, `cloud-connection` 6,
`cli` 7 + 5).
**Ablations**, on head `3100b841`, each through
`scripts/ablation-replace.mjs` (anchor hit ×1, the mutation proven on
disk by blob hash, restored to the HEAD blob with `git diff HEAD`
empty):
| Ablation | Mutation | Result |
|---|---|---|
| suspected-rename detection | `const storedAdded = added.filter(…)` →
`const storedAdded: string[] = []` | 3 failed / 14 passed (17): the
suspected-rename case, the "renamedFrom naming a key the installed
version does not have declares nothing" case, the two-candidates case |
| type judgment on a declared rename | the type-change test limited to
`target === key` (a field that kept its key) | 1 failed / 16 passed
(17): "a declared rename whose type changes is ALSO a type change,
judged on its own" |
| dryRun write guard | step 2a′ (`if (dryRun) return
this.answerUpgradeDryRun(…)`) removed | 3 failed / 3 passed (6): the
re-install, the not-installed and the 503 cases; with the guard gone the
request installs |
## Acceptance notes
- **Two readings go beyond the card's letter.** The edited-row count
covers every overwriting dataset both versions declare, changed or not,
and `update` as well as `upsert`. An unchanged `upsert` dataset writes
the same edits back, and `update` overwrites a matched row like
`upsert`. Each entry says whether the plan lists it as changed.
- **The dry run answers before the confirmation ceremonies.** It does
not say whether the D5e or access check would stop the real install. The
plan is what an installer reads before deciding, so it is answered
either way.
- **An older runtime ignores `dryRun` and installs.** The CLI detects
this from the answer and exits 1, but by then the install has happened.
No guard exists before the send.
- **The value comparison restates the seed loader's `seedValueEquals`.**
That function is private to `@objectstack/metadata-protocol`; its
temporal and blank rules come from the same `@objectstack/core`
functions here. Sinking the comparison to one shared home would be a
follow-up. carrier: none (noted, not filed).
- **Not counted:** a `replace` dataset (it deletes every row on replay);
a dataset's `locale` or `env` filter (the count may include a dataset a
locale filter would skip, which over-reports, never under-reports).
- **Docs.** `content/docs/deployment/cli.mdx` has no row for `--dry-run`
yet. That page is outside this claim's file surface; the seat can card
it.
- **`planPackageUpgrade` is not exported from `@objectstack/objectql`'s
entry.** A host reaches it through
`ctx.getService('manifest').planUpgrade`, the service method the card
names.
## Verification
Head `3100b841`, base `616311c1`. Every test run below went through
`scripts/pm/os-verify-lock.sh` and ended `VERDICT command-exit 0`.
| Run | Result |
|---|---|
| `pnpm --filter @objectstack/metadata-core exec vitest run
--maxWorkers=2` | 24 files, 928 tests passed |
| `pnpm --filter @objectstack/objectql exec vitest run --project local
--maxWorkers=2` | 560 files, 11207 tests passed (on `3dba4119`;
`3100b841` changes only `package-upgrade-environment.test.ts`, 7/7 on
head) |
| `pnpm --filter @objectstack/cloud-connection exec vitest run
--maxWorkers=2` | 73 files, 923 tests passed |
| `pnpm --filter @objectstack/cli exec vitest run --project unit
--maxWorkers=2` | 391 files, 5494 tests passed |
| `… --project integration test/package-install-local-` | 8 files, 83
tests passed (every install-local integration file) |
| `… --project integration test/package-owned-command-readers.test.ts` |
1 file, 8 tests passed |
| typecheck: `metadata-core`, `cloud-connection`, `objectql` (on head),
`cli` | exit 0 each; the test-layer debt ledgers held unchanged (`cli` 3
files / 28 errors, `objectql` 39 / 233). `cli` needs
`NODE_OPTIONS=--max-old-space-size=8192`; at the default heap it runs
out of memory before any verdict |
| `pnpm lint` | exit 0 |
**Gates.** `node scripts/pm/dispatch-gates.mjs --commands` derives 66
families from this diff. All 66 ran with their exit codes recorded
before any pipe, and `--ran` reconciles: "66 derived famil(ies)
accounted for — 66 run, 0 NOT-MEASURED". Four needed a second run, each
for a reason outside the diff, and each then exited 0:
- `check:objectql-double-limit` exited 1 on the first run: the probe
drives every `find` double in the objectql tests, and my driver double's
failure switch threw under it. Fixed in `3100b841` (the failure case now
stubs `engine.find`); the gate then reports "none new".
- `check-plugin-teardown-shape --self-test` exited 3 on a shallow clone;
fetching the commit its self-test names (`621a4876`) fixed it.
- `check:dual-build-cjs-loads` and `check:i18n-coverage` exited 3 for
want of `dist/`; after a full build (`pnpm turbo run build
--filter=!@objectstack/docs`, exit 0) both exited 0.
`check:nul-bytes` exits 0. A self-scan for control bytes over the
changed files finds none (the environment half spells its key separator
as an escape).
**Today, re-measured** on `616311c1`: see Why. Items 1–6 hold; item 7
does not.
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic.
(`packages/spec` untouched.)
- [x] 2. Every name I added uses the casing for its kind.
- [x] 3. No formatter ran; `pnpm lint` passes.
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request (PR 1 of the claim's two).
- [x] 5. A change that publishes carries a changeset, and both
declaration lines are filled in here and in the changeset.
- [x] 6. `packages/spec` touched: n/a.
- [x] 7. No hand edits to `content/docs/references/`,
`packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English.
---
_Generated by [Claude
Code](https://claude.ai/code/session_016WV4cyPqWLpc3AxaZrs98n)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent a11ef71 commit 8847e09
13 files changed
Lines changed: 2116 additions & 99 deletions
File tree
- .changeset
- packages
- cli
- src/commands
- package
- test
- cloud-connection/src
- metadata-core/src
- objectql/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| |||
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
23 | | - | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| |||
39 | 40 | | |
40 | 41 | | |
41 | 42 | | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | 43 | | |
56 | 44 | | |
57 | 45 | | |
58 | 46 | | |
59 | | - | |
60 | 47 | | |
61 | 48 | | |
62 | 49 | | |
| |||
86 | 73 | | |
87 | 74 | | |
88 | 75 | | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
| 76 | + | |
| 77 | + | |
112 | 78 | | |
113 | | - | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
126 | 84 | | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
141 | | - | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
142 | 99 | | |
143 | | - | |
144 | | - | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
153 | 148 | | |
154 | 149 | | |
155 | 150 | | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
156 | 174 | | |
157 | 175 | | |
158 | 176 | | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
159 | 206 | | |
160 | 207 | | |
161 | 208 | | |
| |||
229 | 276 | | |
230 | 277 | | |
231 | 278 | | |
232 | | - | |
233 | | - | |
234 | | - | |
235 | | - | |
236 | | - | |
237 | | - | |
238 | | - | |
239 | | - | |
240 | | - | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
245 | | - | |
246 | | - | |
247 | | - | |
248 | | - | |
249 | | - | |
250 | | - | |
251 | | - | |
252 | | - | |
253 | | - | |
254 | | - | |
| 279 | + | |
255 | 280 | | |
256 | 281 | | |
257 | 282 | | |
| |||
0 commit comments