Skip to content

Commit a5df094

Browse files
feat(cli): os migrate audit-key-material removes key-material columns from older ledger copies (#26772)
Fixes #26687 Clause-②: no Metadata-protocol: none ## 维护者速读 - **改了什么**:新增 `os migrate audit-key-material`,驱动 #26683 已合入的 `migrateKeyMaterialCopies`(热表)与 `migrateArchivedKeyMaterialCopies`(归档数据源),从旧版写入的 `sys_audit_log` / `sys_activity` 账本副本里删除加密密钥材料列;默认 dry run,`--apply` 才写。 - **为什么改**:#26683 只让新写入的副本不再带这些列,旧副本需要运维跑一次命令清理;此前没有入口。 - **风险与回滚**:新命令默认只读,`--apply` 只改副本列、保留 `updated_at`、幂等;另把 `audit-mail-bodies` 的归档判定抽成共享 util(行为与 JSON 输出逐字节不变,其归档集成测试 8/8 绿)。回滚 = revert 本 PR。 - **先看**:`packages/cli/src/utils/ledger-copy-archive.ts`(从 `audit-mail-bodies.ts` 原样搬出的归档判定,风险点在 `examineArchiveTargets` 的 not-connected / unknown 分类)→ `packages/cli/src/commands/migrate/audit-key-material.ts` → `packages/cli/src/commands/migrate/audit-mail-bodies.ts` 的删减 hunk。 - **两分钟验证**:`pnpm --filter @objectstack/dogfood exec vitest run test/audit-key-material.dogfood.test.ts`,预期 4/4 绿(dry run 计数、`--apply` 后 0 份副本带列、重跑无变化)。 - **需要你做的**:无,席位落地。 ## What changes - **`os migrate audit-key-material`** (`packages/cli/src/commands/migrate/audit-key-material.ts`): a new one-shot subcommand that removes the encrypted key-material columns from `sys_audit_log` / `sys_activity` ledger copies 18.2.0 and earlier wrote, then runs the same rewrite over the archive datasource `sys_audit_log` declares. It mirrors `os migrate audit-mail-bodies` flag for flag (`--database-url`, `--apply`, `--yes`, `--force`, `--json`): dry run by default with the read-only boot, the confirmation gate (exit 1 without `--yes` when non-interactive), JSON-only stdout under `--json`, the absent-table read view, and exit 1 when the report counts a failure or an archive target could not be examined. - **One archive classification for both commands** (`packages/cli/src/utils/ledger-copy-archive.ts`): the archive-target lookup, the declared-connect-failure read and the runtime-datasource-row probe moved out of `audit-mail-bodies.ts` unchanged, so `examined` / `none` / `not-connected` / `unknown` mean the same thing in both commands. `audit-mail-bodies.ts` now calls it; its `--json` document keeps the same keys in the same order. - **Registered everywhere `audit-mail-bodies` is**: the one-shot family table, the absent-database roster (with two control rows), the `--json` stdout-purity family, the confirmation-exit roster, and the `composeDeclaredDatasources` docblock in `schema-migrate.ts`. - **Docs**: a command-table row and a `#### os migrate audit-key-material` section in `content/docs/deployment/cli.mdx`, and one line in `content/docs/upgrading.mdx` for operators upgrading from 18.2.0 or earlier. - **Changeset**: `@objectstack/cli` patch (`Metadata-protocol: none`, so `patch` under the v18 grading). ## Tests - `audit-key-material.archive.integration.test.ts` (new, 8): the twin of the mail-bodies archive test, over the real Archiver. Dry run counts hot and archived copies and writes nothing; `--apply` leaves 0 copies carrying the columns in either store, keeps every other key and column, and leaves an unlisted object's same-named column alone; a re-run rewrites nothing; a copy still hot at run time is archived without the columns; no run prints a sentinel; `none` / `not-connected` classification and exit codes. - `audit-key-material.dogfood.test.ts` (new, 4): the real `os` binary over the 18.0 upgrade-harness fixture, which holds 4 `sys_audit_log` and 4 `sys_activity` copies carrying the columns. Dry run counts them byte-identically, `--apply` removes exactly those keys and nothing else, and a re-run is a no-op. - Roster rows added in `data-commands.absent-database.integration.test.ts`, `schema-migrate.one-shot-family.integration.test.ts`, `json-stdout-purity.e2e.test.ts` and `migrate-confirmation-exit.pin.test.ts`. <details><summary>Local runs (head f75fee2)</summary> - `pnpm --filter @objectstack/cli typecheck` (tsc plus `check:test-typecheck`) and `build`: lock VERDICT command-exit 0. - CLI unit tier, `vitest run --project unit --maxWorkers=2`: Test Files 339 passed (339), Tests 4927 passed (4927). - CLI integration tier, the four touched files (`audit-key-material.archive`, `audit-mail-bodies.archive`, `data-commands.absent-database`, `schema-migrate.one-shot-family`): Test Files 4 passed (4), Tests 180 passed (180). - `json-stdout-purity.e2e.test.ts` (nightly tier, run with `OS_TEST_TIERS=nightly`): Tests 71 passed (71). - `migrate-confirmation-exit.pin.test.ts`: Tests 36 passed (36), both `audit-key-material` cases included. - Dogfood `audit-key-material.dogfood.test.ts`: Tests 4 passed (4), 9.6 s. Archive integration: Tests 8 passed (8), 36.9 s (shared agent box). </details> <details><summary>Ablation (fix committed first; restore proven by blob hash)</summary> - **Runner swapped** (`scripts/ablation-replace.mjs`, run mode with its own restore trap): the command's destructuring of the key-material runners was pointed at the mail-body runners, so the command boots and reports but removes no key material. Anchor 1 to 0, blob `370c0585` to `d3752c2d`. - archive integration: 4 failed / 8 (cases 1 to 4; 0, 5, 6, 7 pin the fixture, output hygiene and classification and stay green); - CLI rebuilt, `ablation-dist-preflight.mjs` found the marker in `dist/commands/migrate/audit-key-material.js`; dogfood: 2 failed / 4 (dry-run count, removal after apply). - **Unregistered** (default export removed): `migrate-confirmation-exit.pin.test.ts` 3 failed / 36 (roster equality and both new cases). A first attempt was refused by the tool before running, because its replacement was a substring of the original line and could not be counted; re-run with a countable marker. - Restore: blob back to `370c0585` == HEAD, `git status --porcelain` empty; CLI rebuilt, `--absent` preflight clean over 704 built files; dogfood 4/4 and archive 8/8 green again. </details> ## Acceptance notes - `docs/qa/platform-checklist/areas/cli.json` item `cli.migrate-plan-apply-json` enumerates the `os migrate` subcommands as variants and its source note says 19; with this command there are 20. Left unedited as dispatched; no checklist gate reads the count (`check-platform-checklist.mjs` and the watchdog both exit 0). - The upgrade line names "18.2.0 or earlier" because the writer change ships in the release after 18.2.0; it does not guess that release's number. ## Verification <details><summary>Gate derivation (dispatch-gates, head f75fee2)</summary> - `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands`: 93 commands. All 93 run; final exit 0 each. - Three first refused with exit 3 (`PREREQUISITE NOT MET`, nothing measured: `check:skill-examples`, `check:dual-build-cjs-loads`, `check:i18n-coverage` read `dist/` of packages outside the CLI closure). After those packages were built they re-ran with exit 0. - `dispatch-gates --ran` over the exit-coded record: 93 derived, 93 run, 0 NOT-MEASURED (a derived zero). - Families CI runs over the whole workspace are left to the PR's check-runs. </details> --- _Generated by [Claude Code](https://claude.ai/code/session_012YjcDon6BvR9MN93ZsFq3U)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 76a1275 commit a5df094

13 files changed

Lines changed: 1178 additions & 108 deletions
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
'@objectstack/cli': patch
3+
---
4+
5+
feat(cli): `os migrate audit-key-material` removes the encrypted key-material columns from the audit-ledger copies 18.2.0 and earlier wrote
6+
7+
Clause-②: no
8+
Metadata-protocol: none
9+
10+
**Run `os migrate audit-key-material --apply` once after upgrading from 18.2.0 or earlier.** The audit writer no longer copies the encrypted key-material column of a secret-store or settings row into `sys_audit_log` (`new_value`, `old_value`) or `sys_activity` (both halves of `metadata`). Copies written before that still carry it.
11+
12+
- **The command.** `os migrate audit-key-material` removes those columns with the writer's own helper, through `@objectstack/plugin-audit`'s `migrateKeyMaterialCopies` (the hot tables) and `migrateArchivedKeyMaterialCopies` (the archive datasource `sys_audit_log` archives to). It is a dry run by default (counts, writes no row), `--apply` writes, and its flags are `os migrate audit-mail-bodies`'s, one for one (`--database-url`, `--apply`, `--yes`, `--force`, `--json`). The report is counts only, per table, and never prints a value.
13+
- **What changes in a row.** Only the key-material keys of the copy are removed. Every other key of the copy and every other column of the row stay byte-identical, the row's `updated_at` included. A column of the same name in a copy of any other object is left alone.
14+
- **Idempotent.** A copy without the columns has nothing to remove, so a second run rewrites nothing; re-running finishes an interrupted run. A failed read or write exits `1`, and so does an archive datasource the run could not examine (`not-connected` / `unknown` in the `--json` document's `archive` entries, as for `audit-mail-bodies`).

‎content/docs/deployment/cli.mdx‎

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1293,6 +1293,7 @@ where the data lives.
12931293
| `os migrate organization-ownership` | The v18 ownership ceremony (ADR-0131 D10): write a plan of what each table's rows will become, then `--apply` that reviewed plan. It upgrades a database 17.x or an 18.0 pre-release (`18.0.0-next.*`) created; a v18 server refuses to start on one until it has run. See [below](#os-migrate-organization-ownership) |
12941294
| `os migrate activity-changed-fields` | Stamp the two read-rule columns (`changed_fields`, `milestone_key`) on the `sys_activity` rows written before they existed. Until it runs, those rows are withheld from every reader but the system, so run it at once after upgrading. No flag; idempotent |
12951295
| `os migrate audit-mail-bodies` | Mask the mail bodies (`body_text`, `body_html`) that `sys_audit_log` and `sys_activity` copies of `sys_email` / `sys_identity_email` rows still hold in clear, written before the audit writer masked them. Covers the `archive` datasource `sys_audit_log` archives to as well. Reports counts only. No flag; idempotent. See [below](#os-migrate-audit-mail-bodies) |
1296+
| `os migrate audit-key-material` | Remove the encrypted key-material columns from the `sys_audit_log` and `sys_activity` copies written before the audit writer omitted them. Covers the `archive` datasource `sys_audit_log` archives to as well. Reports counts only. No flag; idempotent. See [below](#os-migrate-audit-key-material) |
12961297
| `os migrate comment-parent-pair` | Stamp the parent pair (`object_name`, `record_id`) on the `sys_comment` rows written before it existed. Until it runs, a broad comment read past 2,000 candidate rows omits comments the reader may see, and its total agrees with the short list, so run it at once after upgrading. No flag; idempotent |
12971298
| `os migrate meta --stored` | Replay the metadata conversion chain over this deployment's `sys_metadata` rows and rewrite the ones still carrying a pre-protocol shape. Hygiene, not a gate — nothing depends on it having run |
12981299
| `os migrate security-catalog-overlays` | List the environment-wide permission-set and position rows (the legacy plurals `permissions` / `positions` included) stored over a name a configured package holds — the rows a v18 cold boot refuses — each with the package that holds its name; `--apply` deletes them, one history tombstone per row. Run it before the first v18 boot, with the flags and environment the deployment boots with: the platform security plugin's sets are held only where `os serve` composes it (an auth secret set, and the `auth` tier on), and the step takes `serve`'s `--preset` and `--dev` with `serve`'s meaning. Adopts nothing; exits 1 while it lists a row |
@@ -1311,7 +1312,7 @@ holds nothing, and answers with empty work and exit 0. `os migrate value-shapes`
13111312
clean over zero records and names the objects it did not read. `os migrate
13121313
recorded-by` has nothing to convert and `os migrate resume` no interrupted runs.
13131314
`os migrate meta --stored` has no stored metadata to examine, `os migrate
1314-
audit-metadata-bodies` and `os migrate audit-mail-bodies` no audit copy to rewrite, and
1315+
audit-metadata-bodies`, `os migrate audit-mail-bodies` and `os migrate audit-key-material` no audit copy to rewrite, and
13151316
`os migrate account-issuer` no account to collide. `os secret orphans`, `os secret rewrap` and `os storage orphans`
13161317
report no secret and no file. A read the command cannot avoid and that fails for any
13171318
other reason still refuses and exits 1. Point `--database-url` at the deployment's
@@ -1639,6 +1640,40 @@ An archive that a host registers in code outside the stack config (a plugin
16391640
that publishes the driver itself) is not reached and reads `none`. To mask it,
16401641
declare it in the stack config's `datasources` with `autoConnect: true`.
16411642
1643+
#### `os migrate audit-key-material`
1644+
1645+
The audit trail keeps a copy of each audited row: `sys_audit_log` in
1646+
`new_value` / `old_value`, and the activity feed in both halves of
1647+
`sys_activity.metadata`. For the secret store and the two settings holders,
1648+
that copy used to carry the row's encrypted key-material column. New copies
1649+
omit those columns. This command removes them from the copies written before
1650+
that.
1651+
1652+
```bash
1653+
os migrate audit-key-material # Dry run: counts, writes no rows
1654+
os migrate audit-key-material --apply # Remove the columns (prompts)
1655+
os migrate audit-key-material --apply --yes --json # CI / scripts
1656+
```
1657+
1658+
Only the key-material keys of the copy are removed: every other key of the copy
1659+
(which record, its key id, algorithm and version), and every other column of the
1660+
row (its `updated_at` included), stays as it was. A column of the same name in a
1661+
copy of any other object is left alone. The report holds counts only and never
1662+
prints a value. The flags are `os migrate audit-mail-bodies`'s, one for one.
1663+
1664+
**Idempotent**: a copy without the columns has nothing to remove, so a second
1665+
run rewrites nothing, and re-running is how an interrupted run is finished. A
1666+
failed read or write exits `1`. Run it once after upgrading, with the server
1667+
stopped, or re-run it once and read 0.
1668+
1669+
**The archive datasource too**, exactly as for
1670+
[`os migrate audit-mail-bodies`](#os-migrate-audit-mail-bodies): after the hot
1671+
run, the command runs the same rewrite over the datasource `sys_audit_log`
1672+
archives to, in the same dry-run or `--apply` mode, and its `--json` document
1673+
carries the same `archive` entries with the same `status` values and exit
1674+
codes. A declared archive this boot does not connect, or an active runtime
1675+
datasource row of that name, reads `not-connected` and exits `1`.
1676+
16421677
#### `os migrate organization-ownership`
16431678
16441679
The v18 ownership ceremony (ADR-0131 D10), and the upgrade path of a database

‎content/docs/upgrading.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -480,6 +480,8 @@ the protocol-18 step, which `os migrate meta --from 17` lists for your sources
480480
| v12.0.0 | [Upgrade checklist](/docs/releases/v12#upgrade-checklist) |
481481
| v9.0.0 | [Upgrade checklist](/docs/releases/v9#upgrade-checklist) |
482482

483+
**Upgrading from 18.2.0 or earlier:** run [`os migrate audit-key-material`](/docs/deployment/cli#os-migrate-audit-key-material) once, a dry run first and then `--apply`; it removes the encrypted key-material columns from the audit-ledger copies those releases wrote.
484+
483485
The v17 and v18 lines are broken out per release. Each v17 minor shipped
484486
breaking changes under the repo's launch-window convention — which ships a
485487
breaking change as `minor` rather than `major`, so the version number is not the

0 commit comments

Comments
 (0)