Repository navigation
Commit 31fe289
feat(objectql,cli): recompute stored home-currency amounts at the rates in force now, report first, rows without a rate listed and skipped (multi-currency M2-3) (#30183)
<!-- claude-projects-attribution: {"github_login":"os-bill"} -->
_Requested by **Bill** · [project
thread](https://claude.ai/code/project/chan_016iMXWSZDzqWRWvWzdNo8wg?thread=cmsg_016iMXWSZDzqWRWvWzdNo8wgJ4XLvydGoPeNPtcqsTzsGU)_
epic:#8345#1 · os-bill · session_01Utp6CZWjnSeed8H4dMjt9d
## 维护者速读
- **改了什么:** 新增 `os migrate recompute-home-amounts --object X [--field]
[--since DATE]
[--apply]`:按「现在生效的汇率」重算已存的本位币金额。默认只出报告(每个组织各自的本位币与条数、哪些行会变、从多少变成多少、哪些行转不了及原因),`--apply`
才写,只写本位币字段与汇率字段,`updated_at` 不动(按 `updated_at` 增量同步的下游看不到,文档已写明),每次
apply 记一条运行级审计,审计插件另记逐行审计行。
- **为什么改:** ADR-0144 D5:改正过的汇率只能经「显式重算」才进到已存的行;给已有数据的对象新加本位币字段时,老行也靠它补齐。
- **风险与回滚:** 不复制任何换算:预览直接调写入时的同一投影函数,apply 走
`engine.update`、由写入自己再算一次,所以重算后的行与新写入一模一样(消融为证)。缺汇率、空币种码、无单据日期、未设本位币的行只列出、不写,命令保持退出码
1 直到补齐。回滚即撤 PR。
- **先看:**
`packages/objectql/src/home-amount-recompute.ts`(`recomputeHomeAmounts`),再看
`engine.ts` 新增的 `homeAmountReadsFor`(委托到
`engine-references-and-master-detail.ts`)。
- **两分钟验证:** `pnpm --filter @objectstack/objectql exec vitest run
src/engine-home-amount-projection.test.ts -t recompute`:改汇率后报告列出
71994.24 → 80000,apply 后与新写入的行相同,再跑一次无变化。
- **需要您做的:** 无。Open item 8(跳过行怎么保持可见)在下方提案并有测试钉住,由 epic PM 带进 ADR 修订。
Fixes #27193
Supersedes #30181 (the same branch; that draft was opened through a door
the fleet does not take for pull requests, so it is closed and this one
is opened through the relay, body unchanged apart from review round 1
below).
R2.1 of epic #8345 (ADR-0144 D9 R2, M2-3). The R2 core (#28096, #27185,
#27198, #28302) shipped in 18.3.0 (ruling 6539345871, letter A′); this
rides a later Version Packages, as the epic PM pairs it.
Clause-②: yes (widening)
Metadata-protocol: none
## What changes
**`@objectstack/objectql` — `recomputeHomeAmounts(engine, logger, {
object, field?, since?, apply?, maxRecords?, maxListed?, source? })`**
(`home-amount-recompute.ts`), exported with
`assertHomeAmountRecomputeSince` (the pure `since` check),
`formatHomeAmountRecomputeReport`, `homeAmountRecomputeComplete` and
`HOME_AMOUNT_SKIP_REASONS`. Modelled on `record-currency-backfill.ts` /
`summary-backfill.ts`: a keyset walk over `id` under the unscoped system
context, one `engine.update` per row that differs.
- **No second copy of the arithmetic.** The preview projects each stored
row with `projectHomeAmounts` (the write's own function) over the
engine's own reads, `ObjectQL.homeAmountReadsFor(context)`: the same
rate-row rule (latest `effective_date` on or before the document date,
in the row's organization), exact division, single half-away rounding to
the home currency's CLDR fraction digits, rate 1 for the home currency,
and a `datetime` document date read as its day in the organization's
`localization.timezone`. The apply writes `{ id, <homeField> }` through
`engine.update`; a payload touching the home field makes the write
project the row again over the stored row, so the value stored is the
write's, never the preview's (ablation A2 below).
- **Per organization.** The organization is the one a write converts the
row under (the row's `organization_id`); each converts into its own home
currency at its own rates, and the report carries one entry per
organization (`homeCurrency`, `amounts`, `unchanged`, `changed`,
`written`, skipped per reason, `outsideSince`). No deployment-wide
answer.
- **Writes only the two projected fields**, under `{ isSystem,
preserveAudit }` with the row's stored `updated_at` (the
`comment-parent-pair` / `audit-key-material` precedent), so `updated_at`
does not move (ablation A3, on SQL). The payload is narrow; the write is
not: what reads the home field moves with it as for a fresh write
(stored formulas, parent summaries, and one per-row `sys_audit_log`
`update` row per rewritten row where the audit writers are installed, as
they are in the command's boot). A consumer syncing by `updated_at` does
not see a recomputed row; `record-currency-codes` does move
`updated_at`. Both stated in the docs.
- **Listed and skipped, never written (parameter 1)**, five reasons:
`rate_missing` (with currency and date), `code_blank` (ruling 6492000994
item 5: an amount beside a blank code is its own class; remedy `os
migrate record-currency-codes`, not a rate; the remedy line warns that
that command writes its one code into every organization's codeless
rows, Open item 13, so the per-organization count is the check first),
`document_date_blank`, `home_currency_unset`, `unconvertible` (a code,
amount or date the write's validator refuses).
- **`since`** filters on the **document date** (`homeAmount.rateDate`,
the date that picks the rate), read as the day the write reads it (a
`datetime` in the organization's timezone); a row with no document date
is outside every window. Pinned at the timezone boundary (ablation A6).
- **Audit:** one run-level `sys_audit_log` entry per applied run, beside
the per-row rows the audit writers add (pinned with the real writers in
`plugin-audit`): `action: 'update'`, `object_name`, no `record_id`,
`tenant_id` empty (the run spans organizations: a deployment-level row),
`metadata` = `{ event: 'home_amount.recompute', source, object, fields,
since, counts, organizations, skippedByReason, skipped (≤ 1000),
failures, truncated }`. The registry is asked first (the
`admin-import-users` posture): with no ledger in the composition the
report says `audit: { written: false, reason: 'ledger_not_mounted' }`; a
refused insert is warned and reported, and the run that already landed
stands. An unclassified error that stops an applied walk partway (a page
read or a rate read that fails) still writes that entry, marked
`aborted` with the counts it reached, before it is thrown on (ablation
A9).
- Refused before any row is read: unknown object (`OBJECT_NOT_FOUND` /
404), an object or `field` that declares no home amount (`INVALID_FIELD`
/ 400), a `since` that is not a real `YYYY-MM-DD` (`VALIDATION_FAILED` /
400). The function takes no CLI dependency, so a non-CLI door can call
it (no HTTP or Setup door is built here; PR #28128 S1 is open).
**`@objectstack/objectql` — `ObjectQL.homeAmountReadsFor(context?)`**
returns `{ serving, organizationOf(row) }`, the reads the write-time
projection makes. The facade stays delegation-only: the body is
`ReferencesAndMasterDetailFacet.homeAmountReadsFor`.
**`@objectstack/cli` — `os migrate recompute-home-amounts --object <o>
[--field <amount>] [--since YYYY-MM-DD] [--apply] [--yes] [--json]
[--force] [--max-records] [--database-url]`**, the
`record-currency-codes` shell one for one (occupancy gate, `--yes`,
read-only dry-run boot, the "no app objects loaded" refusal,
`refuseUndeclaredObjects`, journal-mode note, `--json` envelope).
`--since` is judged before the probe, the prompt and the boot. The boot
takes `buildDataMigrationPlugins({ settings: true, audit: true })`: a
new `settings` arm adds the one-shot settings composition
(`oneShotSettingsPlugin`, never mints a key), because without the
settings service every organization reads as having no home currency.
Exit 1 while any row is listed and skipped, on a dry run as on an apply,
and when `--max-records` cut the walk short (ablation A10). `--since` is
judged by the recompute's own `assertHomeAmountRecomputeSince`, imported
and called before the probe; the command stamps no error code of its
own. Rows added to the three CLI rosters that discover commands from the
tree (`migrate-confirmation-exit.pin`, `schema-migrate.one-shot-family`,
`json-stdout-purity`).
**Ledgers this diff had to teach:**
`federated-injected-column-readers.test.ts` (#21918) gains the
recompute's rows: it asks `isFederatedUnprovisionedInjectedColumn` and
leaves a federated object's unprovisioned `organization_id` /
`updated_at` out of its read and its write (disposition `skips`).
`scripts/engine-double-contract.pinned.json` gains the CLI test double,
whose `update` routes through `assertEngineUpdateDispatch` (coverage
grows, regenerated with `--write`).
**Docs:** `content/docs/data-modeling/currency.mdx` gains "Recomputing
stored home amounts"; `content/docs/deployment/cli.mdx` gains the table
row and the `os migrate recompute-home-amounts` section beside
`record-currency-codes`.
**ADR anchor:**
`scripts/adr-anchors/packages__objectql__src__home-amount-recompute.ts.json`
(ADR-0144: no second copy of the arithmetic; skip classes; Open item 8).
## ADR-0144 open items settled here
For the epic PM to carry into the ADR in its own PR:
- **Open item 8 (a row a backfill skips).** Proposal, pinned: (1) a
skipped row is never written: its home amount and rate stay as stored,
so a never-converted row stays blank, never 0 or a 1:1 conversion; (2)
the run is not complete while any row is skipped:
`homeAmountRecomputeComplete` is false and the command exits 1, on a dry
run and on an apply, until a re-run skips nothing (a
`--max-records`-truncated walk is incomplete the same way); (3) the
applied run's `sys_audit_log` entry records the skipped rows (counts per
organization and reason, and each row with its reason, currency and
date), so the gap outlives the terminal. A per-row marker column was not
chosen: it would be a new author-visible field or an injected column (D8
rule 2), i.e. spec work.
- **Parameter 1 as built:** report and skip a row with no rate in force
on its document date, apply the rest; plus four sibling skip classes
(above), each with its own remedy, a blank code apart from a missing
rate (ruling 6492000994 item 5).
- **`--since` reading:** the document date (`rateDate`), as the write
reads it.
## Declarations re-derived
The order proposed `Clause-②: no` · `Metadata-protocol: none`. Corrected
to **`Clause-②: yes (widening)`**: new `@objectstack/objectql` exports
(`recomputeHomeAmounts` and its report helpers and types, the
`HomeAmountReads` / `HomeAmountServing` / `ExchangeRateRow` types), a
new public engine method (`homeAmountReadsFor`), a new CLI command.
Nothing narrows. `Metadata-protocol: none`: no authorable key, value or
type changes. One changeset, `patch` for both packages. `packages/spec`
is not touched.
## Review round 1 (seat review 6643251084 on #30181 @ 90fabf3)
| Item | Done |
|:--|:--|
| M1 error-code provenance | The CLI's `refuseMalformedSince` and its
`CALENDAR_DATE` are gone; `assertHomeAmountRecomputeSince` is exported
from `home-amount-recompute.ts` (stamped there, the owner the ledger
already lists) and called before the probe.
`check-error-code-provenance.ts` exit 0. No spec ledger touched. |
| M2 duplicate heading | `cli.mdx`: one `#### os migrate
comment-reactions`. |
| M3 404 | Round 1 asserted `httpStatus: 404` by setting it in the
shared `refuseUndeclaredObjects`; that changed every `os migrate
--object` envelope and broke the key-set pin in
`object-scope.integration.test.ts` (#21644). Round 2 (seat withdrew that
half of M3): the util is back to its `90fabf31` form, and the test
asserts what the envelope carries, `code: 'OBJECT_NOT_FOUND'` and exit
1. |
| S1 per-row audit rows |
`plugin-audit/src/home-amount-recompute-audit.integration.test.ts`: real
`installAuditWriters`, SQLite (wasm), 3 rows rewritten → 3 per-row
`update` rows + 1 run-level entry. Docs and the module header say so. |
| S2 incomplete runs | `--max-records` truncation exits 1 (CLI pin,
A10); an aborted applied walk writes its run entry marked `aborted`,
then throws (objectql pin, A9); a dry run that aborts writes nothing
(pin). |
| S3 `updated_at` | One sentence each in `cli.mdx`, `currency.mdx`, the
changeset. |
| S4 docblock | The CLI test names `home-amount-projection.test.ts`. |
| S5 Open item 13 | The `code_blank` remedy line and the docs table warn
that `record-currency-codes` writes one code into every organization. |
| S6 | "What is written" now says dependents of the home field move as
for a fresh write. |
## Review round 2 (seat delta review 6644826096 on #30183 @ 35b01a0)
| Item | Done |
|:--|:--|
| D1 Test Core red | `migrate-object-scope.ts` / `.test.ts` restored to
`90fabf31`; the changeset's envelope bullet dropped; the command's test
asserts `code` and exit 1 only. `object-scope.integration.test.ts` 20 /
20 locally. |
| Stale comment | The type-only import's comment now says the engine
package is imported dynamically inside `run()`, first for the `--since`
check. |
## Verification
Head `8374620a57` (review round 2 on top of `35b01a068e`; round 1 on top
of `90fabf3176`, then `origin/main` `a35fc6656` merged in, no rebase).
On `8374620a57`: cli `object-scope.integration` 20 / 20,
`recompute-home-amounts` 9 / 9, `migrate-object-scope` +
`migrate-confirmation-exit.pin` (83 / 83 across the four files); cli
typecheck 0; eslint 0; `check-error-code-provenance` 0;
`check-changeset-no-major` 0. The rows below are from `35b01a068e`
unless marked.
- `packages/objectql` `src/engine-home-amount-projection.test.ts`: 45 /
45 (28 existing + 17 recompute pins, in-memory driver double over
`applyDriverQueryInMemory`): a corrected rate (dry run lists 71994.24 @
0.1389 → 80000 @ 0.125, writes nothing; apply moves only `amount_home` /
`amount_rate`; a second run is clean); the recomputed row equals a fresh
write; one audit entry per applied run, none for a dry run,
`ledger_not_mounted` without the ledger; a missing rate listed with
currency and date, skipped, the run incomplete, the audit entry naming
it; a home field added over stored rows (rows without a rate stay blank,
never 0); a blank code its own class with the `record-currency-codes`
remedy; blank document date and unset home currency as their own
classes; same currency at rate 1 with no rate read (1.005 → 1.01); a
`datetime` document date across the Shanghai/UTC day boundary, for the
rate and for `since`; `since` on the document date; two organizations,
CNY and JPY homes, counted apart, a correction reaching only its own
organization; four refusals before any read; the pure `since` check with
the run's own envelope; an aborted applied run writing its `aborted` run
entry, and an aborted dry run writing nothing.
- `packages/qa/dogfood` `test/home-amount-projection.test.ts`: 42 / 42 =
3 cells (better-sqlite3, sqlite-wasm, PostgreSQL 16 via
`OS_TEST_POSTGRES_URL`) × (8 existing + 6 recompute pins): corrected
rate with report-then-apply idempotence, the fresh-write equality and
`updated_at` kept; a missing rate beside a filled row; a blank code; the
same currency; the timezone boundary through a `datetime` column; two
organizations under the `isolated` posture, each reading its own rate
rows.
- `packages/plugins/plugin-audit`
`src/home-amount-recompute-audit.integration.test.ts`: 1 / 1 (real
`installAuditWriters`, sqlite-wasm: 3 per-row `update` rows + 1 run
entry).
- `packages/cli`: `recompute-home-amounts.test.ts` 9 / 9 (+
`--max-records` truncation exits 1);
`src/commands/migrate/object-scope.integration.test.ts` 20 / 20 (the
#21644 envelope key-set pin, red in round 1, green again);
`test/migrate-confirmation-exit.pin.test.ts` 48 / 48;
`schema-migrate.one-shot-family.integration.test.ts -t
recompute-home-amounts` 5 / 5; `test/json-stdout-purity.e2e.test.ts`
(`OS_TEST_TIERS=nightly`) 95 / 95.
- Typecheck: `@objectstack/objectql`, `@objectstack/cli` and
`@objectstack/plugin-audit` pass (test-typecheck ledgers held; the cli
one needs `NODE_OPTIONS=--max-old-space-size=8192` on this container).
eslint on every touched file: clean.
- Full `@objectstack/objectql` suite: 548 / 549 files, 10 862 / 10 863
tests on the head before `federated-injected-column-readers.test.ts`
learned the recompute's two column reads; the recompute now asks
`isFederatedUnprovisionedInjectedColumn` before selecting
`organization_id` / `updated_at` and the table has its rows; that file
and the recompute file re-run 50 / 50 on the final head. `pnpm --filter
@objectstack/spec exec tsx scripts/check-error-code-provenance.ts`: exit
0 (review M1).
### Ablations
`scripts/ablation-replace.mjs`, each restored byte-identical (blob ==
HEAD).
| # | Mutation | Result |
|:--|:--|:--|
| A1 | The report's value computed by a local float copy (`amount ÷
rate`, 2 decimals) instead of the projection's | 2 / 42 red (1.005
same-currency; the JPY organization) |
| A2 | The apply payload's home value replaced with `0` | 42 / 42 green:
the stored value is the write's own projection, never the payload's |
| A3 | Apply context without `preserveAudit` (objectql rebuilt under it;
dogfood) | 3 / 3 cells red on `updated_at` |
| A4 | The blank-code class removed | 1 red (the blank code is reported
`unconvertible`) |
| A5 | `homeAmountRecomputeComplete` ignores skipped rows | 1 red |
| A6 | `since` exclusive (`<=`) | 1 red (the timezone-boundary pin) |
| A7 | Organization forced to none | 1 red (two organizations) |
| A8 | The SHARED projection's half-away step removed
(`home-amount-projection.ts`) | the fixed-number pins red (2), while "a
recomputed row equals a fresh write" stays green: write and recompute
move together, one code path |
| A9 | The aborted run's audit write removed from the walk's `catch` | 1
/ 45 red (the aborted-run pin) |
| A10 | The CLI exit ignores `report.truncated` | 1 / 9 red (the
`--max-records` pin) |
### Gates
`dispatch-gates --commands` on head `35b01a068e`: 108 derived commands,
plus `check-error-code-provenance.ts` run by hand because the derivation
does not name it (review P2), each exit code captured before any pipe:
all 109 exit 0. `dispatch-gates --ran`: 108 derived / 108 run / 0
NOT-MEASURED / 0 UNRUN (exit 0). Change set 22 paths, +2016 / −7, under
the 3000-line human-merge threshold.
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic.
(not touched)
- [x] 2. Every name I added uses the casing for its kind.
- [x] 3. No formatter ran; eslint on the touched files passes.
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request (this one supersedes #30181, same branch).
- [x] 5. A change that publishes carries a changeset, and both
declaration lines are filled in here and in the changeset.
- [x] 6. `packages/spec` touched: n/a.
- [x] 7. No hand edits to `content/docs/references/`,
`packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English (the maintainer brief
is Chinese by the seat's convention).
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Utp6CXZRTkUQ6UjyMWfN2U)_
<!-- signed: epic:#8345#1 · os-bill · session_01Utp6CZWjnSeed8H4dMjt9d
-->
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent a35fc66 commit 31fe289
20 files changed
Lines changed: 2012 additions & 5 deletions
File tree
- .changeset
- content/docs
- data-modeling
- deployment
- packages
- cli
- src
- commands/migrate
- utils
- test
- objectql/src
- plugins/plugin-audit/src
- qa/dogfood/test
- scripts
- adr-anchors
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
317 | 317 | | |
318 | 318 | | |
319 | 319 | | |
320 | | - | |
| 320 | + | |
| 321 | + | |
321 | 322 | | |
322 | 323 | | |
323 | 324 | | |
| |||
336 | 337 | | |
337 | 338 | | |
338 | 339 | | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
339 | 370 | | |
340 | 371 | | |
341 | 372 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1307 | 1307 | | |
1308 | 1308 | | |
1309 | 1309 | | |
| 1310 | + | |
1310 | 1311 | | |
1311 | 1312 | | |
1312 | 1313 | | |
| |||
1621 | 1622 | | |
1622 | 1623 | | |
1623 | 1624 | | |
| 1625 | + | |
| 1626 | + | |
| 1627 | + | |
| 1628 | + | |
| 1629 | + | |
| 1630 | + | |
| 1631 | + | |
| 1632 | + | |
| 1633 | + | |
| 1634 | + | |
| 1635 | + | |
| 1636 | + | |
| 1637 | + | |
| 1638 | + | |
| 1639 | + | |
| 1640 | + | |
| 1641 | + | |
| 1642 | + | |
| 1643 | + | |
| 1644 | + | |
| 1645 | + | |
| 1646 | + | |
| 1647 | + | |
| 1648 | + | |
| 1649 | + | |
| 1650 | + | |
| 1651 | + | |
| 1652 | + | |
| 1653 | + | |
| 1654 | + | |
| 1655 | + | |
| 1656 | + | |
| 1657 | + | |
| 1658 | + | |
| 1659 | + | |
| 1660 | + | |
| 1661 | + | |
| 1662 | + | |
| 1663 | + | |
| 1664 | + | |
| 1665 | + | |
| 1666 | + | |
| 1667 | + | |
| 1668 | + | |
| 1669 | + | |
| 1670 | + | |
| 1671 | + | |
| 1672 | + | |
| 1673 | + | |
| 1674 | + | |
| 1675 | + | |
| 1676 | + | |
| 1677 | + | |
| 1678 | + | |
| 1679 | + | |
| 1680 | + | |
| 1681 | + | |
| 1682 | + | |
| 1683 | + | |
| 1684 | + | |
| 1685 | + | |
| 1686 | + | |
1624 | 1687 | | |
1625 | 1688 | | |
1626 | 1689 | | |
| |||
0 commit comments