Skip to content

Commit 314ad95

Browse files
fix(objectql): defer the first lifecycle sweep one interval on a datastore this boot created (#30214)
<!-- claude-projects-attribution: {"github_login":"os-bill"} --> _Requested by **Bill** · [project thread](https://claude.ai/code/project/chan_016iMXWSZDzqWRWvWzdNo8wg?thread=cmsg_016iMXWSZDzqWRWvWzdNo8wgHwNYAiJoiu2ymqfmWX9z1V)_ domain:server#4 · os-bill · session_01Utp6CZMkHvuDeVYgTD73qF ## 维护者速读 - 改了什么:全新建好的数据库,开机 60 秒那次生命周期清扫(保留期读取、表计数、悬空引用审计)不再跑,推迟一个周期(默认一小时)再扫;已有数据的库照旧 60 秒首扫。 - 为什么改:新库里没有任何一行早于这次开机,最短保留窗口是一小时,首扫注定什么都删不掉,却在 cloud 托管内核上花掉第一分钟 160 条语句里的 130 条。 - 风险与回滚:只改调度器的首次触发,直接调用 sweep() 照扫;最坏情况是新库的过期行晚一个周期被清,回滚即还原 lifecycle-service.ts 一处判断。 - 先看:`packages/objectql/src/lifecycle/lifecycle-service.ts` 的 `start()` 与 `firstSweepDueAtInitialDelay()`;再看同目录测试文件末尾 `LifecycleService timers` 里的四个 #29918 用例。 - 两分钟验证:`pnpm --filter @objectstack/objectql exec vitest run src/lifecycle/lifecycle-service.test.ts -t 29918`,预期 4 个用例全绿。 - 需要你做的:无,席位落地。卡的条目 1(持久化最后清扫时间,让重启也省掉首扫)另待裁定,本 PR 不关卡。 Refs #29918 (item 2). Item 1, the persisted last-swept watermark, is not in this PR and waits on a decision (report 6646684881 on the card), so this PR does not close the card. ## What changes - `LifecycleService.start()` now asks, when the initial delay fires, whether the first sweep is due yet (`firstSweepDueAtInitialDelay()`). It is not due when the engine reports that this process created its datastore from empty: `ObjectQL.wasDatastoreCreatedFromEmpty()`, which answers from every driver's schema-sync stats, in process and without issuing a statement. In that case the scheduled first sweep is skipped, and the interval is armed as before, so the first sweep runs one interval later. - `LifecycleEngineLike` gains the optional member `wasDatastoreCreatedFromEmpty?(): boolean`. The real engine already has it. An engine that omits it, or whose answer throws, is treated as not fresh and keeps the old schedule. - The gate sits on the scheduler's first tick, not at the entry of `sweep()`. Direct `sweep()` callers (the dogfood growth gate, `db:clean`-style tooling, and more than a dozen suites in objectql, service-knowledge, service-queue and service-messaging that call it on purpose) keep sweeping. The reference audit is a leg of `sweep()` with no timer of its own (`auditReferences()`; its only other caller is the CLI's orphan-detail sweep), so it is deferred with the same tick. - `OS_LIFECYCLE_DISABLED`, `enabled: false`, `initialDelayMs` and `sweepIntervalMs` keep their meaning. One changeset, `patch` on `@objectstack/objectql`. ## Why A fresh store holds only rows this process wrote, so none is older than the initial delay (60 s), and the duration grammar's smallest window is one hour (`parseLifecycleDuration`). The first sweep on such a store therefore cannot reap anything. The card measured it on cloud's hosted tenant composition as 130 statements in the 50 to 60 s slice of a first boot (17 retention reads, 16 counts, the reference audit and the governance settings reads). Deferring it costs at most one interval of overstay, which the hourly cadence already allows any row. ## Tests Four cases added to the existing `LifecycleService timers` describe in `lifecycle-service.test.ts`. Each counts every data-plane door the sweep reaches: reaps, governance settings reads and the reference audit. - Fresh store: nothing at the initial delay (0 deletes, 0 audits, 0 settings reads), then one sweep one interval later. - Store that existed before the boot, or an engine whose answer throws: sweeps at the initial delay as before. - Direct `sweep()` on a fresh store: still sweeps. - `OS_LIFECYCLE_DISABLED=1` on a fresh store: nothing is armed. ## Acceptance notes - Card "Done when", clause 1 (no lifecycle-sweep statements in a fresh kernel's first minute): met for the measured first-boot reading. With this change no statement is issued at all, not even a watermark read. cloud re-measures under cloud#3413. - Clause 2 (a kernel whose watermark is older than the interval still sweeps) belongs to item 1 and is not met here. A re-boot on an existing store still sweeps at the initial delay. Item 1 has no existing persistence seam that needs neither a new config key nor a new `sys_*` object, so the seat sends it to a decision. - Deviation from the seat's note: the seat asked for the gate at the entry of `sweep()`. It sits on the scheduler's first tick instead, because a gate at the entry would stop every direct `sweep()` caller from sweeping on a fresh store. The reference audit is still deferred, since it only runs inside `sweep()`. ## Verification All readings on head `07cef192` (after one merge of `origin/main`), in the worktree, with the dependency closure built. - `pnpm --filter @objectstack/objectql exec vitest run --maxWorkers=2 src/lifecycle/`: Test Files 5 passed (5), Tests 135 passed (135). - `pnpm --filter @objectstack/objectql test`: Test Files 548 passed (548), Tests 10867 passed (10867), lock VERDICT command-exit 0. - `pnpm --filter @objectstack/objectql typecheck` (`tsc --noEmit` and `check:test-typecheck`): VERDICT command-exit 0. - `pnpm lint`: exit 0. - Derived gate list (`node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`): 59 families, 58 run and exit 0, 1 NOT MEASURED. `pnpm check:dual-build-cjs-loads` exits 3 (PREREQUISITE NOT MET: it needs the whole workspace built) and is left to CI. `check-plugin-teardown-shape.mjs --self-test` exited 3 on the shallow clone, then exit 0 (48 cases) after fetching the commit its fixture reads. `--ran` reconciliation: "59 derived famil(ies) accounted for — 58 run, 1 NOT-MEASURED". <details><summary>Ablation of the freshness gate</summary> Committed fix first (`027c6689`), then `node scripts/ablation-replace.mjs` replaced `if (this.firstSweepDueAtInitialDelay()) void this.sweep();` with an unconditional `void this.sweep();` plus a marker comment (anchor 1 to 0, blob `026c59ebf9a8` to `f36281d0bba1`) and ran `vitest run src/lifecycle/lifecycle-service.test.ts -t 29918`: ``` × [#29918] issues no sweep at the initial delay on a store this boot created, and sweeps one interval later Tests 1 failed | 3 passed | 114 skipped (118) ``` Restore: blob after restore `026c59ebf9a8` equals HEAD, `git diff HEAD` empty. The other three cases stay green under the mutation, as they should: they pin the unchanged paths. The suite imports the service from source by relative path, so no build sits between the mutation and the run. </details> Clause-②: yes (widening) — the exported `LifecycleEngineLike` gains the optional member `wasDatastoreCreatedFromEmpty?(): boolean` (an engine without it keeps the old schedule); no accept set changes. Metadata-protocol: none ## Human Minimum - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic. - [x] 2. Every name I added uses the casing for its kind. - [x] 3. No formatter ran; `pnpm lint` passes. - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. - [x] 6. `packages/spec` touched: its generated artifacts are regenerated and committed. (Not touched.) - [x] 7. No hand edits to `content/docs/references/`, `packages/*/CHANGELOG.md` or `content/docs/releases/`. - [x] 8. Title, body and comments are in English. - [x] The *Before Submitting* list in CONTRIBUTING.md is done. --- _Generated by [Claude Code](https://claude.ai/code/session_01Utp6Ca4cwEoenneeTMpCvy)_ <!-- signed: domain:server#4 · os-bill · session_01Utp6CZMkHvuDeVYgTD73qF --> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 5fa9375 commit 314ad95

3 files changed

Lines changed: 161 additions & 2 deletions

File tree

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
'@objectstack/objectql': patch
3+
---
4+
5+
fix(objectql): the lifecycle sweep no longer runs 60 s after a boot that created its datastore from empty; its first sweep runs one interval later
6+
7+
Clause-②: yes (widening) — the exported `LifecycleEngineLike` gains the optional member `wasDatastoreCreatedFromEmpty?(): boolean` (an engine without it keeps the old schedule); no accept set changes.
8+
Metadata-protocol: none
9+
10+
- `LifecycleService.start()` asks the engine, at the initial delay, whether this process created the datastore from empty (`ObjectQL.wasDatastoreCreatedFromEmpty()`, answered from every driver's schema-sync stats without a statement). When it did, the scheduled first sweep is skipped and the sweep runs on the interval, one interval later. Such a store holds no row older than the boot, and no declared window is shorter than an hour, so the skipped sweep had nothing to reap. It still read every declared object, the `lifecycle` settings and every lookup the reference audit walks: 130 statements in the first minute of a hosted tenant kernel's first boot.
11+
- A store that existed before the boot, or an engine that cannot say, sweeps at the initial delay as before. A direct `sweep()` call always sweeps. `OS_LIFECYCLE_DISABLED`, `initialDelayMs` and `sweepIntervalMs` keep their meaning.
12+
- `LifecycleEngineLike` gains the optional member `wasDatastoreCreatedFromEmpty?(): boolean`. An engine double that omits it keeps the old schedule.
13+
- A re-boot on an existing store still sweeps at the initial delay. Persisting a last-swept watermark for that case is separate work.

‎packages/objectql/src/lifecycle/lifecycle-service.test.ts‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2541,6 +2541,101 @@ describe('LifecycleService timers', () => {
25412541
vi.useRealTimers();
25422542
}
25432543
});
2544+
2545+
/**
2546+
* [#29918] A datastore this process created from empty holds no row older
2547+
* than the boot, so the first scheduled sweep is due one interval later.
2548+
* Every data-plane door the sweep can reach is counted: the reaps, the
2549+
* governance settings reads and the reference audit.
2550+
*/
2551+
function freshnessEngine(answer: () => boolean) {
2552+
const captured = captureEngine([
2553+
{ name: 'sys_job_run', lifecycle: { class: 'telemetry', retention: { maxAge: '30d' } } },
2554+
]);
2555+
const audits: unknown[] = [];
2556+
const settingsReads: string[] = [];
2557+
captured.engine.wasDatastoreCreatedFromEmpty = answer;
2558+
captured.engine.inspectDanglingReferences = async (options: unknown) => {
2559+
audits.push(options);
2560+
return { findings: [], scannedObjects: [], unreadableObjects: [] };
2561+
};
2562+
const getSettings = () => ({
2563+
async get(namespace: string, key: string) {
2564+
settingsReads.push(`${namespace}.${key}`);
2565+
return { value: undefined };
2566+
},
2567+
});
2568+
return { ...captured, audits, settingsReads, getSettings };
2569+
}
2570+
2571+
it('[#29918] issues no sweep at the initial delay on a store this boot created, and sweeps one interval later', async () => {
2572+
vi.useFakeTimers();
2573+
try {
2574+
const { engine, deletes, audits, settingsReads, getSettings } = freshnessEngine(() => true);
2575+
const svc = service(engine, { initialDelayMs: 1_000, sweepIntervalMs: 5_000, getSettings });
2576+
2577+
svc.start();
2578+
await vi.advanceTimersByTimeAsync(1_000);
2579+
expect(deletes).toHaveLength(0);
2580+
expect(audits).toHaveLength(0);
2581+
expect(settingsReads).toEqual([]);
2582+
2583+
await vi.advanceTimersByTimeAsync(5_000);
2584+
expect(deletes).toHaveLength(1);
2585+
expect(audits).toHaveLength(1);
2586+
expect(settingsReads.length).toBeGreaterThan(0);
2587+
svc.stop();
2588+
} finally {
2589+
vi.useRealTimers();
2590+
}
2591+
});
2592+
2593+
it('[#29918] sweeps at the initial delay when the store was there before this boot, or the engine cannot say', async () => {
2594+
vi.useFakeTimers();
2595+
try {
2596+
const answers: Array<() => boolean> = [
2597+
() => false,
2598+
() => {
2599+
throw new Error('no driver reports schema-sync stats');
2600+
},
2601+
];
2602+
for (const answer of answers) {
2603+
const { engine, deletes, audits } = freshnessEngine(answer);
2604+
const svc = service(engine, { initialDelayMs: 1_000, sweepIntervalMs: 5_000 });
2605+
svc.start();
2606+
await vi.advanceTimersByTimeAsync(1_000);
2607+
expect(deletes).toHaveLength(1);
2608+
expect(audits).toHaveLength(1);
2609+
svc.stop();
2610+
}
2611+
} finally {
2612+
vi.useRealTimers();
2613+
}
2614+
});
2615+
2616+
it('[#29918] a direct sweep() on a store this boot created still sweeps', async () => {
2617+
const { engine, deletes, audits } = freshnessEngine(() => true);
2618+
await service(engine).sweep();
2619+
expect(deletes).toHaveLength(1);
2620+
expect(audits).toHaveLength(1);
2621+
});
2622+
2623+
it('[#29918] OS_LIFECYCLE_DISABLED still arms nothing on a store this boot created', async () => {
2624+
vi.useFakeTimers();
2625+
process.env.OS_LIFECYCLE_DISABLED = '1';
2626+
try {
2627+
const { engine, deletes, audits } = freshnessEngine(() => true);
2628+
const svc = service(engine, { initialDelayMs: 1_000, sweepIntervalMs: 5_000 });
2629+
svc.start();
2630+
await vi.advanceTimersByTimeAsync(20_000);
2631+
expect(deletes).toHaveLength(0);
2632+
expect(audits).toHaveLength(0);
2633+
svc.stop();
2634+
} finally {
2635+
delete process.env.OS_LIFECYCLE_DISABLED;
2636+
vi.useRealTimers();
2637+
}
2638+
});
25442639
});
25452640

25462641
/**

‎packages/objectql/src/lifecycle/lifecycle-service.ts‎

Lines changed: 53 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,15 @@ export interface LifecycleEngineLike {
118118
inspectDanglingReferences?(
119119
options?: DanglingReferenceAuditOptions,
120120
): Promise<DanglingReferenceReport>;
121+
/**
122+
* [#29918] Did this process create the datastore it talks to, from empty?
123+
* `ObjectQL.wasDatastoreCreatedFromEmpty()` answers it from every driver's
124+
* schema-sync stats, in process and without a statement. The scheduler asks
125+
* it once, at the initial delay, to decide whether the first sweep is due
126+
* yet. Optional: an engine without it is never treated as fresh, so its
127+
* first sweep runs at the initial delay exactly as before.
128+
*/
129+
wasDatastoreCreatedFromEmpty?(): boolean;
121130
}
122131

123132
export interface LifecycleObjectLike {
@@ -503,7 +512,11 @@ export class LifecycleService {
503512
}
504513

505514
/** Arm the periodic sweep. Idempotent; timers are unref'ed so a kernel
506-
* shutdown is never held open by the lifecycle schedule. */
515+
* shutdown is never held open by the lifecycle schedule.
516+
*
517+
* [#29918] On a datastore this process created from empty, the first sweep
518+
* is due one interval later instead of at the initial delay: see
519+
* {@link firstSweepDueAtInitialDelay}. The interval is armed either way. */
507520
start(): void {
508521
if (!this.enabled || this.timer || this.initialTimer) return;
509522
// A fresh bit per armed run — the one a previous sweep captured stays
@@ -513,13 +526,51 @@ export class LifecycleService {
513526
const initial = this.opts.initialDelayMs ?? DEFAULT_LIFECYCLE_INITIAL_DELAY_MS;
514527
this.initialTimer = setTimeout(() => {
515528
this.initialTimer = undefined;
516-
void this.sweep();
529+
if (this.firstSweepDueAtInitialDelay()) void this.sweep();
517530
this.timer = setInterval(() => void this.sweep(), interval);
518531
this.timer.unref?.();
519532
}, initial);
520533
this.initialTimer.unref?.();
521534
}
522535

536+
/**
537+
* [#29918] Whether the scheduled first sweep runs at the initial delay.
538+
*
539+
* Not when this process created the datastore from empty
540+
* ({@link LifecycleEngineLike.wasDatastoreCreatedFromEmpty}). Every row in
541+
* such a store was written by this process, so none is older than the
542+
* initial delay, and no declared window is that short: the duration grammar's
543+
* smallest unit is an hour ({@link parseLifecycleDuration}). The sweep would
544+
* read every declared object, the governance settings and every lookup the
545+
* reference audit walks, and find nothing to do. On a hosted tenant kernel
546+
* that was 130 statements in the first minute of every first boot. The first
547+
* sweep then runs on the interval, so deferring it costs at most one interval
548+
* of overstay, which the hourly cadence already allows any row.
549+
*
550+
* Asked here, at the initial delay, and nowhere else. Schema sync has run by
551+
* now, so the answer is final. Only the SCHEDULER asks it: a direct
552+
* {@link sweep} call (the dogfood growth gate, `db:clean`-style tooling, a
553+
* test) always sweeps. The reference audit is a leg of that same sweep, so
554+
* it is deferred with it.
555+
*
556+
* An engine that cannot answer, or whose answer throws, is not fresh. That
557+
* keeps the sweep at the initial delay, as before this method existed.
558+
*/
559+
private firstSweepDueAtInitialDelay(): boolean {
560+
let fresh = false;
561+
try {
562+
fresh = this.opts.getEngine()?.wasDatastoreCreatedFromEmpty?.() === true;
563+
} catch {
564+
fresh = false;
565+
}
566+
if (fresh) {
567+
this.opts.logger.debug?.(
568+
'[lifecycle] this process created the datastore from empty; the first sweep runs one interval later',
569+
);
570+
}
571+
return !fresh;
572+
}
573+
523574
/**
524575
* Disarm the schedule AND call off the work.
525576
*

0 commit comments

Comments
 (0)