fix(service-settings)!: a json setting persists a JSON object or array, and the console sends one

.changeset/31241-json-setting-persists-object.md

@@ -0,0 +1,27 @@
+---
+'@objectstack/service-settings': patch
+'@objectstack/objectql': patch
+'@object-ui/console': patch
+---
+
+fix(service-settings)!: a `json` setting persists a JSON object or array, and the console sends one; data lifecycle quotas and retention overrides saved as text are read and applied
+
+Clause-②: yes (narrowing) — the settings save door and the env door start refusing a `json` value that is not a JSON object or array, which they accept today; ADR-0007:123 already declares `json` persists an object, so the narrowing enforces the declared contract and widens nothing
+Metadata-protocol: none
+
+<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped, and no export or type changes. What narrows is the value a settings door accepts for a `json` key: a stored value is runtime state in `sys_setting` / `sys_platform_setting` and an `OS_*` override is a deployment's environment, so there is nothing for a migration to rewrite. Values already stored as JSON text are handled by their one reader (`LifecycleService`): quota text is read as the object it holds, with a warning, and retention-override text waits for an administrator's save. -->
+
+**BREAKING** accept-set narrowing at the settings doors (`@objectstack/service-settings`): a `json` key takes a JSON object or array, as ADR-0007 declares.
+
+- **Before.** `PUT /api/settings/:namespace` stored any value under a `json` key. The console's `json` field sent the textarea's text, so Setup → Data Lifecycle saved `lifecycle.quotas`, `lifecycle.quota_defaults` and `lifecycle.retention_overrides` as strings, answered "saved", and the lifecycle sweep read every quota and override as absent: no `quota-exceeded` alert, no override applied, nothing logged.
+- **Now.** The save door refuses a string, number or boolean under a `json` key with `400 SETTINGS_VALIDATION` and one `invalid_type` field error for the key, and stores nothing. An object, an array and `null` (a clear) are accepted. A value already stored under a key the save does not touch does not block it. An `OS_*` override for a `json` key whose text does not parse to an object or array is not in force: one `error` line names the variable, and the key resolves from the next layer of the cascade.
+- **Who moves.** The console now parses the text at Save: valid JSON is sent as the object or array, blank text as a clear, and text that does not parse blocks Save on that row with "Invalid JSON". A `json` value the page loads as text (and is not locked) is put in the draft as it stands, so the page opens with one unsaved change per such key and a note on its row saying the value is stored as text; Save sends the object. A script that PUTs JSON text for a `json` key gets the 400.
+
+**The remedy.** Send the parsed value, not its text: `{ "quotas": { "sys_audit_log": 1000 } }`, not `{ "quotas": "{\"sys_audit_log\": 1000}" }`. For an `OS_*` override, set JSON text of an object, such as `OS_LIFECYCLE_QUOTAS='{"sys_audit_log":1000}'`.
+
+**Values saved as text before this release** (`@objectstack/objectql`). A deployment that saved the Data Lifecycle page through the old console holds JSON text, which applied nothing. After the upgrade the lifecycle service treats the three keys by what they drive:
+
+- `quotas` and `quota_defaults` only raise alerts, so text that holds a JSON object is read as that object and applied, with one warning per stored value: `lifecycle.quotas is stored as JSON text, not an object … Save the Data Lifecycle settings page again to store it as an object.`
+- `retention_overrides` drives the reaper and the archiver, which delete rows, so its text is **not** applied: the declared retention windows stay in force and no row is reaped or archived on the old windows. One error per stored value names the key (and the organization, for an organization's own value) and the fix: open Setup → Data Lifecycle, check the value and save the page once. The page opens with the stored text already marked as an unsaved change and sends it as an object, so that one Save, with no edit, is what puts the window in force.
+
+A stored value that is not a JSON object still applies nothing, as before, and is now reported once at `error`.

content/docs/protocol/kernel/config-resolution.mdx

@@ -261,6 +261,11 @@ OS_ALLOWED_ORIGINS=["a","b","c"]   # default is array/object → JSON.parse
 ```
 
 If the raw value cannot be parsed for the expected type, the raw string is used.
+The exception is a `json` specifier, which persists a JSON object or array
+(ADR-0007): an override whose text does not parse to one is not in force, one
+`error` line names the variable, and the key resolves from the next layer of the
+cascade. The save door refuses the same values for a `json` key with an
+`invalid_type` field error, so a client sends the parsed value, never its text.
 
 **What is stored beneath a pin is listed by the namespace read.** An override in
 force answers `source: 'env'`, `locked: true`, and `get()` / `getMany()` build

packages/objectql/src/lifecycle/lifecycle-service.governance-map-text.test.ts

@@ -0,0 +1,217 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#31241] A map-valued `lifecycle` setting stored as JSON TEXT is read as the
+ * map it holds.
+ *
+ * `retention_overrides`, `quotas` and `quota_defaults` are `json` specifiers,
+ * which persist an object (ADR-0007). The console's `json` field used to send
+ * the textarea's text and the settings save door stored it, so a deployment
+ * that saved the Data Lifecycle page holds a string where the map belongs.
+ * `loadGovernance` indexed that string, so every override and quota read as
+ * absent, with nothing logged. The save door now refuses a string
+ * (`service-settings`), but rows already stored are not re-judged. This reader
+ * splits them by what the key drives:
+ *
+ *  - `quotas` / `quota_defaults` only alert: text that holds a JSON object is
+ *    applied, with one warning per stored value.
+ *  - `retention_overrides` drives deletion: its text is NOT applied (the
+ *    declared windows stay in force, no row is reaped on the old window) and
+ *    one error per stored value says how to apply it — save the page once.
+ *
+ * Anything else that is not a plain object is ignored as before, now said once
+ * at `error`.
+ */
+
+import { describe, it, expect, vi } from 'vitest';
+import { LifecycleService, type LifecycleObjectLike } from './lifecycle-service.js';
+import { parseLifecycleDuration } from './duration.js';
+
+const FIXED_NOW = 1_700_000_000_000;
+const isoCutoff = (literal: string) => new Date(FIXED_NOW - parseLifecycleDuration(literal)).toISOString();
+
+const JOB_RUN: LifecycleObjectLike = {
+  name: 'sys_job_run',
+  lifecycle: { class: 'telemetry', retention: { maxAge: '30d' } } as any,
+};
+
+/** Settings double: a value map at the global rung, plus per-organization rows. */
+function settingsOf(global: Record<string, unknown>, tenant: Record<string, Record<string, unknown>> = {}) {
+  return {
+    async get(_ns: string, key: string, ctx?: Record<string, unknown>) {
+      const org = ctx?.tenantId as string | undefined;
+      if (org && tenant[org] && key in tenant[org]) return { value: tenant[org][key], source: 'tenant' };
+      if (key in global) return { value: global[key], source: 'global' };
+      return { value: undefined, source: 'default' };
+    },
+  };
+}
+
+/** Engine double: one declared object, a counting driver, captured deletes and candidate reads. */
+function engineOf(rows = 1_500, organizations: string[] = [], stored: Array<{ id: string; created_at: string }> = []) {
+  const deletes: Array<{ where: unknown }> = [];
+  const reads: Array<{ object: string; where: unknown }> = [];
+  /** Ids of `stored` rows a bulk delete's `created_at < cutoff` predicate removed. */
+  const reaped: string[] = [];
+  const driver = { name: 'default', count: vi.fn(async () => rows) };
+  const engine: any = {
+    registry: { getAllObjects: () => [JOB_RUN] },
+    async delete(_object: string, options: any) {
+      deletes.push({ where: options?.where });
+      const cutoff = options?.where?.created_at?.$lt as string | undefined;
+      let deletedCount = 0;
+      if (cutoff) {
+        for (const row of stored) {
+          if (row.created_at < cutoff && !reaped.includes(row.id)) {
+            reaped.push(row.id);
+            deletedCount += 1;
+          }
+        }
+      }
+      return { deletedCount };
+    },
+    getDriverForObject: () => driver,
+    datasource() {
+      throw new Error('no datasource');
+    },
+  };
+  if (organizations.length > 0) {
+    engine.find = async (object: string, options: any) => {
+      if (object === 'sys_organization') return organizations.map((id) => ({ id }));
+      reads.push({ object, where: options?.where });
+      const org = (options?.where as Record<string, unknown> | undefined)?.organization_id;
+      return [{ id: typeof org === 'string' ? `${org}_row` : 'global_row' }];
+    };
+  }
+  return { engine, deletes, reads, reaped };
+}
+
+function serviceOf(engine: unknown, settings: unknown) {
+  const logger = { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() };
+  const svc = new LifecycleService({
+    getEngine: () => engine as any,
+    getSettings: () => settings as any,
+    logger,
+    now: () => FIXED_NOW,
+    initialDelayMs: 1,
+    sweepIntervalMs: 10,
+    onAlert: () => {},
+  });
+  return { svc, logger };
+}
+
+describe('[#31241] a map setting stored as JSON text', () => {
+  it('CONTROL: quotas stored as an object alert, and nothing is logged about them', async () => {
+    const { engine } = engineOf(1_500);
+    const { svc, logger } = serviceOf(engine, settingsOf({ quotas: { sys_job_run: 1_000 } }));
+
+    const report = await svc.sweep();
+
+    expect(report.alerts).toEqual([{ type: 'quota-exceeded', object: 'sys_job_run', rowCount: 1_500, quota: 1_000 }]);
+    expect(logger.warn).not.toHaveBeenCalled();
+    expect(logger.error).not.toHaveBeenCalled();
+  });
+
+  it('quotas stored as the text the old console saved are applied, with one warning naming the key', async () => {
+    const { engine } = engineOf(1_500);
+    const { svc, logger } = serviceOf(engine, settingsOf({ quotas: '{\n  "sys_job_run": 1000\n}' }));
+
+    const first = await svc.sweep();
+    const second = await svc.sweep();
+
+    expect(first.alerts).toEqual([{ type: 'quota-exceeded', object: 'sys_job_run', rowCount: 1_500, quota: 1_000 }]);
+    expect(second.alerts).toEqual(first.alerts);
+    // Once per stored value, not once per sweep.
+    expect(logger.warn).toHaveBeenCalledTimes(1);
+    expect(logger.warn.mock.calls[0][0]).toContain('lifecycle.quotas is stored as JSON text, not an object');
+    expect(logger.warn.mock.calls[0][0]).toContain('Save the Data Lifecycle settings page again');
+    expect(logger.error).not.toHaveBeenCalled();
+  });
+
+  it('a class default stored as text applies through quota_defaults', async () => {
+    const { engine } = engineOf(1_500);
+    const { svc } = serviceOf(engine, settingsOf({ quota_defaults: '{"telemetry": 500}' }));
+
+    const report = await svc.sweep();
+
+    expect(report.alerts).toEqual([{ type: 'quota-exceeded', object: 'sys_job_run', rowCount: 1_500, quota: 500 }]);
+  });
+
+  /** A row 10 days old: inside the declared 30d window, past a 1d override. */
+  const TEN_DAYS_OLD = [{ id: 'row_10d', created_at: new Date(FIXED_NOW - 10 * 86_400_000).toISOString() }];
+
+  it('CONTROL: a retention override stored as an object reaps on its window', async () => {
+    const { engine, deletes, reaped } = engineOf(1_500, [], TEN_DAYS_OLD);
+    const { svc, logger } = serviceOf(engine, settingsOf({ retention_overrides: { sys_job_run: { maxAge: '1d' } } }));
+
+    await svc.sweep();
+
+    expect(deletes).toEqual([{ where: { created_at: { $lt: isoCutoff('1d') } } }]);
+    expect(reaped).toEqual(['row_10d']);
+    expect(logger.error).not.toHaveBeenCalled();
+  });
+
+  it('a retention override stored as text is NOT applied: no row is reaped on it, and one error says how to apply it', async () => {
+    const { engine, deletes, reaped } = engineOf(1_500, [], TEN_DAYS_OLD);
+    const { svc, logger } = serviceOf(engine, settingsOf({ retention_overrides: '{"sys_job_run": {"maxAge": "1d"}}' }));
+
+    await svc.sweep();
+    await svc.sweep();
+
+    // The declared 30d window, not the old 1d text: the 10-day-old row stays.
+    expect(deletes.map((d) => d.where)).toEqual([
+      { created_at: { $lt: isoCutoff('30d') } },
+      { created_at: { $lt: isoCutoff('30d') } },
+    ]);
+    expect(reaped).toEqual([]);
+    expect(logger.error).toHaveBeenCalledTimes(1);
+    const line = logger.error.mock.calls[0][0] as string;
+    expect(line).toContain('lifecycle.retention_overrides is stored as JSON text, not an object');
+    expect(line).toContain('so it is NOT applied');
+    expect(line).toContain('Setup → Data Lifecycle');
+    expect(line).not.toContain('sys_job_run');
+    expect(logger.warn).not.toHaveBeenCalled();
+  });
+
+  it("an organization's retention override stored as text runs no pass of its own, and is named in the error", async () => {
+    const { engine, reads } = engineOf(1_500, ['org_reg', 'org_plain']);
+    const { svc, logger } = serviceOf(
+      engine,
+      settingsOf({}, { org_reg: { retention_overrides: '{"sys_job_run": {"maxAge": "1d"}}' } }),
+    );
+
+    await svc.sweep();
+
+    // One global pass on the declared window; no organization is carved out.
+    const passes = reads.filter((r) => r.object === 'sys_job_run').map((r) => r.where);
+    expect(passes).toEqual([{ created_at: { $lt: isoCutoff('30d') } }]);
+    expect(logger.error.mock.calls[0][0]).toContain("organization org_reg's lifecycle.retention_overrides");
+    expect(logger.error.mock.calls[0][0]).toContain('so it is NOT applied');
+  });
+
+  it("CONTROL: the same organization's override stored as an object runs that organization's pass", async () => {
+    const { engine, reads } = engineOf(1_500, ['org_reg', 'org_plain']);
+    const { svc } = serviceOf(engine, settingsOf({}, { org_reg: { retention_overrides: { sys_job_run: { maxAge: '1d' } } } }));
+
+    await svc.sweep();
+
+    const passes = reads.filter((r) => r.object === 'sys_job_run').map((r) => r.where);
+    expect(passes).toEqual([
+      { created_at: { $lt: isoCutoff('1d') }, organization_id: 'org_reg' },
+      { created_at: { $lt: isoCutoff('30d') }, organization_id: { $nin: ['org_reg'] } },
+    ]);
+  });
+
+  it('text that is not a JSON object applies nothing, and says so once at error', async () => {
+    const { engine } = engineOf(1_500);
+    const { svc, logger } = serviceOf(engine, settingsOf({ quotas: '{ "sys_job_run": 1000' }));
+
+    const first = await svc.sweep();
+    await svc.sweep();
+
+    expect(first.alerts).toEqual([]);
+    expect(logger.error).toHaveBeenCalledTimes(1);
+    expect(logger.error.mock.calls[0][0]).toContain('lifecycle.quotas is not an object');
+    expect(logger.error.mock.calls[0][0]).not.toContain('sys_job_run');
+  });
+});