feat(cloud-connection)!: refuse a managed upgrade that renames or retypes a stored field until it is declared or confirmed

.changeset/30884-schema-upgrade-confirmation.md

@@ -0,0 +1,25 @@
+---
+'@objectstack/spec': patch
+'@objectstack/cloud-connection': patch
+'@objectstack/cli': patch
+---
+
+feat(cloud-connection)!: a managed upgrade that renames or retypes a stored field is refused until it is declared or confirmed
+
+Clause-②: yes (narrowing) — a managed re-install through install-local that installed before is refused `409 SCHEMA_UPGRADE_CONFIRMATION_REQUIRED` when its package upgrade plan lists an undeclared suspected rename or a `needs_confirm` / `destructive` type change, and `501 NOT_IMPLEMENTED` when it declares a field rename (`renamedFrom`) this runtime cannot carry yet. The widening half: the error-code ledger registers `SCHEMA_UPGRADE_CONFIRMATION_REQUIRED` under `@objectstack/cloud-connection`, install-local takes `confirmSchemaUpgrade`, and `os package install` and `os deploy` take `--confirm-upgrade`.
+Metadata-protocol: none
+
+<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped and no export is removed: the refusal judges two package bodies at the install door, so there is nothing for a migration to rewrite and no tombstone. The way through is a token the refusal itself prints. -->
+
+**BREAKING** accept-set narrowing at the install-local door (`POST /api/v1/marketplace/install-local`, `@objectstack/cloud-connection`) for a managed re-install over an installed package.
+
+Before any write, after the access upgrade check, the door compares the installed version (its ledger entry) with the incoming one through `comparePackageVersions` (`@objectstack/metadata-core`), the comparison the dry run and `os diff` read:
+
+- **A suspected rename** (in one object, a stored field removed while a stored field is added, with no `renamedFrom` linking them) or **a type change** the drift categories call `needs_confirm` (both columns stored) or `destructive` (the new type stores no column) answers `409 SCHEMA_UPGRADE_CONFIRMATION_REQUIRED`. `error.details.findings` names each with a one-line `describe` (and, when the runtime's `manifest` service answers `planUpgrade`, the rows the installed column holds); `error.details.findingSet` is the token. The same request with `"confirmSchemaUpgrade": "<findingSet>"` installs, as the additive schema sync leaves the fields, and the ledger entry records `schemaUpgradeConfirmation` (the token, who, when). A token from another finding set is refused again with the fresh one. A type change riding on a declared rename is judged on its own.
+- **The installed side** is the ledger entry's body; an orphaned install (the registry still serves the package, the ledger holds no record) is compared against the body the registry serves. When there is no installed body to compare — the ledger file will not parse, or neither the ledger entry nor the orphan's registry row carries a body — the same `409` names that as its one finding (`installed_record_unreadable` with the file and the cause, or `installed_record_missing`), and its token admits the re-install, which rewrites the record; a rename the incoming version declares still answers the `501` below.
+- **A declared rename** (`renamedFrom` on a field of the new version) answers `501 NOT_IMPLEMENTED`, naming each rename: this runtime cannot yet move a renamed field's values to its new key, and no token answers it.
+- A `safe` type change (the installed field had no stored column), a removed field or object and a seed dataset change are refused by nothing; `dryRun: true` reports them.
+
+A first install, an `unmanaged` install and a re-install whose plan lists none of these answer as before.
+
+**Migration.** A package author whose upgrade is refused either declares the rename on the new field — `renamedFrom: ['<old key>']` — and rebuilds, or confirms the findings as listed: `os package install <artifact> --confirm-upgrade <findingSet>` (`os deploy --confirm-upgrade <findingSet>` against a self-hosted runtime), or `"confirmSchemaUpgrade": "<findingSet>"` in the install-local body. A host that calls install-local itself reads `error.details.findingSet` off the 409 and sends it back. Preview the refusal first with `os package install <artifact> --dry-run`.

content/docs/data-modeling/fields.mdx

@@ -478,7 +478,7 @@ Database indexes are declared at the object level in the object's `indexes[]` ar
 
 | Property | Type | Default | Description |
 | :--- | :--- | :--- | :--- |
-| `renamedFrom` | `string[]` | — | Earlier keys of this field (snake_case), e.g. `renamedFrom: ['company']` on a field now keyed `account_name`: on a package upgrade from a version that has the old key and not this one, the platform carries the old column's data to this field instead of adding an empty column. Each entry once; never the field's own key, a key another field of the object still declares, or an old key another field also claims — each refused at parse. Read only at package upgrade, and declared ahead of the upgrade step that carries the data: until that step ships, no upgrade carries a column on this declaration |
+| `renamedFrom` | `string[]` | — | Earlier keys of this field (snake_case), e.g. `renamedFrom: ['company']` on a field now keyed `account_name`: on a package upgrade from a version that has the old key and not this one, the platform carries the old column's data to this field instead of adding an empty column. Each entry once; never the field's own key, a key another field of the object still declares, or an old key another field also claims — each refused at parse. Read only at package upgrade, and declared ahead of the upgrade step that carries the data: until that step ships, a managed upgrade that declares a rename is refused `501 NOT_IMPLEMENTED`, naming it, so it never installs as a silent empty column |
 
 ### Security
 

content/docs/deployment/cli.mdx

@@ -3396,6 +3396,7 @@ write `-u URL` instead.
 | `--password <password>` | `OS_RUNTIME_PASSWORD` | Account password on the target runtime |
 | `--confirm-global-uniques` | — | Affirm this app's installation-wide `unique` constraints are genuinely platform-wide — see below |
 | `--confirm-access-upgrade <token>` | — | Confirm an upgrade's access findings with the finding-set token the runtime's refusal printed — see below |
+| `--confirm-upgrade <token>` | — | Confirm an upgrade's schema findings (a suspected rename, a type change that is not safe) with the finding-set token the runtime's refusal printed — see below |
 | `--timeout <ms>` | `OS_CLOUD_TIMEOUT_MS` | HTTP timeout in milliseconds, default `120000`. `0` disables it |
 
 **`--confirm-global-uniques` answers a stop; it does not force one past.**
@@ -3427,6 +3428,25 @@ and the upgrade stops again with the new set; the confirmation is recorded in
 the install's ledger entry. Under `group` and `isolated` only the platform
 operator reaches this check. A fresh install is never checked.
 
+**`--confirm-upgrade <token>` confirms exactly the schema findings it names.**
+Upgrading an installed package stops with `SCHEMA_UPGRADE_CONFIRMATION_REQUIRED`
+when the new version would change a stored field in a way the schema sync does
+not carry: a field removed while another is added with no `renamedFrom` linking
+them (if it is a rename, the new field starts empty and the values stay in the
+old column), or a type change that keeps or drops a stored column
+(`needs_confirm` or `destructive`). It also stops when the runtime cannot read
+its record of the installed version, since nothing could be compared. Nothing is
+installed. The command prints each finding and the exact retry, carrying the
+finding set's token. If a field was renamed, declare its old key in the new
+field's `renamedFrom` and rebuild instead; a declared rename is refused with
+`501 NOT_IMPLEMENTED` until the runtime can carry the column's values to the new
+key. As with `--confirm-access-upgrade`, the token stops matching once the
+findings change, and the confirmation is recorded in the install's ledger entry.
+A type change on a field that had no stored column, a removed field and a seed
+data change are never refused; `--dry-run` lists them. `os deploy` passes
+`--confirm-upgrade` through to a self-hosted runtime. A fresh install is never
+checked.
+
 A `404` means the target runtime does not mount `MarketplaceInstallLocalPlugin`
 (from `@objectstack/cloud-connection`). The endpoint is opt-in, so a runtime
 composed without it will not accept installs.

content/docs/references/api/automation-api.mdx

@@ -417,7 +417,7 @@ const result = AutomationApiErrorCode.parse(data);
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **code** | `Enum<'VALIDATION_ERROR' \| 'INVALID_FIELD' \| 'MISSING_REQUIRED_FIELD' \| 'INVALID_FORMAT' \| 'VALUE_TOO_LONG' \| 'VALUE_TOO_SHORT' \| 'VALUE_OUT_OF_RANGE' \| … +361 more>` | ✅ | The inner refusal's code in its WIRE spelling - the ADR-0112 member a record save of the same write answers in `error.code` (e.g. `UNIQUE_VIOLATION`); the closed set `ApiErrorSchema.code` parses against |
+| **code** | `Enum<'VALIDATION_ERROR' \| 'INVALID_FIELD' \| 'MISSING_REQUIRED_FIELD' \| 'INVALID_FORMAT' \| 'VALUE_TOO_LONG' \| 'VALUE_TOO_SHORT' \| 'VALUE_OUT_OF_RANGE' \| … +362 more>` | ✅ | The inner refusal's code in its WIRE spelling - the ADR-0112 member a record save of the same write answers in `error.code` (e.g. `UNIQUE_VIOLATION`); the closed set `ApiErrorSchema.code` parses against |
 | **message** | `string` | ✅ | The inner refusal's own diagnostic text, verbatim, without the node and operation prefixes the run's failure text adds |
 
 ### Allowed Values: `FlowInnerRefusal.code`
@@ -724,6 +724,7 @@ const result = AutomationApiErrorCode.parse(data);
 * `RULE_NOT_FOUND`
 * `RUN_NOT_FOUND`
 * `SAML_REGISTER_FAILED`
+* `SCHEMA_UPGRADE_CONFIRMATION_REQUIRED`
 * `SELF_REGISTRATION_CLOSED`
 * `SERVICE_NOT_REGISTERED`
 * `SETTINGS_ACTION_FAILED`

content/docs/references/api/contract.mdx

@@ -28,7 +28,7 @@ const result = ApiErrorSchema.parse(data);
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **code** | `Enum<'VALIDATION_ERROR' \| 'INVALID_FIELD' \| 'MISSING_REQUIRED_FIELD' \| 'INVALID_FORMAT' \| 'VALUE_TOO_LONG' \| 'VALUE_TOO_SHORT' \| 'VALUE_OUT_OF_RANGE' \| … +361 more>` | ✅ | Error code (e.g. VALIDATION_ERROR; StandardErrorCode ∪ the ledger the serving side registers — ERROR_CODE_LEDGER for framework packages) |
+| **code** | `Enum<'VALIDATION_ERROR' \| 'INVALID_FIELD' \| 'MISSING_REQUIRED_FIELD' \| 'INVALID_FORMAT' \| 'VALUE_TOO_LONG' \| 'VALUE_TOO_SHORT' \| 'VALUE_OUT_OF_RANGE' \| … +362 more>` | ✅ | Error code (e.g. VALIDATION_ERROR; StandardErrorCode ∪ the ledger the serving side registers — ERROR_CODE_LEDGER for framework packages) |
 | **declaredCode** | `string` | optional | The producer-declared code, verbatim, when it is not a member of the closed `code` vocabulary — the open, author-authored channel (app-specific spellings; ADR-0112) |
 | **message** | `string` | ✅ | Readable error message |
 | **userMessage** | `string` | optional | Producer-marked user-facing refusal text, verbatim. Present exactly when the producer opted in at throw time; consumers render it to end users and keep their generic substitution for anything unmarked. Status-agnostic; never replaces `message`. |
@@ -342,6 +342,7 @@ const result = ApiErrorSchema.parse(data);
 * `RULE_NOT_FOUND`
 * `RUN_NOT_FOUND`
 * `SAML_REGISTER_FAILED`
+* `SCHEMA_UPGRADE_CONFIRMATION_REQUIRED`
 * `SELF_REGISTRATION_CLOSED`
 * `SERVICE_NOT_REGISTERED`
 * `SETTINGS_ACTION_FAILED`

content/docs/references/api/error-code-ledger.mdx

@@ -509,6 +509,7 @@ const result = ErrorCode.parse(data);
 * `RULE_NOT_FOUND`
 * `RUN_NOT_FOUND`
 * `SAML_REGISTER_FAILED`
+* `SCHEMA_UPGRADE_CONFIRMATION_REQUIRED`
 * `SELF_REGISTRATION_CLOSED`
 * `SERVICE_NOT_REGISTERED`
 * `SETTINGS_ACTION_FAILED`

packages/cli/src/commands/deploy.ts

@@ -61,8 +61,9 @@
  *      version against this build, printed; nothing is installed. A runtime
  *      that refuses it, or does not serve install-local, stops the deploy here
  *      with its sentence and exit 1;
- *   3. the install, with `--confirm-access-upgrade` passed through; the door's
- *      refusals (an access upgrade to confirm, …) stop the deploy with exit 1;
+ *   3. the install, with `--confirm-access-upgrade` and `--confirm-upgrade`
+ *      passed through; the door's refusals (an access upgrade or a schema
+ *      upgrade to confirm, …) stop the deploy with exit 1;
  *   4. the answer — install-local registers the package in the live kernel
  *      before it answers, so the answer IS "live": `hotLoaded` ⇒ 0; a package
  *      the running kernel could not load ⇒ 1, naming the restart it waits for.
@@ -102,6 +103,7 @@ export default class Deploy extends Command {
     '$ os deploy --env env_abc123    # your account reaches several environments',
     '$ os deploy --no-wait           # return once the install is recorded',
     '$ os deploy --confirm-access-upgrade <finding-set>   # self-hosted: confirm the findings the runtime printed',
+    '$ os deploy --confirm-upgrade <finding-set>          # self-hosted: confirm the schema findings the runtime printed',
   ];
 
   static override flags = {
@@ -140,6 +142,16 @@ export default class Deploy extends Command {
         'Self-hosted runtime target: confirm the access findings of this upgrade with the finding-set token the '
         + "runtime's refusal printed (as `os package install --confirm-access-upgrade`)",
     }),
+    // [epic cloud#3652 Q3, #30884] The schema findings' confirmation, passed
+    // through to the install as `--confirm-access-upgrade` is: the embedded
+    // install's refusal prints `os deploy --confirm-upgrade <token>` as its
+    // retry, so this command must take it.
+    'confirm-upgrade': Flags.string({
+      description:
+        'Self-hosted runtime target: confirm the schema findings of this upgrade (a suspected rename, a type change '
+        + "that is not safe) with the finding-set token the runtime's refusal printed (as `os package install "
+        + '--confirm-upgrade`)',
+    }),
   };
 
   async run(): Promise<void> {
@@ -151,12 +163,13 @@ export default class Deploy extends Command {
       // cloud environment was named.
       const runtime = flags.env ? undefined : await activeRuntimeTarget();
       if (runtime) {
-        await this.deployToRuntime(runtime, flags['confirm-access-upgrade']);
+        await this.deployToRuntime(runtime, flags['confirm-access-upgrade'], flags['confirm-upgrade']);
         return;
       }
-      if (flags['confirm-access-upgrade'] !== undefined) {
+      for (const flag of ['confirm-access-upgrade', 'confirm-upgrade'] as const) {
+        if (flags[flag] === undefined) continue;
         printError(
-          '`--confirm-access-upgrade` confirms a self-hosted runtime\'s refusal; the deploy target is ObjectOS Cloud. '
+          `\`--${flag}\` confirms a self-hosted runtime's refusal; the deploy target is ObjectOS Cloud. `
           + 'Drop the flag, or record the runtime with `os environments use-runtime <url> --token <key>`.',
         );
         this.exit(1);
@@ -285,7 +298,11 @@ export default class Deploy extends Command {
    * The self-hosted half (the header's "A self-hosted runtime target"):
    * build → dry run (plan printed) → install → the `hotLoaded` answer.
    */
-  private async deployToRuntime(runtime: RuntimeTarget, confirmAccessUpgrade: string | undefined): Promise<void> {
+  private async deployToRuntime(
+    runtime: RuntimeTarget,
+    confirmAccessUpgrade: string | undefined,
+    confirmUpgrade: string | undefined,
+  ): Promise<void> {
     printStep(`Deploying to the self-hosted runtime ${runtime.url} (os environments use-runtime)`);
 
     printStep('Building...');
@@ -317,7 +334,10 @@ export default class Deploy extends Command {
     }
 
     printStep(`Installing ${plan.manifestId} ${plan.version} into ${runtime.url}...`);
-    const outcome = await install(confirmAccessUpgrade !== undefined ? ['--confirm-access-upgrade', confirmAccessUpgrade] : []);
+    const outcome = await install([
+      ...(confirmAccessUpgrade !== undefined ? ['--confirm-access-upgrade', confirmAccessUpgrade] : []),
+      ...(confirmUpgrade !== undefined ? ['--confirm-upgrade', confirmUpgrade] : []),
+    ]);
     if (outcome?.kind !== 'installed') {
       printError(`${runtime.url} did not answer the install as a managed package install, so this cannot say ${plan.manifestId} is live (see above).`);
       this.exit(1);