Repository navigation
Commit e063a87
feat(service-datasource): a code-defined datasource reads its credential from OS_DATASOURCE_<NAME>_PASSWORD (#28208)
domain:spec#3 · os-support-ai · session_01WGufzgbJhFoXYsLSe5WSPP
<!-- claude-projects-attribution: {"github_login":"os-support-ai"} -->
_Requested by **Yibo Wang** · [project
thread](https://claude.ai/code/project/chan_01BVUyyPie8gHwoPM18iABwB?thread=cmsg_01BVUyyPie8gHwoPM18iABwBR9NEQPF4XiEBhFuE3M9hKB)_
Fixes #27687
Clause-②: yes (widening) — `@objectstack/service-datasource` exports
`datasourceEnvCredentialVar`, `resolveDatasourceEnvCredentialsSwitch`,
`DATASOURCE_ENV_CREDENTIALS_SWITCH` and the
`DatasourceEnvCredentialSource` type, and
`DatasourceConnectionServiceConfig` gains the optional `envCredentials`;
`@objectstack/spec/data` exports `CODE_DEFINED_CREDENTIAL_ROUTE`.
Nothing an author writes is added or refused, and no existing key
changes meaning.
Metadata-protocol: none
## 维护者速读
1. 代码里声明的数据源(`defineDatasource` / 包内 `*.datasource.ts`)现在在连接时读
`OS_DATASOURCE_<名字>_PASSWORD`,例如 `warehouse` →
`OS_DATASOURCE_WAREHOUSE_PASSWORD`。
2. 只在主机开关 `OS_DATASOURCE_ENV_CREDENTIALS=on` 时读;`os serve` / `os start`
/ `os dev` 默认开,其他组合(云端、多租户、自建 kernel)默认关;其他取值启动即拒绝。
3. 只读代码数据源(按来源集合判定,从不看记录的 `origin`);UI/API 创建的同名数据源、`default` 都不读。
4. 声明了 `credentialsRef` 或 `connector` 时它们优先,失败仍然
fail-closed,不回落到环境变量;变量没设时,失败原因会写出该设哪个变量;值不进日志、不进任何应答。
5. 信任后果:`os serve` 下本地安装的包也读这个槽,所以任何已安装的包都能声明任意空闲名字的数据源,从而读到对应变量。文档和
changeset 已写明。
6. 规范层只改了拒绝文本(5 条凭据拒绝 + 2 条 `password` 提示共用一句新句子),accept 集合不变;changeset
全部 `patch`(`Metadata-protocol: none` 时门禁要求 patch,见下文)。
## The ruling
Maintainer ruling on #27687 (option 5, recorded by the seat in comment
6525963988, claim amended in 6525977981): a datasource defined in code
reads its credential at connect time from a runtime environment variable
whose name the platform derives from the datasource name. The seat's
follow-ups: the switch is opt-in in `os serve` / `os start` with the
library default off and no tenancy coupling, the off value is spelled
`off`, and the install-local route (#28132) keeps reading the slot, with
its trust consequence stated and pinned.
## Measurements on `origin/main` `d70e94ae`
**(a) The seam where the CLI host turns the switch on.** No new public
option was needed. `os serve` loads `@objectstack/service-datasource`
dynamically and builds `DatasourceAdminServicePlugin` itself
(`packages/cli/src/commands/serve.ts`, the `adminMod` block). The
plugin's `init()` reads `process.env`, so `serve` stamps
`OS_DATASOURCE_ENV_CREDENTIALS=on` into its own environment just before
it builds the plugin, when the operator left it unset or empty. That is
the same seam `os start` uses for `OS_CRYPTO_AUTOKEY` (`start.ts:482`).
`os start` spawns `serve` (`start.ts:528`) and `os dev` spawns `serve
--dev` (`dev.ts:848`), so both inherit the stamp. Every other
composition never runs that line and stays off.
**(b) The shape of `secret`.** `DatasourceConnectionSpec.secret?:
string` (`contracts/datasource-driver-factory.ts`) is the cleartext the
connection service hands to `factory.create`. Its readers in
`default-datasource-driver-factory.ts`: postgres puts it in `password`
(`:603`, `:611`), mysql the same (`:830`, `:839`), mongodb puts it in
the `MongoClient` `auth` block (`:945`, `:1047`), and turso reads it
ahead of `config.authToken`. sqlite, sqlite-wasm and memory have no
credential slot. The slot fills exactly this value, so it reaches every
built-in driver that takes a secret. A plugin driver that resolves its
credential per request (OData, through
`DATASOURCE_SECRET_RESOLVER_SERVICE`) does not read it; the docs say so.
**(c) Where `CODE_DATASOURCE_NAMES_SERVICE` is filled, and how
`connect()` gets it.** Three producers, all in `@objectstack/runtime`,
all through `contributeCodeDatasourceNames`
(`code-datasource-names.ts:156`):
- `app-plugin.ts:626`: the boot artifact's declared datasources;
- `default-datasource-plugin.ts:150`: `default`;
- `declared-datasources.ts:160`: a package installed into the running
kernel (install-local, since #28132).
`connect()` gets it as a callback. The admin plugin passes
`isCodeDatasource(name)`, which reads the set through
`codeDatasourceNamesOf(ctx)`, the same reader `servedOrigin` uses. The
set is read at each connect, never cached at `init()`, so a name an
install adds later is honoured.
## What changes
- **`DatasourceConnectionService.connect()`**, after the
`credentialsRef` and connector steps: when neither is declared, the
connect is not `asDefault`, the slot is configured (switch on), the
trigger is `declared-auto`, and the name is in the code set, it reads
`OS_DATASOURCE_<NAME>_PASSWORD`. `<NAME>` is the name upper-cased with
each non-`[A-Z0-9]` character replaced by `_`. A non-empty value becomes
the build's `secret`.
- **`resolveDatasourceEnvCredentialsSwitch()`**: unset or empty is off;
`on` / `off` in any letter case; anything else throws
`OS_DATASOURCE_ENV_CREDENTIALS must be 'on' or 'off', got '<value>'. …`.
The admin plugin calls it in `init()`, so the boot is refused.
- **`os serve`** stamps the `on` default
(`packages/cli/src/utils/datasource-env-credentials-default.ts`).
- **`planCredentialMigration`**'s code-origin remedy names the row's
variable and the switch.
- **Spec prescriptions**: one shared sentence,
`CODE_DEFINED_CREDENTIAL_ROUTE`, appended to the credential refusals and
the two `password` guidance entries (before/after below).
- **Docs**:
- `external-datasources.mdx` §4 gains "A datasource defined in code: the
environment variable". It covers the derivation table, an example, the
switch and its default, the code-only rule, precedence, the unset case,
collisions, the plugin-driver limit, and a warning callout on who can
read the variable (installed packages; why the switch is off on
multi-tenant hosts).
- `drivers.mdx`: the Multi-Datasource paragraph and the read-only sample
now teach the variable instead of a `sys_secret:` handle a code author
cannot mint.
### Deviations from the order, stated
- **Unset variable: the verdict stays `failed-degraded`, not
`failed-credentials`.** The order says "keep the existing
`failed-credentials` verdict", but the existing verdict for a code
datasource connecting with no password is the driver's own connect
failure, `failed-degraded`, so keeping it means `failed-degraded`. Only
the reason changes: it gains "— datasource '<name>' is defined in code
and reads its credential from `OS_DATASOURCE_<NAME>_PASSWORD` at
connect, which is not set on this host". If the database accepts the
connection without a password, nothing changes. A bound datasource still
refuses the boot, and the thrown message names the variable.
- **Collision refused at connect, not "at boot" as a separate pass.**
Datasource names must match `/^[a-z_][a-z0-9_]*$/`, so two names that
pass the schema can never derive one variable. The guard is defensive:
the second connect is refused as `failed-credentials` with both names
and the variable, and its variable is never read. On the boot path that
refusal is a boot failure whenever the datasource is fail-fast.
- **Five refusals carry the sentence, not four**:
`INLINE_CREDENTIAL_REFUSED`, `URL_EMBEDDED_CREDENTIAL_REFUSED`,
`URL_CREDENTIAL_QUERY_PARAM_REFUSED`,
`PASSTHROUGH_INLINE_CREDENTIAL_REFUSED`, and
`PASSTHROUGH_NESTED_CREDENTIAL_REFUSED`. The fifth prescribes the same
"bind it instead" route and would otherwise send a code author to the
same unfollowable door. The `.describe()` on
`refusedInlineCredentialKey` carries it too, which is why four generated
driver reference pages moved.
- **Changeset levels are `patch` on all three packages, not minor.** The
seat asked for service-datasource minor and cli minor, but
`check-changeset-no-major` refuses a `minor` beside `Metadata-protocol:
none` ("`none` -> every introduced entry `patch`"). `none` is the
measured kind: no key, value or metadata type an author writes changes.
## Before / after for each string
The shared sentence (`CODE_DEFINED_CREDENTIAL_ROUTE`, in
`packages/spec/src/data/driver/common.zod.ts`):
> A datasource defined in code (`defineDatasource`) leaves the
credential out of the declaration: it is read at connect time from the
runtime environment variable derived from the datasource name
(`OS_DATASOURCE_WAREHOUSE_PASSWORD` for a datasource named `warehouse`),
which `os serve` / `os start` honour by default
(`OS_DATASOURCE_ENV_CREDENTIALS=on`).
Each string below is unchanged up to its last sentence. The after-text
is the before-text plus ` ` + the sentence above.
| String | Before ends with | After |
| :--- | :--- | :--- |
| `INLINE_CREDENTIAL_REFUSED` | "…always wins over anything embedded in
`config`." | + the sentence |
| `URL_EMBEDDED_CREDENTIAL_REFUSED` | "…do not pass through this publish
door and are unaffected." | + the sentence |
| `URL_CREDENTIAL_QUERY_PARAM_REFUSED` | "…do not pass through this
publish door and are unaffected." | + the sentence |
| `PASSTHROUGH_INLINE_CREDENTIAL_REFUSED` | "…a `${…}` span anywhere in
`options` is itself refused at publish." | + the sentence |
| `PASSTHROUGH_NESTED_CREDENTIAL_REFUSED` | "…a `${…}` span anywhere in
`options` is itself refused at publish." | + the sentence |
| `refusedInlineCredentialKey` `.describe()` | "…never stored in
`config`." | + the sentence |
| `ExternalDatasourceSettingsSchema` guidance `password` | "…it never
dereferences a hand-written path." | + the sentence |
| `DatasourceSchema` guidance `password` | "…a managed datasource
included." | + the sentence |
Every one keeps the Setup → Datasources and `external.credentialsRef`
routes, and none carries a `#NNNN`; the test below checks both.
`planCredentialMigration` code-origin remedy:
- **Before:** "Move the credential in the source that defines this
datasource: bind it through `external.credentialsRef`, or supply it from
the runtime environment."
- **After:** "Move the credential out of the source that defines this
datasource: leave it out of the declaration and set
`OS_DATASOURCE_<NAME>_PASSWORD` in the runtime environment, which the
datasource reads at connect when the host's
`OS_DATASOURCE_ENV_CREDENTIALS` switch is on (the default under `os
serve` / `os start`), or bind it through `external.credentialsRef`." The
real row's variable is substituted.
## Guards and their tests (each ablated)
Unit pins are in
`packages/services/service-datasource/src/__tests__/datasource-env-credential.test.ts`
(24 tests). Kernel wiring is in
`packages/runtime/src/datasource-env-credential.test.ts` (3, a real
admin plugin and the real driver factory). The CLI pins are in
`packages/cli/src/utils/datasource-env-credentials-default.test.ts` (4)
and the spec pins in
`packages/spec/src/data/driver/code-defined-credential-route.test.ts`
(16). Every ablation used `scripts/ablation-replace.mjs` against the
committed tree and restored to `blob == HEAD`.
| Guard | Test | Ablation → red |
| :--- | :--- | :--- |
| A code datasource connects with the variable set | "a code datasource
connects with the variable set", "through `connectDeclared`", runtime
"switch on: the boot artifact's datasource…" | A8 (switch resolver
returns off) → 13 red; A10 (admin wiring `isCodeDatasource` → false,
rebuilt dist) → runtime 1 red |
| Install-local reads the slot (trust consequence) | "a name the host
registers after construction (an install-local package)", runtime
`connectPackageDatasources(…, installed: true)` | A10 → runtime red |
| `registerPool` / runtime-admin never reads it | "the same name created
through the admin door…" | A2 (drop the `declared-auto` check) → 2 red |
| Provenance, never `origin` | "a caller-set `origin: 'code'` is not
provenance" | A3 (drop `isCodeDatasource`) → 1 red |
| Switch off: no read | "a host that did not opt in", runtime "switch
unset" | A10 (runtime); unit: no-slot case reads nothing |
| Switch refuses other values | "refuses %j…" ×5, runtime "a value other
than `on` / `off` refuses the boot" | A8 → 5 red |
| A declared `credentialsRef` wins and stays fail-closed | "a resolving
`credentialsRef` is the secret", "an unresolvable `credentialsRef`… does
NOT fall back" | A4 (drop `!credentialsRef`) → 1 red |
| `default` keeps `OS_DATABASE_URL` | "an `asDefault` connect never
reads the slot" | A11 (drop `!opts.asDefault`) → 1 red |
| Unset variable: the reason names it | "the database refusing the login
answers the variable to set", "an EMPTY variable is unset" | A6 (drop
the reason suffix) → 2 red |
| Collision refused | "the second name is refused with both names and
the variable" | A5 (disable the owner check) → 1 red |
| The value in no log line or answer | "no log line, connect answer or
connection state carries it" | A7 (log the value) → 1 red |
| Every rewritten refusal parses its own example back | spec
`describe.each` over 7 refusal families: the example name is read out of
the message, the credential is left out of the very artefact that was
refused, and the result parses with no secret in it | A1 (drop the
sentence from the nested refusal) → 2 red; A12 (example variable drifts
to `OS_DATASOURCE_WH_PASSWORD`) → 1 red |
| The example is the variable the runtime derives | service "is the
variable the spec refusals' worked example names"
(`datasourceEnvCredentialVar(name) === variable`, and the switch name
matches) | reads spec through `dist`; pinned against the rebuilt dist |
| `os serve` stamps before the plugin is built | cli "stamps once…",
"the stamp precedes the plugin construction" | A9 (remove the stamp) → 2
red |
## Stale text under governed paths (named, not edited)
- `skills/objectstack-data/rules/datasources.md:63`: "## Credentials —
one channel, fail-closed". A code datasource now has the environment
channel too.
- `docs/adr/0015-external-datasource-federation.md` §10 (`:764`),
"Credentials never appear in metadata artefacts. `credentialsRef` points
at `service-secrets`…", and the D7 amendment note (`:10`), "§10 says
credentials reach a datasource only through `credentialsRef`. That stays
true for a database."
- `docs/adr/0062-external-datasource-runtime.md` D3 (`:92`): the
connection service resolves `credentialsRef` through the `SecretBinder`
before building. The env slot is a third route, after it and the D7
borrow.
## Verification
All on head `5eda07f1ec` (the same tree as the measured `4b85a06c79`;
only the commit trailers were reworded to the model-free pair) (base
`d70e94ae`; `origin/main` moved to `c48cd2c1` meanwhile, and none of its
51 changed files is in this diff, so no merge was made).
- `node scripts/pm/dispatch-gates.mjs --commands`: 110 families. All 110
were run and recorded `CMD :: exit N`, and `--ran` reconciles to "110
derived, 110 run, 0 NOT-MEASURED, 0 UNRUN". Four of them first exited 3
(prerequisite not met, nothing measured): `check-plugin-teardown-shape
--self-test` on the shallow clone, and `check:skill-examples`,
`check:dual-build-cjs-loads` and `check:i18n-coverage` before a full
build. After `git fetch --unshallow` and `pnpm build` (115/115 tasks),
all four exit 0. `check-nul-bytes`: OK.
- `@objectstack/spec`: `typecheck` 0, `test` 0 (757 files, 22537 tests),
`test:repo` 0 (87 files, 1448 tests). `check:generated` → `--fix`
regenerated exactly the three it proved stale: `api-surface/`,
`export-origins/`, and
`content/docs/references/data/driver-{mongo,mysql,postgres,turso}.mdx`.
- `@objectstack/service-datasource`: `typecheck` 0, `test` 0 (66 files,
1093 tests).
- `@objectstack/runtime`: `typecheck` 0, `test` 0 (486 files, 7888
passed, 19 skipped).
- `@objectstack/cli`: `typecheck` 0; `vitest run --project unit` 0 (354
files, 5100 tests). The new CLI pin is in the unit tier.
- Workspace typecheck over the dependents: `pnpm --filter
"...^@objectstack/service-datasource" typecheck` 0, covering 27 packages
including dogfood, rest, hono, the examples and `downstream-contract`.
The spec change is additive: one new export and longer strings. Every
spec consumer under that filter, and spec's own `test`/`test:repo`, are
green.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01WGufziLJNDCTSUH9pVN32p)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 69ce01d commit e063a87
24 files changed
Lines changed: 995 additions & 32 deletions
File tree
- .changeset
- content/docs
- data-modeling
- references/data
- packages
- cli/src
- commands
- utils
- runtime/src
- services/service-datasource/src
- __tests__
- spec
- api-surface
- export-origins
- src/data
- driver
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
944 | 944 | | |
945 | 945 | | |
946 | 946 | | |
947 | | - | |
948 | | - | |
949 | | - | |
950 | | - | |
951 | | - | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
952 | 954 | | |
953 | 955 | | |
954 | 956 | | |
| |||
990 | 992 | | |
991 | 993 | | |
992 | 994 | | |
993 | | - | |
994 | | - | |
| 995 | + | |
995 | 996 | | |
996 | 997 | | |
997 | 998 | | |
998 | 999 | | |
999 | 1000 | | |
1000 | | - | |
1001 | | - | |
1002 | | - | |
1003 | | - | |
1004 | | - | |
1005 | | - | |
| 1001 | + | |
| 1002 | + | |
| 1003 | + | |
| 1004 | + | |
| 1005 | + | |
| 1006 | + | |
| 1007 | + | |
| 1008 | + | |
| 1009 | + | |
1006 | 1010 | | |
1007 | 1011 | | |
1008 | 1012 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
258 | 258 | | |
259 | 259 | | |
260 | 260 | | |
261 | | - | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
262 | 264 | | |
263 | 265 | | |
264 | 266 | | |
| |||
274 | 276 | | |
275 | 277 | | |
276 | 278 | | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
277 | 350 | | |
278 | 351 | | |
279 | 352 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
50 | | - | |
| 50 | + | |
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
51 | | - | |
| 51 | + | |
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
67 | 67 | | |
68 | 68 | | |
69 | 69 | | |
70 | | - | |
| 70 | + | |
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
| |||
5279 | 5280 | | |
5280 | 5281 | | |
5281 | 5282 | | |
| 5283 | + | |
| 5284 | + | |
| 5285 | + | |
| 5286 | + | |
| 5287 | + | |
| 5288 | + | |
| 5289 | + | |
| 5290 | + | |
5282 | 5291 | | |
5283 | 5292 | | |
5284 | 5293 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
0 commit comments