You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 18361e5
Browse filesBrowse the repository at this point in the historyBrowse files
feat(catalog): secret flag on AttachOptionSpec for credential attach options
Part A of the attach-credentials proposal. AttachOption/AttachOptionSpec gain
`secret: bool = False`, carried on the wire as a nullable bool column named
"secret" appended after "required". Readers look columns up by name, so a spec
without the column reads as secret=False and older peers ignore it.
`secret` combines with `required`, and is allowed with a default (documented
as discouraged). Credential options (API keys, tokens, passwords) must be
declared secret=True so clients and the DuckDB extension can mask them, keep
them out of cache keys and logs, and supply them from a `vgi_attach` secret.
- the attach-options fixture's required api_key is now secret
- worker page and landing page flag required/secret options and never show a
secret option's default
- docs: "Declaring Attach Options" in docs/catalog-interface.md
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: CLAUDE.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -392,6 +392,8 @@ Set `SENTRY_DSN` (and install `vgi[sentry]`) to forward unhandled exceptions to
392
392
393
393
**Attach options redaction:** by default *no* options are logged in the `catalog.attach` / `catalog.create` breadcrumbs because options routinely carry credentials. Implementers opt in via `CatalogInterface.loggable_attach_options(options) -> Mapping`, which returns a redacted, safe-to-log subset (host, region, bucket — never password/token/secret). When the override returns an empty mapping (the default), the `options` field is omitted from the breadcrumb entirely. See `docs/catalog-interface.md` for details.
394
394
395
+
**Secret attach options:** credential options (API keys, tokens, passwords) MUST be declared `AttachOption(..., secret=True)`. `AttachOptionSpec.secret` rides as a nullable `bool` column named `secret` appended after `required` (an absent column reads as `False`); clients and the DuckDB extension mask such options, keep them out of cache keys and logs, and can supply them from a `vgi_attach` DuckDB secret (`CREATE SECRET (TYPE vgi_attach, SCOPE '<worker url>', api_key '…')`). Secret + required is fine; secret + default is allowed but discouraged.
396
+
395
397
**Releases.**`vgi-serve` populates `release` from `SENTRY_RELEASE` if set, otherwise from `importlib.metadata.version("vgi-python")`. Production deploys should set `SENTRY_RELEASE=$(git rev-parse HEAD)` (or a tagged release identifier) so Sentry's commit-tracking and regression-detection features can correlate events to specific commits. Publishing GitHub releases makes the release-comparison UI more useful.
396
398
397
399
The same enrichment applies to OTel spans when `VGI_OTEL_ENABLED=1` — both backends read from the same `VgiTracer.set_current_span_attributes()` call sites in `vgi/otel.py`. Either, neither, or both can be active in a process.
Copy file name to clipboardExpand all lines: docs/catalog-interface.md
+36Lines changed: 36 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -246,6 +246,42 @@ class MyCatalog(CatalogInterface):
246
246
247
247
---
248
248
249
+
## Declaring Attach Options
250
+
251
+
A worker declares the options its catalogs accept at `ATTACH` time in an `AttachOptions` inner class. Each option is advertised to clients at discovery (`CatalogInfo.attach_option_specs`), so DuckDB can cast and validate it, and frontends can render an options form.
252
+
253
+
```python test="skip"
254
+
from typing import Annotated
255
+
from vgi import Worker
256
+
from vgi.catalog.attach_option import AttachOption
|`required=True`| The caller must supply the option. Declare it with no class-level default; `required` plus a default is rejected. |
268
+
|`secret=True`| The option carries a credential. |
269
+
270
+
**Credential options (API keys, tokens, passwords) MUST be declared `secret=True`.** Clients and the DuckDB extension then mask the value, keep it out of result-cache keys and logs, and can supply it from a `vgi_attach` DuckDB secret, so the `ATTACH` statement never carries it:
`secret` combines with `required`. It is allowed together with a default, but a secret option should normally have none: the default is advertised in plain text to every client at discovery.
278
+
279
+
On the wire, `required` and `secret` are nullable boolean columns appended after the four shared spec columns (`name`, `description`, `type`, `default_value`), in that order. Readers look columns up by name, so a spec from a peer that predates either column reads it as `false`, and older peers ignore columns they don't know.
280
+
281
+
`secret` does not change what the worker itself logs: attach options are never logged unless the catalog opts in through `loggable_attach_options()` (below), which must never return a secret option.
282
+
283
+
---
284
+
249
285
## Logging Attach Options Safely
250
286
251
287
The worker emits structured `_logger.info` records and Sentry breadcrumbs for catalog lifecycle events (`catalog.attach`, `catalog.detach`, `catalog.create`, `catalog.transaction.begin`, `catalog.transaction.commit`, `catalog.transaction.rollback`). `attach_opaque_data` and `transaction_opaque_data` are short-hashed (12-char SHA-256 prefixes) before they reach the log record, the breadcrumb data, or the Sentry scope tags — the raw values never appear in observability output, since they may carry secrets. An operator correlates the short hash back to the catalog via these breadcrumbs.
0 commit comments