Skip to content

Lasagna-140726/recover-crypto - #21

Open
Arcoders wants to merge 46 commits into
LASAGNA-020626/isolation-hardening-and-benchmarksfrom
LASAGNA-140726/recover-crypto
Open

Lasagna-140726/recover-crypto#21
Arcoders wants to merge 46 commits into
LASAGNA-020626/isolation-hardening-and-benchmarksfrom
LASAGNA-140726/recover-crypto

Conversation

@Arcoders

Copy link
Copy Markdown
Owner

No description provided.

Arcoders added 30 commits July 15, 2026 20:54
…internal

The crypto satellite was removed by 620af17 on this line, and the public
surface was then frozen (9f20e74), so its old /crypto and /worm-ledger subpaths
no longer exist. Recover the package and route its two core seams through
/internal rather than re-exposing those frozen subpaths: sealV2WithKey and
openV2WithKey were already on /internal, and this adds WormLedgerWriter/WormDb.
The core public barrel is unchanged, so the frozen surface stays intact.

Reconcile with the post-removal repo: crypto 1.0.0 becomes 0.1.0 with a
>=0.3.0 <1.0.0 peer on saas-tenancy, and it is marked experimental to match
the sibling satellites.

Restore the full removal surface: the 11 invariant guards and their check
wiring, the root build:crypto link, the CI and publish steps, and the docs
(guide, sidebar, commands, stability, contract versions). Restore the two e2e
specs (8 cases) and the demo scaffolding; the worm-ledger and secure-notes
migrations are renumbered to backoffice 0021 and tenant 0005 to avoid
collisions with tables added since.

Add a new @integration/fault_injection tier (4 specs, 6 tests) that injects
real infra faults (KMS unreachable, WORM append dropped, store write dropped,
Redis lock down) and asserts a fail-closed outcome with no residue.

Humanize the recovered comments and the test titles to one voice, and fix the
rowscope migration stub so its template body escapes correctly and renders.
Implements the reserved I7 / threat #12 / OWASP LLM06 tool-calling contract as a
default-deny, fail-closed capability. Inert until the loop is wired into the chat
controller (Phase 9); every piece below is unit-tested (607 green).

- contract v2: AIToolCall/AIToolDefinition, role:'tool' turns, tool_call fragments,
  both wire dialects (Anthropic tools + OpenAI functions), AI_CONTRACT_VERSION -> 2
- multi-round tool loop inside the single streaming-spine producer (one reservation,
  one commit, monotonic ids, aggregated result); per-tenant concurrency cap
- security core: default-deny registry, per-tool authz, prototype-safe argument
  validation, tenancy.run scoped execution + I7 confused-deputy re-assert (read
  before bind), output fencing; 6 Isthmus guards, 6 error codes
- config validation (assertToolsConfig) + ai_tools doctor check
- audit op:'tool' data contract: frozen non-PII event + checksum-preserving
  PgToolAuditSink mapping (toolName->model, round->matchCount, mode->provider)
- memory reconstruction excludes tool_call notices

Action (mutating) tools ship OFF and are refused unconditionally until the Phase 3a
confirmation flow. No new runtime dependency.
…ring surface (WS-AI-11)

Phase 9a — executor observability (discharges the Phase 6/8 deferred debt).
The tool executor now emits the five ai_tool_* integer metrics
(calls/errors/denied/latency, plus budget-exhausted from the loop) and writes
one best-effort op:'tool' audit row per call, both through inert-by-default
injected seams. A fatal gate refusal is metered + audited before the rethrow;
a handler degrade meters an error and audits 'failed'; an I7 scope breach
audits 'error'. principalHash + round thread through from the loop.

Phase 9b — public ./tools surface. A boot-safe authoring module
(defineTool, defineAiTools, readOnlyTool, validateToolInput) plus the erased
authoring types, exported on the ./tools subpath (exports + typesVersions) and
re-exported from the main barrel. The container/router-bound loop + executor
stay on ./routes.

Inert until the controller is wired (Phase 9c). 612 unit specs green,
typecheck clean.
Phases 9c/9d/9e of WS-AI-11. The loop, the gate, the input validator and the
executor have all been inert since Phase 3; this is the wiring that makes them
run, plus the specs that hold the wiring itself honest.

The controller resolves the per-tenant registry behind the default-deny gate,
advertises the read-only subset, and drives the multi-round loop inside the same
single pump. A tool request takes ONE aggregate reservation (perRound x maxRounds)
through the newly exported resolveMaxRounds, so the reservation and the loop's own
round ceiling clamp identically and cannot drift. It acquires liveness under the
per-tenant cap, and consults the rate limiter once per round. A host without
config.ai.tools keeps the byte-for-byte plain closure: no tools field on the
request, no aggregate reservation, no overhead.

A tool loop against a provider that does not declare capabilities.tools now fails
closed with a 403 before the first byte, rather than advertising tools the provider
would silently drop and answering as if tool calling were unavailable. Both shipped
providers declare the capability.

Fix a real fail-open in resolveToolRegistry: it awaited the host resolveTools bare,
so a resolver throw escaped as an unmapped 500 — against tool_gate's own contract
("every refusal is a typed AIException, never a 500") and against its sibling seams,
which both wrap their host hook. It denies with tool_denied now. A resolver that
cannot decide must not read as "this tenant gets no tools", which would answer
ungrounded as though tool calling were unavailable.

Back the v2 contract bump with the EXT-3 shape gate it has owed since Phase 0
(check-extension-contracts was failing: AI is the first surface past v1). Tool
support IS contract v2, so AIProviderRegistry.assertShape refuses a provider that
claims capabilities.tools while declaring a pre-v2 contract — it cannot know the
tool wire shapes, and honoring the claim would route it turns it cannot parse. A v1
provider claiming no tools still registers: every member v2 added is optional, and
the controller never hands it a tool turn.

Also runs eslint --fix over the package, which had been committed unlinted since
36ccd6e (formatting and type-only imports; no behavior change).

642 unit specs green, typecheck clean, check 48/48 guards.
…door (WS-AI-11)

Phase 10's unit tier. The existing specs already pin the known vectors one by one;
these say the FIELD is closed rather than just the holes we thought of.

security_tool_args_fuzz drives the validator with prototype gadgets (top-level,
nested, and array-nested), malformed and oversized JSON, homoglyphs, RTL overrides
and NULs, plus a seeded random sweep. The contract asserted is total: every input
ends either as a typed tool_input_invalid or as a reconstruction holding only
declared, well-typed keys — never a crash, never a polluted Object.prototype, never
an undeclared key reaching a handler. The sweep is mulberry32-seeded so a red run
reproduces instead of flaking.

security_tool_result_injection_fuzz covers the other direction: a result carries
whatever sits in the tenant's tables, which an attacker may have written. Forged
closing fences (case-varied, split, doubled), system-prompt mimics and hostile
values nested in a real aggregate all stay inside one matched fence on a role:'tool'
turn. It also pins the truncation boundary specifically — padding the payload so a
forged tag lands exactly where the bound cuts — since neutralization being
length-preserving and running after the bound is what stops truncation from
stitching a tag back together. Injection text is kept, not scrubbed: role separation
is the control, and a customer legitimately named "</tool_result>" must still read.

security_tool_client_cannot_forge_result pins the front door. Both halves matter:
parseChatBody rejects a role:'tool' turn, AND it reads only role + content, so
toolCalls smuggled onto a valid assistant turn are structurally dropped rather than
forwarded. Without this a client could hand the model a fabricated "fact" that looks
like it came from the company's database with no tool run and no authz consulted.
The mirror case is pinned too — the loop's own server-authored tool turn must still
be accepted — so a regression to a blanket ban would fail rather than silently
disable tool calling.

performance_tools_bounded_queries pins the satellite's own overhead as O(1) per call
and O(rounds) per request: one audit row and a fixed metric set regardless of result
size, and one provider call per round rather than per tool call. Audit fan-out here
would land on the shared backoffice table every tenant contends on.

664 unit specs green, typecheck clean, lint clean, check 48/48.
…e (WS-AI-11)

Phase 11. The threat model still told readers tools were unimplemented, which after
WS-AI-11 shipped is worse than saying nothing: a security review would have mapped
LLM06 onto "no agent-action surface exists" while the loop was live.

check-ai-invariant-7 makes I7 structural rather than aspirational. It scans for the
four facts the invariant rests on, each cheap to delete by accident and silent when
deleted: the handler is bound in runScoped, the ambient scope is re-asserted BEFORE
the bind, the registry default-denies an absent config, and authorization is
consulted per call. It checks ORDER, not presence, for the re-assert — that is the
whole point, since reading the active scope inside the bind compares the just-set
scope to itself and passes forever while checking nothing. That exact tautology was
a real bug caught in the Phase 3/4 review, so the guard's red spec reproduces it
along with a moved-file case, because a scan that finds nothing and reports OK is
worse than no scan.

docs/guides/satellites/ai-tools.md is the authoring guide: the readOnlyTool
quickstart, the default-deny posture table, per-tool authorization with filters, the
bounds and their ceilings, the tool_call frame a client must not render as a token,
and the honest limits (the model still chooses; fencing is defense in depth, not the
control; parseInput is sync so vine cannot satisfy it). The action-tool section says
plainly that enabling the kill-switch does not turn writes on.

docs_ai_surface_documented closes a real hole: core's config_documented walks
top-level keys, so it sees config.ai and stops — everything nested under
config.ai.tools could ship undocumented with every gate green. It parses
AIToolsConfig and asserts each key is documented, pins ./tools in both halves of the
export map, and asserts the security page no longer claims tools are post-1.0.

Also updates the coverage matrix (vector 12 moves off "tracked so it cannot be
forgotten when WS-AI-11 lands" to its real red + chaos specs), the ai-security rows
for #12 / I7 / LLM06, the production hardening checklist, and ARCHITECTURE.md's I7
and tradeoff rows.

677 unit specs green, check 49/49, test:integrity green, docs:build with zero dead
links.
…-11)

Phase 10's integration tier. The unit specs pin the executor's gate order against
doubles; these pin the property that only exists at runtime — a tool reads the
schema of the tenant that asked, and keeps doing so when other tenants' calls are
in flight at the same instant.

Each handler resolves its own connection from the AMBIENT scope, exactly as a
TenantBaseModel query does, so a bind that never happened or bound the wrong tenant
reads the wrong schema and the assertion catches it. Nothing mocks the routing.

isolation_two_tenant_tool_no_leak covers the two-tenant case plus the real
confused-deputy shape: an executor bound to A, invoked while the process already
sits inside B's scope, must refuse rather than rebind — and the handler must never
run. It also asserts identical row text in both schemas stays disjoint (the
separation is the schema, not a difference in the data) and that no scope survives
the call.

isolation_tool_cross_tenant_fuzz is the one a unit test cannot reach. Scoping rides
an AsyncLocalStorage, and ALS is exactly what breaks under concurrent async work: a
bind leaking across an await, a handler resuming on another call's context. One call
at a time always looks correct, so the interleaving IS the test — N tenants x PER
calls, mulberry32-shuffled under bounded concurrency, each re-reading its scope
after a real query round-trip. Seeded, so a failing interleaving reproduces.

performance_tools_concurrent_tenants fills the empty performance/integration slot
with two properties that only show up at scale: different tenants' calls must
genuinely overlap (a process-wide lock would keep every functional test green while
serialising the fleet behind the slowest query, visible only as latency), and the
Phase-2a cap must bound in-flight work per tenant without touching anyone else. The
timing bound is loose on purpose — a "nothing is globally serialised" probe, not a
benchmark.

The runScoped seam is a real AsyncLocalStorage rather than the kernel's tenancy.run,
which also connects the tenant and runs the bootstrapper lifecycle; the kernel's own
suite proves that, and it needs provisioned tenants this harness does not have. ALS
is what tenancy.run is built on, so the property under test is the satellite's own.

Also drops seven stale scaffold READMEs. Each said "Delete it once specs live here"
while sitting in a slot with up to 38 specs. core, billing and reporting already
follow that rule; this brings ai in line.

677 unit green, 58 integration passed / 4 skipped (pgvector, local), check 49/49.
The convention is that a test slot carries a README only while it is empty:
the placeholder keeps the directory visible and tracked until specs arrive,
and core has none in any slot that is filled. Eleven READMEs had outlived
that, six of them still instructing the reader to "delete it once specs live
here" from inside a directory with up to four specs in it.

Nine were in crypto and two in ai, so this is not one package drifting. The
ai pair is the same drift the WS-AI-11 cleanup missed, because that pass only
looked for the explicit delete instruction and these say "placeholder until
the first spec lands here" instead. Both sentences are false in a filled slot.

crypto/@guarantees/performance/unit keeps its README: that slot really is
empty, so the placeholder is doing its job. scaffold_test_tree.ts only writes
a README into a directory that holds no files, so nothing regenerates these.
The tool loop had no chaos coverage. Its guarantees were argued in unit specs
against doubles, which is where the WS-AI-11 security review found that
`await tool.handler()` never raced the abort signal: a handler ignoring its
signal pinned the pump, the reservation and the concurrency slot. That class
of defect only shows up when something real breaks mid-flight, so this tier
breaks real things. It is non-gating and runs on a [chaos] commit, like the
core and crypto tiers it copies.

Four faults, each against real Postgres and real Redis:

- provider_aborts_mid_tool_use: the provider drops part-way through a tool
  call's arguments. A truncated block is discarded rather than executed, and
  its sibling on the same round still runs, so the discard is selective. The
  OpenAI dialect behaves differently and the spec says so plainly: it emits
  the truncated call with unparseable JSON, and validateToolInput is what
  refuses it.
- tool_executor_backend_down: a handler whose database is unreachable degrades
  to a bounded error result and the loop continues, while a gate refusal stays
  fatal. Both sides of that split are pinned, since collapsing either one is a
  real bug.
- redis_down_during_tool_round_reserve: a round that cannot be metered refuses
  and never re-enters the provider. It drives both outage shapes, including the
  one ioredis actually produces, where exec() RESOLVES carrying per-command
  errors. A rail guarding only against a rejection would read that as a pass.
- client_disconnect_mid_tool_execution: the runWithAbort regression, pinned.
  Removing the race turns this tier red.

The specs use a real AsyncLocalStorage as the runScoped seam rather than
tenancy.run, which connects the tenant and needs provisioned tenants this
harness does not have. ALS is what tenancy.run is built on, so what is under
test is the satellite's own property: that the executor binds the scope around
the handler and re-asserts it first.

Every schema, connection and Redis key derives from a per-run randomUUID and
teardown drops only what it created, so a run cannot collide with the demo e2e
on the shared local database.

Two honest limits are recorded in the specs: the tools are mode 'read', so
nothing here proves action-tool durability, and the transport drop is injected
at the parser's byte source rather than by tearing down a real socket.
The denial-of-wallet defense has two halves. The spend half, the rate limiter,
trips guard.ai_rate_limited and an operator can watch it. The flood half, the
Phase 2a admission cap, threw a bare 429 and emitted nothing, so the one rail
that stops a tenant opening tool loops until the connection pool gives out was
invisible. You could not tell "the cap is biting" from "nobody is asking".

The refusal now emits guard.ai_too_many_concurrent, carrying the in-flight
count and the cap that refused it. Severity is warn for the same reason its
sibling ai_rate_limited is warn: an admission cap trips in normal operation
under load, so it is monitored by rate rather than per event.

The registry's Record<AiGuardId, TripRecipe> matrix is compile-forced, so the
guard came with its trip and happy recipes. Each recipe builds its own watcher,
since the count it asserts on has to be its own.

This was flagged during the Phase 3/4 security review and deferred twice to
avoid widening the guard matrix mid-phase. It lands on its own, before the
Phase 3a action-tool work, because it touches the watcher and the registry
rather than anything 3a adds.
reconstructAssistantText skipped a list of known control events, so anything
not on the list counted as the assistant's prose. That direction is backwards
for this function. A recorded frame feeds two consumers: the client, and this
path, whose output is encrypted into conversation memory and re-injected into
the next prompt. Under a deny-list every new event is content until somebody
remembers to add it, and forgetting is silent.

It had already happened once. The tool_call notice was being concatenated into
memory until Phase 8 added it to the list, which fixed the instance and left
the shape. The next event was going to be Phase 3a's confirmation frame, whose
data is a live signed capability with a five minute life: reconstructed into
memory it would ride into the model's context and could come back out as text,
past the one gate hosts are told is the last word on output.

So it now allows the default event instead. Only that event carries prose,
SseWriter.writeFragment resolves an absent event to the same default, and
tool_loop's own accumulator already allow-lists this way, so the two agree by
construction rather than by maintenance. A new control event is inert the day
it is added.

The spec that pins it uses an event name that does not exist, which is the
whole point: a deny-list cannot pass that test.
The first piece of human-in-the-loop confirmation: a pure module that mints a
capability for one proposed mutation and verifies it on the turn that follows.
Nothing is wired yet. Action tools are still refused unconditionally.

The token carries no claims. Verification re-derives the tenant, the principal,
the tool name and the argument hash from the request being served, and reads
only the nonce and the expiry off the wire. A token for another tenant, another
user, another tool or other arguments does not fail a check, it is simply a
different value and nothing matches. There is no field to compare, so there is
no field to forget to compare. That is the whole reason it is shaped this way
rather than as a claims token.

The expiry travels in the clear because it is a MAC input: editing it to buy
more time changes the expected MAC and the token stops working. Nothing else
is on the wire, so a token captured from an access log names neither the
tenant, nor the user, nor the action.

effectKey is the token's own MAC rather than a hash of what it binds. Keyed by
the binding, a deliberate repeat of the same action would be indistinguishable
from a replay and the second one would vanish. Every mint carries a fresh
nonce, so one token is one effect and the ledger that lands next can refuse a
re-fire without refusing a genuine second request.

Argument canonicalization sorts object keys and leaves arrays alone. That is
load-bearing rather than tidy: the model re-proposes the action on the
confirming turn and key order does not survive the round trip, so an
order-sensitive hash would make confirmation fail at random.

Verification never throws. It sits on a mutation path, and a malformed token
becoming a 500 there would be the worst of both.

The specs try to spend tokens where they should not work rather than asserting
on internals. Removing argsHash from the MAC turns them red, which is the
propose-A-confirm-B attack arriving.
The second piece of action-tool confirmation, and the one that carries the
replay guarantee. Still not wired: action tools remain refused.

A confirmation token is a bearer credential and a bearer credential cannot be
burned, so the token cannot be what stops an action firing twice. This can. It
is a Postgres table with a unique effect_key, and the claim is a single INSERT
with ON CONFLICT DO NOTHING RETURNING, so the set-if-absent is atomic in one
statement. A SELECT then INSERT would leave a window where two requests both
read absent, both insert, and the loser only finds out after its handler had
already run.

The gap being closed is narrow. The response cache is side-effect-free on
replay, but a stream that dies after the effect fired and before it completed
is never cached, so the client retries, the model re-proposes, the human
confirms again and the mutation happens twice. That hole is there no matter how
the confirmation is designed, which is why this is a separate mechanism.

Keyed by the token's own MAC rather than by what the token binds. Every mint
carries a fresh nonce, so a conflict can only mean this exact token already
fired. Keyed by (tenant, principal, tool, arguments) instead, a deliberate
repeat of the same action would look identical to a replay and vanish.

It fails closed: an action whose fence cannot be written is refused. When the
backoffice database is unreachable, action tools are unavailable and read tools
are untouched. That is a real availability cost and it is the right direction,
since a mutation that cannot be made at-most-once should not be made.

A row still reading claimed is reported as unknown rather than smoothed into a
failure or a success. The process died mid-effect and whether it landed is
genuinely unknown; calling it failed would invite a retry that doubles it. Same
reason a failed effect stays fenced instead of becoming retryable.

Two new codes land on opposite sides of FATAL_CODES, which is a hand-maintained
Set the compiler does not check: a bad confirmation is permanent, an unwritable
ledger is transient. Getting that backwards would invite a client to hammer the
MAC on a mutation path, so a spec pins it.

The table is in backoffice like the audit chain, but deliberately without the
append-only triggers: a claimed row is updated once when it settles. The audit
row is the evidence an action happened; this row is the fence.
Opens the door that assertActionAllowed has been holding shut, and puts four
locks on it. Still not reachable: nothing emits a confirmation challenge yet,
so every action refuses with tool_confirmation_required and there is no way to
obtain a token. That is deliberate. The gate and the check that enforces it had
to land together, or actionTools.enabled would briefly have meant "run writes
without asking anyone".

The four rules, all default-deny:

- The kill-switch. Off by default, and off means unadvertised, not just
  refused: the model never learns the tool exists, so it cannot propose a write
  the operator disabled and no human is shown a confirmation for one.
- summarizeArgs is mandatory. This reads like a config nit and is not. The
  human's decision is only as good as what they are shown, so if the model
  wrote that line an injection could author its own confirmation prompt and the
  whole flow is a rubber stamp. Refusing per tool beats a doctor warning, which
  does not stop a rubber stamp that already shipped, and beats failing the boot,
  which would push hosts to turn confirmation off instead.
- acknowledgeUnauthorizedTools does not cover writes. That escape hatch exists
  so a host can try read tools out before wiring authorization, where isolation
  still holds and the worst case is reading its own data. Extended to writes it
  would be one boolean, set once for a demo, authorizing every mutation the
  model can reach.
- The confirmation itself, checked against the arguments, per call.

requiresConfirmation: false is refused rather than honored, which narrows the
phase. The fence is keyed by the confirmation token's own nonce, so one token
is one effect. An auto-executing action has no token, therefore no nonce, and
nothing left to key a fence by that behaves: by the arguments, a deliberate
repeat is indistinguishable from a retry and vanishes; by anything fresh, it is
not a fence. At-most-once is not reachable there without a client-supplied
idempotency key, so the choice was a subtly wrong semantic or a refusal.

The audit ordering inverts for actions: intent is written fail-closed BEFORE
the effect. A read audits best-effort afterwards because losing the record of a
read costs a log line. A mutation that ran with no durable record of intent is
one nobody can account for.

All of it decided in tool_gate beside the other gates. The executor owns the
ordering and the side effects and throws nothing itself, which is also what
keeps the no-silent-guard boundary honest.

authorizeToolScope now takes the tool rather than its name, so the mode comes
from the definition and a caller cannot forget it and land on the weaker path.
Worth noting: this package does not typecheck its tests, so that signature
change left them passing a string, silently, until they were fixed by hand.
The finding this closes: only core and doc-coverage typechecked their specs.
Every other package's tsconfig include covered src and providers and nothing
else, so ten packages had never once run their tests through tsc. It was not
hypothetical. A signature change to authorizeToolScope earlier this session
left every spec passing the string 'read' where an object was now required, and
the suite stayed green because 'read'.name is undefined and an undefined mode
happened to read as the default. Doctor specs were calling run() with no
argument against a run(ctx) contract. The whole noUncheckedIndexedAccess and
exactOptionalPropertyTypes hardening had never touched a spec.

check-typecheck-covers-tests pins the shape core has always had and the reason
there are two config files rather than one: tsconfig.json checks with tests
included and no outDir, tsconfig.build.json emits with tests excluded. It fails
if a package's typecheck include stops covering tests, if that config gains an
outDir (which would make including tests ship them), if the build config starts
covering tests, or if the build script stops pointing at the build config. The
failure it exists to prevent is somebody quietly dropping tests/ from an include
to silence a noisy spec, which reads as a one-line cleanup and turns the suite
back into unchecked JavaScript.

Two guards read tsconfig files, and a tsconfig is JSONC: comments are legal and
now used to document why the split is there. JSON.parse chokes on them, which is
how check-satellite-migrations started failing the moment the configs were
documented. read-jsonc.mjs is the shared reader so that is fixed once rather
than per guard. check-satellite-migrations also now reads tsconfig.build.json
where it exists, because after the split that is the config that governs
emission; reading the typecheck config would test a file that no longer emits.
Splits each satellite's single tsconfig into the two core has always had:
tsconfig.json now typechecks the tests too (no outDir, so it cannot emit),
tsconfig.build.json emits only the shipped surface (tests excluded), and the
build script points at the build config. The architectural specs import the
repo-root scripts/check-*.mjs auditors, so the packages that have those get
allowJs (checkJs off) or tsc reads the guards as implicit any and silently stops
checking every call into them.

Then fix every error the first-ever typecheck of these specs turned up, around
300 across the ten packages. The bulk is noUncheckedIndexedAccess honesty in
assertions: frames[0]?.data over frames[0].data, so a wrong index fails the
assertion loudly rather than throwing a confusing TypeError. Nothing was
weakened to pass. No as-any around a subject, no ts-ignore, no dropped
assertion, no skipped test; a mutation probe on each package confirmed a
deliberate type error in a spec now fails the typecheck and names the file.

Two of the errors were real latent bugs the typecheck exposed, both in test
support, not production. In crypto's rekek fake a class field initializer read
this.reportCursor before the constructor assigned it, so it was always
undefined and the cursor branch it was meant to exercise never ran; the fix
installs the value in the constructor body. The other was a fake whose type had
drifted from the contract it doubles. Production src was not touched in any
package: where a spec suggested a real src type was wrong, it was reported, not
edited.
The ai-tools guide shipped with WS-AI-11 but was never referenced from the
overview page, so the integrity spec that pins every guide to the index caught
it. It belongs in the "also documented" callout beside AI security, being a
guide within the AI section rather than a tenant-attached feature satellite.
…g (WS-AI-11 Phase 3a)

The tool loop now plans every call in a round before running any of them, so a
round is never half-applied around a pending confirmation: the executor's new
`plan` phase classifies each call — run, challenge, or a fatal refusal thrown
during the scan — and only once the whole round is clear does the loop invoke the
run thunks. An action awaiting a human is emitted as a `tool_confirmation_required`
SSE frame carrying the host `summarizeArgs` line and the minted token, which the
client echoes back in `X-Ai-Tool-Confirmation`.

- tool_loop: single-pass `planRound`; `ToolLoopExecutor.plan` replaces `execute`;
  `ToolCallPlan`/`ToolConfirmationChallenge` types; the challenge frame rides its
  own event so `reconstructAssistantText` never folds the live token into memory.
- tool_gate: `resolveActionConfirmation` (returns confirmed|challenge, throws the
  fatal cases) replaces the always-throwing `assertActionConfirmed`;
  `renderActionSummary` runs `summarizeArgs` fail-closed and bounded.
- tool_executor: `#planOne` returns a plan; the effect (ledger claim, fail-closed
  intent audit, scoped+timed handler, fenced result) lives in a `run` thunk that
  only fires in phase 2. `forRequest` returns `{ plan, execute }`; `execute` is the
  read/compat shim that runs a single call to completion.

Still gated at runtime until the controller wires the MAC key + ledger + header
(Phase 3a step 5): with neither present an action is refused tool_action_unavailable.

Gates: ai unit 735, check 50/50, typecheck/eslint/prettier clean.
…S-AI-11 Phase 3a)

The chat controller now reads the client's X-Ai-Tool-Confirmation tokens and
threads them into the executor's forRequest, so a confirmed action runs and an
unconfirmed one is challenged. A confirming request (any token present) is NEVER
served from nor written to the idempotency cache: the cache key MACs
{tenant, principal, session, headerKey} and NOT the token, so a client keeping its
Idempotency-Key across the confirming retry — what every HTTP retry layer does —
would otherwise get a cache hit and replay the SAME challenge frame forever, never
reaching the executor. No error, no metric distinguishes that livelock from working,
so the presence of a token suppresses the scope entirely (no lookup, no save).

The provider wires the confirmation MAC key (derived from APP_KEY, like the
idempotency key) and a backoffice-schema AiActionLedger into ToolExecutorService,
but ONLY when audit is on: an action's intent must be durably recorded before it
runs, so with audit off the machinery stays undefined and every action refuses
tool_action_unavailable (503).

Tests: a proposed action emits a tool_confirmation_required frame through the real
controller + loop + SSE writer and runs no handler; a confirming request is neither
cache-served nor cached.

Gates: ai unit 738, check 50/50, typecheck/eslint/prettier clean.
…l doctor (WS-AI-11 Phase 3a)

FATAL_CODES is a hand-maintained Set the compiler does not check against the code
union, so a new code silently defaults to retryable. A spec now pins every
AIErrorCode's isRetryable() exhaustively (the table must cover the union exactly),
which nails the three Phase 3a codes whose neighbours classify oppositely: a missing
and a bad confirmation are fatal, while the ledger being unreachable
(tool_action_unavailable) is retryable.

The ai_tools doctor check carried a now-stale info claiming action tools are refused
unconditionally because the confirmation flow "is not yet shipped". It now reports the
real Phase 3a posture: a warn when actions cannot run (audit off ⇒ tool_action_unavailable,
or a static registry action tool missing summarizeArgs / setting requiresConfirmation:false),
and an honest info — with the confirmation-stops-autonomy-not-injection limit — when
they can. Only the static registry is boot-checked; a resolveTools hook is per-request.

Gates: ai unit 742, check 50/50, typecheck/eslint/prettier clean.
…I-11 Phase 3a)

The ai-tools guide's action-tools section claimed mutating tools are "refused
unconditionally today" because the confirmation flow "has not shipped". It now documents
the real flow: what it takes to enable one (kill-switch, authorizeTool allow,
summarizeArgs, a resolvable principal, audit on), the tool_confirmation_required frame
round-trip and the X-Ai-Tool-Confirmation header (with a callout to scrub it from access
and proxy logs), and the honest limits verbatim — confirmation stops autonomy not
injection, at-most-once not exactly-once, and a GDPR purge does not revoke pending tokens.

ai-security.md's vector #12 / I7 / LLM06 no longer say the model can "read but never
write". The docs-surface integrity spec now also pins the three Phase 3a error codes, so
the mutating-tool surface cannot drift back to undocumented once it shipped.

Gates: ai unit 742, check 50/50, lint clean.
…wn (WS-AI-11 Phase 3a)

The deferred fault-injection spec for the phase. An action writes its intent FAIL-CLOSED
before its effect, so if the audit DB is unreachable the mutation must not happen. This
drives the real loop + executor over real Postgres with a real side effect (an INSERT):
the audit sink throws the same audit_write_failed the real PgToolAuditSink → AiAuditWriter
chain throws, and the spec proves the effect row is never written, the loop aborts with
audit_write_failed, and the at-most-once fence was claimed first (the claimed-but-unsettled
row is the honest "unknown" tombstone). A healthy-audit control writes the row exactly once.

Runs green against real Postgres (skips when unavailable, like the tier's siblings); the
neighbouring backend-down spec's stale "hard-gated until Phase 3a" note now points here.

Gates: ai fault tier 18 passed, check 50/50, typecheck/eslint/prettier clean.
A full car-rental SaaS built on @adonisjs-lasagna/*, exercising the whole
platform: two auth realms, schema-per-tenant isolation, the nine satellites,
a telematics plugin, and two Inertia + React consoles.

The fleet assistant answers operational questions through the AI satellite's
read-only tools (current_date, count_bookings, count_vehicles,
list_available_vehicles, revenue_summary, top_rented_vehicles) and grounds
document questions on the per-tenant RAG store. There is no live-data snapshot;
the tools carry every operational answer.
…ocks)

The fleet assistant now renders replies as markdown with hand-rolled ```stat
and ```chart blocks (no deps), led by a FLEET_SYSTEM_PROMPT that sets the
answer format and tells the model each tool returns everything in one call.
That one-call-per-tool discipline stops the round loop from exhausting the
tool budget on multi-metric reports; ceilings go to 4 rounds x 3 tools so a
"give me a report" turn can pull revenue + fleet + bookings + a ranking
without tripping tool_budget_exhausted mid-report.
The public IsthmusGuardTripped event is broadcast process-wide and any plugin
can subscribe. Two root fixes at the single emit seam:

- S1: bound the broadcast metadata (16 keys / 256 chars, fixed constants, a clip
  is counted as dropped{metadata_bounded}) and tokenize foreign tenant ids with a
  per-process HMAC (tokenizeTenantId, /sdk) so the ContextSeal's requestResolvedId
  and the AI scope-mismatch's `active` never leak a real cross-tenant id to a
  listener. The precise ids stay server-side in the typed exception.
- S3: add a registry dispatchPolicy classification enforced in the shared emit;
  a guard reachable at high volume from pre-tenant request input
  (guard.tenant_identifier) is count-only, so a flood can never consume the shared
  per-severity dispatch window and suppress another tenant's alerts. The trip
  still records on multitenancy_isthmus_rejected_total.

Prove-then-fix specs: metadata contract, dispatch starvation (with negative
self-test), updated seal-emission / degrade / conservation-law / matrix specs.
…ookup

WS-1: pin "the shield is cheap" as an operation-count invariant (never
wall-clock) in core's previously-empty @guarantees/performance slot: the
ContextSeal adds zero SQL round-trips and emit() is O(1) per trip. Add an Isthmus
micro-bench (emit broadcast/count-only, allow, tokenizeTenantId) to the Docker-free
micro tier.

WS-2: mirror the kernel's BY_ID map in the AI and crypto guard registries for
O(1) id lookup (kept `as const satisfies`, no naive Object.freeze that would
widen the id union), plus a type-level assertion that AiGuardId/CryptoGuardId can
never silently widen to string and break the emission matrix's compile-time
completeness.
…r awareness

S2: the absolute connection ceiling is now truly unbypassable. Split the admission
check so atHardCeiling() always throws while only the soft cap responds to the
renamed bypassSoftCap flag (was bypassHardCap, which let every operational path
skip the ceiling = a Postgres-exhaustion DoS). Add a ceiling-derived
operationalConnectionBudget that bulk/parallel paths draw from, and bring the
satellite-migration temp clone under it.

F1: a bounded, cap-aware migration worker pool. `migration:tenant:run --concurrency`
runs tenants through mapWithConcurrency clamped to the operational budget, with
per-tenant failure isolation and an exit code that reflects any failure. Default 1
keeps the sequential UX unchanged.

F2: an opt-in connection warm pool (isolation.warmPool + defineWarmPool). Pre-opens
connections for OPERATOR-declared hot tenants (never request-derived, so no warm
DoS) through the capped connect(), bounded by the operational budget and refused by
the ceiling, failing closed per tenant.

F3: PgBouncer awareness (isolation.pgBouncer). A fail-closed boot probe records the
posture (auto never assumes transaction pooling, detection never raises a cap) and
a doctor check reports it; the tenant GUC path is transaction-pooling-safe by
construction.

Specs: ceiling-unbypassable (unit + real-PG fault), migration pool, warm-pool
behavior + cold-cliff op-count, PgBouncer posture + fail-closed probe.
Document the Isthmus security model in the guard-registry reference: the listener
trust boundary (foreign-id tokenization + bounded metadata, S1), the count-only
classification with an alert-on-rejected-counter recipe (S3), and the unbypassable
ceiling vs the soft cap (S2). Record the rejected performance-roadmap proposals as
a decision note so they are not re-litigated (the WS-2 C2 note).

Correct the connection-ceiling docs (configuration, scaling-limits,
custom-isolation-driver) that described the old bypassable behavior, and document
the operational connection budget, the migration --concurrency flag, the warm pool
(with defineWarmPool), and the PgBouncer awareness block + doctor check.
Make the tenant doctor tell platform failure apart from a single tenant's
problem, and turn `--fix` into a repair that never destroys data.

healTenant keystone (services/tenant_healer.ts): provision-up-only, idempotent,
re-reads under a TOCTOU guard and refuses a soft-deleted tenant, quarantines to
`failed` on error, and never calls destroy/reset/rollback. Every repair path now
composes it: the doctor's fixable heals, the new `tenant:heal` command, and
`isolation.migrateOnProvision` (new companies are born migrated, so the
half-provisioned state that caused the original 503 cannot recur).

Scope-aware tri-state verdict (doctor_service + types): a platform-scoped error
fails (`status:'fail'`), a lone tenant-scoped error only degrades. admin's
healthReport maps `fail` to 503 and everything else to 200, so one broken tenant
no longer takes the whole operator console offline. The operator UI mirrors the
tri-state (Healthy / Degraded / Attention).

New checks: tenant_lifecycle (reconciles the status/deletedAt axes, Invariant A)
and migration_drift (tenants behind head). Existing checks gain a scope, mark
their fixable issues as heals, and shed false positives (skip provisioning, treat
retained schemas as not-orphan, include suspended). Invariant A is also fixed at
the source: status and deletedAt move together across every delete and restore
path.

utils/lazy_logger.ts: importing the logger service at module top level crashes
the unit runner (it top-level-awaits app.booted). The fixable envelope, the
healer and the circuit-breaker check reach the logger lazily instead.

Chaos/fault tier for the keystone, which surfaced and fixed a real defect: heal
was flipping a healthy tenant to `failed` on benign migration-lock contention.
Lucid guards every migration with a single global, non-blocking Postgres advisory
lock (PG_TRY_ADVISORY_LOCK('1')), so any two overlapping tenant migrations
contend and the loser gets E_UNABLE_ACQUIRE_LOCK / E_UNABLE_RELEASE_LOCK. The
lock is what keeps the ledger uncorrupted, so heal now rethrows that contention
without quarantining, and the fault tier proves both a genuine fault (quarantine
plus recovery) and benign contention (no spurious quarantine).

Also corrects a pre-existing stale test (isolation_context_seal_fail_closed) that
predated the isthmus tokenization of the resolved request id.
…dger reconcile

Machine-enforce "never harm the patient" across the doctor/healer path so a
routine --fix can no longer quarantine a healthy tenant on benign migration
drift, and so migrations relocated into satellite packages reconcile the
ledger without touching data.

Part A — status exhaustiveness. TENANT_STATUSES tuple in contracts.ts derives
TenantStatus + isKnownTenantStatus. tenantLifecycleDisposition() is an
exhaustive switch that fails closed to reject-not-ready (compile-time :never,
runtime warn) rather than throwing on an out-of-tuple DB status. The three
middleware floor sites route through it. check-tenant-statuses.mjs guards the
tuple; tenant_state_matrix pins the disposition surface.

Part B — benign-collision net + migration_relocated reconcile. The healer
classifies duplicate-object SQLSTATEs (42P07/42P06/42710/42701/42723) as
collisions, returns them instead of throwing, and never quarantines or flips
failed->active on a benign net. migration_drift_check detects relocation only
after the inline file is gone; --reconcile-ledger rewrites the ledger name in a
REPEATABLE READ txn under the migration advisory lock, 15 gates, zero DDL,
data byte-identical, with reversible audit rows. buildMigrationAliasMap
reproduces the satellite target name byte-for-byte (crypto/ai migrationAliases).

Part C — safe-fix registry. SAFE_FIXES declare an effectClass; the per-code
fix-coverage guard backward-pairs every fixable code to its declaration and
bans a fixable code hiding in the surface-only set. health_lattice composes the
per-check verdicts.

All gates green: build:all, typecheck x13, npm run check (51 guards),
unit+arch 1797, integration 404, fault 20, 4 api-extractor reports regenerated.
Live-verified on karimoto: --fix no longer quarantines Acme/carivo/Sahara;
--reconcile-ledger rewrote 6 ledger rows zero-DDL, data byte-identical.
Arcoders added 11 commits July 19, 2026 20:12
The design document and study material for taking @adonisjs-lasagna/ai to
enterprise-grade at the root. It mirrors the design/data-protection-satellites
convention: a README index, a 00-foundation constitution, and six numbered topic
docs, each closing with an honesty bound and the open decisions owned by the
operator.

- README + 00-foundation: the framing principle (the operator earns security, the
  library ships mechanisms), the governing direction (no patches, everything a seam
  or a named constant, fail posture by explicit policy), the baseline ledger of
  what already ships (so no wave rebuilds a working control), and the section
  template every topic doc is measured against.
- 01 threat model: the OWASP LLM Top 10 (2025) delta on top of the shipped
  18-vector table, plus the probability x impact matrix that ranks the genuine gaps
  and sets the wave order.
- 02-06: the root-cause design per pillar (resilience foundation, injection defense,
  audit consumption, gated data-at-rest) and the execution plan with per-wave
  acceptance tests, the verification recipe, and the success metrics.

Design and study material only; the implementation lands wave by wave.
… type the redis seams (Wave 0)

Wave 0 of the enterprise-hardening initiative: the type and error foundation, no
behavior change. Proven by the existing regression specs staying green (744 unit
tests) plus a full typecheck.

- Retryability is now compile-forced. FATAL_CODES was a hand-maintained ReadonlySet
  the compiler did not check against the code union, so a new code silently
  defaulted to retryable (its own comment flagged the footgun). Replace it with
  RETRYABILITY: Record<AIErrorCode, 'fatal' | 'retryable'>, total over the union
  exactly like STATUS_BY_CODE, so adding a code without classifying it is a compile
  error. isRetryable() reads the total map. The behavior_ai_exception spec is the
  runtime mirror; its framing is updated (the footgun is gone).

- Extract the ~143-line #runStream into a thin orchestrator plus #preflight / #setup
  / #commit / #pump / #classifyCaught / #settleAndRelease, threading one StreamRun
  state object. Behavior-preserving: resilience_stream_extension and
  behavior_stream_extension are the regression guards.

- Type the three request-path Redis seams. New services/redis_seam.ts exports
  interface-segregated slices (AiRedisMemory / AiRedisLock / AiRedisProbe, and their
  union AiRedisLike) so each seam declares only the commands it issues and no test
  double must implement commands it never calls. The three getRedis seams drop
  Promise<any> for the slice they use, so a command typo is now a compile error.
…age classifier, unmetered-spend fail-closed (Wave 1)

Wave 1 of the enterprise-hardening initiative: fix the resilience seams at the
root so the two latent 500s dissolve instead of being patched per caller. Gated by
754 unit tests, a full typecheck, 51 guards, and the api-report.

1.1 One resilience-policy seam over every request-path Redis read. A new
`runResilient` closure (bound at wiring to the kernel `ResilienceService.run`,
injected like `getRedis` so no module value-imports the eager core barrel) routes
the conversation-memory load, the idempotency lookup/epoch read, and the rate-limit
consume, PRESERVING each posture: memory and idempotency fail-open (to [] / no
replay), rate-limit fail-closed (mapped back to the typed
`AIException('rate_limit_unavailable')`). The posture is satellite-owned via a
validated `config.ai.resilience.{memory,idempotency,rateLimit}.policy` block whose
defaults reproduce today's semantics exactly, so an absent block changes nothing and
a `fail-closed` idempotency override is now possible. The idempotency-lookup-500 is
dissolved: fail-open is the POLICY, not a catch a caller must remember. Only the
store op is wrapped, never the JSON parse. `runResilient` is optional with a local
passthrough default, so every existing service spec is unchanged.

1.2 Classify DB transport errors at the vector-store boundary. Core's
`isDependencyOutageError` is exported on `/sdk` (single source of truth, api-report
golden regenerated), and one private `#exec` funnel routes every rawQuery (plain and
in-transaction, plus the transaction openers) so a mid-query backend death becomes a
new typed, retryable `vector_store_unavailable` (503), added to the total
`AI_ERROR_CODES` / `STATUS_BY_CODE` / `RETRYABILITY` tables. A real query error passes
through unchanged. The vector-outage-500 is dissolved for all callers at the source.

1.3 Unmetered aiTokens becomes a fail-closed boot abort. The provably-unbudgeted,
unacknowledged, non-dynamic posture (previously a warning that scrolled past) now
aborts boot via `assertAiTokensBudgetOrAbort` (emits `guard.ai_config_invalid`,
throws `config_missing`), the same default-deny posture the membership mount uses.
The `acknowledgeUnbudgetedAiTokens` escape hatch and the info-only
dynamic/operator-ceiling postures still let boot proceed, so it never false-aborts.
…fication (Wave 2, part)

Wave 2 of the enterprise-hardening initiative: expand the chaos tier and remove the
one-spec-per-vector limitation.

- The threat-vector coverage matrix (`ai_threat_vector_coverage_matrix.spec.ts`) now
  lets a vector carry MULTIPLE chaos specs: `chaosSpec` accepts `string | string[] |
  null`, the fs-existence sweep flattens it, and an empty array counts as a coverage
  gap needing an honest-limit reason. This is the root fix to the single-string
  limitation so the dedicated fault tier can attach several distinct failures to one
  vector.

- New fault spec `redis_down_request_path_reads_fail_open.spec.ts` proves the Wave-1.1
  postures through the REAL kernel `ResilienceService` inside a booted app (not the
  local passthrough the unit spec uses): with Redis down, the conversation-memory load
  degrades to [] and the idempotency lookup to no-replay, the `DependencyDegraded`
  observability fires (verified in the run log), and neither 500s. A healthy control
  against real Redis confirms the reads still work, so the fail-open is a genuine
  degradation. Registered on vectors 1 (memory) and 5 (replay).

Gates: ai unit 754, fault tier 21, full typecheck, 51 guards, eslint/prettier.

The remaining planned fault specs (provider 429/5xx, provider timeout, plain-chat
socket-drop, vector-store outage, malformed SSE) exercise the streaming spine and the
vector-store `#exec` boundary, which are already covered by the unit specs
`resilience_stream_extension` and `resilience_vector_store_outage` and by the passing
integration suite; promoting them to dedicated real-spine fault specs is a follow-up.
…ble input classifier seam (Wave 3)

Wave 3 of the AI enterprise-hardening plan (design/ai-enterprise-hardening/
03-injection-defense.md). It makes the structural prompt-injection boundary
OBSERVABLE and adds a defense-in-depth input classifier seam, and deliberately
ships NO hardwired semantic regex wall as the boundary.

3a — structural boundary made observable (neutralize AND observe, never block).
neutralizeFence (context_builder) and neutralizeToolFence (tool_executor) now emit
guard.ai_injection_structural on a real fence-token rewrite: the one deliberate
neutralize-and-observe guard (failMode closed, severity warn), so a corpus/tool
probing for a fence breakout is no longer invisible. The neutralization behavior is
unchanged. The guard emits tenant-less (a pure builder holds no tenant, and a
neutralize-and-observe signal must not inflate the ai_guard_rejections bridge); the
controller/executor emit the dedicated per-tenant AI_INJECTION_STRUCTURAL_METRIC
from the returned neutralized flag (mirrors AI_OUTPUT_REDACTED_METRIC).

3b/3c — the InjectionClassifier async host contract (the seam). New AIInjectionConfig
{ classifier?, onError?, scanRetrieved? } on AiConfig, symmetric to redactOutput on
the output side, boot-validated (assertInjectionConfig routes through the single
guard.ai_config_invalid choke). enforceInjectionClassifier (new gateway/injection_gate.ts)
runs at the chat() input pre-flight over each user turn (and, opt-in, the assembled
retrieval block), before any reserve/rate-limit/mint. A block verdict is fail-CLOSED:
new injection_detected code (400, fatal in the compile-forced RETRYABILITY map),
guard.ai_injection_detected (severity high), AI_INJECTION_DETECTED_METRIC, audit
failed_preflight with zero spend. The classifier's OWN error is fail-OPEN by default
(it is NOT the boundary — structural role separation plus I4 is), emitting
AI_INJECTION_DETECTOR_ERROR_METRIC; onError:'closed' is the visible strict opt-out.

3d — info-only ai_injection doctor check (ai_injection_check.ts) reports the live
posture (classifier wired / structural-only, scanRetrieved, onError) and never warns,
because the no-theater default is correct, not a misconfiguration.

Tests: emission-matrix recipes for both guards; structural signal preserved-and-observed
(retrieval + tool fence, happy path silent); classifier block/fail-open/fail-closed/
malformed driven through the real controller with zero-spend + load-bearing "boundary
was never the classifier" invariant; enforceInjectionClassifier seam logic; boot
validation; doctor posture; tool-executor ai_injection_structural metering.

Gates: ai unit 789, integration 58/4-skipped, fault 21, build:all + typecheck 0,
npm run check 51 guards, eslint/prettier clean.
…port, incremental verify, retention checkpoints, anomaly watcher (Wave 4)

Wave 4 of the AI enterprise-hardening plan (design/ai-enterprise-hardening/
04-audit-consumption.md). The write-side chain (per-tenant seq + sha256, advisory
-locked, DB-trigger append-only, fail-closed) was already enterprise-grade; this
builds the five CONSUMPTION surfaces on top and holds one invariant above all: no
consumption path ever rewrites a seq / checksum / prev_checksum. It reads and
appends new artifacts (checkpoints, exports, alerts); it never mutates a chained row.

3.1 Read/query API. New AiAuditReader (SELECT-only, same injected tenancy deps as the
writer, so it inherits qualified-table + re-assert-before-raw-SQL for free): query()
is one SELECT of AI_AUDIT_COLUMNS with EVERY filter ?-bound, clamped paging
(DEFAULT/MAX_AI_AUDIT_PAGE_SIZE, MAX_AI_AUDIT_PAGE OFFSET ceiling), scope re-assert
(a bound tenantId disagreeing with the active scope trips ai_scope_mismatch). New
admin-gated ai_audit_controller.ts + GET /ai/audit route, DEFAULT-DENY via a new
AIAuditConfig.authorizeAudit (distinct from authorizeAIAccess), tenant-scoped over HTTP.

3.2 Export. New utils/ai_audit_export.ts (NDJSON default + CSV with core's
formula-injection escaping) + AiAuditReader.exportStream() keyset pager, in
(tenant_id, seq) chain order so the file is SELF-VERIFIABLE by an external re-walk (it
carries seq/checksum/prevChecksum). New tenant:ai:audit:export command, fail-closed
on a write error.

3.3 Checkpoint-aware verify. verify() gains an optional per-tenant seed + a seq > ?
lower bound (single-tenant seeded read cost ∝ tail, not all history); a pure fold-seed
change — no stored row is ever rewritten. A seeded walk is byte-for-byte as strong as
a full walk over its tail.

3.4 Retention. New tenant:ai:audit:archive (the default path, PRUNES NOTHING):
verify → export to WORM/SIEM in chain order → advance a signed checkpoint in the new
additive ai_audit_checkpoints table (migration stub 0002). Physical DETACH-PARTITION
pruning stays designed-but-off-by-default (open decision #3), not wired here.

3.5/3.6 Alerting. New ai_audit_anomaly_watcher.ts: sliding-window guard-trip velocity
per (tenant, principal, guard), fail-OPEN and off the request path, bounded key map,
fires guard.ai_anomaly + fans a content-free summary to onAnomaly or the host audit
destinations; wired in provider ready()/shutdown(). reportScheduledVerify raises a
found break to guard.ai_audit_chain_broken (critical). config.ai.audit.verify.schedule
is a VALIDATED cron field (the host-owned `... || alert` line stays the recommended path).

3.7 Correctness: toAuditLogEntry now anchors the first-class 'ai' actor, not 'system'.
3.8 Cleared the stale audit_seam TODO(WS-AI-7) docblock.

Two new guards (guard.ai_audit_chain_broken critical, guard.ai_anomaly high, fail-open)
with registry literals, emit sites, emission-matrix recipes, no_silent green (the reader
is allowlisted: it emits the already-registered ai_scope_mismatch).

Gates: ai unit 826, integration 58/4-skipped, fault 21, build:all + typecheck 0,
npm run check 51 guards, eslint/prettier clean. NOTE: 3.7 references core's uncommitted
AuditActorType 'ai' widening (Karimoto work in the working tree). Docs updates remain.
…t typecheck errors

Two defects shipped in the audit-consumption work, both gate-blockers unrelated
to the data-at-rest wave that follows:

- The retention-checkpoint migration stub (Wave 4, b9081b2) reused ordinal 0002,
  colliding with the action-ledger stub (499cade), and its comment carried raw
  backticks tempura parses as a template literal ("Unexpected identifier
  'tenant'"). So `node ace configure` could neither order the batch
  (check-migration-order) nor render the stub (check-stub-render). Renumber it to
  0003 and drop the backticks.

- Four test-fake type errors the tests-covering typecheck config surfaces: three
  fakes return a value where a void `EmitMetric` is expected, and the audit-reader
  fake uses a self-referential `typeof client` that infers implicit any. Give the
  fakes void bodies, a typed emitter handler, and an explicit client interface, so
  `npm run typecheck` is green.
…tent encryption, gated default-off (Wave 5)

Ships the two at-rest seams and their validated config, both DEFAULTING to
exactly today's behavior; enabling either on a host is a separate go. crypto is
an OPTIONAL peer, resolved by dynamic import (the admin->sso pattern), and boot
fails CLOSED when an at-rest DEK is selected without it installed — never a
silent fallback to the fleet APP_KEY.

Memory (conversation_memory_service): encryptMemory/decryptMemory/
decryptMemoryPrevious become async and carry a leading tenantId, rippling
#decodeExchange + load. The default 'app-key' mode wraps writeSecret/readSecret
in Promise.resolve (byte-identical), proven by the existing memory regressions
staying green. The 'tenant-dek' mode seals under crypto's per-tenant DEK
(subject=tenantId, category ai:conversation-memory) via its fail-closed
encryptField/decryptField facade; a shredded DEK degrades a read to empty
(crypto-erase, fail-safe), a KeyProvider outage surfaces the distinct
ai_memory_dek_unavailable metric, and the OLD_APP_KEY grace is retired on that
path (KEK rotation lives in the KeyProvider).

Embeddings (vector_store_service): optional sealContent/openContent + encrypt
Content/encryptMetadata flags. insert() seals content (and metadata via
to_jsonb(?::text)) after provisioning the tenant DEK once, sequentially, before
the fan-out — crypto's provisioning lock is fail-open, so N concurrent first-time
seals would race and collide on the partial UNIQUE. content_hash is untouched
(it hashes caller plaintext upstream, so dedup survives), and a metadata
serialization failure keeps its prior fatal propagation rather than a seal 503.
search() opens the O(limit) returned rows and drops an un-openable one fail-safe
(ai_embedding_content_undecryptable). A metadata-scoped retrieval over encrypted
metadata is refused fail-closed with the new guard.ai_embedding_metadata_scope_
conflict; a seal failure fails the ingest closed (embedding_seal_failed, 503),
writing no plaintext.

Adds two error codes (embedding_metadata_scope_conflict fatal 400,
embedding_seal_failed retryable 503), one guard, and red-first acceptance tests
(default-off parity, tenant-scoped seam + shred degrade, encrypted insert/search,
metadata-scope refusal, seal-failure fail-closed). Honest residual: the plaintext
vector stays ~invertible (physical isolation remains the real embedding control),
and the design's LRU DEK cache + SubjectShredded invalidation are deferred to the
enablement go (a perf optimization needing a crypto-side emitter event; without a
cache, erasure is trivially correct — a shred reads empty).

Gates: ai unit 844, integration 58/4-skip, fault 21, typecheck 0, check 51 guards.
Cover the gated, default-off at-rest options in the AI guide and the
threat model: the per-tenant conversation-memory DEK (memory.encryption),
embeddings content-at-rest (embedding.encryptContent / encryptMetadata),
the new guard.ai_embedding_metadata_scope_conflict and the two metrics,
plus the honest blast-radius and vector-inversion limits. Documents what
shipped (no in-process DEK cache yet) in the docs' existing voice.
Sweep the satellite's comments, JSDoc, and test titles/messages so they
read like a teammate wrote them: drop em-dashes and the (WS-AI-N) /
(Wave N) / (G2) / (#N) cross-reference tags, and turn acceptance-checklist
docblocks into plain prose that explains the why in accessible English.
Comment, title, and operator-message text only; no behavior, identifiers,
or asserted values changed. The guard registry's evidence.ref keeps its
design provenance, since that field exists to cite a guard's origin.
Add the branch to the push trigger so the full suite runs while it bakes.
Revert this line before merging to master.
@Arcoders
Arcoders changed the base branch from master to LASAGNA-020626/isolation-hardening-and-benchmarks July 25, 2026 13:43
@github-actions

Copy link
Copy Markdown

docs:doctor

✓ Tier 1 (gate): 0 dead member(s) in prose, coverage floor met
⚠ Tier 2 (advisory): 19 review item(s)
  - docs/guides/bootstrappers/index.md (-> saas-tenancy/services#BootstrapperRegistry): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/cookbook/custom-isolation-driver.md (-> saas-tenancy/services#IsolationDriverRegistry): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/cookbook/per-tenant-worker-concurrency.md (-> saas-tenancy/services#TenantQueueService): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/health.md (-> saas-tenancy/services#CircuitBreakerService): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/jobs.md (-> billing#ProcessBillingEventJob): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/jobs.md (-> saas-tenancy/jobs#InstallTenant): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/plugins.md (-> saas-tenancy/services#CapabilityRegistry): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/plugins.md (-> saas-tenancy/services#HookRegistry): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/satellites/impersonation.md (-> saas-tenancy/services#ImpersonationService): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/satellites/metrics.md (-> saas-tenancy/services#MetricsService): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/satellites/quotas.md (-> saas-tenancy/services#QuotaService): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/satellites/reporting.md (-> saas-tenancy/services#MetricsService): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/scheduler.md (-> saas-tenancy/jobs#TenantJob): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/tenant-identification.md (-> saas-tenancy/services#SyncTenantResolver): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/tenant-identification.md (-> saas-tenancy/services#TenantResolverRegistry): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/guides/testing.md (-> saas-tenancy/services#BootstrapperRegistry): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/reference/services.md (-> saas-tenancy/services#QuotaService): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/reference/upgrade-to-0.3.md (-> backup#CloneService): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
  - docs/start/tutorial/reporting.md (-> saas-tenancy/services#MetricsService): contract changed since the doc was last reviewed
      Action: re-review the doc, then run docs:doctor --update-freshness.
      Suppress: <!-- doc:freshness-ignore reason="..." -->
Coverage: explained 95%, exemplified-only 0%, uncovered 5% (346 public symbols)

Arcoders added 5 commits July 25, 2026 15:57
Wave 5 added `typeof import('@adonisjs-lasagna/crypto')` to the ai
provider, so the ai build now needs crypto's declarations. Both the
root build:all chain and the ci.yml satellite-build step ordered
build:ai before build:crypto, which fails a clean build with TS2307
(it only passed locally because a stale crypto build/ was on disk).
Move build:crypto ahead of every satellite that may depend on it.
The AI satellite attributes an assistant-driven action to a first-class
actorType: 'ai' (the staff principal it acted for is the row's actorId),
rather than folding it into 'system' or a metadata flag, so "the assistant
did this" stays queryable and exportable. Widen AuditActorType to include
'ai' and cover the round-trip to a persisted actor_type='ai'. The ai
package's committed audit writer already depends on this actor.
These core and benchmark files landed in the Isthmus hardening and
tenant-doctor waves before CI ran on this branch, so their prettier
violations were never caught. Formatting only, no behavior change;
brings `npm run lint` back to green.
Wave 4 registered `tenant:ai:audit:export` and `tenant:ai:audit:archive`
but never added them to the CLI reference, which the commands-documented
integrity spec enforces. Add both rows so every registered ace command is
documented.
apps/rental is a workspace but was never added to knip.json, so knip
scanned none of its source and flagged every declared dependency as
unused, failing the knip:deps gate. Add its entry points (adonisrc,
bin, start, config, database, app, commands, inertia, tests) so knip
sees the imports and gates the app like the other workspaces.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant