feat(backend): W11-store and prove every byte the platform fetches for a manifest

[lucas-xzp]

Problem

§2.8 of the bot-config-manifest plan is a hard requirement from audit and reconciliation: content the platform fetches on a manifest's behalf must be kept as the platform's own durable copy, and every step after fetch must read that copy — retried delivery re-reads here instead of re-fetching, a source-side fault can never pollute a delivery in progress, and keep_last has somewhere to live. The backend has no such store today (issue #1510, work item W11; builds on W2's merged fetcher — zero remaining dependencies, no coordination).

Solution

core/bot_config_manifest/content/ — one mechanism behind all three consumers of §2.8 (audit, delivery, keep_last), not a copy per consumer:

• Bytes live in a content-addressed blob tree — <root>/blobs/<hex[:2]>/<hex64>. The digest (sha256:<hex>, the fetcher's own vocabulary, now shared as fetch/limits.DIGEST_RE) IS the address: identical bytes are written once, ever; the write is atomic (temp + os.replace in the shard directory) and heals truncation/append corruption beneath an existing address while the correct bytes are in hand. Not in the database — schema §5 lets one entry be 100–200 MiB, and a column holding that is a self-destructing design.
• ac_manifest_content is the append-only provenance log, one row per store event: the bot axes (avernet_tenant, env, entity_id, bot_id) behind the same tenant guard as W1's table, the digest, both URLs (entry source after ${BOT_*} substitution, and the final hop — differing values mean a redirect happened), the credential name only (W3's identifier; a value never crosses into this layer), size, fetch time, modifier. No unique key: fetching the same digest twice is two audit events, not one row to overwrite.
• read(digest) is the single read path, shared by delivery and audit: read in chunks with the hash on the same pass, returned whole (peak ≈ 2× the blob at the §5 cap — a stated v1 trade; the consumer materialises the full payload anyway) — the store returns bytes it can prove or fails loudly; a missing address is terminal and the store never re-fetches (§2.8's decoupling). keep_last will be W4's apply-record digest plus this read — one addressing, no second copy.
• Every refusal lands before any side effect: receipt-vs-bytes (re-hash + size), URL sanitization, and column-width validation (a wire-controlled Content-Type header has no length cap of its own, and SQLite tests cannot catch varchar overflow — strict mode would surface a 500 after the blob landed; non-strict would silently truncate an append-only audit row forever) all happen before the blob write. URLs in provenance carry scheme://host[:port]/path — never userinfo, never a query string (where signed-source tokens live); the reconciliation anchor is the digest.
• Retention is stated against the audit requirement, not defaulted: v1 retains rows and blobs unconditionally; a window, when audit names one, is a DDL-comment change plus a sweep in that PR.
• The blob root is config-borne (user_config.bot_config_manifest.content_store_dir, neutral ./data/manifest_content; an env overlay decides NAS), parsed purely by content.settings.content_store_root_from_config for W4's composition root to inject — core reads no environment, per the ruling on W2's allowlist. DI binds the repository protocol now; the service is a declared machine part in the same shape as the fetcher before its orchestrator existed.

Validation

• 37 tests in the new package (store contract matrix + repository over real SQLite incl. cross-tenant isolation on a shared bot_id + config seam), pinning: roundtrip, receipt-mismatch refusal with zero side effects, corrupted-blob and missing-address behavior on the read path, write-once dedup with two provenance rows, URL sanitization (query/userinfo/fragment/port/path), name-only credentials, tz normalization, width refusals, empty-payload roundtrip, orphan-blob self-heal.
• Batch review (per the review-before-push rule): 7 findings — 1 HIGH (width validation before the blob write: SQLite tests structurally cannot catch it) + 3 MEDIUM (write ordering, the dedup shortcut's blind trust, error messages echoing unparseable URLs) + 3 LOW — all resolved in this PR; the ledger is in the spec.
• Full backend suite after a clean rebase onto the day's dev (which gained docs(backend): deliver the cross-engine convergence contract, declare cli_tools in the artifact (W12 + part of W9) #1734/feat(task): unify static-plan execution and harden callback recovery #1745/fix(aicoding): 应用 Bot 多工作流完整物化 AIX_DEVFLOW_INFO_LIST 到容器 #1559/chore(skills): add runtime projection latency logs #1756 mid-flight; zero file overlap): 16261 passed, 59 skipped; line coverage 88.14%; changed-line coverage 97.84% (226/231, ≥80%) — reproduced locally via scripts/ci_test.sh.
• Architecture gates green: boundaries/oversized/user_config AST guard/shipped-config/repository-contracts provides (both READMEs updated)/flow_coverage entry extended to cover W11; golden effective-config snapshots regenerated for the deliberate new key (all three profiles gain exactly one neutral line).

Spec

src/backend/specs/2026-08-31-w11-content-store/ — spec (acceptance map + design decisions + the batch-review ledger), plan, tasks.

Closes #1510.

[totalfrank]

Reviewed against W11's acceptance list in docs/bot-config-manifest/work-items.zh-CN.md and against the merged waves (W1, W2, W3, W4, W10). I checked the branch out, installed the suite and ran it: the 43 new tests pass, and tests/community/architecture + core/bot_config_manifest + config is 571 passed, so nothing here is broken as built. The blob layer is a good mechanism — content addressing, atomic write, verify-on-read, refusals before side effects, credential name only — and the compatibility work is genuinely done: the tenant guard follows W1's pattern (and works from inside W4's apply thread, which re-binds the tenant contextvar at wrap time), DIGEST_RE is shared with W2 rather than copied, the 128-char credential_name matches W3's ac_source_credential.name, and ContentScope's axes match W4's ApplyContext.

Findings inline, roughly in the order I'd fix them.

Schema, and the reason it can't wait (comments on the CREATE TABLE and on the header). The retention policy makes this table a one-shot decision: rows are never updated and never deleted, so a column added later is permanently NULL for everything written before it, with no backfill this policy permits. Two things are missing while it's still free to add them:

• No apply_id and no entry identity. W4's apply_models.py:95 literally describes apply_id as "what a per-entry table would join on, if provenance ever needs one" — this is that table. And §2.7's keep_last is per entry, while the only per-entry-ish column here is source_url, stored sanitized, so two entries differing only in query string collapse and an entry whose source moved loses its own history. The README's answer (per-entry digests in the apply report's JSON) is the second addressing that the 一套机制，不是两套 acceptance line is trying to prevent.
• A row means "fetched", not "delivered". Under §3.2's all-or-nothing category overwrite an entry can be stored here and never materialised. That's fine as a mechanism, but the DDL header, the README section and the service docstring all say "what this bot received", and that over-reports.

Correctness / robustness: IPv6 sources land in the audit log as unparseable URLs (https://[2001:db8::1]:8443/x → https://2001:db8::1:8443/x, verified on this branch); ContentIntegrityError subclasses the documented 400-class base, so the obvious mapping reports disk corruption as a caller error; no fsync before os.replace, which is the difference between an atomic rename and a durable copy, and matters here precisely because this layer never re-fetches; .tmp-* files survive a crash forever under a no-sweep policy.

Boundary placement: a >256-char Content-Type or a >2048-char URL discards a fully fetched and verified 200 MiB payload at the last step. Nothing caps URL length at PUT (the DDL's "doc-wide URL ceiling" doesn't exist in the repo), so a document W1 accepted can only fail at apply — which crosses this module's own stated rule, "This surface never accepts something it cannot apply."

Nits: DIGEST_RE's $ matches a trailing newline, now that it's the shared address vocabulary; records_for's negative-limit clamp answers "no receipts" for an audit read; idx_digest has no reader; the source_ref column named in the DDL comment doesn't exist; the dedup shortcut's stated cost model is off.

Two things not worth inline comments but worth a look before W5 wires this up:

1. The blob root is node-local by default while the provenance log is shared. ./data/manifest_content resolves against the process CWD, so with more than one backend replica a blob written on node A is invisible to a delivery retry that lands on node B — and by design that's ContentMissingError, terminal, never a re-fetch. The yaml comment offers a NAS overlay as an option; given this store's contract, W5's wiring should probably make the shared volume a stated deployment requirement (or the read path's missing case a documented "re-fetch is the caller's decision"), not a possibility an overlay may take up.
2. When W5 constructs the service, a ContentScope.from_apply_context(ctx) would keep the three duplicated axes from drifting apart from ApplyContext.

Nothing here is a blocker on the mechanism itself — the schema points are the ones I'd want settled before the table exists in prod.

---

Generated by Claude Code

[totalfrank]

The axes here are the bot's, and nothing ties a receipt to the apply or to the entry it was fetched for. Two consequences worth settling before this table ships, because its own retention policy (never updated, never deleted) means a column added later is permanently NULL for every row written before it — there is no backfill this table's stated policy permits.

1. W4 already ships the join key, and this table doesn't take it. ac_bot_config_manifest_apply.apply_id carries the comment "also what a per-entry table would join on, if provenance ever needs one" (repository/apply_models.py:95). This is that table. Without apply_id, "what did apply X deliver" is unanswerable, and receipts from a retried apply interleave with the original's under nothing but gmt_create.

2. keep_last is per entry, and an entry has no identity here. work-items §2.7: 「keep_last means 'reuse what we materialised for this entry last time'」, and the §2.8 acceptance line is that this store is the one keep_last reads — 一套机制，不是两套. The only per-entry-ish column is source_url, and it is stored sanitized: two entries whose sources differ only in the query string collapse to one value, and an entry whose source moved (the very case keep_last covers) no longer matches its own history. The README's answer — "the per-entry digests that wave's apply records carry" — puts the entry→digest mapping in the apply report's JSON blob, i.e. a second, unindexed addressing outside the store, which is what that acceptance line is trying to prevent.

A nullable apply_id plus category + entry_identity (W4 already computes EntryResult.identity) costs nothing now and turns both questions into indexed reads.

Related, same area: idx_digest is justified as "which fetches ever produced this content", but the repository exposes no query that can use it — records_for is bot-scoped only. Either add the read or drop the index until a reader exists.

---

Generated by Claude Code

[totalfrank]

Worth being precise about what one row means, because this header and the README both promise slightly more than the code delivers.

A row is a fetch/store event, not a delivery. Under §3.2's all-or-nothing category overwrite, an entry can be fetched, verified and written here and then never materialised — because a sibling entry in the same category failed and the whole category was skipped. So 「这个 bot 到底收到了什么」 read off this table alone over-reports: what it actually answers is "what the platform fetched on this bot's behalf".

Not necessarily a schema change (the apply report holds the outcomes) — but the DDL header, README.md's new W11 section and the service docstring all say "what this bot received", and a future auditor will read those, not this thread. Either reword them, or carry the delivery outcome — which is another argument for the apply_id link in my comment on the CREATE TABLE below.

---

Generated by Claude Code

[totalfrank]

An IPv6 source is written into the audit log as a URL nothing can read back. httpx.URL.host returns an IPv6 host unbracketed, so the reassembled authority loses the brackets and the port becomes ambiguous:

>>> # httpx 0.28.1, run against this branch's service
>>> svc.store(fetched_from("https://[2001:db8::1]:8443/a.bin"), ...).fetched_url
'https://2001:db8::1:8443/a.bin'

It is reachable, not theoretical: _validate_url accepts an IPv6 literal, _refused_address only refuses non-global addresses, and _collect sets FetchedObject.url = str(url) with the brackets intact — this function is where they are lost. And by this table's own policy the row is never updated and never deleted, so it can't be corrected afterwards.

host = f"[{parsed.host}]" if ":" in parsed.host else parsed.host
authority = f"{host}:{parsed.port}" if parsed.port is not None else host

plus an IPv6 case in test_stored_urls_carry_path_but_never_query_or_userinfo, which currently parametrises query/userinfo/fragment/port/path but no literal address.

Two smaller fidelity notes on the same function, since the log is append-only and this is the moment to decide: httpx.URL also decodes punycode (xn--fsq.example → 例.example) and unquotes the path (/%2e%2e/x → /../x), so what lands in provenance is httpx's re-rendering rather than the bytes that went on the wire.

---

Generated by Claude Code

[totalfrank]

Now that this is the shared address vocabulary and not just the fetcher's private check, the $ deserves tightening: in Python $ also matches immediately before a trailing newline.

>>> DIGEST_RE.match("sha256:" + "a"*64 + "\n")
<re.Match ...>   # matches

So sha256:<hex>\n passes _require_valid_digest, and read() answers ContentMissingError ("the digest names nothing", the 404 case) instead of ContentStoreError("untrusted content address") — the two mean different things to a caller — and the newline rides into the error message and from there into a log line. Same at the other call site: GuardedFetcher.fetch accepts a newline-suffixed expected_digest as a valid pin and then fails it later as FetchFailedError("digest mismatch") rather than refusing it as malformed config.

re.compile(r"\Asha256:[0-9a-f]{64}\Z") (or switching both call sites to .fullmatch) closes both. No traversal risk either way — the charset is hex — so this is a taxonomy/hygiene fix, not a security one.

---

Generated by Claude Code

[totalfrank]

The hierarchy contradicts the docstrings. ContentStoreError says "(400-class)", this one says "(500-class)", and it subclasses the first:

>>> issubclass(ContentIntegrityError, ContentStoreError)
True

So the obvious mapping in W5's consumer — except ContentStoreError: -> 400 — reports platform-side disk corruption as a caller error, which is backwards for the one failure in this file that should page someone. A caller has to know to catch the subclass first, and nothing here says so.

Cleanest is a sibling base (ContentStoreError(ValueError) for the 400 class, ContentStoreFault(RuntimeError) for integrity); minimum is a line in this docstring stating that the mapping must catch ContentIntegrityError before its base. ContentMissingError under the 400 class is fine.

---

Generated by Claude Code

[totalfrank]

Refusing the whole store() over content_type is the wrong severity for that field, and it hands the source side a lever. The bytes are already fetched, streamed under the cap, hashed and verified; a Content-Type of 257+ characters — nothing between the wire and here caps it at 256 — throws all of that away. content_type is advisory metadata, not a reconciliation anchor (the digest is, as the DDL says). Truncating it, or storing NULL plus a log line, keeps the receipt and loses nothing that matters.

The URL cap is the same shape with a bigger blast radius, and it crosses a rule this module states about itself. README: "This surface never accepts something it cannot apply." Nothing caps URL length at PUT — schema/limits.py deliberately carries only the write-time limits and neither it nor entries.py bounds a source URL — so a manifest with a 3000-character source is accepted, validates, fetches up to 200 MiB, and is refused only here. (The DDL's "2048 chars is the doc-wide URL ceiling", line 66, describes a ceiling that doesn't exist anywhere in the repo; the only real bound is the 64 KiB document cap, which leaves plenty of room.) A redirect Location longer than 2048 does the same after a successful fetch.

Refusing at the store means the expensive work is always done first. The rule-preserving fix is a URL length limit at W1's admission (cheapest, and it keeps that README sentence true); the second-best is W2 refusing an over-long hop before it streams a body. Keeping the check here as the last line of defence is right — it just shouldn't be the first place the length is noticed.

---

Generated by Claude Code

[totalfrank]

os.replace buys atomicity of the rename, not durability of the data. With no fsync on the temp fd before the replace (and none on the directory after), a power loss can leave the blob visible at its content address holding zero or partial bytes. That is exactly the case §2.8 exists to prevent — 「重试下发绝不重新拉取」 — and this layer never re-fetches, so an in-flight retry cannot recover; only a new apply that fetches the same bytes again heals it, and only because the size check on line 305 happens to catch a truncated file. For a store whose whole claim is "the platform's own durable copy", two lines seem worth it:

with os.fdopen(fd, "wb") as handle:
    handle.write(data)
    handle.flush()
    os.fsync(handle.fileno())

(plus an optional directory fsync if you want the rename itself to survive too).

Crash path, same block: a process that dies mid-write leaves a .tmp-* file of up to 200 MiB in the shard directory forever — tmp.unlink(missing_ok=True) only covers in-process exceptions, and the retention policy is "no delete, no sweep, no TTL", so nothing will ever collect them. Either sweep stale .tmp-* in the shard dir on store, or state in the retention comment that temp files are the one thing a future sweep may remove — otherwise the next reader will correctly conclude the policy forbids it.

---

Generated by Claude Code

[totalfrank]

Nit on the reasoning rather than the decision, which may well stand: "a full re-hash of a 200-MiB blob on every re-store would tax every audit fetch" — a re-store is not an audit fetch, and read() re-hashes on every single call anyway, so the audit path already pays that cost unconditionally. The real trade is stat-vs-read on the dedup path only. Worth rewording so the next reader doesn't inherit the wrong cost model and "optimise" read() on the strength of it.

Worth noting alongside: on this shortcut store() returns success, and inserts a provenance row asserting the content is stored, without ever having verified the bytes on disk — while holding the correct bytes in hand. Fine as a stated trade; just make sure the docstring's "hands back bytes it can prove, or it fails" is understood as a read() guarantee only, never a store() one.

---

Generated by Claude Code

[totalfrank]

max(0, limit) turns a negative limit into "return nothing", which for an audit read is the quietest possible wrong answer: a caller that computes limit = total - offset and goes negative is told this bot has no receipts, which in an audit context is a claim, not an empty page.

The protocol docstring's stated goal — a negative can never mean "everything" — is met just as well by refusing (ValueError), and refusing can't be mistaken for a fact about the bot. Same argument for limit == 0.

---

Generated by Claude Code

[totalfrank]

This sentence names a column the table doesn't have — there is no source_ref anywhere in the DDL below. As written it reads as "the column exists and W7 will populate it", and someone will grep for it.

It's also the one acceptance line of #1510 that isn't fully met: 「来源、解析出的 ref/SHA 或 digest、拉取时间，以及字节本身」. For https sources the digest is the whole answer, so v1 is defensible — but say that as a decision ("git refs land with W7, which adds the column then") rather than as a reference to a column that exists. And given this table's own never-update/never-delete retention, add the consequence out loud: every row written before W7 is permanently NULL for that column, with no backfill this policy permits.

---

Generated by Claude Code