changelog: catch the log up on the design-system floor and the NATS outage

Two entries the log was missing. 2026-08-06_01 was written mid-flow and four
more things shipped the same day after it; the NATS fix shipped 08-08 with no
entry at all.

2026-08-06_02 leads with gate A20, because it is the one a reader outside the
team can learn something from. Seventeen micro-frontends inherit design tokens
from the shell at runtime, and nobody had asked what happens to a member that
deploys before the shell catches up. The answer was the worst available one:
not an error, not a fallback, but black text on a transparent background,
reported as success, and invisible locally because the developer's machine
always has the latest theme. The entry shows the browser measurement including
the counterfactual, because "the degraded case looked fine" proves nothing on
its own. Carries the three smaller items behind it — React evicted from the one
config line that was its entire footprint, the shell's typecheck passing for the
first time, and five services that had all named a function registerHandlers.

2026-08-08_01 is the debugging story. It leads with the eliminations rather than
the fix, since seven wrong answers are the part a reader can reuse, and it owns
the one that was our own instrument-misreading rather than quietly dropping it.
The two broker counters that collapsed the search space are the centrepiece.

Both follow the audience cascade — ledes kept to subtitle length with the
context moved into Why Care?, marketing voice up top dissolving into engineering
detail further down, and a What's Next? that states plainly what did NOT get
fixed: 112 instances of the same idiom remain, and the shared-package fix for
them is gated on a Dockerfile change.

Files changed:
- changelog/2026-08-06_02_Federated-Tokens-Get-A-Floor-And-React-Finally-Leaves-The-Building.md (new)
- changelog/2026-08-08_01_The-Corpora-Were-Never-Missing-One-Bad-Message-Had-Killed-The-Subject.md (new)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019a8tSPbFdvF1pKtADnWyDg

Author: mpstaton

changelog/2026-08-06_02_Federated-Tokens-Get-A-Floor-And-React-Finally-Leaves-The-Building.md

@@ -0,0 +1,123 @@
+---
+date_created: 2026-08-06
+date_modified: 2026-08-08
+title: "Federated tokens get a floor — and React finally leaves the building"
+lede: "Ask a hard question about a micro-frontend architecture and you sometimes learn the answer is 'it renders invisible text and tells you everything is fine.' We measured that failure in a browser, then gave every design token a floor so it can't happen."
+publish: true
+authors:
+  - Michael Staton
+augmented_with:
+  - Claude Code on Claude Opus 5
+files_changed:
+  - packages/theme/token-baseline.css
+  - packages/federation/src/index.ts
+  - scripts/generate-token-baseline.mjs
+  - tsconfig.json
+  - tsconfig.base.json
+  - shell/src/css.d.ts
+  - shell/src/App.svelte
+  - services/record-surrealdb-resolver/src/handlers.ts
+  - services/row-store/src/handlers.ts
+tags:
+  - Augment-It
+  - Design-System
+  - Module-Federation
+  - Design-Tokens
+  - Accessibility
+  - Refactor
+---
+
+# Federated tokens get a floor
+
+## Why Care?
+
+augment-it is seventeen micro-frontends that load into one shell at runtime, each built and deployed on its own schedule. That independence is the whole point — but it raises a question we hadn't answered: **the shell owns the design tokens, so what happens to a micro-frontend that ships before the shell catches up?**
+
+The honest answer turned out to be *the worst possible one*. Not an error. Not a fallback. The component renders **black text on a transparent background** and reports success. In production. And you can't see it locally, because your machine has the latest theme.
+
+That's now impossible. Every design token has a floor underneath it.
+
+## What's New?
+
+- **A20 closed** — every federated token is registered with `@property` and an initial value, so a missing or stale shell degrades to a legible dark surface instead of nothing
+- **`token-baseline.css` is generated, not hand-written** — `pnpm tokens:baseline` derives it from `theme.css`; `pnpm tokens:check` fails CI if they drift
+- **Federated bundles got 3KB smaller** — they carry a 24-token floor instead of the whole theme
+- **React is gone** — one config line was its entire footprint, in a codebase that bans it
+- **The shell's typecheck passes for the first time**
+- **Five services stopped sharing a function name** — `registerHandlers` was ambiguous five ways
+
+## The question that started it
+
+> *"How do we make sure the microfrontends inherit the global federated classes so there's no failure when one thing breaks somewhere?"*
+
+Good question. CSS custom properties inherit down the document, so a member picks up whatever the shell declares. Fine when everyone's current. But token **removal** and token **introduction** are not symmetric:
+
+| Change | Old member, new shell | New member, old shell |
+|---|---|---|
+| **Remove** a token | Alias it forever. Fine. | n/a |
+| **Add** a token | n/a | **Resolves to nothing.** |
+
+"Nothing" is not a metaphor. An undeclared custom property is *invalid at computed-value time* — `color` falls back to inherited, `background-color` to transparent.
+
+## We measured it rather than argued about it
+
+Rather than reason from the spec, we drove the real built stylesheets through a browser in three configurations:
+
+```
+healthy (shell theme + member bundle)
+  tokens resolve to the shell's values · all three modes distinct ✅
+
+degraded (member bundle ONLY — a stale shell)
+  #e8eaf0 on #13151b · contrast 15.17:1 · neither transparent ✅
+
+counterfactual (a token neither declared nor registered)
+  rgb(0,0,0) on rgba(0,0,0,0) ← black text, transparent background
+```
+
+That third line is the bug, reproduced. It's also why we ran it: without the counterfactual, "the degraded case looked fine" proves nothing — it might have worked for unrelated reasons.
+
+## How the floor works
+
+CSS has had the answer since `@property` shipped. Registering a custom property gives it a **typed initial value** that applies when nothing declares it:
+
+```css
+@property --color-surface {
+  syntax: '<color>';
+  inherits: true;
+  initial-value: #13151b;
+}
+```
+
+Now `var(--color-surface)` can never resolve to nothing. Worst case it's the floor. The shell's real declarations still win whenever they exist — so this costs exactly nothing when the stack is healthy, and saves you when it isn't.
+
+The shell stays the canonical source. Members carry only the floor. Standalone entries still load the full theme, because they have no shell to inherit from.
+
+### Generated, because a hand-maintained copy would drift by Thursday
+
+```bash
+pnpm tokens:baseline   # regenerate from theme.css
+pnpm tokens:check      # CI: fail if stale
+```
+
+The generator flattens each token through its Tier-1 reference to a literal — `initial-value` forbids `var()` — and picks `<color>` or the universal syntax per token. That last part matters: registering a box-shadow as `<color>` invalidates the whole rule and CSS drops it *silently*. Zero dependencies, pure Node.
+
+## Under the hood: three smaller things
+
+**React was one line.** The root `tsconfig.json` set `"jsx": "react-jsx"` — the only React reference anywhere in a repo where React is a hard prohibition, with zero `.tsx` files and no `react` dependency. Scaffold vestige that survived because nothing extends that file, so nobody ever read it. Now `"jsx": "preserve"`: TSX stays legal, no runtime is bound.
+
+**The shell's typecheck had never passed.** Two errors, and the interesting part is *why nobody fixed them*: the shell needed a `css.d.ts` shim, but its tsconfig omitted `src/**/*.d.ts` from `include` — so adding the shim wouldn't have helped. Two halves of one bug. Fixing it also revealed the shell as the **eighteenth** copy of a tsconfig we'd converged earlier that day.
+
+**Five services, one function name.** `registerHandlers` existed five times, once per NATS service, and three of them ranked in the codebase graph's top ten by connectivity — the most-connected symbols in the system were also the least searchable. Now service-qualified.
+
+That last one came with a lesson: our refactor doc said there were *three*, because three was what the graph's top-ten showed. God-node rankings report the top of a distribution, not a census.
+
+## Verified
+
+- 19 packages build · **1,587 files typecheck clean** (up from 1,494 with the shell failing outright) · all suites pass
+- `record-collector`'s built CSS is **byte-identical** before and after the federation change — same content hash
+
+## What's Next?
+
+The floor covers the 24 tokens that exist today. Members can still invent tokens *after* a baseline is generated, so the 939 `var()` calls with no inline fallback are worth sweeping — belt and braces.
+
+And the bigger one: 1,386 of 1,392 CSS selectors are unique to a single app. There's almost nothing to deduplicate; there are seventeen independently invented vocabularies. The component library is greenfield extraction, not consolidation.

changelog/2026-08-08_01_The-Corpora-Were-Never-Missing-One-Bad-Message-Had-Killed-The-Subject.md

@@ -0,0 +1,139 @@
+---
+date_created: 2026-08-08
+date_modified: 2026-08-08
+title: "The corpora were never missing — one bad message had killed the subject"
+lede: "A client's research corpora vanished from the workspace. The data was fine, the database was fine, the service was healthy and logging nothing. Two counters found it: a single unanswerable message had taken the whole request channel down until restart."
+publish: true
+authors:
+  - Michael Staton
+augmented_with:
+  - Claude Code on Claude Opus 5
+files_changed:
+  - services/record-surrealdb-resolver/src/nats-loop.ts
+  - services/record-surrealdb-resolver/test/nats-loop.test.ts
+  - services/record-surrealdb-resolver/src/surreal.ts
+  - services/record-surrealdb-resolver/src/domains.ts
+  - context-v/issues/One-Stuck-Message-Kills-A-NATS-Subject-Until-Restart.md
+tags:
+  - Augment-It
+  - NATS
+  - Resilience
+  - Debugging
+  - Error-Handling
+  - Microservices
+---
+
+# The corpora were never missing
+
+## Why Care?
+
+A workspace opened and its research corpora weren't there. Nine of them — months of collected reading on adult literacy, workforce development, rural income — gone from the surface. The console said:
+
+```
+❯ domain.list: timeout
+```
+
+The obvious readings are all wrong. The data hadn't been lost. The database was reachable in about a second. The service was running, healthy, `RestartCount=0`, and writing **nothing at all** to its logs.
+
+This is the failure mode that should scare you most in a message-driven system: **silent, total for one channel, and invisible everywhere you'd normally look.** Every dashboard is green. Only the replies stop.
+
+It's fixed, it has regression tests, and the diagnosis is written down so the next person doesn't re-walk it.
+
+## What's New?
+
+- **`domain.list` answers again** — reach-edu's 9 corpora, humain-vc's 7, 16 unfiltered
+- **A hardened consumer** (`nats-loop.ts`) that can't be killed by one bad message
+- **A bounded database handshake** — the unbounded one is what parked the whole channel
+- **9 regression tests** for failure modes the old suite structurally could not reach
+- **A written diagnosis** listing the seven causes we ruled out, so nobody re-walks them
+
+## The seven things it wasn't
+
+Most of debugging is elimination, and the eliminations are the useful part:
+
+| Suspected | Verdict |
+|---|---|
+| The client's data was lost | **No.** All 9 in the database |
+| Something client-specific | **No.** Every client timed out, including no filter |
+| A rename we'd shipped that week | **No.** Different file, untouched |
+| Service crashed | **No.** Up, zero restarts, clean boot |
+| Database unreachable | **No.** Connect → sign in → query in **1,107ms** |
+| A full-table migration on cold start | **No.** Measured **538ms** |
+| Nothing subscribed to the channel | **No** — and this one was *our* mistake |
+
+That last row is worth owning. We read the message broker's monitoring output, saw an empty `subs_detail` array, and concluded nothing was listening. Wrong key. The real one is `subscriptions_list_detail`, and the subscription was there all along. A misread instrument sent us down a blind alley for a while.
+
+## The two numbers that found it
+
+The broker tracks messages in and out per connection. Take a reading, send one request, take another:
+
+```
+BEFORE: in_msgs=4 out_msgs=15
+PROBE:  timeout
+AFTER:  in_msgs=4 out_msgs=16
+```
+
+`out_msgs` went up — the broker **delivered** the message. `in_msgs` didn't — the service **replied with nothing**.
+
+So the message arrived, was consumed, and vanished. That single observation collapses the search space: it isn't networking, it isn't the database, it isn't the data. It's the code between receiving a message and answering it.
+
+## What was actually wrong
+
+Every handler in the service was registered like this:
+
+```ts
+void (async () => {
+  const sub = nc.subscribe(subject);
+  for await (const msg of sub) {
+    const args = msg.json();          // ← outside the try
+    try {
+      const db = await getDb();       // ← no timeout
+      ...
+    } catch { respond({ ok: false }) }
+  }
+})();
+```
+
+Three defects that individually look survivable and together are fatal:
+
+**`for await` is strictly sequential.** It processes one message completely before pulling the next. So one call that never settles doesn't slow the channel — it *stops* it. Forever. Later messages get delivered and dropped.
+
+**The database handshake had no deadline.** `connect()` over a WebSocket can stall indefinitely. When it did, the connection was never cached, and the loop parked on it with nothing to retry.
+
+**The JSON parse sat outside the `try`.** One malformed payload throws out of the loop entirely — and because the loop is a bare `void (async () => {})()` with no `.catch()`, that death is an unhandled rejection nobody sees. The subscription stays registered, so monitoring keeps reporting the channel as healthy while nothing consumes it.
+
+## The fix, and one deliberate non-fix
+
+`nats-loop.ts` now owns the parse, a per-message deadline, the always-answer guarantee, and the loop's own death.
+
+It takes any `AsyncIterable` of reply-shaped messages — which is the whole trick. **The original bug was unreachable from our existing test suite** because every test went through a real database and none exercised the loop. Making the loop take a plain async iterable made all three failure modes testable with no broker and no database.
+
+We **kept it sequential on purpose.** Processing messages concurrently would also fix head-of-line blocking, but it would reorder writes on channels like `domain.create` — a real semantic change this bug doesn't require. Bounded-sequential turns "dead forever" into "one slow message," which is the actual defect.
+
+## Under the hood: proof it's fixed
+
+Container rebuilt, then probed live:
+
+```
+domain.list reach-edu        1857ms  ok=true  domains=9
+   upward-mobility, grant-prospecting-tools, future-of-work,
+   workforce-development, frontier-job-demand, agent-workflow-maxxing,
+   adult-literacy-numeracy, ncad-forge, rural-income-boosts
+domain.list humain-vc         111ms  ok=true  domains=7
+domain.list (no filter)       161ms  ok=true  domains=16
+
+malformed payload               3ms  ok=false  "not json{{" is not valid JSON
+domain.list reach-edu (after) 112ms  ok=true  domains=9   ← SUBJECT SURVIVED
+```
+
+The last two lines are the regression. Under the old code that malformed payload would have ended the channel for the rest of the process's life. It now answers in 3ms and the channel keeps serving.
+
+**87 tests across 7 suites**, all passing.
+
+## What's Next?
+
+We fixed one file. The same idiom appears **112 times** across ten services — including three more files in the very service we just fixed, which serve the organization, affiliation and person channels.
+
+One thing does protect them today: the bounded database handshake sits in front of every one of those loops, so the *hang* is closed service-wide. What's still exposed is the malformed-payload path.
+
+The rollout is tracked, and it has a prerequisite worth naming: our services build from their own directory with plain `npm install` and **cannot see the shared packages directory at all.** So "put the helper in a shared package" isn't free — it's a Dockerfile change first. That's the same seam we hit earlier this week when consolidating duplicated utilities, which is a good sign it's the real constraint and not a one-off.