Skip to content

Commit 4676e5f

Browse files
authored
Merge pull request #89 from lossless-group/refactor/corpora-curator-rename
corpora-curator: the rename, the type-filter bug (#88), and federated component libraries
2 parents 254765f + dd85be1 commit 4676e5f

91 files changed

Lines changed: 5182 additions & 423 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

DEPLOYMENT.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,13 @@ along the way — lives in
1414
|---|---|---|
1515
| `https://augment.didi.sh` | `shell` | The app itself — what users visit |
1616
| `wss://ws.augment.didi.sh/ws` | `workspace-service` | WebSocket endpoint every remote connects to directly |
17-
| `https://strategy-curator-production.up.railway.app/remoteEntry.js` | `strategy-curator` | Federated remote (static JS asset, not user-facing) |
17+
| `https://corpora-curator-production.up.railway.app/remoteEntry.js` | `corpora-curator` | Federated remote (static JS asset, not user-facing) |
1818
| `https://chat-production-3378.up.railway.app/remoteEntry.js` | `chat` | Federated remote (static JS asset, not user-facing) |
1919

2020
Both `shell` and `workspace-service` **must** stay on `*.didi.sh` — the
2121
`didi_session` cookie `id.didi.sh` issues is scoped to `Domain=.didi.sh`,
2222
and every federated remote's WS connections go to `workspace-service`
23-
directly. `strategy-curator` and `chat` are just static JS hosts loaded
23+
directly. `corpora-curator` and `chat` are just static JS hosts loaded
2424
cross-origin into the shell's page; they don't need to share the cookie
2525
domain themselves.
2626

@@ -50,7 +50,7 @@ environment (`production`).
5050
| `content-ingest` | `services/content-ingest/Dockerfile` | Dockerfile || none |
5151
| `prompt-runner` | `services/prompt-runner/Dockerfile` | Dockerfile || none |
5252
| `shell` | `shell/Dockerfile` | Dockerfile | 3100 | `augment.didi.sh` |
53-
| `strategy-curator` | `apps/strategy-curator/Dockerfile` | Dockerfile | 3017 | Railway-generated |
53+
| `corpora-curator` | `apps/corpora-curator/Dockerfile` | Dockerfile | 3017 | Railway-generated |
5454
| `chat` | `apps/chat/Dockerfile` | Dockerfile | 3006 | Railway-generated |
5555

5656
**Every service is Dockerfile-built, including the three frontends**
@@ -62,7 +62,7 @@ set (they need the full pnpm workspace context — `shared monorepo`
6262
pattern), and their Dockerfiles `COPY` the whole repo before running
6363
`pnpm --filter <pkg> build`.
6464

65-
**Only two remotes are actually wired for this deploy**: `strategyCurator`
65+
**Only two remotes are actually wired for this deploy**: `corporaCurator`
6666
and `chat`. The other twelve remotes `shell/rsbuild.config.ts` knows about
6767
(`recordCollector`, `promptTemplateManager`, …) stay hardcoded to
6868
`localhost` — they belong to flows this single-tenant instance doesn't use.
@@ -112,8 +112,8 @@ credential handling).
112112
| `record-surrealdb-resolver` | `NATS_URL`, `SURREAL_URL`, `SURREAL_NS`, `SURREAL_DB`, `SURREAL_USER`, `SURREAL_PASS` |
113113
| `content-ingest` | `NATS_URL`, `CLIENTS_ROOT=/clients`, `JINA_API_KEY` (paid-tier extraction) |
114114
| `prompt-runner` | `NATS_URL`, `ANTHROPIC_API_KEY` |
115-
| `shell` | `PUBLIC_WS_URL=wss://ws.augment.didi.sh/ws`, `PUBLIC_ID_BASE=https://id.didi.sh`, `PUBLIC_STRATEGY_CURATOR_REMOTE`, `PUBLIC_CHAT_REMOTE` (all build-time — baked in via Docker `ARG`/`ENV`, not read at runtime) |
116-
| `strategy-curator` | `PUBLIC_WS_URL`, `PUBLIC_STRATEGY_CURATOR_ASSET_PREFIX` (build-time) |
115+
| `shell` | `PUBLIC_WS_URL=wss://ws.augment.didi.sh/ws`, `PUBLIC_ID_BASE=https://id.didi.sh`, `PUBLIC_CORPORA_CURATOR_REMOTE`, `PUBLIC_CHAT_REMOTE` (all build-time — baked in via Docker `ARG`/`ENV`, not read at runtime) |
116+
| `corpora-curator` | `PUBLIC_WS_URL`, `PUBLIC_CORPORA_CURATOR_ASSET_PREFIX` (build-time) |
117117
| `chat` | `PUBLIC_WS_URL`, `PUBLIC_CHAT_ASSET_PREFIX` (build-time) |
118118

119119
`PUBLIC_*` vars on the three frontends only take effect on the **next
@@ -204,7 +204,7 @@ scoping is a logged follow-up.
204204

205205
**The Augment-from-DB remotes** (`org-workbench`, `search-and-add`,
206206
`search-results`) deploy as three more static-asset services — same shape
207-
as `chat`/`strategy-curator`: repo-root build context, dockerfilePath
207+
as `chat`/`corpora-curator`: repo-root build context, dockerfilePath
208208
`apps/<name>/Dockerfile`, no rootDirectory, build-time vars
209209
`PUBLIC_WS_URL` + `PUBLIC_<NAME>_ASSET_PREFIX=https://<own-domain>`, and
210210
three matching `PUBLIC_<NAME>_REMOTE=https://<domain>/remoteEntry.js`
@@ -267,10 +267,10 @@ keyed by ID).
267267
string, not `undefined`**`?? default` in TypeScript doesn't catch
268268
it. Two real bugs from this: `shell/rsbuild.config.ts`'s remote-URL
269269
fallbacks had to change from `??` to `||`, and the same for the
270-
`PUBLIC_WS_URL` fallback in `shell`/`strategy-curator`/`chat`.
270+
`PUBLIC_WS_URL` fallback in `shell`/`corpora-curator`/`chat`.
271271
- **Federated remotes need `output.assetPrefix`, not just
272272
`dev.assetPrefix`.** Missing it in production meant `chat`'s and
273-
`strategy-curator`'s async sub-chunks resolved as relative paths
273+
`corpora-curator`'s async sub-chunks resolved as relative paths
274274
against the **shell's** origin instead of their own, 404ing into the
275275
shell's SPA-fallback HTML (`SyntaxError: Unexpected token '<'` — that
276276
HTML being `eval`'d as JS). Only reproduces cross-origin; local

DESIGN.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ revisions:
1313
- 2026-08-01 — 0.0.3.2. **Architecture review remediation (Michael).** Five contradictions, four of them introduced by 2.0/2.1. **F11 restated as a prohibition on literals** — the old wording ("resolves to a Tier-1 name") flags 13 declarations against the runtime, 11 of which are the `color-mix()` idiom *Derived tokens* recommends; composition over Tier 1 **or Tier 2** is now explicitly legal. **Effect tokens settled as Tier 3** and given their own table; the Tier-2 count corrects 24 → 21, and F11 binds Tiers 2 and 3. **P1 corrected** — it said components consume Tier 2 only, which forbade the effect tokens every card uses. **Staleness compares majors**, so "2.1 is additive" is now true rather than contradicted by the schema. **Tier 4 rule 4 made checkable** — it forbade "a colour" while rule 3 mandated composing from colour tokens. Gates raised but not decided — **A20** token introduction across independent deploys (the mirror of A16 — F10 made the shell the sole injector and nothing covers a member outrunning the deployed theme), **A21** logical vs physical properties, plus owners and dates on A19 and the `warn` exit, and the eleven `product-wide` a11y defects read as federal until the audit assigns them.
1414
- 2026-08-01 — **0.1.0.0 — Phases 0 and 1 IMPLEMENTED.** First version where the runtime moved rather than the prose. **Phase 0:** `scripts/design-drift.mjs` exists (`pnpm design:drift`) — the F1–F11 checks and the contrast measurement, zero dependencies, reading the member registry from this document's frontmatter. **Until now F1–F11 were prose; they are now checks.** **Phase 1:** ten missing colour tokens shipped in all three mode blocks; `--color-bg` aliased (closing 43 declarations that painted nothing in production); the four phantom warn dialects unified; `--focus-ring` added and the `strategy-curator` light-mode focus defect closed; global `prefers-reduced-motion` and `forced-colors` blocks; the shell's inline FOUC guard. **A19 resolved** — the three ramps renumbered so step numbers track lightness, verified a pure permutation by `--resolve` diff. **F11 debt cleared.** **R11 fired and was fixed:** contrast measurement found five real light-mode failures; `--color__ink-500` and `--color__amber-ink` darkened, and **all 108 text-on-surface pairs now pass 4.5:1**. **`--color-text-dim` rejected** — no legal value exists below muted in light. New gate **A22** — `--color-border` measures 1.2–1.5:1 and does not meet 3:1 for control boundaries.
1515
- 2026-08-01 — 0.1.0.1. **Post-implementation review remediation.** The Phase 1 review found the code sound and **the document wrong in seven places** — it still described the palette as it was before Phase 1 changed it. Worst case — §Colors warned the ramps were non-monotonic and cited `--color__graphite-700` as `#13151b`, when it is now `#232634`, **so an agent reading the document got the wrong hex**. Also corrected — the A19 block still offering two options "neither yet chosen"; the Tier-1 table missing the five tokens Phase 1 added; the Elevation ladder marking three shipped tokens 🔶; the effect-token table listing 3 of 4 and still claiming the F11 debt was open; the frontmatter `colors:` block missing all 10 new tokens; and the Tier-2 table having no role rows for them. **Root cause — nothing checks the document against the runtime** — the drift script compares CSS to CSS. Added **F1a** to the drift script, which catches a member *consuming* a Tier-1 name (F1 only caught *declaring* one); it found three real violations, all reading `var(--font__mono)` instead of `--font-mono`, now fixed. **That check would have caught a near-miss — A19 was safe only because it happened not to touch the font tokens.**
16+
- 2026-08-08 — 0.1.0.2. **`strategy-curator` → `corpora-curator`, prefix `sc` → `cc`.** Registry row and member table updated; 383 class occurrences across 56 names renamed in lockstep with the gallery catalog's `rootClass`, so the F2/F3 containment audit stays at zero. The app was never strategy-specific — `strategy` is one of `strategy | topic | thesis | market-segment | category`, and humain-vc has only ever run it on `thesis`. **The domain-type vocabulary was deliberately NOT renamed**: it is a data value in two external client submodules, in per-client `DEFAULT_DOMAIN_TYPE`, and in on-disk folder names. Earlier revision entries and the closed-defect log keep the old name on purpose — they record what was true when written. See [[context-v/refactors/Rename-Strategy-Curator-To-Corpora-Curator.md]].
1617
token_status:
1718
colors: shipped
1819
typography: "family shipped; scale proposed"
@@ -44,7 +45,7 @@ federation:
4445
- { name: person-enrichment, path: apps/person-enrichment, prefix: pe, root_class: ".pe-app", tier: A, debt: high, status: registered, doc: apps/person-enrichment/DESIGN.md }
4546
- { name: response-reviewer, path: apps/response-reviewer, prefix: resp, root_class: ".resp-app", tier: A, debt: med, status: registered, doc: apps/response-reviewer/DESIGN.md }
4647
- { name: chat, path: apps/chat, prefix: chat, root_class: ".chat-app", tier: A, debt: med, status: registered, doc: apps/chat/DESIGN.md }
47-
- { name: strategy-curator, path: apps/strategy-curator, prefix: sc, root_class: ".sc-app", tier: B, debt: high, status: registered, doc: apps/strategy-curator/DESIGN.md }
48+
- { name: corpora-curator, path: apps/corpora-curator, prefix: cc, root_class: ".cc-app", tier: B, debt: high, status: registered, doc: apps/corpora-curator/DESIGN.md }
4849
- { name: record-collector, path: apps/record-collector, prefix: rc, root_class: ".rc-app", tier: B, debt: low, status: registered, doc: apps/record-collector/DESIGN.md }
4950
- { name: pack-runner, path: apps/pack-runner, prefix: pr, root_class: ".pr-app", tier: B, debt: high, status: registered, doc: apps/pack-runner/DESIGN.md }
5051
- { name: enhanced-records-list, path: apps/enhanced-records-list, prefix: erl, root_class: ".erl-app", tier: B, debt: low, status: registered, doc: apps/enhanced-records-list/DESIGN.md }
@@ -1004,7 +1005,7 @@ local component → promotion candidate → shared-ui primitive
10041005
| `person-enrichment` | `pe` | `.pe-app` | 6 | The repeater, save flash, form vocabulary | **A** | high | [[apps/person-enrichment/DESIGN.md]] |
10051006
| `response-reviewer` | `resp` | `.resp-app` | 2 | Review pane, highlight spans, reading measure | **A** | med | [[apps/response-reviewer/DESIGN.md]] |
10061007
| `chat` | `chat` | `.chat-app` | 4 | Mode variants, constrained-width column | **A** | med | [[apps/chat/DESIGN.md]] |
1007-
| `strategy-curator` | `sc` | `.sc-app` | 4 | Hairline source rows, tag bar | **B** | high | [[apps/strategy-curator/DESIGN.md]] |
1008+
| `corpora-curator` | `cc` | `.cc-app` | 4 | Hairline source rows, tag bar | **B** | high | [[apps/corpora-curator/DESIGN.md]] |
10081009
| `record-collector` | `rc` | `.rc-app` | 2 | `auto-fit` card grid, family grouping | **B** | low | [[apps/record-collector/DESIGN.md]] |
10091010
| `pack-runner` | `pr` | `.pr-app` | 2 | The run row | **B** | high | [[apps/pack-runner/DESIGN.md]] |
10101011
| `enhanced-records-list` | `erl` | `.erl-app` | 0 | **The only real `<table>`** | **B** | low | [[apps/enhanced-records-list/DESIGN.md]] |
@@ -1596,7 +1597,7 @@ apps/<name>/
15961597
**Don't** hand-roll an overlay. Eight were; zero trap focus, zero restore it, five cannot be closed with Escape.
15971598

15981599
**Do** give every input a real `<label for>`.
1599-
**Don't** style a `<span>` to look like a label. That pattern exists here deliberately — the CSS styles `label` and `.sc-label` identically — and it produces controls a screen reader cannot name.
1600+
**Don't** style a `<span>` to look like a label. That pattern exists here deliberately — the CSS styles `label` and `.cc-label` identically — and it produces controls a screen reader cannot name.
16001601

16011602
**Do** announce state changes with a live region.
16021603
**Don't** assume a visible banner is a communicated banner. This product has two live regions and neither is an error.

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ augment-it/
5151
│ ├── org-workbench/ :3014 # "Augment from DB" — org-first workbench: search to a canonical org, work its links/streams/corpus/people card in place
5252
│ ├── person-enrichment/ :3015 # Sparse-person triage — name/socials/org/web presence, one attendee at a time
5353
│ ├── search-and-add/ :3016 # Editable-term web search + stream scan, paired with org-workbench — one-click add to the launching entity
54-
│ ├── strategy-curator/ :3017 # "Corpora Curator" — build a strategy/thesis corpus from the SurrealDB canonical layer, live multi-operator sync
54+
│ ├── corpora-curator/ :3017 # "Corpora Curator" — build a strategy/thesis corpus from the SurrealDB canonical layer, live multi-operator sync
5555
│ ├── highlight-collector/ # planned — collect highlights from AI responses (scaffold)
5656
│ └── insight-manager/ # planned — manage insights across responses (scaffold)
5757
@@ -106,7 +106,7 @@ augment-it/
106106
- **Lenses.** A *lens* is a federated remote that re-presents the active record set under a different affordance shape — sort/filter, inline-edit, per-row corpus add — without leaving the record. `sort-filter-lens` is the first; registered as a third member of `AUGMENT_COMPOSITE` alongside PTM + Pack Runner. Lenses auto-fall-back to the newest non-archived record set when localStorage points at an archived one, so they survive `/promote-snapshot` cleanly.
107107
- **Funder-content corpus.** Per-client, per-funder directory of source materials backing each row. Two entry vectors land into the same shape: the chat `/inbox <url>` verb (with active-client context) and the per-row inline "+ URL" affordance in the lens. Both run fire-and-forget through `services/content-ingest/`, Jina-extract markdown, preserve original PDFs as LFS binaries, and stamp `record_uuid` + `published_at` into frontmatter. Manual-paste URLs land regardless of domain (operator curation trumps the same-host rule, which only binds pack outputs).
108108
- **Corpus chips tell the truth.** `corpus.list_for_record` joins by `corpus_funder_slug` as primary (one dir scan) with `record_uuid` lineage as fallback — chips stay accurate across `/promote-snapshot` cuts. `/promote-snapshot` itself derives `corpus_*` columns from filesystem state when cutting a new record set, and stitches `predecessor_record_set_id` for lineage walks.
109-
- **The canonical layer — a second, DB-first way to build a corpus.** Every flow above starts from an uploaded CSV. `strategy-curator` ("Corpora Curator" in the UI) starts the other direction: pick or create a **domain** — a `type`-discriminated grouping (`strategy`, `thesis`, `topic`, `market-segment`, …) stored in SurrealDB, not the filesystem — and gather **sources** into it (Jina-fetched metadata, full-content fetch, tags, pasted extracts). `record-surrealdb-resolver` owns the DB side (the `domains`/`sources`/`source_usages` tables plus `persons`/`organizations`/`affiliations`); `content-ingest` mirrors every write to an on-disk corpus file. `person-db-resolver`, `record-db-resolver`, and `affiliation-rating-resolver` are the sibling canonical-layer flows — matching/creating people and organizations, then rating the relevance of the affiliation between them — all writing into the same shared entity graph rather than a per-upload CSV schema.
109+
- **The canonical layer — a second, DB-first way to build a corpus.** Every flow above starts from an uploaded CSV. `corpora-curator` ("Corpora Curator" in the UI) starts the other direction: pick or create a **domain** — a `type`-discriminated grouping (`strategy`, `thesis`, `topic`, `market-segment`, …) stored in SurrealDB, not the filesystem — and gather **sources** into it (Jina-fetched metadata, full-content fetch, tags, pasted extracts). `record-surrealdb-resolver` owns the DB side (the `domains`/`sources`/`source_usages` tables plus `persons`/`organizations`/`affiliations`); `content-ingest` mirrors every write to an on-disk corpus file. `person-db-resolver`, `record-db-resolver`, and `affiliation-rating-resolver` are the sibling canonical-layer flows — matching/creating people and organizations, then rating the relevance of the affiliation between them — all writing into the same shared entity graph rather than a per-upload CSV schema.
110110
- **Augment from DB — the org-first workbench.** The newest canonical-layer flow inverts the CSV pipeline completely: start from an **organization already in SurrealDB**. `org-workbench` autocompletes to an org (names, aliases, or domains), then shows one card that views AND edits in place — identity/social links, pulse streams (`media_streams`), corpus items, and a reveal of every affiliated person with nested links of their own. Adding a person generates the `affiliations` edge + observation automatically (`person.affiliate`, org pre-bound — no affiliation UI needed, N orgs per person by construction). Every list carries a 🔍 that opens `search-and-add` in a paired tile: the search term stays always-editable, providers swap via the registry palette (SearXNG default, Exa a chip away), and each result row one-click-adds to exactly the list that launched the search. Streams go further — a per-stream **scan** fires the entity-pulse blog machinery at the stream URL and badges anything already in the `content_items` ledger, so "what's new on their blog" is one click and only genuinely-new items get added. Spec: `context-v/specs/Augment-From-DB-Flow.md` (Shipped).
111111
- **Live multi-operator sync.** Domain and source mutations on the canonical layer broadcast over NATS (`domain.created`, `source.added`, …); every connected session in the same workspace refetches automatically. Two people signed into the same client see each other's edits without a refresh.
112112
- **didi.sh identity + actor attribution.** Sign-in is magic-link only (no passwords, invite-only), via the shared `id.didi.sh` service — one account works across this app and its sibling Lossless VC-tooling apps. A signed-in session's `didi_id` rides every capability call and gets stamped as `created_by`/`updated_by` on canonical writes (DB rows and corpus frontmatter alike), so every mutation on the canonical layer carries who did it. The in-app chat agent ("didi") writes through the same envelope, tagged `via: didi-agent`, so an agent-driven edit is distinguishable from a manual one.
@@ -171,7 +171,7 @@ Run a single group directly with `pnpm test` in its package:
171171
| B, D, H | `services/workspace` | session tenancy, workspace registry, chat corpora slab |
172172
| E, J | `services/record-surrealdb-resolver` | canonical CRUD + the alignment-audit diff |
173173
| F | `services/content-ingest` | corpus markdown files |
174-
| G | `apps/strategy-curator` | curator surface state (Svelte 5 runes) |
174+
| G | `apps/corpora-curator` | curator surface state (Svelte 5 runes) |
175175
| I | `e2e` | full backend chain over a real WebSocket |
176176

177177
Notes:

0 commit comments

Comments
 (0)