You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
doc(README): App structure was missing 6 apps, 2 services, and correct ports
The ASCII app-structure tree hadn't kept pace with the canonical-layer
build-out: affiliation-rating-resolver, person-db-resolver,
person-enrichment, record-db-resolver, records-surface, and
strategy-curator (the "Corpora Curator") were never added, and
record-surrealdb-resolver + decile-mcp were missing from services/.
Several listed ports were also stale from before the port renumbering
(record-collector :3001 -> :3002, shell :3000 -> :3100, etc.) — pulled
every port fresh from each app's rsbuild.config.ts rather than trusting
the old doc.
Narrative sections updated to match: Backend tech-stack bullets now
name SurrealDB Cloud (canonical entity layer) and didi.sh (shared
identity plane). Core concepts gained three new bullets — the
canonical/DB-first corpus path alongside the original CSV-first one,
live multi-operator NATS sync, and didi.sh actor attribution — and the
stale "Federation-host shell" bullet was rewritten to describe the
FLOWS registry that replaced the old single ROTATION array. Get
started's shell URL was still pointing at :3000.
Files changed:
- README.md
Copy file name to clipboardExpand all lines: README.md
+26-13Lines changed: 26 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,40 +26,50 @@ The codebase is a federated set of small Svelte 5 microfrontends mounted into a
26
26
27
27
-**NATS** as the message bus — every service subscribes to a subject via the `@nats-io/transport-node` v3 client (migrated off the legacy `nats@2` package); the browser talks to the backend via the workspace capabilities router
-**Anthropic** for free-form prompt enrichment (via `prompt-runner`)
29
+
-**SurrealDB Cloud** as the canonical entity layer — persons, organizations, and domains/sources (the corpus catalog: strategies, theses, topics) live here, shared across every client workspace and resolved via `record-surrealdb-resolver`. This is the "start from data already in the canonical layer" side of the app, alongside the original CSV-first pipeline — see [Core concepts](#core-concepts)
30
+
-**didi.sh** as the shared identity plane — magic-link sign-in, org/membership-gated access, actor attribution (`created_by`/`updated_by`) on every canonical write. One account works across this app and its sibling Lossless VC-tooling apps (dididecks, memopop)
31
+
-**Anthropic** for free-form prompt enrichment (via `prompt-runner`) and the in-app chat agent ("didi") — same service, two jobs
30
32
-**Search providers are pluggable** (via `social-search`): a `connectors/` seam with a common `Connector` interface, dispatched per-fire with an optional `provider_override`. **SearXNG** (self-hosted, no API key) is the default for the social packs; **Tavily** stays wired in as a peer for content-RAG packs. Response Reviewer's by-record view exposes both — each record has a SearXNG row and a Tavily row of per-pack run icons, so any source can be re-fired on any record through either provider (additive; never overrides accepted data)
│ ├── content-ingest/ # Jina-extracted .md + binary PDFs for both the funder-content corpus and the domain/strategy-thesis corpus; record_uuid + published_at stamping, /promote-snapshot
67
+
│ ├── record-surrealdb-resolver/ # Canonical entity layer — persons, organizations, domains/sources (the corpus catalog), affiliations — against the shared SurrealDB Cloud instance
-**Packs and bundles.** A **pack** is the atomic enrichment unit — one source, one prompt-snippet template, one extraction schema, one render config. A **bundle** is a workflow composition of packs with orchestration, carry-forward between passes, and a single chat verb that fires the whole thing. The contracts are locked in `context-v/blueprints/Packs-and-Bundles-Pattern.md`.
91
101
-**Sibling-payload responses.** Every response record carries both prose AND an optional structured `Candidate` (url, display_name, confidence 0–100, snippet, source_metadata), plus an `outcome` enum (`found | not_found | error | skipped | pending`). The renderer in Response Reviewer branches on outcome.
92
102
-**Inline correction + human-supply on one surface.** The by-record view in Response Reviewer lets the user edit URLs, edit display_names, edit entity-names (the row's identity column), or supply a URL the pack didn't find — all riding the same `response.set_structured` subject.
93
-
-**Federation-host shell.** The shell handles peek-deck tile rotation, co-existence (50/50 splits), paired-only remotes (`pack-runner`, `chat` — not in the rotation, reached via the `augment-it:navigate` event), and runtime cross-remote communication via `window` events + localStorage.
103
+
-**Federation-host shell, multiple Flows.** The shell used to walk one hardcoded `ROTATION`; it now owns a `FLOWS` registry (`shell/src/flows.svelte.ts`) — each Flow (`Improve a CSV`, `Build Corpora`, `Augment a CSV of Event Attendees`, `Augment a CSV of People`, `Rate Affiliations`, …) is a named use-case with its own rotation, switched via the header's "Flows" popdown. Within a Flow the shell still handles peek-deck tile rotation, co-existence (50/50 splits), and runtime cross-remote communication via `window` events + localStorage. A single-tenant (pinned) deploy defaults to its one relevant Flow rather than the first-registered one.
94
104
-**Lenses.** A *lens* is a federated remote that re-presents the active record set under a different affordance shape — sort/filter, inline-edit, per-row corpus add — without leaving the record. `sort-filter-lens` is the first; registered as a third member of `AUGMENT_COMPOSITE` alongside PTM + Pack Runner. Lenses auto-fall-back to the newest non-archived record set when localStorage points at an archived one, so they survive `/promote-snapshot` cleanly.
95
105
-**Funder-content corpus.** Per-client, per-funder directory of source materials backing each row. Two entry vectors land into the same shape: the chat `/inbox <url>` verb (with active-client context) and the per-row inline "+ URL" affordance in the lens. Both run fire-and-forget through `services/content-ingest/`, Jina-extract markdown, preserve original PDFs as LFS binaries, and stamp `record_uuid` + `published_at` into frontmatter. Manual-paste URLs land regardless of domain (operator curation trumps the same-host rule, which only binds pack outputs).
96
106
-**Corpus chips tell the truth.**`corpus.list_for_record` joins by `corpus_funder_slug` as primary (one dir scan) with `record_uuid` lineage as fallback — chips stay accurate across `/promote-snapshot` cuts. `/promote-snapshot` itself derives `corpus_*` columns from filesystem state when cutting a new record set, and stitches `predecessor_record_set_id` for lineage walks.
107
+
-**The canonical layer — a second, DB-first way to build a corpus.** Every flow above starts from an uploaded CSV. `strategy-curator` ("Corpora Curator" in the UI) starts the other direction: pick or create a **domain** — a `type`-discriminated grouping (`strategy`, `thesis`, `topic`, `market-segment`, …) stored in SurrealDB, not the filesystem — and gather **sources** into it (Jina-fetched metadata, full-content fetch, tags, pasted extracts). `record-surrealdb-resolver` owns the DB side (the `domains`/`sources`/`source_usages` tables plus `persons`/`organizations`/`affiliations`); `content-ingest` mirrors every write to an on-disk corpus file. `person-db-resolver`, `record-db-resolver`, and `affiliation-rating-resolver` are the sibling canonical-layer flows — matching/creating people and organizations, then rating the relevance of the affiliation between them — all writing into the same shared entity graph rather than a per-upload CSV schema.
108
+
-**Live multi-operator sync.** Domain and source mutations on the canonical layer broadcast over NATS (`domain.created`, `source.added`, …); every connected session in the same workspace refetches automatically. Two people signed into the same client see each other's edits without a refresh.
109
+
-**didi.sh identity + actor attribution.** Sign-in is magic-link only (no passwords, invite-only), via the shared `id.didi.sh` service — one account works across this app and its sibling Lossless VC-tooling apps. A signed-in session's `didi_id` rides every capability call and gets stamped as `created_by`/`updated_by` on canonical writes (DB rows and corpus frontmatter alike), so every mutation on the canonical layer carries who did it. The in-app chat agent ("didi") writes through the same envelope, tagged `via: didi-agent`, so an agent-driven edit is distinguishable from a manual one.
97
110
98
111
## Setup
99
112
@@ -122,7 +135,7 @@ Start the full stack (Docker services + every federated remote + shell) with one
122
135
pnpm stack up
123
136
```
124
137
125
-
The shell will be available at [http://localhost:3000](http://localhost:3000). Every remote is also independently reachable at its own port (`:3001`–`:3006`, `:3009`) — useful for debugging Module-Federation cross-origin errors which the shell's DevTools console scrubs to `'Script error.'`.
138
+
The shell will be available at [http://localhost:3100](http://localhost:3100) (`:3000` is deliberately skipped — reserved locally for Open WebUI). Every remote is also independently reachable at its own port (`:3002`–`:3017`, see [App structure](#app-structure) for the full list) — useful for debugging Module-Federation cross-origin errors which the shell's DevTools console scrubs to `'Script error.'`.
126
139
127
140
Other common commands:
128
141
@@ -139,7 +152,7 @@ The `scripts/dev.sh` script prints the full URL list on start.
139
152
140
153
The humain-vc single-tenant instance runs live on **Railway** at
0 commit comments