|
| 1 | +--- |
| 2 | +title: "Search-Results Queue — a right-rail microfrontend where concurrent agent searches land, signal, and get triaged" |
| 3 | +lede: "Agent searches run minutes; the operator shouldn't. Every search fired anywhere becomes a collapsed card in a persistent right rail — status, elapsed vs. typical wait, arrival signal — expandable to act, dismissed when done. Fire many, walk away, triage on arrival." |
| 4 | +date_created: 2026-07-24 |
| 5 | +date_modified: 2026-07-24 |
| 6 | +authors: |
| 7 | + - Michael Staton |
| 8 | +augmented_with: |
| 9 | + - Claude Code on Claude Fable 5 |
| 10 | +semantic_version: 0.0.0.1 |
| 11 | +issue_of_record: "[[../issues/Concurrent-Agent-Searches-Queue-Into-A-Search-Results-Column]]" |
| 12 | +tags: |
| 13 | + - Spec |
| 14 | + - Augment-It |
| 15 | + - Search-Results |
| 16 | + - Didi-Crawl |
| 17 | + - Microfrontends |
| 18 | + - Concurrency |
| 19 | + - Workspace |
| 20 | +status: Draft |
| 21 | +--- |
| 22 | + |
| 23 | +# Search-Results Queue — the `search-results` remote |
| 24 | + |
| 25 | +## Summary |
| 26 | + |
| 27 | +Today an agent search is a **synchronous invoke**: the button disables, one |
| 28 | +column is hijacked, and the operator babysits a 60–210s wait (observed live: |
| 29 | +links 87s; team 147s and 211s). This spec makes searches **asynchronous |
| 30 | +jobs**: a `search.submit` capability returns a `search_id` immediately, the |
| 31 | +workspace service owns a durable search registry, completion broadcasts as a |
| 32 | +WS event, and a new `search-results` remote renders the registry as a queue |
| 33 | +of cards in a persistent right rail — collapsed while running, signalling on |
| 34 | +arrival, expanding into the accept surfaces, dismissed by the operator when |
| 35 | +dealt with. |
| 36 | + |
| 37 | +## Goals |
| 38 | + |
| 39 | +1. **Fire-and-forget from every door.** The 🤖 buttons (links, streams, |
| 40 | + team), and later chat, enqueue a search and return the operator's |
| 41 | + attention immediately. |
| 42 | +2. **Concurrency as the norm.** N searches in flight across N orgs; the |
| 43 | + queue is the worklist. |
| 44 | +3. **Honest waiting.** Every card shows status (`queued / running / done / |
| 45 | + failed`), elapsed time, and typical duration for its target — the |
| 46 | + antidote to the frozen "crawling…" line. |
| 47 | +4. **Arrival is an event, not a discovery.** Done cards signal (badge + |
| 48 | + count in the rail header); the operator notices without polling. |
| 49 | +5. **Results survive everything.** Tab refresh, remount, WS reconnect, even |
| 50 | + closing the browser: the registry lives server-side (persisted like |
| 51 | + sessions and the active workspace), TTL-bounded. This finishes what the |
| 52 | + claim protocol (#41) started. |
| 53 | +6. **Act in place, then clear.** Expand → per-row ➕ (links/streams) or the |
| 54 | + staged-people accept gates (team) → mark complete → card leaves the |
| 55 | + queue. |
| 56 | + |
| 57 | +## Non-goals (this spec) |
| 58 | + |
| 59 | +- **Progress traces inside a run** — the per-card progress line renders |
| 60 | + whatever `search.progress` events exist, but emitting rich narration from |
| 61 | + prompt-runner is [[../issues/Crawl-Progress-Is-A-Black-Box-Needs-Traces-The-Operator-Can-Watch]]'s |
| 62 | + build (Phase 4 hooks it in). |
| 63 | +- **Manual 🔍 term searches in the queue** — search-and-add keeps the |
| 64 | + interactive term/palette flow; unification is an open question below. |
| 65 | +- **JetStream-grade durability** — the registry persists to the workspace |
| 66 | + volume, not a message log; a crash mid-crawl marks the search `failed` |
| 67 | + with a retry affordance, which is honest and sufficient. |
| 68 | +- **Cross-operator queues** — the registry is per-workspace, not per-user, |
| 69 | + in v1 (single-operator reality today). |
| 70 | + |
| 71 | +## Decisions |
| 72 | + |
| 73 | +- **D1 — The registry lives in the workspace service.** A `searches.ts` |
| 74 | + module: in-memory map + write-through JSON at `SEARCH_STORE_PATH` |
| 75 | + (`/data/searches.json`, the sessions/active-workspace volume precedent). |
| 76 | + Server-side because: survives every client failure mode, one source of |
| 77 | + truth for N surfaces, chat can enqueue, and completion events broadcast on |
| 78 | + the existing WS event machinery. |
| 79 | +- **D2 — Submit/execute split.** `search.submit` validates, writes the |
| 80 | + registry entry (`queued`), returns `{search_id}` and kicks execution |
| 81 | + without awaiting it: the workspace dispatches the existing |
| 82 | + `organization.crawl.requested` NATS request itself (600s timeout, as |
| 83 | + today) and on settle updates the entry (`done` + results | `failed` + |
| 84 | + error) and broadcasts. prompt-runner is UNCHANGED in Phase 1 — the async |
| 85 | + boundary moves into workspace, not the runner. |
| 86 | +- **D3 — Events over polling.** `search.updated {search_id, status}` joins |
| 87 | + `BROADCAST_SUBJECTS`-style WS event frames. The remote refetches the |
| 88 | + registry on events (+ once on mount). No polling loops. |
| 89 | +- **D4 — A persistent right rail, mirroring the chat rail.** The shell |
| 90 | + mounts `search-results` as a right-side companion with its own toggle |
| 91 | + (`🔎 queue` beside `💬 chat`) and localStorage visibility — NOT a numbered |
| 92 | + rotation step, because searches are fired from every flow and must be |
| 93 | + reachable from every flow. The rail header shows a done-count badge even |
| 94 | + while collapsed... the rail, not the cards, is the "furthest right column". |
| 95 | +- **D5 — Cards carry their provenance.** `{search_id, entity {org_slug, |
| 96 | + display_name}, target, status, submitted_at, started_at?, finished_at?, |
| 97 | + error?, result_summary {count}, typical_ms}` — enough to render collapsed |
| 98 | + without fetching results. Results fetch on expand (`search.results`). |
| 99 | +- **D6 — Typical durations are hardcoded v1** from live observations |
| 100 | + (links/streams ~90_000ms, team ~200_000ms), displayed as "elapsed 1:20 · |
| 101 | + typically ~1:30". A rolling per-target average in the registry is the |
| 102 | + cheap v2. |
| 103 | +- **D7 — Accept actions reuse today's verbs, per-remote copies of the UI.** |
| 104 | + Links/streams rows: ResultRow-shaped list with ➕ → |
| 105 | + `organization.links.add` / `streams.add` (kind/name carried through, as |
| 106 | + search-and-add does since v1.2). Team: a StagedPeople-shaped section with |
| 107 | + the same candidates-gate accept → `person.apply` + `person.affiliate` + |
| 108 | + link adds. Copied-and-adapted per the knots-style no-shared-runtime rule; |
| 109 | + this is deliberately more fuel for the component library |
| 110 | + ([[../issues/No-Component-Library-UI-Improvised-Not-Component-Based]]). |
| 111 | +- **D8 — Mark complete = dismiss.** `search.dismiss {search_id}` deletes |
| 112 | + the entry (results included). No archive in v1; the canonical layer |
| 113 | + already holds everything the operator accepted. Failed cards offer |
| 114 | + **Retry** (resubmits same entity+target) alongside dismiss. |
| 115 | + |
| 116 | +## Capability contract |
| 117 | + |
| 118 | +All served by the workspace service itself (no NATS round-trip for |
| 119 | +registry ops; the crawl dispatch inside execution reuses the existing |
| 120 | +subject and timeout): |
| 121 | + |
| 122 | +| Capability | Args | Returns | |
| 123 | +|---|---|---| |
| 124 | +| `search.submit` | `{entity: {org_slug, display_name?}, target: 'links'\|'streams'\|'team', client}` | `{ok, search_id}` — immediate | |
| 125 | +| `search.list` | `{client}` | `{ok, searches: SearchCard[]}` (D5 shape, no results) | |
| 126 | +| `search.results` | `{search_id}` | `{ok, status, results?\|people?, filtered_note?, source_urls?, error?}` | |
| 127 | +| `search.dismiss` | `{search_id}` | `{ok}` | |
| 128 | +| WS event | — | `search.updated {search_id, status, org_slug, target}` broadcast to all sessions | |
| 129 | + |
| 130 | +Execution note: workspace's executor calls `organization.crawl.requested` |
| 131 | +over NATS exactly as `dispatch('organization.crawl', …)` does today (same |
| 132 | +600s budget, same reply shapes per target); the crawl capability stays |
| 133 | +chat-legal and untouched for direct use. |
| 134 | + |
| 135 | +## Architecture — the new remote |
| 136 | + |
| 137 | +`apps/search-results/` (scaffold template: `apps/search-and-add`; port |
| 138 | +**3017**; federation name `searchResults`): |
| 139 | + |
| 140 | +``` |
| 141 | +src/ |
| 142 | + App.svelte — rail shell: header (count badge, clear-done), card list |
| 143 | + SearchCard.svelte — collapsed row: target chip · org · status · elapsed/typical · signal dot |
| 144 | + expanded: results region per target + Mark complete / Retry |
| 145 | + ResultsAccept.svelte — links/streams: rows with ➕ (copy-adapt ResultRow/ResultsList) |
| 146 | + TeamAccept.svelte — team: staged-people accept gates (copy-adapt StagedPeople) |
| 147 | + lib/search-client.ts — submit/list/results/dismiss wrappers + typical-duration map |
| 148 | + lib/types.ts — SearchCard, per-target result shapes (mirrors) |
| 149 | + mount.ts / app.css — per the remote scaffold conventions |
| 150 | +``` |
| 151 | + |
| 152 | +Shell (`shell/src/`): `SEARCH_RESULTS_REMOTE` beside `CHAT_REMOTE` |
| 153 | +(remotes.ts); right-rail slot + `🔎 queue` toggle beside the chat toggle |
| 154 | +(App.svelte); rail layout mirrors the chat rail's (chat left · stage center |
| 155 | +· queue right). |
| 156 | + |
| 157 | +Door rewiring (org-workbench): `makeCrawl(...)` and PeopleReveal's `crawl()` |
| 158 | +stop invoking `organization.crawl` synchronously and instead |
| 159 | +`search.submit` + (if rail hidden) flip it visible. The search-and-add crawl |
| 160 | +mode and its envelope `crawl` flag are removed once the queue lands |
| 161 | +(the manual term/scan modes remain). |
| 162 | + |
| 163 | +## Implementation phases |
| 164 | + |
| 165 | +### Phase 1 — Registry + async execution (service only) |
| 166 | +`searches.ts` (registry, persistence, TTL sweep ~24h), the four |
| 167 | +capabilities, the executor, the `search.updated` broadcast; capability map + |
| 168 | +timeouts. Prove over NATS/WS with a scripted submit → event → results → |
| 169 | +dismiss round-trip against the Aspen safe target. |
| 170 | + |
| 171 | +### Phase 2 — The rail |
| 172 | +Scaffold `apps/search-results`; shell registration + toggle; cards render |
| 173 | +the registry live (submit via a temporary script or Phase 3 doors); status, |
| 174 | +elapsed vs typical, arrival badge; expand fetches results read-only; dismiss |
| 175 | +works. Browser-drive: submit two searches, watch both cards progress, |
| 176 | +expand the finished one. |
| 177 | + |
| 178 | +### Phase 3 — Accept surfaces + door switchover |
| 179 | +ResultsAccept (links/streams ➕ with kind/name carry-through) and TeamAccept |
| 180 | +(candidate gates, consume-on-accept per gh #37); org-workbench doors switch |
| 181 | +to `search.submit`; retire search-and-add's crawl mode; entity-updated |
| 182 | +events keep firing on accepts so cards and counts refresh. |
| 183 | + |
| 184 | +### Phase 4 — Progress line (joint with #35) |
| 185 | +prompt-runner emits `search.progress` beats; executor relays them onto the |
| 186 | +card's progress line. Separately planned; the card reserves the slot. |
| 187 | + |
| 188 | +## Constraints & assumptions |
| 189 | + |
| 190 | +- Wire discipline unchanged: slugs/uuids only, client tagging on accepts, |
| 191 | + candidates-never-auto-write. |
| 192 | +- The claim protocol (#41) stays — it protects every *other* long invoke; |
| 193 | + the queue simply stops using long invokes for searches. |
| 194 | +- Federation rules per [[../blueprints/Module-Federation-Rsbuild-Dev-Loop-Gotchas]]: |
| 195 | + no shared runtime, remote owns its Svelte, theme.css before app.css, |
| 196 | + new remote = shell rebuild + fresh browser load. |
| 197 | +- `docker compose` frontends untouched (rsbuild dev servers); dev.sh gains |
| 198 | + the :3017 line. |
| 199 | + |
| 200 | +## Open questions (deliberately few) |
| 201 | + |
| 202 | +- [ ] Does dismissing a `done` card that still has unaccepted rows warn |
| 203 | + ("3 candidates not reviewed")? Lean yes, one inline confirm. |
| 204 | +- [ ] Chat door: `organization.crawl` stays for didi v1, or didi learns |
| 205 | + `search.submit` so chat-fired crawls also land in the rail? Lean the |
| 206 | + latter, one slab edit, Phase 3. |
| 207 | +- [ ] Rolling typical-duration averages (D6 v2) — worth it when? After the |
| 208 | + rail proves itself. |
| 209 | + |
| 210 | +## Related |
| 211 | + |
| 212 | +- [[../issues/Concurrent-Agent-Searches-Queue-Into-A-Search-Results-Column]] — issue of record (operator's design, absorbed here) |
| 213 | +- [[Augment-From-DB-Flow]] §v1.2 — the crawl capability this queues |
| 214 | +- [[../issues/Invokes-Survive-Reconnects-The-Claim-Protocol]] · [[../issues/Crawl-Replies-Can-Be-Lost-Eternal-Spinner-No-Client-Timeout]] — the reliability substrate and its lesson (results belong server-side) |
| 215 | +- [[../issues/Crawl-Progress-Is-A-Black-Box-Needs-Traces-The-Operator-Can-Watch]] — Phase 4's partner |
| 216 | +- [[../issues/No-Component-Library-UI-Improvised-Not-Component-Based]] — the copies D7 knowingly adds |
0 commit comments