|
| 1 | +--- |
| 2 | +date_created: 2026-07-24 |
| 3 | +date_modified: 2026-07-24 |
| 4 | +title: "Didi crawls the web — three targets, one relevance brief, candidates never writes" |
| 5 | +lede: "The workbench's 🤖 era begins: one organization.crawl capability (Sonnet + Anthropic server-side web search) crawls for relevant identity links, pulse streams, or team members, steered by a per-workspace operator-editable relevance brief — proven live against Aspen in 65 seconds with the brief visibly shaping what came back." |
| 6 | +publish: true |
| 7 | +authors: |
| 8 | + - Michael Staton |
| 9 | +augmented_with: |
| 10 | + - Claude Code on Claude Fable 5 |
| 11 | +files_changed: |
| 12 | + - services/prompt-runner/src/crawl.ts |
| 13 | + - services/prompt-runner/src/server.ts |
| 14 | + - services/workspace/src/chat.ts |
| 15 | + - services/workspace/src/capabilities.ts |
| 16 | + - services/record-surrealdb-resolver/src/resolver.ts |
| 17 | + - services/record-surrealdb-resolver/src/handlers.ts |
| 18 | + - apps/search-and-add/src/App.svelte |
| 19 | + - apps/search-and-add/src/lib/search-client.ts |
| 20 | + - apps/search-and-add/src/lib/types.ts |
| 21 | + - apps/org-workbench/src/AdditiveList.svelte |
| 22 | + - apps/org-workbench/src/OrgCard.svelte |
| 23 | + - apps/org-workbench/src/App.svelte |
| 24 | + - apps/org-workbench/src/PeopleReveal.svelte |
| 25 | + - apps/org-workbench/src/StagedPeople.svelte |
| 26 | + - apps/org-workbench/src/BriefPanel.svelte |
| 27 | + - apps/org-workbench/src/lib/org-client.ts |
| 28 | + - apps/org-workbench/src/lib/types.ts |
| 29 | + - apps/org-workbench/src/app.css |
| 30 | +tags: |
| 31 | + - Didi-Crawl |
| 32 | + - Org-Workbench |
| 33 | + - Search-And-Add |
| 34 | + - Relevance-Brief |
| 35 | + - Prompt-Runner |
| 36 | + - Affiliations |
| 37 | +--- |
| 38 | + |
| 39 | +# Didi crawls the web |
| 40 | + |
| 41 | +## What shipped ([#33](https://github.com/lossless-group/augment-it/issues/33)) |
| 42 | + |
| 43 | +The operator's insight driving the whole build: *web-search-equipped agents |
| 44 | +get this retrieval shape mostly right, quickly* — so the manual |
| 45 | +query-composing 🔍 gains an agentic sibling. One capability, |
| 46 | +`organization.crawl`, served by prompt-runner (the LLM gateway): a single |
| 47 | +Sonnet turn with Anthropic server-side web search composes the queries, |
| 48 | +searches, filters by the relevance brief, and returns **candidates — never |
| 49 | +writes**. Three targets: |
| 50 | + |
| 51 | +- **"crawl for relevant identity links"** → 🤖 on the links list → the |
| 52 | + search-and-add rail's new **crawl mode** (scan-mode's twin: no term bar, |
| 53 | + same results list, per-row ➕ — now carrying the model's `kind` into the |
| 54 | + write instead of re-inferring it). |
| 55 | +- **"crawl for relevant pulse streams"** → same door; a stream's real title |
| 56 | + ("Today's Credentials") rides the ➕ as its `name`. |
| 57 | +- **"crawl for relevant team members"** → 🤖 on People → candidates stage in |
| 58 | + the new `StagedPeople` section: accept runs the candidate check — |
| 59 | + no-match rows flow straight through `person.apply` + `person.affiliate` |
| 60 | + (team-page URL as the observation's source), ambiguous rows open the |
| 61 | + pick-or-create gate. The people-policy filter is never silent |
| 62 | + (`filtered_note` renders under the staged rows). |
| 63 | + |
| 64 | +## The relevance brief |
| 65 | + |
| 66 | +`relevance_briefs` in SurrealDB behind `client.brief.get`/`set` — one prose |
| 67 | +document per workspace client carrying the topical scope and the people |
| 68 | +policy, edited in place via the workbench header's 📋 panel, loaded |
| 69 | +server-side into every crawl so the button door and the (chat-legal, per |
| 70 | +`CHAT_CAPABILITY_NAMES`) chat door read the same intent. reach-edu's brief |
| 71 | +is seeded with the operator's stated policy: education & workforce |
| 72 | +development, all major leadership plus covering team members. |
| 73 | + |
| 74 | +## Proven live |
| 75 | + |
| 76 | +Smoke over NATS against the Aspen safe target: brief upsert + read-back |
| 77 | +clean; the links crawl returned in **65 seconds** with 8 correctly-kinded |
| 78 | +candidates, all deduped against what the org already had — and one that |
| 79 | +only a brief-steered agent would rank: the Education & Society Program's |
| 80 | +LinkedIn, tagged "directly relevant to education and workforce development |
| 81 | +focus." Candidates only; the canonical layer untouched. |
| 82 | + |
| 83 | +Typechecks and svelte-checks clean across three services and both remotes; |
| 84 | +all three service containers rebuilt. Remaining human rung: the operator |
| 85 | +walk-through (crawl a thin org's three lists end to end; edit the brief and |
| 86 | +watch selection change) — and the chat-rail conversational door rides the |
| 87 | +didi-chat team-page plan. |
0 commit comments