attempt(augment-from-db, stream-scan, step5): scan a pulse stream, badge the known, one-click the new — and the spec goes Shipped

Spec step 8 lands as a mode of search-and-add (decision D3), and with it
all five phases of Augment-From-DB-Flow.md are implemented — one day,
one signed-off spec, five plans, five proofs.

The cheap trick: runOfficialBlogPack already had the exact seam.
curated_index_urls skips discovery entirely when the operator has
curated the index pages — and a stream scan IS "the index is curated:
it's the stream URL." organization.stream.scan is a thin wrapper: fire
the blog machinery at the stream, dedup via the new content.urls.check
on record-surrealdb-resolver (social-search has no DB access by design;
cross-service NATS read per the domains.ts precedent), reply items with
already_in_corpus flags.

On the card: AdditiveList grew an optional per-entry action; the
Pulse-streams list wires it as "scan," dispatching a stream-carrying
envelope through the same localStorage+event+navigate contract as every
🔍. In search-and-add: a stream envelope flips the surface into scan
mode — TermBar/palette give way to the stream URL + Re-scan, known rows
badge "in corpus" with the ➕ parked, fresh rows one-click into
org_corpus. Social walls ride the same call flagged experimental (spec
non-goal); blog/RSS/newsroom is the proven path.

Proof, live against the stream Phase 2 seeded: scan of
aspeninstitute.org/blog/ → 8 dated July-2026 posts → corpus.add one →
re-scan shows already_known:1 with the added item's flag flipped true.
svelte-check 0/0 both remotes (one prop-destructure miss caught + fixed),
all builds, three service typechecks, Phase 1 proof 7/7.

Spec: context-v/specs/Augment-From-DB-Flow.md → Shipped (v0.1.0.0);
operator browser walk-throughs + parked open questions noted in its
post_ship_note.
Plan: context-v/plans/Augment-From-DB-Phase-5-Stream-Scan-Mode.md (Shipped).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014zrFkWSVgTkoQyiobdBjrd

Author: mpstaton

apps/org-workbench/src/AdditiveList.svelte

@@ -14,13 +14,16 @@
     kindHint = 'auto-detected from URL',
     onadd,
     onsearch,
+    entryaction,
   }: {
     title: string;
     entries: ShapedLink[];
     kindHint?: string;
     onadd: (url: string, kind?: string) => Promise<void>;
     // Optional 🔍 — launches search-and-add pre-scoped to this list (Phase 3).
     onsearch?: () => void;
+    // Optional per-entry action (Phase 5 — "scan" on pulse streams).
+    entryaction?: { label: string; fn: (entry: ShapedLink) => void };
   } = $props();
 
   let adding = $state(false);
@@ -105,6 +108,11 @@
         <li class="ow-entry">
           <span class="ow-kind">{e.kind}</span>
           <a class="ow-url" href={e.url} target="_blank" rel="noreferrer">{host(e.url)}</a>
+          {#if entryaction}
+            <button type="button" class="ow-entry-action" onclick={() => entryaction.fn(e)}>
+              {entryaction.label}
+            </button>
+          {/if}
           <span class="ow-date">{(e.added_at ?? '').slice(0, 10)}</span>
         </li>
       {/each}

apps/org-workbench/src/OrgCard.svelte

@@ -80,6 +80,16 @@
       kindHint="kind (auto: blog_index/rss/newsroom/…)"
       onadd={makeAdd(addOrgStream)}
       onsearch={makeSearch('streams', (n) => `"${n}" blog`)}
+      entryaction={{
+        label: 'scan',
+        fn: (stream) =>
+          requestSearch({
+            entity: { type: 'organization', org_slug: org.slug, display_name: displayName },
+            target: 'corpus',
+            seed_term: '',
+            stream: { url: stream.url, kind: stream.kind },
+          }),
+      }}
     />
     <AdditiveList
       title="Corpus items"

apps/org-workbench/src/app.css

@@ -63,6 +63,8 @@
 .ow-url { color: var(--color-link, #8ab4f8); text-decoration: none; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
 .ow-url:hover { text-decoration: underline; }
 .ow-date { margin-left: auto; flex-shrink: 0; color: var(--color-text-muted, #9aa0aa); font-size: 0.72rem; font-variant-numeric: tabular-nums; }
+.ow-entry-action { flex-shrink: 0; font-size: 0.7rem; padding: 1px 8px; border-radius: 999px; border: 1px solid var(--color-border, #2a2c33); background: var(--color-bg, #14151a); color: var(--color-text-muted, #cfd3da); font-family: inherit; cursor: pointer; }
+.ow-entry-action:hover { background: var(--color-border, #2a2c33); color: var(--color-text, #e7e7e7); }
 
 .ow-empty { color: var(--color-text-muted, #9aa0aa); font-size: 0.8rem; margin: 0.4rem 0 0; }
 .ow-error { color: var(--color-error-text, #f3a3a3); font-size: 0.8rem; margin-top: 0.4rem; }

apps/org-workbench/src/lib/types.ts

@@ -71,4 +71,8 @@ export type SearchRequestDetail = {
   target: 'links' | 'corpus' | 'streams';
   seed_term: string;
   intent?: string;
+  // Phase 5 — when present, search-and-add enters scan mode: the stream URL
+  // is scanned via organization.stream.scan instead of a term search, and
+  // ➕ lands items in org_corpus (target is 'corpus' for scan envelopes).
+  stream?: { url: string; kind?: string };
 };

apps/search-and-add/src/App.svelte

@@ -15,7 +15,7 @@
   import ProviderPalette from './ProviderPalette.svelte';
   import ResultsList from './ResultsList.svelte';
   import { searchContext } from './lib/search-context.svelte';
-  import { fireSearch, fetchConnectors, addResult, verbFor } from './lib/search-client';
+  import { fireSearch, fetchConnectors, addResult, verbFor, scanStream } from './lib/search-client';
   import type { ConnectorInfo, ConnectorResult } from './lib/types';
 
   const TOKEN_KEY = 'augment-it:session-token';
@@ -36,6 +36,9 @@
   let firedArrival = -1;
 
   const req = $derived(searchContext.request);
+  // Phase 5 — scan mode: the envelope carries a stream. No term, no palette;
+  // the stream URL is the query and organization.stream.scan is the fire.
+  const scanMode = $derived(Boolean(req?.stream?.url && req?.entity.type === 'organization'));
   const entityLabel = $derived(
     req
       ? req.entity.display_name ??
@@ -52,6 +55,7 @@
   });
 
   async function fire() {
+    if (scanMode) return void scan();
     if (!term.trim()) return;
     firing = true;
     fireError = null;
@@ -72,6 +76,28 @@
     }
   }
 
+  async function scan() {
+    if (!req?.stream?.url || req.entity.type !== 'organization') return;
+    firing = true;
+    fireError = null;
+    try {
+      const r = await scanStream({
+        org_slug: req.entity.org_slug,
+        stream_url: req.stream.url,
+        stream_kind: req.stream.kind,
+        client,
+      });
+      results = r.results;
+      firedVia = `stream scan (${r.already_known} already in corpus)`;
+    } catch (err) {
+      fireError = err instanceof Error ? err.message : String(err);
+      results = [];
+      firedVia = null;
+    } finally {
+      firing = false;
+    }
+  }
+
   // A fresh envelope (mount-time localStorage read counts, via arrival 0 vs
   // firedArrival -1) seeds the term and auto-fires exactly once. Later
   // operator edits + re-fires never re-trigger this.
@@ -82,7 +108,7 @@
         term = req.seed_term;
         results = [];
         firedVia = null;
-        void fire();
+        void (scanMode ? scan() : fire());
       }
     }
   });
@@ -139,8 +165,19 @@
       {/if}
       <span class="saa-ws status-{status}">{status}</span>
     </div>
-    <TermBar bind:term {firing} onfire={fire} />
-    <ProviderPalette {connectors} bind:selected={selectedProvider} />
+    {#if scanMode && req?.stream}
+      <div class="saa-scanbar">
+        <span class="saa-scan-label">scanning stream</span>
+        <a class="saa-scan-url" href={req.stream.url} target="_blank" rel="noreferrer">{req.stream.url}</a>
+        {#if req.stream.kind}<span class="saa-scan-kind">{req.stream.kind}</span>{/if}
+        <button type="button" class="saa-fire" disabled={firing} onclick={scan}>
+          {firing ? 'scanning…' : 'Re-scan'}
+        </button>
+      </div>
+    {:else}
+      <TermBar bind:term {firing} onfire={fire} />
+      <ProviderPalette {connectors} bind:selected={selectedProvider} />
+    {/if}
     {#if fireError}<div class="saa-error saa-fire-error">{fireError}</div>{/if}
   </header>
 

apps/search-and-add/src/ResultRow.svelte

@@ -18,6 +18,9 @@
   let added = $state(false);
   let error = $state<string | null>(null);
 
+  // Scan mode: the corpus already holds this URL — badge it, park the ➕.
+  const known = $derived(result.known === true);
+
   async function add() {
     adding = true;
     error = null;
@@ -44,18 +47,19 @@
   <div class="saa-row-main">
     <a class="saa-row-title" href={result.url} target="_blank" rel="noreferrer">{result.title}</a>
     <span class="saa-row-host">{host(result.url)}</span>
+    {#if known}<span class="saa-known">in corpus</span>{/if}
     {#if result.published_date}<span class="saa-row-date">{result.published_date.slice(0, 10)}</span>{/if}
     {#if result.content}<p class="saa-row-snippet">{result.content.slice(0, 220)}</p>{/if}
     {#if error}<div class="saa-error">{error}</div>{/if}
   </div>
   <button
     type="button"
     class="saa-add"
-    class:added
-    disabled={adding || added}
+    class:added={added || known}
+    disabled={adding || added || known}
     onclick={add}
-    title={added ? 'added to the entity' : 'add to the entity'}
+    title={known ? 'already in the corpus' : added ? 'added to the entity' : 'add to the entity'}
   >
-    {added ? '✓' : adding ? '…' : '+'}
+    {added || known ? '✓' : adding ? '…' : '+'}
   </button>
 </li>

apps/search-and-add/src/app.css

@@ -50,6 +50,15 @@
 .saa-add.added { border-color: var(--color-ok-text, #6ee7a8); color: var(--color-ok-text, #6ee7a8); cursor: default; }
 .saa-add:disabled:not(.added) { opacity: 0.6; cursor: default; }
 
+/* scan mode (Phase 5) */
+.saa-scanbar { display: flex; align-items: center; gap: 0.6rem; margin-top: 0.6rem; flex-wrap: wrap; }
+.saa-scan-label { font-size: 0.78rem; color: var(--color-text-muted, #9aa0aa); }
+.saa-scan-url { color: var(--color-link, #8ab4f8); font-size: 0.85rem; text-decoration: none; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; max-width: 60%; }
+.saa-scan-url:hover { text-decoration: underline; }
+.saa-scan-kind { font-size: 0.7rem; padding: 1px 6px; border-radius: 3px; background: var(--color-border, #2a2c33); color: var(--color-text-muted, #cfd3da); }
+.saa-scanbar .saa-fire { margin-left: auto; }
+.saa-known { font-size: 0.68rem; padding: 1px 6px; border-radius: 3px; margin-left: 0.5rem; background: var(--color-ok-bg, #173a26); color: var(--color-ok-text, #6ee7a8); }
+
 .saa-empty { color: var(--color-text-muted, #9aa0aa); font-size: 0.85rem; }
 .saa-error { color: var(--color-error-text, #f3a3a3); font-size: 0.8rem; margin-top: 0.35rem; }
 .saa-fire-error { margin-top: 0.5rem; }

apps/search-and-add/src/lib/search-client.ts

@@ -21,6 +21,37 @@ export async function fireSearch(args: {
   return { provider: r.provider ?? 'unknown', results: r.results ?? [] };
 }
 
+export async function scanStream(args: {
+  org_slug: string;
+  stream_url: string;
+  stream_kind?: string;
+  client: string;
+}): Promise<{ results: ConnectorResult[]; already_known: number }> {
+  const r = (await workspace.invoke('organization.stream.scan', args)) as {
+    ok: boolean;
+    items?: {
+      url: string;
+      title: string;
+      snippet: string;
+      published_date: string | null;
+      already_in_corpus: boolean;
+    }[];
+    meta?: { already_known: number };
+    error?: string;
+  };
+  if (!r.ok) throw new Error(r.error || 'organization.stream.scan failed');
+  return {
+    results: (r.items ?? []).map((i) => ({
+      url: i.url,
+      title: i.title,
+      content: i.snippet,
+      published_date: i.published_date ?? undefined,
+      known: i.already_in_corpus,
+    })),
+    already_known: r.meta?.already_known ?? 0,
+  };
+}
+
 export async function fetchConnectors(): Promise<ConnectorInfo[]> {
   const r = (await workspace.invoke('connectors.inventory', {})) as {
     ok: boolean;

apps/search-and-add/src/lib/types.ts

@@ -12,16 +12,21 @@ export type SearchRequestDetail = {
   target: 'links' | 'corpus' | 'streams';
   seed_term: string;
   intent?: string;
+  // Phase 5 — scan mode: when present, the stream URL is scanned via
+  // organization.stream.scan instead of a term search.
+  stream?: { url: string; kind?: string };
 };
 
 // Normalized result every connector returns (services/social-search
-// connectors/types.ts ConnectorResult, verbatim).
+// connectors/types.ts ConnectorResult, verbatim) — plus the scan mode's
+// already-in-corpus flag, absent on term-search results.
 export type ConnectorResult = {
   url: string;
   title: string;
   content: string;
   score?: number;
   published_date?: string;
+  known?: boolean;
 };
 
 // connectors.inventory wire shape — ConnectorRegistration minus `fire`.

changelog/2026-07-22_05_Stream-Scan-Lands-And-Augment-From-DB-Is-Feature-Complete.md

@@ -0,0 +1,56 @@
+---
+date_created: 2026-07-22
+date_modified: 2026-07-22
+title: "Stream-scan lands and Augment-from-DB is feature-complete — scan a pulse stream, badge the known, one-click the new"
+lede: "Spec step 8 ships as a mode of search-and-add: a per-stream Scan button on the org card fires the official-blog machinery with the stream URL as the authoritative index, dedups against content_items, and badges what the corpus already holds — proven live against The Aspen Institute's blog, flag-flip and all. All five phases of the spec are now implemented."
+publish: true
+authors:
+  - Michael Staton
+augmented_with:
+  - Claude Code on Claude Fable 5
+files_changed:
+  - services/social-search/src/stream-scan.ts
+  - services/social-search/src/server.ts
+  - services/record-surrealdb-resolver/src/resolver.ts
+  - services/record-surrealdb-resolver/src/handlers.ts
+  - services/workspace/src/capabilities.ts
+  - apps/org-workbench/src/AdditiveList.svelte
+  - apps/org-workbench/src/OrgCard.svelte
+  - apps/org-workbench/src/lib/types.ts
+  - apps/org-workbench/src/app.css
+  - apps/search-and-add/src/App.svelte
+  - apps/search-and-add/src/ResultRow.svelte
+  - apps/search-and-add/src/lib/search-client.ts
+  - apps/search-and-add/src/lib/types.ts
+  - apps/search-and-add/src/app.css
+  - context-v/plans/Augment-From-DB-Phase-5-Stream-Scan-Mode.md
+  - context-v/specs/Augment-From-DB-Flow.md
+tags:
+  - Progress-Update
+  - Augment-From-DB
+  - Stream-Scan
+  - Entity-Pulse
+  - Feature-Complete
+---
+
+# Stream-scan lands — and the flow the spec drew is all there
+
+Phase 5 of [[../context-v/specs/Augment-From-DB-Flow.md]], and with it the spec goes **Shipped**: all five phases implemented in one day against a Signed-Off spec, each phase its own plan, commit, and proof.
+
+## What shipped
+
+**The cheap trick that made this a half-day, not a week:** `runOfficialBlogPack` already had the exact seam — `curated_index_urls`, where operator-curated index pages skip discovery entirely. A stream scan IS "the index is already curated: it's the stream URL." So `organization.stream.scan` is a thin wrapper: fire the blog machinery at the stream, then dedup via the new **`content.urls.check`** on record-surrealdb-resolver (social-search has no DB access by design — the cross-service NATS read follows the domains.ts → content-ingest precedent).
+
+**On the org card:** `AdditiveList` grew an optional per-entry action, and the Pulse-streams list wires it as **scan** — one click dispatches a stream-carrying envelope through the same localStorage + event + navigate contract as every 🔍.
+
+**In search-and-add:** a stream envelope flips the surface into scan mode — TermBar and palette give way to the stream URL + Re-scan; result rows carry an **"in corpus"** badge with the ➕ parked; fresh items one-click into `org_corpus` through the existing add path, and the flag flips on the next scan.
+
+Social walls (LinkedIn/Facebook) ride the same call but stay experimental per the spec's non-goal — blog/RSS/newsroom is the dependable path, and it's the one proven.
+
+## Proof
+
+Live, end-to-end, against the stream Phase 2's proof seeded: scan of `aspeninstitute.org/blog/` → **8 real posts with publish dates** (July 2026, fresh) · `organization.corpus.add` one of them · re-scan → `already_known: 1` and the added item's flag flipped to true. Full sweep green: svelte-check 0/0 on both remotes (one caught-and-fixed prop-destructure miss in AdditiveList — svelte-check earning its keep), all builds, three service typechecks, Phase 1 proof re-run 7/7.
+
+## Where the flow stands
+
+All five phases of [[../context-v/specs/Augment-From-DB-Flow.md]] are implemented and pushed as `attempt(augment-from-db, …, step1–5)`. Remaining before calling the *product* (not the code) done: the operator browser walk-throughs each plan names — the six-flow front door, the org card's ➕s, the 🔍→pairing→add loop, add-person with automatic affiliation, and the scan badge — plus the spec's parked open questions (fire-log persistence, pack-template seed terms, pinned-deploy remote URLs) for a future pass.