Skip to content

Commit 1f31fb5

Browse files
mpstatonclaude
andcommitted
new(context-v, issues): corpus adds don't fetch metadata — no cue, no inspector; the write-side half of the content_items hole
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UvYzx7vDWeafnkAi2nEQeb
1 parent cc5f9be commit 1f31fb5

1 file changed

Lines changed: 79 additions & 0 deletions

File tree

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
title: "Corpus adds don't fetch metadata — and the row gives no cue either way, and there's no inspector to see or fix it"
3+
lede: "Adding a corpus item to New America raised the question 'did it fetch the metadata?' — and the honest answer is no, and nothing on the row would tell you either way. The ➕ path writes url/kind/domain and stops; the Jina metadata fetch we already run for domain-corpus sources never fires for entity corpus."
4+
date_created: 2026-07-24
5+
date_modified: 2026-07-24
6+
authors:
7+
- Michael Staton
8+
augmented_with:
9+
- Claude Code on Claude Fable 5
10+
semantic_version: 0.0.0.1
11+
tags:
12+
- Issue
13+
- Usability
14+
- Augment-It
15+
- Corpus
16+
- Org-Workbench
17+
- Content-Items
18+
- Metadata
19+
status: Open · Jotted
20+
---
21+
22+
# Corpus adds don't fetch metadata; no cue; no inspector
23+
24+
## The answer to the operator's question
25+
26+
**It didn't.** `organization.corpus.add` / `person.corpus.add`
27+
`findOrCreateContent` writes `url`, `kind`, `url_domain`, and counters to the
28+
`content_items` ledger — full stop. No title, no fetch, no enrichment. This
29+
is the same hole [[Corpus-Items-Not-Visible-On-Person-Cards-Coverage-Hard-To-Assess]]
30+
noted from the read side ("content_items has no title field"); this doc is
31+
the write side of it.
32+
33+
## Three wants (one flow)
34+
35+
1. **Fetch metadata on add — the way we already do it.** The precedent is
36+
content-ingest's `source.add`: a Jina metadata fetch (title, excerpt,
37+
authors, publisher, bibliographic fields — `services/content-ingest/src/corpus.ts`,
38+
the "METADATA-ONLY file" path). Entity corpus adds should ride the same
39+
machinery: the ➕ (and search-and-add's per-row add) triggers the fetch,
40+
`content_items` grows the metadata columns (title, authors?,
41+
published_at?), and every corpus render hydrates from the ledger.
42+
2. **A visual cue on the row.** Fetched → the row shows the title (host+path
43+
stays the un-fetched fallback per
44+
[[List-Rows-Show-Hostname-Only-Same-Domain-Entries-Indistinguishable]]);
45+
un-fetched or failed → a visible mark (badge/dot) that says "bare URL,
46+
metadata pending/failed". The operator should never have to ask the
47+
question this issue opens with.
48+
3. **An inspector.** Click a corpus row → see what the ledger holds and
49+
adjust it: title, kind, date, maybe authors. The in-place patch shipped
50+
for streams ([[Pulse-Streams-Need-Editable-Kind-And-User-Facing-Names]])
51+
is the edit precedent; a corpus row wants the same, plus the fetched
52+
fields. Human adjusts, per [[human-in-drivers-seat]] — Jina's guess is a
53+
candidate, not truth.
54+
55+
## Machinery notes (jotted)
56+
57+
- `content_items` is SCHEMALESS — adding `title` etc. is free; the work is
58+
the fetch trigger + hydration reads + the row UI.
59+
- Where does the fetch run? The resolver could call content-ingest over NATS
60+
at add time (sync — the cue is immediate), or a fire-and-forget that
61+
patches the ledger after (async — needs the pending/failed cue anyway).
62+
The failed/pending badge is wanted regardless, so async is viable.
63+
- `organization.detail` / `affiliation.detail` / `organization.affiliations`
64+
would hydrate title via `content_id``content_items` (the join noted in
65+
the #20 issue).
66+
- Backfill: existing bare corpus entries (all of them, today) want a
67+
re-fetch pass — same additive discipline as the person-name backfill
68+
(fill missing metadata, never overwrite operator-adjusted fields).
69+
70+
## Open questions
71+
72+
- [ ] Sync fetch on add (immediate cue, slower ➕) vs async patch (fast ➕,
73+
needs pending state) — or Jina-on-add with a short timeout and async
74+
retry?
75+
- [ ] Does the inspector edit `content_items` (shared ledger — edits visible
76+
to every referencing entity) or the per-entity corpus entry? Probably the
77+
ledger for metadata, the entry for kind.
78+
- [ ] Does search-and-add's one-click add fire the same fetch (it should —
79+
same verb underneath)?

0 commit comments

Comments
 (0)