|
| 1 | +--- |
| 2 | +date_created: 2026-06-29 |
| 3 | +date_modified: 2026-06-29 |
| 4 | +title: "Sources capture their own bibliography — authors, publisher, date" |
| 5 | +lede: "Fetch a source and it now fills in its own author(s), publisher, and publication date, straight from Jina's structured metadata — authors as an array, all editable, all written to both the file and the registry. Plus a green save-pulse on every field, and a fix so renamed files actually stay connected." |
| 6 | +publish: true |
| 7 | +authors: |
| 8 | + - Michael Staton |
| 9 | +augmented_with: |
| 10 | + - Claude Code on Claude Opus 4.8 (1M context) |
| 11 | +files_changed: |
| 12 | + - services/content-ingest/src/jina.ts |
| 13 | + - services/content-ingest/src/corpus.ts |
| 14 | + - services/record-surrealdb-resolver/src/domains.ts |
| 15 | + - apps/strategy-curator/src/curation.svelte.ts |
| 16 | + - apps/strategy-curator/src/SourceDetail.svelte |
| 17 | + - apps/strategy-curator/src/types.ts |
| 18 | + - apps/strategy-curator/src/app.css |
| 19 | + - context-v/specs/Strategy-Curator-Entry-Point-for-Augment-It.md |
| 20 | +tags: |
| 21 | + - Progress-Update |
| 22 | + - Strategy-Curator |
| 23 | + - Source-Curation |
| 24 | + - Jina |
| 25 | + - Bibliography |
| 26 | + - SurrealDB |
| 27 | + - reach-edu |
| 28 | +--- |
| 29 | + |
| 30 | +## Why Care? |
| 31 | + |
| 32 | +A source isn't just a URL — it's a citation. To ground anything in it you need the |
| 33 | +author, the publisher, and the date. Until now the curator caught the title and |
| 34 | +nothing else, so every source needed those typed in by hand. That's exactly the |
| 35 | +tedious, error-prone work the tool exists to remove. |
| 36 | + |
| 37 | +Now: fetch a source and its **author(s), publisher, and publication date appear on |
| 38 | +their own**, pulled from the page's structured metadata. You still own them — every |
| 39 | +field is editable — but you start from filled-in, not blank. |
| 40 | + |
| 41 | +## What's New? |
| 42 | + |
| 43 | +- **Auto-filled bibliography.** `source.add` and `source.fetch` extract author, |
| 44 | + publisher (`og:site_name`), and published date from Jina and write them to both the |
| 45 | + markdown frontmatter and the SurrealDB registry. |
| 46 | +- **Authors is an array.** One author → a one-element list; multiple authors → multiple |
| 47 | + entries. (Jina hands back a string for one and an array for several; we normalize.) |
| 48 | +- **Editable everything.** Title, Filename, Author(s), Publisher, Published date — all |
| 49 | + in the source form, all hand-correctable, each writing through to file + registry. |
| 50 | +- **Green save-pulse.** Hit Enter (or click away) in a field and its border pulses green, |
| 51 | + then fades — a "saved ✓" right where you typed. No pulse if the save errored. |
| 52 | +- **Renamed files stay connected.** Fixed a gap where an in-session source could lose its |
| 53 | + link to its file on disk. |
| 54 | + |
| 55 | +## Under the Hood — the one-line bug that hid everything |
| 56 | + |
| 57 | +The bibliographic data was always in Jina's response — we were just asking for the |
| 58 | +wrong format. `r.jina.ai` returns a markdown preamble (title + date only) *or* a JSON |
| 59 | +body whose `data.metadata` carries author and `og:site_name`. We switched to JSON, wrote |
| 60 | +all the extraction code... and it did nothing. The author and publisher kept coming back |
| 61 | +empty. |
| 62 | + |
| 63 | +The cause: a stale `Accept: 'text/markdown'` header. An earlier edit replaced the |
| 64 | +response-*handling* code but the window started one line below the header, so the request |
| 65 | +still asked for markdown. Every fetch ran the JSON parser against a markdown body, |
| 66 | +`JSON.parse` threw, and the code fell to the partial fallback path. **All the right code, |
| 67 | +dead behind a failing parse.** One character — `application/json` — and authors, |
| 68 | +publisher, and date all lit up: |
| 69 | + |
| 70 | +```yaml |
| 71 | +authors: |
| 72 | + - "Alana Semuels" |
| 73 | +publisher: "The Atlantic" |
| 74 | +published_date: "2016-06-02" |
| 75 | +``` |
| 76 | +
|
| 77 | +The lesson logged for next time: when a feature is "implemented but inert," check the |
| 78 | +*request*, not just the handler — and verify the running container actually has the line |
| 79 | +you think it does. |
| 80 | +
|
| 81 | +## The "renamed files won't connect" fix |
| 82 | +
|
| 83 | +A source added during a session came back from the resolver without its `source_slug`, so |
| 84 | +its Filename field sat empty and rename had nothing to point at — even though the database |
| 85 | +knew the file perfectly well. Two fixes: the `source.add` / `fetch` / `retry` responses now |
| 86 | +echo `source_slug`, and the UI derives the slug from `corpus_path` as a fallback. A file |
| 87 | +that exists never shows a blank filename again. |
| 88 | + |
| 89 | +## What's Next |
| 90 | + |
| 91 | +- A **paid Jina key** (`JINA_API_KEY` in `augment-it/.env`) for more consistent extraction |
| 92 | + and higher rate limits — the free tier honors the JSON request less reliably. |
| 93 | +- The still-pending **interstitial detector** and **auto text-extraction from attached PDFs** |
| 94 | + from the prior entry, [[2026-06-29_01_Strategy-Curator-Source-Surface-Curate-Sources-By-Domain-Type]]. |
| 95 | + |
| 96 | +## Files Touched |
| 97 | + |
| 98 | +content-ingest's `jina.ts` (JSON extraction) and `corpus.ts` (bib frontmatter + authors |
| 99 | +list), the resolver's `domains.ts` (registry columns + response echo), and the |
| 100 | +strategy-curator UI (`curation.svelte.ts`, `SourceDetail.svelte`, `types.ts`, `app.css`). |
| 101 | +Spec bumped to 0.0.0.6 with an Increment 4 section. |
0 commit comments