Skip to content

Commit f38e7d7

Browse files
mpstatonclaude
andcommitted
changelog(2026-08-02_02): Jina learns two metadata profiles
Ship note for #78 — source-kind routing (structured vs opengraph) so academic sources keep their date + publisher instead of the operator hand-filling them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018UYTYu4MAFZ7iyr2VTo2kq
1 parent c2ba35a commit f38e7d7

1 file changed

Lines changed: 49 additions & 0 deletions

File tree

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
---
2+
title: "Jina learns two metadata profiles — academic sources stop losing their dates and publishers"
3+
lede: "The metadata parser was reading only blog/OpenGraph keys, so scholarly articles — whose data hides under Highwire/Dublin-Core/PRISM tags — arrived with no date and a weak publisher, and the operator hand-filled them. Now it detects the source KIND (academic-paper / article / company-landing / web-page), routes to the right parser profile, and resolves every field across all conventions. A Springer article now fills its authors, publisher, and date automatically."
4+
date_created: 2026-08-02
5+
date_modified: 2026-08-02
6+
publish: true
7+
authors:
8+
- Michael Staton
9+
augmented_with:
10+
- Claude Code on Claude Opus 4.8
11+
files_changed:
12+
- services/content-ingest/src/jina.ts
13+
- services/content-ingest/test/jina-extract.test.ts
14+
tags:
15+
- Augment-It
16+
- Content-Ingest
17+
- Jina
18+
- Metadata
19+
- Bug-Fix
20+
---
21+
22+
# Jina learns two metadata profiles
23+
24+
Jina always returned rich metadata for every source — we just weren't reading
25+
it. The parser looked for blog/OpenGraph keys (`article:published_time`,
26+
`og:site_name`), which scholarly pages don't set: they carry their metadata
27+
under Highwire, Dublin-Core, and PRISM tags (`citation_*`, `dc.*`, `prism.*`).
28+
So academic sources lost their publish date and got a weak publisher, and you
29+
filled them in by hand.
30+
31+
Now `jina.ts` **detects the source kind** and **routes to one of two parser
32+
profiles**:
33+
34+
- **`structured`**`citation_*` / `dc.*` / `prism.*`, for scholarly articles
35+
and anything with a DOI.
36+
- **`opengraph`**`og:*` / `article:*` / generic, for blogs, news, company
37+
landing pages, and plain web pages.
38+
39+
Each field resolves across an ordered list of aliases spanning every
40+
convention, so a value missing under one still resolves under another. Every
41+
source now records its detected `source_kind` and `parser_profile`, and the
42+
extractor accepts a `forceProfile` override — the foundation for a "re-parse
43+
as…" button when detection is wrong (fast-follow, #79).
44+
45+
A live Springer article that used to arrive dateless now fills its four
46+
authors, `Springer US`, and `2023-05-27` automatically.
47+
48+
Verified with a vitest suite whose academic fixture is the real metadata
49+
captured live from that article. Reaches augment.didi.sh on the next redeploy.

0 commit comments

Comments
 (0)