Skip to content

Commit 6cf0d12

Browse files
committed
Add runtime fixture for annotation colors
1 parent db18f42 commit 6cf0d12

7 files changed

Lines changed: 252 additions & 3 deletions

File tree

docs/RELEASE_CHECKLIST.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,6 @@
2020

2121
## Before Public Stable
2222

23-
- [ ] Add positive runtime fixtures for all supported annotation colors.
23+
- [x] Add positive runtime fixtures for all supported annotation colors.
2424
- [ ] Decide whether translation providers remain menu-only or move to a settings pane.
2525
- [ ] Add release signing/distribution beyond GitHub prerelease assets if publishing beyond local beta users.

docs/SPEC.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@ for generated test artifacts after cleanup.
117117

118118
## Known Residual Risks
119119

120-
- The six color hex values must match Zotero's stored annotation colors. Green has been positively runtime-verified on item `9761`; other colors need positive annotation-level runtime fixtures before release confidence is complete.
120+
- The six color hex values must keep matching Zotero's stored annotation colors. A controlled Zotero runtime fixture positively verified green, yellow, blue, purple, red, and gray on 2026-06-20; rerun the fixture if Zotero changes its stored annotation palette.
121121
- Candidate quality is heuristic. Domain-specific phrase extraction is not a dictionary or language model.
122122
- The 30-candidate confirmation threshold is heuristic and may need adjustment after real use.
123123
- The `google-free` translation provider is inaccurate, context-free, and may be rate-limited or blocked. It should remain opt-in, clearly warned, and framed as a weak aid, not as Vocab Flow's core value.

docs/issues/vocab-flow-gap-audit.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -183,6 +183,14 @@ Vocab Flow must not treat every underlined sentence as a final vocabulary list.
183183
- Final rebuilt XPI was recopied to the default profile after documentation updates; final local/profile hash: `31114930e10d8aebc138aa726129be463953f52b696ec29ca2f332f949cc0050`.
184184
- Final rebuilt XPI after candidate-stage clarity, bilingual wording, translation-aid positioning, v0.1.0 release prep, plugin icon replacement, and deterministic packaging was recopied to the default profile; final local/profile hash: `5515f32cb536bac99d9be67724d8904185ad2b0c97ecc239fec8b44deaca9eb3`.
185185
- Runtime QA for the v0.1.0-beta.1 release candidate verified green candidate extraction on item `9761`, accept flow, OpenAI-compatible BYO callback through a localhost mock endpoint, final Korean meaning fill, cleanup of notes `10011` and `10012`, active generated-note DB counts at zero, and final local/profile XPI hash `ed59c077c289b985b8ebf1f46d33caec3cbeafcc0ea0b35c94f6756c8b5d0afe`.
186+
- Controlled runtime color fixture after v0.1.1-beta.1 release prep:
187+
- `scripts/zotero-runtime-color-fixture.js` creates a temporary Zotero journal article, a linked PDF attachment, and six real underline annotations with `Zotero.Annotations.saveFromJSON`.
188+
- The fixture invokes the Vocab Flow color command path through `VocabFlowMenuManager.runColorForTesting` for green, yellow, blue, purple, red, and gray.
189+
- Zotero `Tools > Developer > Run JavaScript` returned `ok: true` at `2026-06-20T15:15:04.490Z`.
190+
- Results: green `#5fb236` note `10162` candidates `rheology, actuator`; yellow `#ffd400` note `10163` candidates `yellow, dielectric, elastomer`; blue `#2ea8e5` note `10164` candidates `blue, piezoelectric, polymer`; purple `#a28ae5` note `10165` candidates `purple, anisotropic, hydrogel`; red `#ff6666` note `10166` candidates `valence, actuator`; gray `#aaaaaa` note `10167` candidates `gray, ionic, conductor`.
191+
- Each color-specific run checked that candidates included its own marker term and did not include the other fixture color marker terms.
192+
- Cleanup deleted generated note IDs `10162,10163,10164,10165,10166,10167`; fixture item IDs were `10154,10155,10156,10157,10158,10159,10160,10161`.
193+
- Read-only immutable DB check after cleanup found `active_candidate_notes=0` and `active_fixture_items=0` for the fixture item range.
186194

187195
## Acceptance Checklist
188196

@@ -203,6 +211,7 @@ Vocab Flow must not treat every underlined sentence as a final vocabulary list.
203211
- [x] Generated note lookup is robust to missing Zotero tags when the Vocab Flow ownership marker is present.
204212
- [x] Users are not forced to reserve green underlines; multiple color menus and `vocab` tag mode are available.
205213
- [x] Runtime validation confirms color-specific menu registration, green color extraction, empty purple/tag paths, all-underlines extraction, accept flow, and cleanup.
214+
- [x] Controlled Zotero runtime fixture positively verifies green, yellow, blue, purple, red, and gray underline extraction with cleanup.
206215
- [x] Product/UX spec exists and captures non-goals, source-selection rules, data-safety invariants, and residual risks.
207216
- [x] Color-specific feedback names the selected color in candidate notes and empty-result toasts.
208217
- [x] Korean-first candidate exclusion works through `제외` and short `x` tokens.
Lines changed: 201 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
/*
2+
* Zotero Developer > Run JavaScript fixture for Vocab Flow color extraction.
3+
*
4+
* This script creates a temporary Zotero item with one PDF attachment and one
5+
* underline annotation for every supported Vocab Flow candidate color. It then
6+
* invokes the built Vocab Flow menu command path inside Zotero and verifies
7+
* that each color command produces candidates from only its matching annotation.
8+
*
9+
* Run inside Zotero, not Node:
10+
* Tools -> Developer -> Run JavaScript
11+
* paste this file, click Run
12+
*/
13+
14+
const VOCAB_FLOW_COLOR_FIXTURE = {
15+
colors: [
16+
{ name: 'green', color: '#5fb236', marker: 'green rheology actuator' },
17+
{ name: 'yellow', color: '#ffd400', marker: 'yellow dielectric elastomer' },
18+
{ name: 'blue', color: '#2ea8e5', marker: 'blue piezoelectric polymer' },
19+
{ name: 'purple', color: '#a28ae5', marker: 'purple anisotropic hydrogel' },
20+
{ name: 'red', color: '#ff6666', marker: 'red valence actuator' },
21+
{ name: 'gray', color: '#aaaaaa', marker: 'gray ionic conductor' }
22+
],
23+
repoRoot: globalThis.VOCAB_FLOW_REPO_ROOT ?? '/Users/choemun-yeong/workspace/projects/zotero/zotero-vocab-flow'
24+
};
25+
26+
async function runVocabFlowColorFixture() {
27+
const startedAt = new Date().toISOString();
28+
const createdItemIDs = [];
29+
const generatedNoteIDs = [];
30+
let parentItem = null;
31+
let attachmentItem = null;
32+
33+
function assert(condition, message) {
34+
if (!condition) throw new Error(message);
35+
}
36+
37+
async function loadRuntimeExports() {
38+
let code = await IOUtils.readUTF8(`${VOCAB_FLOW_COLOR_FIXTURE.repoRoot}/addon/bootstrap.js`);
39+
code = code.replace(
40+
'return __toCommonJS(bootstrap_exports);',
41+
[
42+
'bootstrap_exports.__runtimeTest = {',
43+
' VocabFlowMenuManager,',
44+
' readUnderlineTexts,',
45+
' extractForItem,',
46+
' generateVocabCandidates,',
47+
' writeCandidateNote,',
48+
' discardCandidateNote',
49+
'};',
50+
'return __toCommonJS(bootstrap_exports);'
51+
].join('\n')
52+
);
53+
const VocabFlowRuntime = eval(code + '\nVocabFlowBootstrap;');
54+
assert(
55+
VocabFlowRuntime.__runtimeTest?.VocabFlowMenuManager,
56+
`Vocab Flow runtime test exports not available; keys=${Object.keys(VocabFlowRuntime || {}).join(',')}`
57+
);
58+
return VocabFlowRuntime.__runtimeTest;
59+
}
60+
61+
async function saveItem(item) {
62+
const id = await item.saveTx();
63+
createdItemIDs.push(id);
64+
return id;
65+
}
66+
67+
async function createFixtureItem() {
68+
const libraryID = Zotero.Libraries.userLibraryID;
69+
parentItem = new Zotero.Item('journalArticle');
70+
parentItem.libraryID = libraryID;
71+
parentItem.setField('title', `Vocab Flow color runtime fixture ${startedAt}`);
72+
const parentID = await saveItem(parentItem);
73+
74+
attachmentItem = new Zotero.Item('attachment');
75+
attachmentItem.libraryID = libraryID;
76+
attachmentItem.parentID = parentID;
77+
attachmentItem.setField('title', 'Vocab Flow color fixture.pdf');
78+
attachmentItem.attachmentContentType = 'application/pdf';
79+
attachmentItem.attachmentLinkMode = Zotero.Attachments.LINK_MODE_LINKED_FILE;
80+
attachmentItem.attachmentPath = 'vocab-flow-color-fixture.pdf';
81+
const attachmentID = await saveItem(attachmentItem);
82+
parentItem.getAttachments = () => [attachmentID];
83+
84+
for (let index = 0; index < VOCAB_FLOW_COLOR_FIXTURE.colors.length; index += 1) {
85+
const fixture = VOCAB_FLOW_COLOR_FIXTURE.colors[index];
86+
const annotation = await Zotero.Annotations.saveFromJSON(attachmentItem, {
87+
key: Zotero.DataObjectUtilities.generateKey(),
88+
type: 'underline',
89+
text: fixture.marker,
90+
color: fixture.color,
91+
pageLabel: '1',
92+
sortIndex: `00001|${String(index + 1).padStart(6, '0')}|00001`,
93+
position: {
94+
pageIndex: 0,
95+
rects: [[72, 72 + index * 18, 240, 84 + index * 18]]
96+
},
97+
tags: []
98+
});
99+
createdItemIDs.push(annotation.id);
100+
}
101+
102+
return parentItem;
103+
}
104+
105+
async function getGeneratedNotes() {
106+
const noteIDs = new Set(parentItem.getNotes?.() ?? []);
107+
const dbNoteIDs = await Zotero.DB.columnQueryAsync(
108+
'SELECT itemID FROM itemNotes WHERE parentItemID=? AND note LIKE ?',
109+
[parentItem.id, '%data-vocab-flow-candidates="review"%']
110+
);
111+
for (const id of dbNoteIDs ?? []) noteIDs.add(id);
112+
return [...noteIDs]
113+
.map((id) => Zotero.Items.get(id))
114+
.filter((note) => note && !note.deleted && String(note.getNote?.() ?? '').includes('data-vocab-flow-candidates="review"'));
115+
}
116+
117+
function extractTerms(note) {
118+
const html = String(note.getNote?.() ?? '');
119+
return [...html.matchAll(/data-vocab-flow-candidate="([^"]+)"/g)].map((match) => match[1]);
120+
}
121+
122+
async function cleanup() {
123+
for (const note of await getGeneratedNotes()) {
124+
generatedNoteIDs.push(note.id);
125+
await note.eraseTx();
126+
}
127+
for (const id of createdItemIDs.slice().reverse()) {
128+
const item = Zotero.Items.get(id);
129+
if (item && !item.deleted) await item.eraseTx();
130+
}
131+
}
132+
133+
try {
134+
const item = await createFixtureItem();
135+
const runtime = await loadRuntimeExports();
136+
const toasts = [];
137+
const manager = new runtime.VocabFlowMenuManager({
138+
extractForItem: (selectedItem, options) => runtime.extractForItem(selectedItem, undefined, { notify: false, ...options }),
139+
toast: (message) => toasts.push(message),
140+
showGeneratedNote: () => {}
141+
});
142+
const results = [];
143+
144+
for (const fixture of VOCAB_FLOW_COLOR_FIXTURE.colors) {
145+
const preflightTexts = runtime.readUnderlineTexts(item, { scope: 'color', color: fixture.color });
146+
const preflightCandidates = runtime.generateVocabCandidates(preflightTexts).map((candidate) => candidate.label);
147+
await manager.runColorForTesting(fixture.color, { items: [item] });
148+
const notes = await getGeneratedNotes();
149+
assert(
150+
notes.length === 1,
151+
`${fixture.name} should leave exactly one generated candidate note, found ${notes.length}; texts=${JSON.stringify(preflightTexts)} candidates=${JSON.stringify(preflightCandidates)} toasts=${JSON.stringify(toasts)}`
152+
);
153+
const terms = extractTerms(notes[0]);
154+
const joined = terms.join(' | ').toLowerCase();
155+
assert(joined.includes(fixture.marker.split(' ')[1]), `${fixture.name} candidates did not include marker term from ${fixture.marker}`);
156+
157+
for (const other of VOCAB_FLOW_COLOR_FIXTURE.colors) {
158+
if (other === fixture) continue;
159+
assert(!joined.includes(other.marker.split(' ')[1]), `${fixture.name} candidates leaked marker from ${other.name}`);
160+
}
161+
162+
results.push({
163+
color: fixture.name,
164+
hex: fixture.color,
165+
noteID: notes[0].id,
166+
candidates: terms
167+
});
168+
generatedNoteIDs.push(notes[0].id);
169+
await notes[0].eraseTx();
170+
}
171+
172+
await cleanup();
173+
return {
174+
ok: true,
175+
startedAt,
176+
parentItemID: parentItem.id,
177+
attachmentItemID: attachmentItem.id,
178+
results,
179+
cleanup: {
180+
generatedNoteIDs,
181+
createdItemIDs,
182+
activeGeneratedNotes: (await getGeneratedNotes()).length
183+
}
184+
};
185+
} catch (error) {
186+
await cleanup();
187+
return {
188+
ok: false,
189+
startedAt,
190+
errorMessage: String(error?.message ?? error),
191+
error: String(error && error.stack ? error.stack : error),
192+
cleanup: {
193+
generatedNoteIDs,
194+
createdItemIDs,
195+
activeGeneratedNotes: (await getGeneratedNotes()).length
196+
}
197+
};
198+
}
199+
}
200+
201+
return runVocabFlowColorFixture();

src/annotationReader.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,15 @@
33

44
export const DEFAULT_CANDIDATE_COLOR = '#5fb236';
55
export const DEFAULT_CANDIDATE_TAG = 'vocab';
6+
export const SUPPORTED_CANDIDATE_COLORS = [
7+
'#5fb236',
8+
'#ffd400',
9+
'#2ea8e5',
10+
'#a28ae5',
11+
'#ff6666',
12+
'#aaaaaa'
13+
] as const;
14+
615
const CANDIDATE_COLOR_LABELS: Record<string, string> = {
716
'#5fb236': '초록',
817
'#ffd400': '노란',

test/annotationReader.test.ts

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import test from 'node:test';
22
import assert from 'node:assert/strict';
3-
import { DEFAULT_CANDIDATE_COLOR, readUnderlineTexts } from '../src/annotationReader';
3+
import { DEFAULT_CANDIDATE_COLOR, SUPPORTED_CANDIDATE_COLORS, readUnderlineTexts } from '../src/annotationReader';
44

55
function annotation(type: string, text: string, sortIndex: string, color = DEFAULT_CANDIDATE_COLOR, tags: string[] = []) {
66
return { annotationType: type, annotationText: text, annotationSortIndex: sortIndex, annotationColor: color, getTags: () => tags };
@@ -65,6 +65,24 @@ test('collects a chosen candidate color without requiring tag typing', () => {
6565
]);
6666
});
6767

68+
test('collects each supported candidate color without leaking other colors', () => {
69+
const textsByColor = new Map(SUPPORTED_CANDIDATE_COLORS.map((color) => [color, `${color} rheology actuator`]));
70+
const annotations = SUPPORTED_CANDIDATE_COLORS.map((color, index) => annotation(
71+
'underline',
72+
textsByColor.get(color)!,
73+
String(index + 1).padStart(5, '0'),
74+
color
75+
));
76+
77+
Zotero.Items.get = () => ({ isPDFAttachment: () => true, getAnnotations: () => annotations });
78+
79+
for (const color of SUPPORTED_CANDIDATE_COLORS) {
80+
assert.deepEqual(readUnderlineTexts({ getAttachments: () => [15] }, { scope: 'color', color }), [
81+
textsByColor.get(color)
82+
]);
83+
}
84+
});
85+
6886
test('collects vocab-tagged underlines regardless of color', () => {
6987
const pdf = {
7088
isPDFAttachment: () => true,

test/runtimeColorFixture.test.ts

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
import test from 'node:test';
2+
import assert from 'node:assert/strict';
3+
import fs from 'node:fs';
4+
import { SUPPORTED_CANDIDATE_COLORS } from '../src/annotationReader';
5+
6+
test('runtime color fixture script covers every supported candidate color', () => {
7+
const script = fs.readFileSync('scripts/zotero-runtime-color-fixture.js', 'utf8');
8+
9+
for (const color of SUPPORTED_CANDIDATE_COLORS) {
10+
assert.ok(script.includes(color), `missing fixture color ${color}`);
11+
}
12+
});

0 commit comments

Comments
 (0)