Skip to content

Commit bfb5035

Browse files
hellpanderrrclaude
andcommitted
docs: capture Morpheus parity findings; de-flake dark-mode e2e
The dark-mode macronizer test clicked straight after goto(), racing the module script that top-level-awaits __wasmReady — intermittently failed on a 60s timeout and passed on retry. Wait for #macronize_btn like the other tests do. Suite drops 1.6m -> 29s with no flake. CLAUDE.md: new 'Macronizer / Morpheus traps' section recording the four diagnostics that cost this session (comma in the Morpheus accented form, the entriesCache staleness, the cruncher.js-vs-.data manifest that makes byte-grepping the blob useless, and the GREEK language default that makes a working cruncher look dead). Annotated the existing 'E2E must wait' lesson as enforced rather than duplicating it. docs/ISSUES.md: new, with stable IDs for the r/latin feedback and CodeRabbit leftovers so they stop being re-derived from the thread. 7 open, 2 fixed. NEXT.md: handoff, with the ruled-out dead ends spelled out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 1cb1771 commit bfb5035

4 files changed

Lines changed: 126 additions & 1 deletion

File tree

wiktionary_pron/CLAUDE.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,10 +134,17 @@ Each of these cost real debugging time in past sessions. Check this list before
134134
- **Never shim `document` in `scripts/tests/setup.cjs`** — wasmoon's Emscripten glue uses `typeof document` for environment detection and dies with "Invalid URL" under Node. Shim it per-test-file only in suites that don't load wasmoon (see `unit/lexicon_decode.test.js`).
135135
- **Lexicon test words must be letters only.** Every lookup path strips `[^\p{Letter}\p{Mark}-]` — synthetic words like `word000042` contain digits, get cleaned to `word`, and silently never match. Cost a failed "language isolation" test until spotted.
136136
- **Macronizer results have empty `textContent`** — words render via `<span class="ipa" content="...">` painted with CSS `attr(content)`. Assert the `content` attribute, not text. (Same idiom in the transcriber's line mode.)
137-
- **E2E must wait for `#lang` to be enabled before any interaction**`main.js` top-level-awaits the wasmoon engine; clicking earlier hits elements with no listeners attached. Symptom: clear/dark-mode/persistence tests fail with stale values.
137+
- **E2E must wait for `#lang` to be enabled before any interaction**`main.js` top-level-awaits the wasmoon engine; clicking earlier hits elements with no listeners attached. Symptom: clear/dark-mode/persistence tests fail with stale values. 2026-08-05: recurred on **macronizer.html**, where the gate is `#macronize_btn` being enabled — the dark-mode test clicked straight after `goto()` and was intermittently flaky (60s timeout, passed on retry). ✅ enforced by the readiness wait in `e2e/macronizer.spec.js`.
138138
- **Browser Latin ≠ Node Latin**: the browser flow macronizes input first (provinciarum → prōvinciārum), so e2e IPA expectations carry length marks that the Node suite's don't.
139139
- **RU/UK stress-transfer skips multi-form dictionary entries by design** — test it with a single-form word (голова → голова́), not вода (record "во́да, вода́" has a comma → skipped).
140140

141+
**Macronizer / Morpheus traps** (2026-08-05)
142+
- **Morpheus returns `"accented_stem,lemma"`** — e.g. `currito_,curro`. The comma must be stripped from the *accented* form (Python does `accented.split(",")[0]`, `postags.py:434-436`). Keeping it corrupts `accentedUnderscore`, breaks DP alignment, and silently mis-macronizes every out-of-wordlist word. Fixed in `MorpheusAnalyzer.parseAnalysisLine`.
143+
- **`WordlistEngine.addEntry` must invalidate `entriesCache`**`ensureAnalyzed` caches the *empty* lookup for a missing word, then Morpheus writes the row; without invalidation `getAccents` re-reads the stale empty cache and falls through to the `tag_to_endings` guess. This is why `currito` came out `currītō` on first visit.
144+
- **Don't byte-grep `cruncher.data` for filenames** — the preload manifest lives in **`cruncher.js`** (`{"files":[...]}` with offsets), not in the `.data` blob. Grepping the blob returns 0 hits for files that *are* packed, which sent a whole session down a false "missing stemlib" trail. Verify by reading the manifest and `cmp`-ing byte ranges.
145+
- **The cruncher defaults to GREEK.** `morpheus_init()` only sets `MORPHLIB`; you must call `morpheus_set_language(32768)` (LATIN) before `morpheus_analyze`, or it looks in `stemlib/Greek/` and returns 0 analyses for *every* word — including trivial ones like `aqua`. Native equivalent: the `-L` flag. A probe that skips this "proves" Morpheus is broken when it isn't.
146+
- **Don't run `build-morpheus-wasm.sh` against a read-write mount of the repo** — it does `rm -rf stemlib/Greek` and will delete 711 tracked files from your working tree. Mount read-only or copy first.
147+
141148
**Code traps**
142149
- **Regex char classes with `-` between Unicode literals form ranges**: `[^\p{L}\p{M}'’-‿]` parsed `’-‿` as U+2019–U+203F and stripped ASCII hyphens from every word. Put `-` last in the class. (Was a live bug in `sanitize()` for years.)
143150
- **`#header > a > i` selects the HOME link's icon** — the dark-mode toggle on macronizer.html restyled the wrong button for this reason. Use `#dark_mode i`.

wiktionary_pron/NEXT.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Next
2+
3+
_Updated 2026-08-05 — branch `main`_
4+
5+
## State
6+
Fixed the macronizer parity bug reported on r/latin: `currito``curritō` and
7+
`diffregit``diffrēgit` now match Winge's original. Two interacting port bugs
8+
(M-001 Morpheus `,lemma` suffix never stripped; M-002 `addEntry` left a stale
9+
`entriesCache`). Verified first-visit *and* return-visit in a real browser;
10+
81 node tests + 3 macronizer e2e green on exit code. Site `main` = `1cb1771`
11+
(live on Pages), engine `macronizer-ui-support` = `797f779`. Both pushed, both
12+
working trees clean.
13+
14+
## Open threads
15+
- **M-003, the obvious next one**: scansion feet disappear when a line number
16+
trails the line. Display-only — start at `scannedFeet[lineIdx]` in
17+
`macronizer.html` (~line 1150). Reproduce with Catullus 1 in hendecasyllable
18+
mode; line 4 shows no foot.
19+
- **Reply to the r/latin thread.** Both macronization bugs are genuinely fixed
20+
and were real bugs in the port — say so. See `docs/ISSUES.md` M-003..M-007
21+
for the rest of that feedback.
22+
- Remaining findings have IDs in `docs/ISSUES.md` (7 open) — read it rather
23+
than re-deriving from the thread.
24+
25+
## Running / unfinished
26+
Nothing running. A local static server may still be on :8993
27+
(`curl -s -o/dev/null -w "%{http_code}" http://127.0.0.1:8993/wiktionary_pron/macronizer.html`);
28+
harmless, and `npm run test:e2e` starts its own.
29+
30+
## Don't redo
31+
- **Morpheus is NOT broken.** A long stretch of last session chased "missing
32+
Latin stemlib indices in `cruncher.data`". False. Two independent agents
33+
refuted it and the shipped WASM analyzes `currito`/`diffregit`/`aqua`
34+
correctly. The byte-grep that "proved" it was meaningless — filenames live in
35+
`cruncher.js`'s manifest, not the `.data` blob.
36+
- **Always `morpheus_set_language(32768)` before analyzing.** Skipping it makes
37+
the cruncher read `stemlib/Greek/` and return 0 for every word, which looks
38+
exactly like a broken build.
39+
- **`macrons.txt`, `tag_to_endings`, the RFTagger model and `toAscii` are all
40+
byte-identical to upstream Alatius.** Already diffed. Not the problem.
41+
- **The `macronizer-py-compare` Docker image is not a usable reference** — its
42+
cruncher defaults to Greek and the build strips `stemlib/Greek`, so it fails
43+
on every word and "agrees" with any wrong output.
44+
- **The scansion of `iam tum, cum ausus es unus Italorum` is not a parity bug.**
45+
Prose and scan modes give identical output in both engines.
46+
- **Never run `build-morpheus-wasm.sh` against a read-write mount of the engine
47+
repo** — its `rm -rf stemlib/Greek` deleted 711 tracked files last session
48+
(restored).

wiktionary_pron/docs/ISSUES.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Issues
2+
3+
Findings that outlived the session in which they were discovered. IDs are
4+
stable and never renumbered; fixed rows stay, with `Status: FIXED` and the
5+
evidence that closed them.
6+
7+
Totals: 7 open, 2 fixed (9 total).
8+
9+
---
10+
11+
## M-001 — Morpheus accented form keeps the `,lemma` suffix
12+
**Status: FIXED** (2026-08-05, engine `797f779` / site `1cb1771`)
13+
`parseAnalysisLine` stored `currito_,curro` as the accented form instead of
14+
`currito_`. Python reference strips it (`postags.py:434-436`). Corrupted DP
15+
alignment for every out-of-wordlist word on return visits.
16+
17+
## M-002 — `addEntry` left a stale `entriesCache`
18+
**Status: FIXED** (2026-08-05, same commits)
19+
`ensureAnalyzed` cached the empty lookup for a missing word; the Morpheus row
20+
written immediately after was invisible to `getAccents`, which fell through to
21+
the `tag_to_endings` heuristic. Broke first visits: `currito``currītō`.
22+
23+
## M-003 — Scansion feet vanish when a line number trails the line
24+
**Status: OPEN.** Reported on r/latin. Reproduced on Catullus 1: line 4
25+
renders no metrical notation while the macronized words stay correct.
26+
Cause is display-side — `scannedFeet[lineIdx]` in `macronizer.html` (~line
27+
1150) misaligns when a trailing line-number token shifts the line index.
28+
Engine macronization is *not* affected. Not a parity bug.
29+
30+
## M-004 — Output is not editable
31+
**Status: OPEN.** Adoption blocker per r/latin feedback: users macronize to
32+
catch typos and normalize spellings (`-īs``-ēs`) and cannot correct the
33+
result. Winge's original site made the output `contenteditable` in May 2017
34+
plus per-vowel click-to-toggle. Substantial build: contenteditable +
35+
re-macronize + re-export + surviving the cycle/click handlers.
36+
37+
## M-005 — Word popup shows no dictionary definition
38+
**Status: OPEN.** Users cannot tell *populus* (people) from *populus* (poplar),
39+
or the two *malus* lemmas, from the readings list alone. Add a dictionary link
40+
per reading — `accentedSources` already carries lemma+tag per row.
41+
42+
## M-006 — Popup buries the useful section under debug detail
43+
**Status: OPEN.** r/latin feedback: "Possible readings" is the only part users
44+
want; the RFTagger/Morpheus detail reads as debug output. Move readings to the
45+
top, collapse the rest behind `<details>`. Also the heading is misleading — it
46+
lists *distinct macronizations*, not all morphological readings (readings that
47+
differ only in a short vowel collapse into one row).
48+
49+
## M-007 — Individual words/lines cannot be selected or copied
50+
**Status: OPEN.** Only the bulk "copy" button works. Structural: words render
51+
via `<span class="ipa" content="…">` painted with CSS `attr(content)`, so the
52+
selectable text is empty, and the popup/cycle handlers own the click.
53+
54+
## M-008 — `rftagger.js` ships an assertions (debug) build
55+
**Status: OPEN.** CodeRabbit finding on PR #7. `assert()` bodies,
56+
`checkStackCookie`, `runtimeDebug` and the missing/unexported symbol tables are
57+
all present, unlike the release-style `cruncher.js` beside it. Inflates the
58+
one-time download and keeps assertions on hot paths. Fix belongs in the engine
59+
repo's build, then re-sync `dist/`.
60+
61+
## M-009 — Deployed WASM assets are untracked
62+
**Status: OPEN.** `macronizer/wasm/cruncher.*` is untracked (not ignored), so
63+
the bytes actually served have no verifiable provenance. Commit them, or record
64+
the build SHA that produced them.

wiktionary_pron/e2e/macronizer.spec.js

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,12 @@ test.describe("macronizer", () => {
4141
// Regression: the toggle used to target '#header > a > i', which is the
4242
// HOME link's icon — the sun appeared on the wrong side of the header.
4343
await page.goto(PAGE);
44+
// The toggle's listener is attached by the module script that top-level-awaits
45+
// __wasmReady, so clicking straight after goto() is a race: the click lands on
46+
// a button with no handler and body never gets the class. Wait for readiness.
47+
await expect(page.locator("#macronize_btn")).toBeEnabled({
48+
timeout: 240_000,
49+
});
4450
await page.click("#dark_mode");
4551
await expect(page.locator("body")).toHaveClass(/dark_mode/);
4652
await expect(page.locator("#dark_mode i")).toHaveClass(/icon-sun/);

0 commit comments

Comments
 (0)