Skip to content

[LLM] Implement reader engine, sentence segmentation, and session state machine - #17

Open
menny wants to merge 1 commit into
toolbox-enginesfrom
reader-engine
Open

menny wants to merge 1 commit into
toolbox-enginesfrom
reader-engine

Conversation

@menny

@menny menny commented Oct 17, 2026

Copy link
Copy Markdown
Member

Overview

This PR implements the interactive reader engine in //core/reader, providing sentence segmentation, automated interlinear glossing, token inspection, and the session state machine for both curated lessons and custom/imported texts.

Components Implemented

  1. Reader Domain Models (//core/reader:ReaderModels.kt):
    • ReaderMode: SENTENCE_FOCUS and PARAGRAPH.
    • ReadingSource: Curated(lesson), CustomText(customText), and RawText(title, content).
    • EnrichedSentence: tokens, interlinear glosses, translation scaffolding, and grammar callouts.
    • TokenInspection: lemma, POS, English gloss, cognate rule notes, false cognate trap cards, clitic surgery deconstructions, and vault status.
    • ReaderSessionState & SessionSummary.
  2. Sentence Segmenter (SentenceSegmenter.kt):
    • Robust Spanish sentence boundary segmentation handling inverted question/exclamation marks (¿, ¡) and abbreviation boundary preservation (Dr., Sr., Dra., etc.).
  3. Interlinear Aligner (InterlinearAligner.kt):
    • On-demand literal 1-to-1 word gloss generation for freeform texts derived from dictionary entries, clitic deconstructions, and false cognate disambiguations.
  4. Token Inspector (TokenInspector.kt):
    • Deep inspection builder linking tokens to cognate rules, clitic surgery, and learner vault encounter logging.
  5. Session Sentence Builder (SessionSentenceBuilder.kt):
    • Builds enriched sentences for both curated lessons and custom passages.
  6. Reader Session Engine (ReaderSessionEngine.kt):
    • Pure Kotlin state machine coordinating sentence navigation (next, previous, jump), mode switching, interlinear and translation toggles, token selection/deselection, vault status updates, and session summary metrics.
  7. Unit Tests:
    • SentenceSegmenterTest: Spanish sentence segmentation, inverted marks, and abbreviations.
    • InterlinearAlignerTest: 1-to-1 literal gloss alignment, false cognate prioritization, clitic compounds.
    • ReaderSessionEngineTest: Curated and custom text loading, navigation, toggles, token inspection, encounter logging, and vault actions.

Verification

  • bazelisk run //:format and bazelisk run //:format.check passed.
  • bazelisk build --config=lint //... (Detekt) passed with 0 issues.
  • bazelisk test //... passed all 25 tests across the workspace.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant