@@ -69,6 +69,22 @@ for text in texts:
6969Model initialization is expensive. Warm normalization calls on an existing
7070instance are much faster than loading a new instance for each request.
7171
72+ ## See the difference
73+
74+ The hard part is not turning ` twenty ` into ` 20 ` . It is deciding what the same
75+ spoken surface means in context:
76+
77+ | Transcript | Premove output |
78+ | --- | --- |
79+ | ` can you look up order d l t two nine eight two ` | ` can you look up order DLT2982 ` |
80+ | ` I need to change flight m d o three five one ` | ` I need to change flight MDO351 ` |
81+ | ` the meeting starts at seven thirty six ` | ` the meeting starts at 7:36 ` |
82+ | ` the cash price in dollars was seven thirty six ` | ` the cash price in dollars was $7.36 ` |
83+ | ` my verified number is eight one four two three one four ` | ` my verified number is 814-2314 ` |
84+
85+ The last two rows use the same spoken value. Sentence context selects a time
86+ or a dollar amount without exposing categories or gold metadata to the model.
87+
7288## Command-line interface
7389
7490Normalize one transcript:
@@ -105,45 +121,19 @@ normalized transcripts. Diagnostics and errors use stderr.
105121
106122## Why contextual ITN?
107123
108- Obvious spoken values can often be normalized with fixed rules:
124+ Fixed rules are good at defining valid written forms. They are not enough when
125+ a number sequence can be a time, an identifier, a count, or part of a phone
126+ number. Premove generates valid alternatives with deterministic rules, scores
127+ them against the complete sentence, and selects a compatible set of edits.
109128
110- ``` text
111- twenty dollars → $20
112- ```
129+ ## Architecture
113130
114- The harder cases have several plausible written forms. For example, a number
115- sequence can represent a time, an identifier, a count, or part of a phone
116- number. Premove generates valid alternatives with deterministic rules, then
117- uses the surrounding sentence to score the intended interpretation.
118-
119- ``` text
120- ASR transcript
121- ↓
122- structured Rust candidates
123- ↓
124- full-sentence contextual scores
125- ↓
126- exact interval decoding
127- ↓
128- written transcript
129- ```
130-
131- ## How it works
132-
133- ``` text
134- Spoken ASR text
135- │
136- ▼
137- Rust candidate generation
138- │ TIME / MONEY / PHONE / ID / ...
139- ▼
140- DeBERTa-v3-large contextual scorer
141- │
142- ▼
143- Exact maximum-score decoder
144- │
145- ▼
146- Written transcript
131+ ``` mermaid
132+ flowchart TD
133+ A[ASR transcript] --> B[Rust candidate graph<br/>valid written forms]
134+ B --> C[DeBERTa contextual scorer<br/>full-sentence context]
135+ C --> D[Exact interval decoder<br/>non-overlapping edits]
136+ D --> E[Normalized transcript]
147137```
148138
149139** Rust candidate generation** deterministically proposes valid written
0 commit comments