Skip to content

Commit 1ad45b9

Browse files
authored
fix(ai-components): restart useHeardText for a new text and treat Infinity as fully heard (#3349)
### Goal Bring the `useHeardText` fixes from the v15 port (#3348) to the v14 line, which shipped the hook in 14.14.1 (#3343, #3344) and shares the same issues. When the same mounted component rendered a new reply, the hook kept the previous reply's reveal length, `skip()` and `live`, so the new text could appear already revealed. `chars: Infinity` revealed nothing, contrary to the type doc. The docs also suggested returning `undefined` for text the voice source does not know yet, which shows the full text before it is heard and defeats pacing. ### Implementation details Cherry-pick of `835c3eb66` from `feat/ai-components-use-heard-text`. The hook and its tests were byte-identical between the two lines before that commit, and the README hunks applied without conflict. - The reveal state now records the text it refers to. When `text` is replaced by a different text, the reveal starts over: the length shown and `skip()` are reset, and `live` is read again. Growing or truncating the text keeps the reveal, capped at the new length. The reset happens during render, so the new text never renders with the previous reveal. - `chars: Infinity` reveals the whole text; NaN, negative and non-numeric counts still reveal nothing. - Docs (type doc and `README.md`): `undefined` is only for a text that is not spoken at all, and a text that will be spoken but is not known yet must return `{ chars: 0, done: false }`. `subscribe` is bound once (remount with a `key` to switch sources) and is the only wake-up for a settled hook. `heard` is called during render, so it must be free of side effects. - 6 new tests in `useHeardText.test.ts` cover the new-text reset, truncation, `skip()` and `live` reset, and `Infinity`. No public API change: the exported types and signature are unchanged. ### UI Changes None. The hook's reveal behaviour changes only for a component that renders a different text, and for `chars: Infinity`.
1 parent 2380514 commit 1ad45b9

3 files changed

Lines changed: 192 additions & 43 deletions

File tree

‎src/plugins/AIComponents/README.md‎

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -588,17 +588,17 @@ type HeardTextSubscribe = (listener: () => void) => () => void;
588588
const useHeardText: (params: UseHeardTextParams) => UseHeardTextResult;
589589
```
590590

591-
`heard(now)` returns how many characters of the text the user has heard by `now` (a `performance.now()` timestamp) and whether the speech for it has finished. `chars` counts Unicode code points (`Array.from(text).length`), not UTF-16 code units. Return `undefined` when the text is not paced, and `{ chars: 0, done: false }` for a spoken text whose audio has not started yet.
591+
`heard(now)` returns how many characters of the text the user has heard by `now` (a `performance.now()` timestamp) and whether the speech for it has finished. `chars` counts Unicode code points (`Array.from(text).length`), not UTF-16 code units; `Infinity` means the whole text. Return `undefined` only for a text that is not spoken at all. A text that will be spoken but whose audio has not started, or that the voice source does not know yet, must return `{ chars: 0, done: false }`: while `heard` returns `undefined` the full text is shown, and the reveal never moves back past what has been shown. `heard` is also called during render, so keep it free of side effects.
592592

593593
#### Props
594594

595-
| Name | Type | Required | Description |
596-
| ------------ | -------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------ |
597-
| `text` | `string` | yes | The full text to reveal. |
598-
| `heard` | `HeardTextFunction` | yes | The speech progress of `text`. |
599-
| `subscribe` | `HeardTextSubscribe` | no | Change notifications from the speech progress source; each one makes the hook check `heard` again. |
600-
| `generating` | `boolean` | no | `true` while the text is still being written. The full text is revealed once `heard` reports `done` and this is falsy. |
601-
| `live` | `boolean` | no | Defaults to `true`. `false` reveals the full text at once, for example for a message loaded from history. Read at mount. |
595+
| Name | Type | Required | Description |
596+
| ------------ | -------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------- |
597+
| `text` | `string` | yes | The full text to reveal. |
598+
| `heard` | `HeardTextFunction` | yes | The speech progress of `text`. |
599+
| `subscribe` | `HeardTextSubscribe` | no | Change notifications from the speech progress source; each one makes the hook check `heard` again. See below. |
600+
| `generating` | `boolean` | no | `true` while the text is still being written. The full text is revealed once `heard` reports `done` and this is falsy. |
601+
| `live` | `boolean` | no | Defaults to `true`. `false` reveals the full text at once, for example for a message loaded from history. See below. |
602602

603603
#### Returns
604604

@@ -610,10 +610,11 @@ const useHeardText: (params: UseHeardTextParams) => UseHeardTextResult;
610610

611611
Behaviour:
612612

613-
- It reveals the first `floor(chars)` code points of `text` and never moves backwards. Emoji and other astral characters are never split.
614-
- While the text is paced and its speech is not done, it samples `heard` on every animation frame, and stops when the speech is done or the component unmounts. `subscribe` notifications and changes of `text` or `generating` make it check again.
613+
- It reveals the first `floor(chars)` code points of `text` and never moves backwards while `text` grows or is truncated. Emoji and other astral characters are never split.
614+
- When `text` is replaced by a different text (the next reply rendered by the same component), the reveal starts over: the length shown and `skip()` are reset, and `live` is read again. `live` is otherwise read only at mount.
615+
- While the text is paced and its speech is not done, it samples `heard` on every animation frame, and stops when the speech is done or the component unmounts. `subscribe` notifications and changes of `text` or `generating` make it check again. Without `subscribe`, nothing else does: pass it when `heard` can change after the speech is done, or start returning a value after returning `undefined`.
615616
- If `heard` returns `undefined` at first and a value later, the reveal continues from the length already shown.
616-
- `heard` and `subscribe` are read through refs: passing new functions on every render (inline arrow functions) does not restart the pacing. `subscribe` is called once and unsubscribed on unmount.
617+
- `heard` and `subscribe` are read through refs: passing new functions on every render (inline arrow functions) does not restart the pacing. `subscribe` is called once and unsubscribed on unmount; to switch to another speech source, remount the component (for example with a `key`).
617618

618619
#### Example
619620

‎src/plugins/AIComponents/__tests__/useHeardText.test.ts‎

Lines changed: 85 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,14 +72,31 @@ describe('useHeardText', () => {
7272
expect(result.current.heardText).toBe(TEXT.slice(0, 4));
7373
});
7474

75-
it('treats non-finite and negative counts as nothing heard', () => {
75+
it('treats NaN, negative and non-numeric counts as nothing heard', () => {
7676
let progress: HeardTextProgress = { chars: Number.NaN, done: false };
7777
const { result } = renderHeardText({ heard: () => progress, text: TEXT });
7878
expect(result.current.heardText).toBe('');
7979

8080
progress = { chars: -3, done: false };
8181
advance(FRAME_MS);
8282
expect(result.current.heardText).toBe('');
83+
84+
progress = { chars: Number.NEGATIVE_INFINITY, done: false };
85+
advance(FRAME_MS);
86+
expect(result.current.heardText).toBe('');
87+
88+
// a JS caller returning no count at all
89+
progress = { done: false } as unknown as HeardTextProgress;
90+
advance(FRAME_MS);
91+
expect(result.current.heardText).toBe('');
92+
});
93+
94+
it('treats an infinite count as the whole text heard', () => {
95+
const { result } = renderHeardText({
96+
heard: () => ({ chars: Number.POSITIVE_INFINITY, done: false }),
97+
text: TEXT,
98+
});
99+
expect(result.current).toMatchObject({ heardText: TEXT, isPaced: true });
83100
});
84101

85102
it('never moves the reveal backwards', () => {
@@ -290,6 +307,73 @@ describe('useHeardText', () => {
290307
expect(result.current.heardText).toBe(TEXT);
291308
});
292309

310+
describe('when the text is replaced by a different text', () => {
311+
const NEXT_TEXT = 'Sure, here is the forecast for tomorrow.';
312+
313+
it('starts a new reveal instead of carrying over the length already shown', () => {
314+
let progress: HeardTextProgress = { chars: 100, done: false };
315+
const heard: HeardTextFunction = () => progress;
316+
const { rerender, result } = renderHeardText({ heard, text: TEXT });
317+
expect(result.current.heardText).toBe(TEXT);
318+
319+
// the next reply, whose audio has not started yet
320+
progress = { chars: 0, done: false };
321+
rerender({ heard, text: NEXT_TEXT });
322+
expect(result.current).toMatchObject({ heardText: '', isPaced: true });
323+
324+
progress = { chars: 4, done: false };
325+
advance(FRAME_MS);
326+
expect(result.current.heardText).toBe('Sure');
327+
});
328+
329+
it('starts a new reveal after the previous text was shown unpaced', () => {
330+
let progress: HeardTextProgress | undefined = undefined;
331+
const heard: HeardTextFunction = () => progress;
332+
const { rerender, result } = renderHeardText({ heard, text: TEXT });
333+
expect(result.current).toMatchObject({ heardText: TEXT, isPaced: false });
334+
335+
progress = { chars: 2, done: false };
336+
rerender({ heard, text: NEXT_TEXT });
337+
expect(result.current).toMatchObject({ heardText: 'Su', isPaced: true });
338+
});
339+
340+
it('clears a previous skip', () => {
341+
const heard: HeardTextFunction = () => ({ chars: 3, done: false });
342+
const { rerender, result } = renderHeardText({ heard, text: TEXT });
343+
act(() => result.current.skip());
344+
expect(result.current.heardText).toBe(TEXT);
345+
346+
rerender({ heard, text: NEXT_TEXT });
347+
expect(result.current.heardText).toBe('Sur');
348+
});
349+
350+
it('reads live again for the new text', () => {
351+
const heard: HeardTextFunction = () => ({ chars: 3, done: false });
352+
const { rerender, result } = renderHeardText({ heard, live: false, text: TEXT });
353+
expect(result.current.heardText).toBe(TEXT);
354+
355+
rerender({ heard, live: true, text: NEXT_TEXT });
356+
expect(result.current.heardText).toBe('Sur');
357+
});
358+
359+
it('keeps the reveal when the text is truncated, and does not reuse the cut length', () => {
360+
let progress: HeardTextProgress = { chars: 12, done: false };
361+
const heard: HeardTextFunction = () => progress;
362+
const { rerender, result } = renderHeardText({ heard, text: TEXT });
363+
expect(result.current.heardText).toBe(TEXT.slice(0, 12));
364+
365+
// e.g. the reply was cut short when the user stopped it
366+
rerender({ heard, text: TEXT.slice(0, 8) });
367+
expect(result.current.heardText).toBe(TEXT.slice(0, 8));
368+
369+
// growing again from the truncated text continues from 8, not from 12
370+
progress = { chars: 3, done: false };
371+
rerender({ heard, text: `${TEXT.slice(0, 8)} again and again` });
372+
advance(FRAME_MS);
373+
expect(result.current.heardText).toBe(TEXT.slice(0, 8));
374+
});
375+
});
376+
293377
describe('with emoji and other astral characters', () => {
294378
// waving hand + skin tone modifier, mathematical script X, grinning face, the
295379
// US flag (two regional indicators) and a BMP character: all but the last are

‎src/plugins/AIComponents/hooks/useHeardText.ts‎

Lines changed: 95 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,14 @@ export type HeardTextProgress = {
1515

1616
/**
1717
* Returns how much of the text the user has heard by `now` (a `performance.now()`
18-
* timestamp). Return `undefined` when the text is not paced (not spoken, or not known
19-
* to the voice source yet); return `{ chars: 0, done: false }` for a spoken text whose
20-
* audio has not started yet.
18+
* timestamp).
19+
*
20+
* Return `undefined` only for a text that is not spoken at all: the full text is then
21+
* shown, and the reveal never moves back past what has been shown. A text that will be
22+
* spoken but whose audio has not started, or that the voice source does not know yet,
23+
* must return `{ chars: 0, done: false }`, or it is shown in full before it is heard.
24+
*
25+
* Also called during render, so keep it free of side effects.
2126
*/
2227
export type HeardTextFunction = (now: number) => HeardTextProgress | undefined;
2328

@@ -39,10 +44,16 @@ export type UseHeardTextParams = {
3944
generating?: boolean;
4045
/**
4146
* Defaults to `true`. `false` reveals the full text at once, for example for a message
42-
* loaded from history. Read once, when the hook mounts.
47+
* loaded from history. Read when the hook mounts and when `text` is replaced by a
48+
* different text, not on every change.
4349
*/
4450
live?: boolean;
45-
/** Optional change notifications from the speech progress source. */
51+
/**
52+
* Optional change notifications from the speech progress source. Without it, `heard`
53+
* is only checked again while the text is paced and its speech is not done, and when
54+
* `text` or `generating` change. Subscribed once; to switch to another source, remount
55+
* the component (for example with a `key`).
56+
*/
4657
subscribe?: HeardTextSubscribe;
4758
};
4859

@@ -56,13 +67,22 @@ export type UseHeardTextResult = {
5667
};
5768

5869
type PacingState = {
59-
/** Number of code points revealed. Never decreases. */
70+
/** Number of code points revealed. Never decreases while the text continues. */
6071
count: number;
6172
isPaced: boolean;
6273
/** No more ticking is needed until the inputs change or the source notifies. */
6374
settled: boolean;
6475
};
6576

77+
/** The reveal of one text. Starts over when the text is replaced by a different text. */
78+
type RevealState = PacingState & {
79+
/** `live` as read when the reveal started. */
80+
live: boolean;
81+
skipped: boolean;
82+
/** The text `count` refers to. */
83+
text: string;
84+
};
85+
6686
const now = () =>
6787
typeof performance !== 'undefined' && typeof performance.now === 'function'
6888
? performance.now()
@@ -121,9 +141,10 @@ const sample = ({
121141
return { count: Math.max(previousCount, textLength), isPaced: true, settled: true };
122142
}
123143

124-
const heardChars = Number.isFinite(progress.chars)
125-
? Math.max(0, Math.floor(progress.chars))
126-
: 0;
144+
// `+Infinity` means the whole text has been heard; NaN, negative and non-numeric
145+
// counts mean nothing has
146+
const flooredChars = Math.floor(progress.chars);
147+
const heardChars = flooredChars > 0 ? flooredChars : 0;
127148

128149
return {
129150
count: Math.max(previousCount, Math.min(heardChars, textLength)),
@@ -132,6 +153,39 @@ const sample = ({
132153
};
133154
};
134155

156+
const startReveal = ({
157+
generating,
158+
heard,
159+
live,
160+
text,
161+
textLength,
162+
}: {
163+
generating: boolean;
164+
heard: HeardTextFunction;
165+
live: boolean;
166+
text: string;
167+
textLength: number;
168+
}): RevealState => ({
169+
...sample({ generating, heard, live, previousCount: 0, skipped: false, textLength }),
170+
live,
171+
skipped: false,
172+
text,
173+
});
174+
175+
/**
176+
* Whether `text` continues the reveal of `previous.text`: it extends or truncates the part
177+
* already shown. Anything else (the next reply rendered by the same component) is a new
178+
* text, whose reveal must not start from the length shown for the previous one.
179+
*/
180+
const continuesReveal = (previous: RevealState, text: string) => {
181+
const offsets = getCodePointOffsets(previous.text);
182+
const shown = previous.text.slice(
183+
0,
184+
offsets[Math.min(previous.count, offsets.length - 1)],
185+
);
186+
return text.startsWith(shown) || shown.startsWith(text);
187+
};
188+
135189
const noop = () => undefined;
136190

137191
/**
@@ -141,20 +195,24 @@ const noop = () => undefined;
141195
* backend and has no dependency on the chat SDK.
142196
*
143197
* - Reveals the first `floor(chars)` Unicode code points of `text` (emoji and other
144-
* astral characters are never split) and never moves backwards.
198+
* astral characters are never split) and never moves backwards while `text` grows or
199+
* is truncated. When `text` is replaced by a different text (the next reply rendered
200+
* by the same component), the reveal starts over: the length shown, `skip()` and
201+
* `live` are reset.
145202
* - While the text is paced and its speech is not done, samples `heard` on every
146203
* animation frame (a 16 ms timeout without `requestAnimationFrame`). It stops ticking
147204
* when the speech is done or the component unmounts; `subscribe` notifications, and
148205
* changes of `text` or `generating`, make it check again.
149206
* - Reveals the full text once `heard` reports `done` and `generating` is falsy, when
150-
* `live` is `false` at mount, and after `skip()`.
207+
* `live` is `false`, and after `skip()`.
151208
* - When `heard` returns `undefined`, `isPaced` is `false` and `heardText` is the full
152209
* text. If it becomes paced later, the reveal continues from the length already shown.
153210
*
154211
* `heard` and `subscribe` are read through refs, so passing new function identities on
155212
* every render (inline arrow functions) does not restart the pacing. `subscribe` is
156213
* called once on mount (and again only if it goes from absent to present) and its
157-
* unsubscribe function runs on unmount.
214+
* unsubscribe function runs on unmount; to switch to another speech source, remount the
215+
* component. `heard` is also called during render, so keep it free of side effects.
158216
*
159217
* @example
160218
* ```tsx
@@ -186,21 +244,23 @@ export const useHeardText = ({
186244
const subscribeRef = useRef(subscribe);
187245
subscribeRef.current = subscribe;
188246

189-
const [isLive] = useState(live);
190-
const [skipped, setSkipped] = useState(false);
191-
const [pacingState, setPacingState] = useState<PacingState>(() =>
192-
sample({
193-
generating,
194-
heard,
195-
live: isLive,
196-
previousCount: 0,
197-
skipped,
198-
textLength,
199-
}),
247+
const [revealState, setRevealState] = useState<RevealState>(() =>
248+
startReveal({ generating, heard, live, text, textLength }),
200249
);
201-
const pacingStateRef = useRef(pacingState);
250+
let reveal = revealState;
251+
if (revealState.text !== text) {
252+
// Updated during render so that the new text never renders with the previous reveal.
253+
reveal = continuesReveal(revealState, text)
254+
? { ...revealState, count: Math.min(revealState.count, textLength), text }
255+
: startReveal({ generating, heard, live, text, textLength });
256+
setRevealState(reveal);
257+
}
258+
const revealRef = useRef(reveal);
259+
revealRef.current = reveal;
202260
const wakeRef = useRef<() => void>(noop);
203261

262+
const { live: isLive, skipped } = reveal;
263+
204264
useEffect(() => {
205265
let cancelFrame: (() => void) | undefined;
206266
let disposed = false;
@@ -209,7 +269,7 @@ export const useHeardText = ({
209269
cancelFrame = undefined;
210270
if (disposed) return;
211271

212-
const previous = pacingStateRef.current;
272+
const previous = revealRef.current;
213273
const next = sample({
214274
generating,
215275
heard: heardRef.current,
@@ -224,8 +284,9 @@ export const useHeardText = ({
224284
next.isPaced !== previous.isPaced ||
225285
next.settled !== previous.settled
226286
) {
227-
pacingStateRef.current = next;
228-
setPacingState(next);
287+
const updated = { ...previous, ...next };
288+
revealRef.current = updated;
289+
setRevealState(updated);
229290
}
230291

231292
if (!next.settled) cancelFrame = scheduleFrame(tick);
@@ -249,13 +310,16 @@ export const useHeardText = ({
249310
return subscribeRef.current?.(() => wakeRef.current());
250311
}, [hasSubscribe]);
251312

252-
const skip = useCallback(() => setSkipped(true), []);
313+
const skip = useCallback(
314+
() => setRevealState((previous) => ({ ...previous, skipped: true })),
315+
[],
316+
);
253317

254318
return {
255-
heardText: pacingState.isPaced
256-
? text.slice(0, codePointOffsets[Math.min(pacingState.count, textLength)])
319+
heardText: reveal.isPaced
320+
? text.slice(0, codePointOffsets[Math.min(reveal.count, textLength)])
257321
: text,
258-
isPaced: pacingState.isPaced,
322+
isPaced: reveal.isPaced,
259323
skip,
260324
};
261325
};

0 commit comments

Comments
 (0)