feat(message-toolbar): add assistant reply actions and read aloud - #518
feat(message-toolbar): add assistant reply actions and read aloud#518AtomsH4 wants to merge 29 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds mobile-parity assistant reply actions (copy / regenerate / read aloud / stop) to the chat message toolbar, including speakable-text projection, language hinting, and voice selection for Expo TTS, while keeping playback state ephemeral to the chat workspace.
Changes:
- Add assistant message toolbar actions (copy, regenerate, read aloud/stop) and wire them from
ChatWorkspaceintoMessageList/AssistantMessageRow. - Implement read-aloud text projection + Markdown/citation stripping, language hinting, voice resolution, and chunked speech session management with lifecycle cleanup.
- Add i18n strings and Jest coverage for the new TTS utilities and hook (with limited coverage for the UI toolbar wiring).
Reviewed changes
Copilot reviewed 24 out of 25 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/frontend/i18n/locales/zh-cn.json | Adds localized strings for new assistant message actions and read-aloud errors. |
| src/frontend/i18n/locales/en-us.json | Adds English strings for new assistant message actions and read-aloud errors. |
| src/frontend/features/chat/workspace/utils/splitReplyReadAloudText.ts | Splits spoken text into safe chunks with preferred boundaries and surrogate-pair safety. |
| src/frontend/features/chat/workspace/utils/resolveReplyReadAloudVoice.ts | Selects the best matching system voice by base language, exact locale, and quality. |
| src/frontend/features/chat/workspace/utils/tests/splitReplyReadAloudText.test.ts | Tests chunk sizing and boundary selection, including emoji/surrogate handling. |
| src/frontend/features/chat/workspace/utils/tests/resolveReplyReadAloudVoice.test.ts | Tests locale normalization, exact-match preference, quality ordering, and non-mutation. |
| src/frontend/features/chat/workspace/hooks/useReplyReadAloud.ts | Introduces the read-aloud session hook with chunk playback, cancellation, and lifecycle cleanup. |
| src/frontend/features/chat/workspace/hooks/tests/useReplyReadAloud.test.tsx | Adds extensive tests for session races, cleanup behavior, chunk sequencing, and errors. |
| src/frontend/features/chat/workspace/ChatWorkspace.tsx | Wires toolbar actions (copy/regenerate/read-aloud) into MessageList, with user feedback alerts. |
| src/frontend/features/chat/workspace/tests/ChatWorkspace.test.tsx | Updates workspace test scaffolding/mocks for regenerate and read-aloud hook presence. |
| src/frontend/features/chat/README.md | Documents workspace ownership boundaries and read-aloud session behavior. |
| src/frontend/components/messagePresentation/types.ts | Adds AssistantMessageActions + AssistantReadAloudInput and threads actions through list props. |
| src/frontend/components/messagePresentation/README.md | Documents read-aloud projection boundaries and consumer-owned playback state. |
| src/frontend/components/messagePresentation/messageRow/utils/resolveAssistantReadAloudLanguage.ts | Adds script-signal language hinting (explicit targetLanguage first, then CJK/Latin heuristics). |
| src/frontend/components/messagePresentation/messageRow/utils/projectAssistantMessageReadAloud.ts | Projects speakable assistant reply text while excluding code, citations, URLs, etc. |
| src/frontend/components/messagePresentation/messageRow/utils/copyAssistantMessageText.ts | Produces a “controlled copy” payload from visible assistant message parts. |
| src/frontend/components/messagePresentation/messageRow/utils/tests/resolveAssistantReadAloudLanguage.test.ts | Tests explicit language, script detection, thresholds, and documented limitations. |
| src/frontend/components/messagePresentation/messageRow/utils/tests/projectAssistantMessageReadAloud.test.ts | Tests translation preference, Markdown cleanup, citation stripping, and reject cases. |
| src/frontend/components/messagePresentation/messageRow/utils/tests/copyAssistantMessageText.test.ts | Tests copy projection includes only intended visible content. |
| src/frontend/components/messagePresentation/messageRow/components/AssistantMessageRow.tsx | Adds the assistant message toolbar UI and connects to AssistantMessageActions. |
| src/frontend/components/messagePresentation/messageRow/components/tests/AssistantMessageRow.test.tsx | Adds i18n mocking but does not yet cover new toolbar interactions. |
| src/frontend/components/messagePresentation/index.ts | Re-exports new action/input types from the module entrypoint. |
| src/frontend/components/messagePresentation/components/MessageList.tsx | Accepts assistantActions, passes to assistant rows, and updates list extraData. |
| pnpm-lock.yaml | Locks the new expo-speech dependency. |
| package.json | Adds expo-speech dependency for read-aloud support. |
Files not reviewed (1)
- pnpm-lock.yaml: Generated file
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| {actions && message.status !== 'pending' ? ( | ||
| <View className="flex-row items-center" testID="assistant-message-toolbar"> | ||
| {copyText ? ( | ||
| <Button | ||
| accessibilityLabel={t(isCopied ? 'chat.messageActions.copied' : 'common.copy')} | ||
| hitSlop={6} | ||
| icon={isCopied ? <CheckIcon /> : <CopyIcon />} | ||
| onPress={() => actions.onCopy({ messageId: message.id, text: copyText })} |
| <MessageList | ||
| assistantActions={assistantActions} | ||
| key={listRenderKey} |
|
补充从项目规范、React composition 和 React Native 实践角度的 review。整体方向合理,但建议合并前处理下面这些问题:
Cherry UI 使用是符合规范的:这里正确使用了 这个 PR 涉及 30 个文件、约 2.1k 行新增,把工具栏、文本 projection、语言/voice 策略和 native TTS 生命周期放在了一起,review 和回归范围都偏大。建议作者用 拆好分支后可以用 |
|
修改后需要按照最新的AGENTS.md对代码进行调整,添加了一些规范 |
Summary
Why
Assistant replies lacked mobile toolbar parity with the desktop implementation. Using the system default TTS language also caused Chinese and other CJK replies to be spoken with an incorrect voice or cadence.
User impact
Users can copy, regenerate, start, and stop reading assistant replies from the message toolbar. Read-aloud now prefers a compatible installed system voice and explains how to install a missing voice when the device cannot provide one.
Validation
pnpm typecheck:appoxfmtandoxlintpnpm packages:build:app:assembleRelease