Skip to content

Commit f4a5534

Browse files
hhugo-gradiumclaude
andcommitted
feat(gradium): multiplex TTS replies over one shared websocket
Every reply opened its own websocket: TLS and websocket handshake, API key resolution, then the session. The API supports several sessions on one socket (`close_ws_on_eos: false`, one `client_req_id` per session, every message and reply tagged with it), so keep one socket per TTS instance and run every reply, including parallel ones, as a session on it. - A reader task owns the socket and routes each frame to the session it is tagged with; sessions run in parallel. Frames for a session nobody listens to any more are dropped, an error frame among them is logged, a malformed frame is skipped. When the socket dies or is closed, every open session is handed the error and the next session opens a new socket. - `prewarm()` opens the socket when the agent activity starts, so the first reply does not pay the handshake either. - A session that does not complete, because the agent interrupted it, it timed out waiting for the server, or it failed, is cancelled with the API's `cancel` message (gradium-serve #4604): the server stops synthesizing it, releases the worker and answers `cancelled` as the session's last frame, so a retry on the same socket does not run next to an orphan. The socket and the other sessions on it are unaffected. - Only the wait for the server's `ready` is bounded by conn_options.timeout. After that a session waits as long as the socket lives: the server is silent while it waits for text, and a dead socket is noticed through the heartbeat, now a fixed interval since there is no receive timeout left for it to stay ahead of. - `aclose()` fails the in-flight sessions, closes the streams and the socket. - The setup message is built in one place for both streams, and ChunkedStream no longer carries arguments it does not use. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1 parent 2bcba15 commit f4a5534

2 files changed

Lines changed: 667 additions & 217 deletions

File tree

0 commit comments

Comments
 (0)