Repository navigation
Commit f4a5534
feat(gradium): multiplex TTS replies over one shared websocket
Every reply opened its own websocket: TLS and websocket handshake, API
key resolution, then the session. The API supports several sessions on
one socket (`close_ws_on_eos: false`, one `client_req_id` per session,
every message and reply tagged with it), so keep one socket per TTS
instance and run every reply, including parallel ones, as a session on
it.
- A reader task owns the socket and routes each frame to the session it
is tagged with; sessions run in parallel. Frames for a session nobody
listens to any more are dropped, an error frame among them is logged,
a malformed frame is skipped. When the socket dies or is closed, every
open session is handed the error and the next session opens a new
socket.
- `prewarm()` opens the socket when the agent activity starts, so the
first reply does not pay the handshake either.
- A session that does not complete, because the agent interrupted it,
it timed out waiting for the server, or it failed, is cancelled with
the API's `cancel` message (gradium-serve #4604): the server stops
synthesizing it, releases the worker and answers `cancelled` as the
session's last frame, so a retry on the same socket does not run next
to an orphan. The socket and the other sessions on it are unaffected.
- Only the wait for the server's `ready` is bounded by
conn_options.timeout. After that a session waits as long as the socket
lives: the server is silent while it waits for text, and a dead socket
is noticed through the heartbeat, now a fixed interval since there is
no receive timeout left for it to stay ahead of.
- `aclose()` fails the in-flight sessions, closes the streams and the
socket.
- The setup message is built in one place for both streams, and
ChunkedStream no longer carries arguments it does not use.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>1 parent 2bcba15 commit f4a5534
2 files changed
Lines changed: 667 additions & 217 deletions
File tree
- livekit-plugins/livekit-plugins-gradium/livekit/plugins/gradium
- tests
0 commit comments