Repository navigation
Conversation
…the agent ends the call Under a simulation the simulator asks for the agent's onSimulationEnd verdict over the session channel only after the run ends. When the agent ends the call itself (EndCallTool -> session.shutdown(), USER_INITIATED), AgentSession close tore down the SessionHost right away and the job then deleted the room, so finalize_simulation failed with "room session transport is closed" and the verdict was dropped. On a USER_INITIATED close under a simulation, SessionHost now keeps serving until finalize_simulation is answered, a run_input arrives (left unanswered, which is how the simulator learns the call ended), or 10s pass. Mirrors livekit/agents.
🦋 Changeset detectedLatest commit: 6265c57 The changes in this PR will be included in the next version bump. This PR includes changesets to release 41 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if (reason === CloseReason.USER_INITIATED) { | ||
| // only an agent ending its own session leaves the simulator yet to finalize; a | ||
| // job shutdown has to stay within the supervisor's shutdownProcessTimeout | ||
| await this.sessionHost?._holdForSimulationFinalize(); |
There was a problem hiding this comment.
🔴 Room deletion drops simulation verdict
With deleteRoomOnClose enabled, RoomIO deletes the room before _holdForSimulationFinalize begins. The simulator cannot deliver finalizeSimulation, so the agent's verdict is lost.
Learn more
A session's Close listeners run before the new hold. RoomIO.onAgentSessionClose starts room deletion directly when deleteRoomOnClose is enabled. The host uses that room's byte-stream handler to receive finalization, so waiting afterward cannot preserve the channel. The end-call tool uses a job shutdown callback instead, which runs after the session closes.
Example: A simulated agent starts with inputOptions: { deleteRoomOnClose: true } and calls session.shutdown(). The Close listener deletes the room while the host waits for finalization; the later request cannot reach the host and the verdict is omitted.
Recommended fix: Defer RoomIO's deletion until after the simulation hold and response, while retaining existing deletion behavior for other close reasons. Test with a real AgentSession and RoomIO using deleteRoomOnClose: true.
Was this helpful? React with 👍 or 👎 to provide feedback.
| if (reason === CloseReason.USER_INITIATED) { | ||
| // only an agent ending its own session leaves the simulator yet to finalize; a | ||
| // job shutdown has to stay within the supervisor's shutdownProcessTimeout | ||
| await this.sessionHost?._holdForSimulationFinalize(); |
There was a problem hiding this comment.
🟡 External shutdown stalls simulated jobs
When a simulated job shuts down externally, finalizeSession calls session.close(), which chooses USER_INITIATED. The hold then waits ten seconds without finalization, delaying room disconnect and shutdown callbacks.
Learn more
The shutdown path finalizeSession calls session.close() regardless of why the job stopped. AgentSession.close schedules closing with USER_INITIATED if no close is already in progress. The new reason check thus treats an externally requested job shutdown as an agent hangup under a simulation. With no request to release the hold, cleanup runs only after its 10-second timeout.
Example: A worker terminates a simulated job while the agent session remains active. Job cleanup calls session.close(), and the room stays connected for another ten seconds instead of disconnecting immediately.
Recommended fix: Pass the actual job shutdown reason into session cleanup or distinguish an agent-initiated close from the cleanup call before entering the hold. Add a lifecycle test for external job shutdown with a real session and host.
Was this helpful? React with 👍 or 👎 to provide feedback.
Problem
When an agent ends the call itself during a simulation, its
onSimulationEndverdict is silently dropped.The simulator asks for the agent's verdict with
finalize_simulationonly after the run is over and it has judged the transcript, which takes a few seconds of LLM calls. When the agent hangs up (for example withEndCallTool),AgentSessioncloses itsSessionHostright after the Close event, andEndCallTool's defaultdeleteRoom = truethen deletes the room. By the time the simulator sendsfinalize_simulation, nothing is listening, and the run is reported using only the simulator's own verdict.Fix
This is the same change as livekit/agents#7986. A session that the agent closes itself (
CloseReason.USER_INITIATED) under a simulation keeps itsSessionHostserving requests until the simulator finalizes, for at most 10s. Job shutdown awaitssession.close()before it disconnects and runs shutdown callbacks, sodeleteRoomnow waits for the verdict too.finalize_simulation.onSimulationEndruns and its verdict is returned.run_input. The simulator is still driving turns. The host doesn't answer, so the simulator sees the agent leave and recordsagent_ended, as it does today.The wire protocol is unchanged.
Verification
remote_session.test.tsmirror the Python ones: the hold ends on finalize,run_inputreleases it without a reply, it gives up after 10s, and it's a no-op outside a simulation.vitest run agents/src/voice agents/src/jobpasses (1494 tests), as dopnpm typecheckandpnpm api:check.