Skip to content

[Fix] Fast sessions stop after interrupted inference retries - #1966

Draft
roomote-roomote[bot] wants to merge 2 commits into
developfrom
fix/recover-interrupted-fast-retries-037yhi7r65hmg
Draft

[Fix] Fast sessions stop after interrupted inference retries#1966
roomote-roomote[bot] wants to merge 2 commits into
developfrom
fix/recover-interrupted-fast-retries-037yhi7r65hmg

Conversation

@roomote-roomote

Copy link
Copy Markdown
Contributor

​Opened on behalf of @daniel-lxs. Follow up by mentioning @roomote-roomote, in the web UI, or in Slack.

What changed

  • Mark Fast inference retries as automatically recoverable only while a human turn remains side-effect-free, and revoke that eligibility durably before any native or integration tool runs.
  • Re-enqueue orphaned eligible retries through the existing durable parent-event inbox after the responding lease expires, then claim and resume them under a new conversation-lock owner using the canonical request, images, attachment context, and supplemental thread context.
  • Keep the existing interruption closeout for legacy, side-effecting, unsupported, raced, or repeatedly interrupted recoveries so ambiguous work is never replayed into duplicate answers or ownership conflicts.

Why this change was made

Fast already retried transient provider failures in process, but a process exit or lost turn owner converted the durable retry marker into “The inference retry was interrupted before it completed. Please send the request again.” Replaying every marker would be unsafe because a prior attempt may already have executed externally visible tools.

Impact

Users on web, Slack, and Discord no longer need to resend requests when infrastructure interrupts a retry before any tool call. Attempts with uncertain side effects still fail closed, and targeted regression coverage exercises retry eligibility, shutdown handoff, lease-gated queue admission, lock-owned recovery, context replay, and repeated-interruption fallback.

@roomote-community

roomote-community Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

No new code issues found. See task

  • Preserve retry recovery context when updating a visible retry notice (packages/cloud-agents/src/server/fast-agent/fast-agent-service.ts:1633).

Reviewed fdeba84

Comment thread packages/cloud-agents/src/server/fast-agent/fast-agent-service.ts
@roomote-roomote

roomote-roomote Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Fixed the BullMQ Test timeout and the recovery-context review finding in fdeba84. See the fixer task.

  • The reconciliation test now mocks durable queue admission, so the CI unit-test environment no longer waits for an unavailable Redis service.
  • Every active retry-notice rewrite preserves attachment and thread recovery context; terminal settlement still removes it.
  • Full BullMQ suite: 473 passed with Redis pointed at a closed port.
  • Fast service suite: 104 passed.
  • Cloud-agents and BullMQ typechecks, formatting, lint, and repository pre-push checks passed.
  • The addressed review thread is resolved, the follow-up Roomote review found no new issues, and the complete CI rerun is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant