Skip to content
This repository was archived by the owner on Aug 3, 2026. It is now read-only.

fix: bound batch preflight retries and timeouts - #683

Merged
smtmfft merged 4 commits into
mainfrom
fix/batch-preflight-timeout-retry
Apr 2, 2026
Merged

fix: bound batch preflight retries and timeouts#683
smtmfft merged 4 commits into
mainfrom
fix/batch-preflight-timeout-retry

Conversation

@johntaiko

Copy link
Copy Markdown
Contributor

Why

  • Prevent batch guest input requests from hanging indefinitely when upstream RPC stalls or times out.
  • Surface stuck preflight and SP1 network waits as explicit failures instead of leaving tasks stuck in Registered.

How

  • Add configurable RPC connect/request timeouts plus bounded retry with exponential backoff in the batch preflight provider.
  • Wrap backend request execution with per-request top-level timeouts so stalled guest input or proof jobs resolve to Failed.
  • Make SP1 network proof wait timeout configurable and add an early batch preflight log before the first RPC fetch.

Tests

  • cargo fmt --all
  • cargo check -p raiko-reqactor --features sp1
  • cargo check -p raiko-host --features sp1

Copilot AI review requested due to automatic review settings March 31, 2026 09:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens batch preflight/proving execution against upstream stalls by introducing bounded RPC timeouts/retries and top-level request timeouts, so tasks fail explicitly instead of hanging indefinitely.

Changes:

  • Add per-request backend timeouts (env-configurable) so guest input/proof/aggregation jobs transition to Failed on stalls.
  • Add RPC client connect/request timeouts plus bounded retry w/ exponential backoff for batched RPC fetching in preflight.
  • Make SP1 network wait_proof timeout configurable and add an early batch preflight log line before RPC fetching.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

File Description
reqactor/src/backend.rs Wraps each request execution in a tokio::time::timeout with env-configurable durations to prevent stuck jobs.
provers/sp1/driver/src/lib.rs Replaces hard-coded SP1 network wait timeout with an env-configurable function.
core/src/provider/rpc.rs Introduces RPC client timeouts and a retry helper, applying it to batched block/account/storage/proof fetches.
core/src/preflight/mod.rs Adds a log line before the first batch preflight RPC fetch for better observability.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread core/src/provider/rpc.rs
Comment thread provers/sp1/driver/src/lib.rs Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

@smtmfft smtmfft left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think retry isn’t essential (though it doesn’t hurt), as the first timeout error will be caught by the task DB, and a new task will be triggered later by following client requests.

Comment thread core/src/provider/rpc.rs Outdated
@smtmfft
smtmfft added this pull request to the merge queue Apr 2, 2026
Merged via the queue into main with commit a714846 Apr 2, 2026
5 checks passed
@smtmfft
smtmfft deleted the fix/batch-preflight-timeout-retry branch April 2, 2026 02:16
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants