Value
Users should be able to quickly understand what a specific rollout is doing while gym eval run is still running. This would make it easier to diagnose hangs, slow rollouts, unexpected model behavior, and tool failures without waiting for completion. Some users have reported a single task may run for several hours, and are unable to identify if the task is hung or still ongoing, which can result in wasted compute.
Current Behavior
Gym reports aggregate progress and incrementally writes completed rollouts. However, it does not provide a standardized way to inspect an individual rollout while it is still in progress.
Server logs may contain relevant information, but concurrent rollout activity is interleaved and varies by agent.
Desired Behavior
Users should be able to select or focus on a rollout and see its latest available activity without waiting for it to finish.
This could include:
- Task and rollout identifiers
- Current status and elapsed time
- Recent model and tool activity
- Errors or retries
- Whether the rollout appears stalled
Value
Users should be able to quickly understand what a specific rollout is doing while gym eval run is still running. This would make it easier to diagnose hangs, slow rollouts, unexpected model behavior, and tool failures without waiting for completion. Some users have reported a single task may run for several hours, and are unable to identify if the task is hung or still ongoing, which can result in wasted compute.
Current Behavior
Gym reports aggregate progress and incrementally writes completed rollouts. However, it does not provide a standardized way to inspect an individual rollout while it is still in progress.
Server logs may contain relevant information, but concurrent rollout activity is interleaved and varies by agent.
Desired Behavior
Users should be able to select or focus on a rollout and see its latest available activity without waiting for it to finish.
This could include: