Instrument evaluation with timers - #1145
Draft
alexmillane wants to merge 2 commits into
Draft
Conversation
Merged
cvolkcvolk
added a commit
that referenced
this pull request
Aug 27, 2026
## Summary Add a reusable timer utility ## Detailed description - Extract Alex's timer utility and focused tests from #1145. - Provide named wall-time statistics, NVTX ranges, optional CUDA synchronization, table output, and JSON helpers. - Leave experiment, policy, video, and OSMO instrumentation out of this PR. - Existing runtime behavior is unchanged until callers adopt the utility. --------- Signed-off-by: alex <amillane@nvidia.com> Signed-off-by: Clemens Volk <cvolk@nvidia.com> Co-authored-by: alex <amillane@nvidia.com>
- Copy the nvblox_next Timer into isaaclab_arena/utils/timer.py, adding a JSON writer and a cross-process merge helper for combining per-Run records. - Instrument the Experiment Runner at three levels: Experiment phases, per-Run environment/policy construction and rollout, and the per-step policy and env calls in rollout_policy. - Print a timing table at the end of an Experiment and write the same data to arena_experiment_timings.json beside arena_experiment_result.json. - Collect each completed Run's timings in the OSMO output stage and aggregate them into one Experiment timings file holding per-Run records and totals. Signed-off-by: alex <amillane@nvidia.com>
- Add EnvStepTimerWrapper, a gym wrapper that records the wall time of the env below it, and insert it under the recorders in wrap_env_for_video so rollout/env_step_without_video excludes all recording work. - Time the camera recorder's per-step frame copy and encoder writes as video/camera_frame_write, and its per-episode encoder shutdown as video/camera_finalize. - The viewport recorder's render cost is the remainder between the outer rollout/env_step and these timers. Signed-off-by: alex <amillane@nvidia.com>
alexmillane
force-pushed
the
alex/poc/timing
branch
from
September 3, 2026 07:54
a025e8c to
ad54848
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Times the evaluation pipeline using the merged
Timerutility.Detailed description
Timerutility this builds on has since landed onmain, so this branch now only adds the instrumentation.EnvStepTimerWrapperbeneath the video recorders so the recording cost (camera frame writes and encoder finalize) separates from the env step itself.arena_experiment_timings.jsonbeside each Experiment Runner output, and the OSMO collect task merges every completed Run's timings into one Experiment-level file with per-timer totals.