Commit 59df9b8
Makes DataGenConfig.max_num_failures actually bound a Mimic generation run (#7433)
# Description
`DataGenConfig.max_num_failures` is documented as *"Maximum number of
failures allowed before stopping generation"* and eighteen shipped Mimic
environment configs set it to `25`. Nothing reads it.
```console
$ git grep -c max_num_failures
source/isaaclab/isaaclab/envs/mimic_env_cfg.py:1 # the definition
source/isaaclab_mimic/isaaclab_mimic/envs/*.py:18 # eighteen assignments
```
No third entry: there is no read anywhere in the repository, and the
field has been inert since Isaac Lab Mimic was introduced in #179.
`env_loop` counts `num_failures` and then never consults it; its only
termination is
```python
check_val = num_success if generation_guarantee else num_attempts
if check_val >= generation_num_trials:
```
With `generation_guarantee = True` (which those same eighteen configs
also set), a task whose success rate is low retries without any bound,
and the one knob that claims to stop that does nothing. On our own data
generation this cost 337 attempts for 10 demos before we discovered the
cap was not real.
### Reproduction, on a stock task
```bash
./isaaclab.sh -p scripts/imitation_learning/isaaclab_mimic/generate_dataset.py \
--task Isaac-Stack-Cube-Franka-IK-Rel-Mimic-v0 \
--input_file ./datasets/annotated_dataset.hdf5 \
--output_file /tmp/out.hdf5 \
--generation_num_trials 30 --num_envs 10 --headless
```
`FrankaCubeStackIKRelMimicEnvCfg` sets `max_num_failures = 25`.
Observed:
| | successes | failures | attempts |
|---|---|---|---|
| before | 30 | **50** | 80 |
| after (`max_num_failures = 25`) | 12 | **25** | 37 |
Before the change the run passed the configured cap and kept going,
ending at twice it. After, it stops on the attempt that reaches 25
failures and says so:
```
Reached 25 failures (max_num_failures=25) after 12/30 successes. Exiting.
```
### On the default
Wiring the field up exactly as written would abort the primary
documented workflow. At the ~36% success rate measured above,
`--generation_num_trials 1000` needs on the order of 1800 failures, so a
cap of 25 would end the run after roughly forty attempts. Since the
eighteen assignments were inert when they were written and no current
behaviour depends on them, they are removed here and the default becomes
`None`, meaning no limit. **Runs behave exactly as they do today unless
a limit is explicitly requested.**
If maintainers would rather the shipped configs keep an active cap, that
is a one-line change per config and I am happy to make it — it just
needs a value that suits large runs.
### Tests
`source/isaaclab_mimic/test/test_generation_failure_cap.py` drives
`env_loop` with a fake environment whose `step()` consumes scripted
attempt outcomes and refills the action queue, so the loop's own
termination logic runs unmodified with no simulator or dataset behind
it; a step-count fuse turns an unbounded loop into a failed assertion.
Six cases: the cap stops a run that never succeeds; `None` keeps the
guarantee unbounded as before; enough successes still end the run first;
a cap reached short of the target ends it; attempt-based termination
with the guarantee off is untouched (×2).
Against the unpatched loop the two cap-dependent cases fail (`'fuse' ==
'exited'`, and `(3, 4) == (2, 4)` — the loop overran the cap and
collected an extra success) and the four asserting unchanged behaviour
pass.
## Type of change
- Bug fix (non-breaking change which fixes an issue)
## Checklist
- [x] I have read and understood the contribution guidelines
- [ ] I have run the `pre-commit` checks with `./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have added a changelog fragment under
`source/<pkg>/changelog.d/` for every touched package
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01JFz91VMgmrjS4XR6f9Q9Z2
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>1 parent befb2d4 commit 59df9b8
22 files changed
Lines changed: 200 additions & 20 deletions
File tree
- source
- isaaclab_mimic
- changelog.d
- isaaclab_mimic
- datagen
- envs
- test
- isaaclab
- changelog.d
- isaaclab/envs
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
123 | 123 | | |
124 | 124 | | |
125 | 125 | | |
| 126 | + | |
126 | 127 | | |
127 | 128 | | |
128 | 129 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
42 | | - | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
43 | 56 | | |
44 | 57 | | |
45 | 58 | | |
| |||
Lines changed: 14 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
Lines changed: 12 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
140 | 140 | | |
141 | 141 | | |
142 | 142 | | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
143 | 155 | | |
144 | 156 | | |
145 | 157 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
35 | | - | |
36 | 35 | | |
37 | 36 | | |
38 | 37 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
32 | | - | |
33 | 32 | | |
34 | 33 | | |
35 | 34 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | | - | |
34 | 33 | | |
35 | 34 | | |
36 | 35 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
31 | | - | |
32 | 31 | | |
33 | 32 | | |
34 | 33 | | |
| |||
Lines changed: 0 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
31 | 30 | | |
32 | 31 | | |
33 | 32 | | |
| |||
0 commit comments