Skip to content

[Fix] Publish and fetch the checkpoints a task's components declare - #7485

Draft
hujc7 wants to merge 4 commits into
isaac-sim:developfrom
hujc7:jichuanh/pretrained-feature-extractor-ckpt
Draft

[Fix] Publish and fetch the checkpoints a task's components declare#7485
hujc7 wants to merge 4 commits into
isaac-sim:developfrom
hujc7:jichuanh/pretrained-feature-extractor-ckpt

Conversation

@hujc7

@hujc7 hujc7 commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

play --checkpoint pretrained crashed on the Shadow Hand camera tasks: the vision CNN the policy
needs was never published beside it, and nothing in the tooling knew the file existed. This PR adds
the smallest mechanism that fixes it — a component declares what it writes, and the tooling
publishes and fetches it with the policy.

The refactor that this mechanism invites is deliberately not here; it is stacked in
#7509 — Replace the pretrained checkpoint functions with CheckpointBundle.

Description

1. Declaring a checkpoint

A component declares what it writes on its own config:

@configclass
class FeatureExtractorCfg:
    checkpoint: Checkpoint = Checkpoint(name="feature_extractor", run_glob="cnn_*.pth")
    """The trained CNN, published beside the policy checkpoint."""

get_declared_checkpoints walks the resolved environment config to find every declaration, so a
task declares nothing and the component that writes the file owns its name. Each file is published
beside the policy as <policy stem>_<name><extension> and fetched with it. Checkpoint.resolve
hands the component its local file, so no component needs to know the naming convention.

Checkpoint also covers weights that already exist (url=): a frozen encoder is fetched but never
published by the checkpoint tooling.

2. Published checkpoints

Both companion pairs are trained and published, so the fix has something to fetch:

Task Backends
Isaac-Reorient-Cube-Shadow-Camera newtonmjwarp+newton, physx+rtx
Isaac-Reorient-Cube-Shadow-Camera-Direct newtonmjwarp+newton, physx+rtx

Each is a policy .pt plus its _feature_extractor.pth.

3. Verification

  • 82 tests pass across test_pretrained_checkpoint.py, test_checkpoints.py,
    test_train_and_publish_checkpoints.py, test_shadow_hand_camera_presets.py and
    test_pretrained_checkpoint_lookup.py. isaaclab -f clean.
  • The declaration tests pin discovery through nested component configs, exclusion of url weights,
    and that a published file keeps the extension the component declared.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Release backport

  • Backport this pull request to the active release branch after it merges into develop

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@github-actions github-actions Bot added enhancement New feature or request isaac-lab Related to Isaac Lab team labels Sep 2, 2026
@hujc7

hujc7 commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 3, 2026
@hujc7

hujc7 commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 3, 2026
@hujc7

hujc7 commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 4, 2026
play --checkpoint pretrained crashed on the Shadow Hand camera tasks: the
vision CNN the policy needs was never published beside it, and nothing in
the tooling knew the file existed.

A component now declares what it writes with a Checkpoint on its own
config. The tooling walks the resolved environment config to find every
declaration, so a task declares nothing, and publishes each file beside
the policy as <policy stem>_<name><extension>. The download path fetches
them with the policy.
@hujc7
hujc7 force-pushed the jichuanh/pretrained-feature-extractor-ckpt branch from 8a871d9 to eda9b46 Compare September 4, 2026 04:22
@hujc7 hujc7 changed the title [Fix] Provide the vision CNN checkpoint that pretrained Shadow Hand camera playback needs Publish and fetch the checkpoints a task's components declare Sep 4, 2026
@hujc7

hujc7 commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 4, 2026
@hujc7

hujc7 commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 4, 2026
"Auxiliary" ranked these files below the policy and did not generalise:
any component can declare a run artifact, and the policy is not special
among them. The discovery and path helpers, their parameter, and the
collect locals now say declared.
@hujc7
hujc7 force-pushed the jichuanh/pretrained-feature-extractor-ckpt branch from 17433c7 to b87603e Compare September 4, 2026 07:59
@hujc7 hujc7 changed the title Publish and fetch the checkpoints a task's components declare [Fix] Publish and fetch the checkpoints a task's components declare Sep 4, 2026
@hujc7

hujc7 commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 4, 2026
…ned-feature-extractor-ckpt

# Conflicts:
#	source/isaaclab_rl/isaaclab_rl/utils/pretrained_checkpoint.py
#	source/isaaclab_rl/test/test_pretrained_checkpoint.py
@hujc7

hujc7 commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant