Skip to content

[Newton] Delegate homogeneous world prefix generation to ModelBuilder.replicate() - #7453

Merged
kellyguo11 merged 13 commits into
isaac-sim:developfrom
camevor:newton-world-prefixes
Sep 4, 2026
Merged

[Newton] Delegate homogeneous world prefix generation to ModelBuilder.replicate()#7453
kellyguo11 merged 13 commits into
isaac-sim:developfrom
camevor:newton-world-prefixes

Conversation

@camevor

@camevor camevor commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Description

This depends on newton-physics/newton#4012 and pins the Newton 1.6 development build that first provides ModelBuilder.replicate(..., label_prefixes=...).

The homogeneous Newton cloning path previously replicated the model and then walked every replicated entity label to replace the prototype environment with its destination environment. It now rebases only the retained prototype labels, restores them after the call, and lets ModelBuilder.replicate() assign each copy's destination prefix while it is created. String-valued custom attributes still use the existing targeted rewrite because Newton cannot identify which strings are entity paths.

The destination itself is the source of identity: prefixes are derived from the queued source/destination mapping. No clone-plan environment metadata, scene discovery, fallback clone path, or retained prototype mutation is introduced. Labels exactly at a directly cloned asset root, including Newton's _free_joint and _articulation companion labels, are expressed relative to the nearest templated destination ancestor and remain on the same fast path.

The Newton commit pin is mirrored in workspace and wheel installation inputs so every supported install receives the API this path requires.

The final PR is +195/-218, net -23 LOC.

Performance

GPU 0, 4096 worlds, two samples per PR head. The targeted measurement uses one 48-body/48-shape Newton prototype and times replication plus all required label publication:

Head Targeted path median Versus develop
develop base (3a31389a9) 1.428 s
original PR (cbb17ab6e) 1.160 s -18.8%
cleaned PR (844f4c4db) 1.171 s -18.0%

The cleanup is within 1.0% (11 ms) of the original PR result and preserves the removed-pass gain. The 257 ms reduction from develop is consistent with the original 215 ms observation on Isaac-Velocity-Flat-G1.

Whole-workload check used:

CUDA_VISIBLE_DEVICES=0 uv run --no-sync isaaclab zero_agent \
  --task Isaac-Lift-KukaAllegro --num_envs 4096 --max_steps 1 \
  --device cuda:0 --visualizer none physics=newton_mjwarp
Head Warm wall median Scene creation median
original PR 21.30 s 6.324 s
cleaned PR 21.39 s 6.345 s

That end-to-end difference is within startup noise; the cleanup introduces no measured workload regression.

Type of change

  • Startup performance enhancement

Validation

  • 81 focused clone, Newton, visualization-state, wheel-metadata, and install tests passed.
  • Ruff formatting and lint checks passed.
  • Exact-root, sparse-environment, hook-authored-label, custom-attribute, and retained-prototype cases are covered.
  • The changelog fragment is present for the changed Newton package.

Checklist

  • I have read and understood the contribution guidelines
  • My changes generate no new warnings
  • I have added tests that prove the fix is effective
  • I have added the required changelog fragment
  • My name already exists in CONTRIBUTORS.md

Rewriting every merged label after replication is a second full pass over what
replication just produced, and it scales with environment count. `replicate()`
now takes a per-world label prefix, so the prototype's labels are rebased once
and each copy comes out named for the environment it lands in; `ClonePlan`
records `env_template` to supply that boundary.

A prototype the boundary cannot express keeps the old path, and the string
custom attributes are still rewritten here.
@camevor
camevor requested a review from a team August 31, 2026 17:02
@github-actions github-actions Bot added bug Something isn't working isaac-lab Related to Isaac Lab team infrastructure labels Aug 31, 2026
@greptile-apps

greptile-apps Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR propagates the clone plan’s environment template into backend replication and delegates homogeneous Newton entity naming to ModelBuilder.replicate(). It also qualifies per-world site labels with their destination environment and updates Newton to the required pinned 1.6 development commit.

  • Stores env_template on ClonePlan and passes it to replication contexts.
  • Rebases homogeneous prototype labels once and supplies per-environment label prefixes during Newton replication.
  • Carries destination templates through site registration so bodyless per-world sites receive environment-qualified labels.
  • Preserves the existing rewrite path when labels cannot be represented by a per-world prefix.
  • Updates tests, changelog fragments, and the Newton dependency lock entry.

Confidence Score: 5/5

The PR appears safe to merge, with no actionable changed-code defect identified.

The new template and label-prefix data are propagated consistently through clone planning, backend construction, site registration, replication, and fallback rewriting, while body bindings and custom-attribute rewriting remain intact.

Important Files Changed

Filename Overview
source/isaaclab/isaaclab/cloner/clone_plan.py Persists the environment destination template in every clone-plan construction path.
source/isaaclab/isaaclab/cloner/replicate_session.py Propagates the clone plan’s environment template to each backend replication context.
source/isaaclab_newton/isaaclab_newton/cloner/newton_clone_utils.py Adds prototype-label rebasing, delegated per-environment naming, and environment-qualified site-label generation while retaining fallback rewriting.
source/isaaclab_newton/isaaclab_newton/cloner/replicate.py Threads the environment template through Newton model construction and skips redundant entity rewriting only when replication already named the copies.
source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py Extends pending site registrations with destination-template identity and carries it into injected per-world sites.
source/isaaclab_newton/isaaclab_newton/sim/views/newton_site_frame_view.py Includes destination templates in resolved bodyless-site specifications and registration.
pyproject.toml Pins Newton to the development commit providing label-prefix replication support.
uv.lock Updates only Newton’s source and version while preserving its resolved dependency set.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[ClonePlan with env_template] --> B[Backend replication context]
    B --> C[Build Newton source prototypes]
    C --> D{Homogeneous and labels rebase cleanly?}
    D -- Yes --> E[ModelBuilder.replicate with per-env prefixes]
    D -- No --> F[Replicate then rewrite labels]
    E --> G[Rewrite string custom attributes]
    F --> G
    B --> H[Register per-world sites with destination template]
    H --> I[Environment-qualified site labels]
    G --> J[Final Newton model]
    I --> J
Loading

Reviews (1): Last reviewed commit: "Update newton pin" | Re-trigger Greptile

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The replication-prefix optimization is well scoped, but two compatibility issues need correction: the fast path mutates retained prototype labels, and NewtonReplicateContext.__init__ silently rebinds existing positional arguments by inserting env_template before device.

  • Design and architecture: Delegating homogeneous label prefixing to ModelBuilder.replicate() with a guarded fallback is sound, but rebasing the source builder in place makes retained prototype state depend on the replication path. Rebase a copy or restore labels after replication.
  • API: ClonePlan.env_template is defaulted and propagated consistently to in-tree contexts. However, inserting env_template before existing positional parameters in NewtonReplicateContext.__init__ breaks positional-call compatibility and the new argument is omitted from the docstring. Preserve the existing parameter order or make the new argument keyword-only.
  • Implementation: The batched and per-world site paths and three-value replication result are wired through their consumers and tests. The remaining implementation defect is that _rebase_to_env permanently changes builders later retained in NewtonManager._cl_protos, exposing environment-relative labels to single-model consumers.

Minor fixes needed. Posted 2 actionable findings inline.

Automated review; human maintainers own approval decisions.

# env root, instead of rewriting every label in every world afterwards.
label_prefixes = None
prototype_env = clone_path.match(sources[0], env_template) if env_ids is not None else None
if prototype_env is not None and _rebase_to_env(source_builder, env_template.format(prototype_env.instance)):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Warning · Implementation — Rebase mutates retained prototype builders

_rebase_to_env rewrites source_builder labels in place, and that same object is returned in source_builders, stored as NewtonManager._cl_protos, and handed out by copy_newton_clone_source for single-model consumers. On the delegated path the retained prototype now carries env-relative labels (Robot/base) instead of the clone-plan source path, so prototype label state differs between the two cloning paths. Rebase a copy, or restore the labels after replicate().

self,
stage: Usd.Stage,
global_paths: tuple[str, ...] = (),
env_template: str = DEFAULT_ENV_TEMPLATE,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Warning · Api — New parameter inserted before existing positional arguments

env_template is inserted between global_paths and device in NewtonReplicateContext.__init__, so an existing positional call such as NewtonReplicateContext(stage, paths, "cuda:0") silently binds the device string to the template and falls back to device="cpu". The USD and OvPhysX contexts append the parameter last. Append it after the existing parameters (or make it keyword-only) and document it in the Args block, which currently omits it.

@camevor camevor self-assigned this Aug 31, 2026
@camevor camevor removed the bug Something isn't working label Aug 31, 2026

@StafaH StafaH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is moving the newton pin, we won't be able to backport this to our release branch, it'll have to stay in develop until GA. Is that okay?

Comment thread source/isaaclab/isaaclab/cloner/usd.py Outdated
from pxr import Gf, Sdf, Usd, UsdGeom, Vt

from ._fabric_notices import disabled_fabric_change_notifies
from .cloner_cfg import DEFAULT_ENV_TEMPLATE

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use absolute imports here and elsewhere (feel free to change other imports in the file to match)

xforms = _compose_world_xforms(positions_np, quaternions_np, source_xform_inv)
builder.replicate(source_builder, num_worlds, xforms=xforms)

# One source populating every world is the shape replication can name itself: rebase the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wording here is a bit confusing "every world is the shape replication can name itself:"

@ooctipus

ooctipus commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Nice finding chris , there is soem duplication is stored attribute I will help to make this pr leaner

@github-actions github-actions Bot added the bug Something isn't working label Sep 2, 2026
@ooctipus

ooctipus commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 2, 2026
@ooctipus

ooctipus commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 3, 2026
@ooctipus

ooctipus commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

run-ci

1 similar comment
@kellyguo11

Copy link
Copy Markdown
Contributor

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 3, 2026
@ooctipus

ooctipus commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 4, 2026
@ooctipus

ooctipus commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 4, 2026
Signed-off-by: Kelly Guo <kellyg@nvidia.com>
@kellyguo11

Copy link
Copy Markdown
Contributor

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 4, 2026
@kellyguo11

Copy link
Copy Markdown
Contributor

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 4, 2026
@kellyguo11
kellyguo11 merged commit 0eb2589 into isaac-sim:develop Sep 4, 2026
53 checks passed
ooctipus added a commit that referenced this pull request Sep 4, 2026
# Description

This is a narrow follow-up to merged #7453. It keeps that Newton fast
path and makes the existing `ClonePlan` the single mapping consumed by
USD, Newton, PhysX, and OvPhysX replication.

Planning records `context_rows`, mapping each simulation-owned
clone-context type to the plan rows it consumes.
`cloner.replicate(plan)` retrieves those already-registered contexts
from `SimulationContext`, orders them by replication priority, and
passes each the same plan. Backends no longer rebuild the mapping
through local `queue(...)` or `queue_mapping(...)` state.

Clone planning is a NumPy host-side control plane. `ClonePlan`, its
constructors, strategies, queries, and raw clone APIs use NumPy arrays;
backend code no longer performs Torch device synchronization or
`detach().cpu().tolist()` round trips. Runtime owners convert once when
they need a device tensor. `InteractiveScene` derives its environment
roots and device-side origins from the same plan instead of calculating
a second grid.

The public structure stays small:

- `ClonePlan.global_paths` remains the explicit declaration for shared
scene prims.
- `ClonePlan.env_ids` and `positions` remain optional for query-only
plans; execution validates what it needs.
- Per-asset `cloning_contexts`, `cfg_rows`, `ReplicateSession`, and the
existing scene-construction lifecycle remain.
- Standalone tooling keeps the raw USD, Newton, PhysX, and OvPhysX
replication functions.
- OVRTX camera/export/SDP architecture remains deferred to the renderer
cutover; this PR only removes Torch round trips from its mechanical
plan-array consumption.
- No stage discovery, fallback context construction, registry-wide duck
routing, or second clone mapping is introduced.

## Migration

Custom clone contexts now implement `replicate(plan)` and must be
registered with `SimulationContext.get_or_create_backend(...)` before
dispatch. Remove context-local `queue(...)` / `queue_mapping(...)` calls
and the former backend `PHYSICS_CONTEXT` aliases; raw standalone
replication functions remain available.

The high-level `stage=` argument is removed from `cloner.replicate(...)`
and `ReplicateSession`; each simulation-owned context already owns its
stage.

Clone arrays and raw clone API arrays are now NumPy. The unused public
`device` arguments are removed from `CloneCfg`, plan constructors,
`grid_transforms`, `ReplicateSession`, clone strategies, and raw backend
replication functions. Custom clone strategies now implement
`(combinations: np.ndarray, num_clones: int) -> np.ndarray`.

## Size

| Added | Deleted | Net |
| ---: | ---: | ---: |
| 1,267 | 1,700 | **-433** |

These counts are for this PR alone against current `develop`; merged
#7453 code and dependency changes are no longer duplicated in the diff.

## Performance

The targeted benchmark warms with 128 worlds, then times replication and
required label publication for one 48-body/48-shape Newton prototype at
4,096 worlds. Measurements taken while #7453 was under review used
physical GPU 0 and the same Newton revision:

| Revision | Targeted median | Versus pre-#7453 develop |
| --- | ---: | ---: |
| develop before #7453 | 1.428 s | — |
| #7453 | 1.171 s | -18.0% |
| this follow-up before restacking | 1.181 s | **-17.3%** |

The restack removed the duplicated #7453 implementation rather than
reimplementing it. A file-by-file audit confirms this PR no longer
changes #7453's label-prefix fast-path files, and the
one-source/all-true Newton route still delegates to
`ModelBuilder.replicate(...)`.

## Type of change

- Breaking clone-context and NumPy API simplification
- Documentation update

## Validation

- `84 passed` — clone-plan algebra
- `7 passed` — replicate-session lifecycle
- `15 passed` — focused Newton label-prefix/root-name coverage from
#7453
- Repository formatting, lint, RST, spelling, license, changelog, LFS,
and hygiene hooks passed against current `upstream/develop`
- `git diff --check` passed
- Static audit found no Torch import or CPU/list materialization in the
core, Newton, PhysX, or OvPhysX clone paths

## Release backport

- [ ] <!-- backport-active-release --> Backport this pull request to the
active release branch after it merges into `develop`

## Checklist

- [x] I have read and understood the contribution guidelines
- [x] I have run the relevant pre-commit checks
- [x] I have made corresponding documentation changes
- [x] I have added focused contract, routing, and lifecycle tests
- [x] I have added a changelog fragment for every touched source package
- [x] My name already exists in `CONTRIBUTORS.md`
hujc7 added a commit to hujc7/IsaacLab that referenced this pull request Sep 4, 2026
develop pins Newton to 24bd8635 via isaac-sim#7453, which sits 16 commits before
newton#4017 and so lacks the ArticulationView fix this branch exists to deliver.
1.6.0rc1 is a descendant of that commit, so keeping the release pin loses
nothing develop gained.

Drop two things develop now does better: it expands package_roots to include the
extras-qualified wheel trees, which covers the prebundle mirrors this branch
walked by hand, and its P-ADMM fix applies the sparse flags only for
penalty_update_method rather than for every field.
hujc7 added a commit to hujc7/IsaacLab that referenced this pull request Sep 4, 2026
isaac-sim#7453 pinned the Newton build in [project].dependencies specifically to cover
"both workspace and wheel installs". Restoring the loose >=1.2.0 bound during the
develop merge dropped the wheel half of that guarantee: [tool.uv].override-
dependencies only reaches uv sync, and 1.6.0rc1 is a prerelease, so a default
resolve of the published metadata skipped it and picked the newest stable, 1.5.1
-- which lacks newton#4017, the fix this bump exists to deliver.

Pin the same release the override forces. A default resolve of the new metadata
returns newton==1.6.0rc1. isaacsim-core==6.0.1.0 requires newton[sim]==1.2.1, so
an override is still needed to install the isaacsim extra, exactly as on develop.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working infrastructure isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants