[Experimental] Add Newton model to USD exporter - #4
Draft
hujc7 wants to merge 12 commits into
Draft
Conversation
Isaac Lab applies most configuration directly to the solver rather than to the stage, so a scene that has been loaded and overridden no longer has a USD file describing what is actually being simulated. Add export_model_to_usd(), the inverse of ModelBuilder.add_usd(). Core physics is authored with standard UsdPhysics schemas and Newton-specific properties as newton:* attributes, at the prim paths the model was imported from, so a reimport reproduces the same model. The correctness contract is model idempotence rather than USD fidelity: the importer normalizes as it reads, so the exported stage differs from the source by construction, but reimporting it must change nothing. Known gaps are documented in the module and fail loudly rather than exporting something wrong: D6/ball/distance joints, planes, heightfields and SDF shapes, cloned multi-world scenes, and shape ordering, which follows USD stage traversal.
Comparing model arrays shows the numbers match but not that the export reproduces the same physics, and it reports spurious differences for rebuilt acceleration structures and for shape orderings that differ without changing behavior. Add a rollout check that steps the source and exported models under identical initial conditions and compares body trajectories, treating a quaternion and its negation as the same rotation. The fixture's joint carries an offset frame so the trajectory is sensitive to joint geometry; without it the check cannot distinguish a correct export from one that drops joint frames. The two layers are complementary. The rollout misses parameter drops that do not change this particular motion, such as mass in a pendulum under gravity, which the array comparison catches.
Every existing check starts from an unmodified load, so an exporter that re-derived its output from the source USD would pass all of them while reproducing the asset rather than what is being simulated. That is the property the feature exists for: Isaac Lab applies most configuration by writing into the model after the stage is parsed. Add a check that writes distinctive values into the model, as the asset classes do, and asserts the export carries them rather than the source file's values. Also document that free-joint degrees of freedom carry no exportable state: a floating body is expressed in USD by the absence of a joint, so per-DOF values written to a free joint have nowhere to be authored.
Isaac Lab's cloner builds one Newton world per environment, all sharing the source asset's prim paths. The exporter resolved indices as if the model held a single world, so a cloned scene exported only the first environment's entities under every environment's prim paths and reported success: a 3-world model of 6 bodies came back as 2. Select the world explicitly. Entities are chosen by their world membership plus the worldless global content, which is the same rule for an uncloned model, and the count of exportable entities is checked against the source's prim paths so a shortfall fails instead of shipping a partial file. Free joints and sites are exempt: a floating body is expressed in USD by the absence of a joint, and a site is a bare frame with no geometry. Every task's ground is a plane, which raised NotImplementedError; author it as a Z-axis UsdGeom.Plane, the form the importer reads back. Physics materials are authored for every shape, since Isaac Lab randomizes friction per shape regardless of whether it collides. Also export the entry point from isaaclab_newton.sim; the lazy-export stub had omitted it, so the documented import path raised AttributeError.
export_model_to_usd() needs the path maps that ModelBuilder.add_usd() returns, and every build path discarded them: the clone-plan session dropped the return of ctx.replicate(), and the per-prototype import result never left _build_source_builder. The exporter could not be called on a real environment at all. Keep the results and merge them into one environment's provenance. Global content is indexed against the model directly; each prototype's indices are lifted by the landing offset replicate_builder_mapping already computes before appending it, so a world with several prototypes (robot, object, table) resolves exactly. NewtonManager.get_stage_info() exposes the result in the shape add_usd() returns, so it feeds the exporter unchanged.
PhysX and OVPhysX keep the stage authoritative for structure, but every runtime write (drive gains, masses, armature, friction, limits) goes to the solver's buffers and never reaches a prim. Saving the stage of a running scene therefore emits a file that looks complete while carrying the spawn-time value of everything overridden since. Author those properties back onto the prims they came from, reading through BaseArticulationData so the same code serves any backend. Recovering prim paths is the one backend-specific step, so each backend supplies them through ArticulationPrimPaths. Values are joined to prims by name, because the view is in backend order and the data in public order. Angular drive gains are per degree on the stage and per radian in the simulation; converting on export matters for every driven revolute joint, and a task whose only revolute joint is passive cannot reveal it.
The PhysX tensor view records the prim each link and degree of freedom was built from, so the paths are read straight off it and handed to the shared exporter.
An OVPhysX binding reports names and the articulation prims it matched, not per-link paths, so they are resolved by indexing the articulation subtree by prim name. A body or joint with no prim fails the export rather than leaving it partial.
github-actions
Bot
added
documentation
Both PhysX and OVPhysX populate the joint friction the solver applies from PhysxJointAxisAPI (static and dynamic friction effort, viscous coefficient), not from the legacy physxJoint:jointFriction scalar the exporter wrote, so randomized friction reimported as zero. The per-axis schema also shadows the joint-level armature on OVPhysX, and its viscous coefficient is per degree per second on angular axes: the exporter now writes the triple under the drive axis, armature in both places, and viscous scaled like drive gains. Applying a schema to a prim that is itself an articulation root invalidates every PhysX articulation view on the stage for the rest of the session. export_articulation_to_usd therefore authors onto a flattened snapshot; write_articulation_state_to_stage takes the target stage explicitly and applies schemas only where absent. Reads precede writes, prim paths on OVPhysX resolve from a typed index that walks up from the root prim, and each backend wrapper forwards the stage argument.
ModelBuilder.approximate_meshes(keep_visual_shapes=True), which the USD importer runs on visible meshes with a collision approximation, copies the mesh as a visual-only shape labelled <label>_visual that has no prim of its own, so the coverage guard refused every scene with such an asset. The resolution of a world's entities to prim paths moves into a public resolve_world_prim_paths returning WorldPrimPaths; twins resolve to a visual-only <prim>_visual sibling of their source and are authored as such. Coverage is checked entity by entity rather than by count, unsupported geometry is rejected by name, and joints outside the articulation are exported with physics:excludeFromArticulation.
Retaining import provenance made replicate_builder_mapping return the world-0 landing offsets as a third element against its declared two-tuple, which broke four develop cloner tests and every caller that unpacks two values. The offsets are recorded only on request by a private implementation; replicate_builder_mapping keeps its contract and the two provenance callers use replicate_builder_mapping_with_provenance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
export_model_to_usd()— the inverse ofModelBuilder.add_usd()— so a scene Isaac Lab hasloaded and overridden can be written back to a USD file describing what is actually being simulated.
Prototype for roadmap task T4.14 (Export to USD); no GitHub issue tracks it today.
UsdPhysicsschemas, Newton-specific properties asnewton:*isaaclab_assetsships, not just synthetic fixturesCorrectness contract
Model idempotence, not USD fidelity:
The importer normalizes as it reads (unit conversion, shape-scale baking, fixed-joint collapsing),
so the exported stage differs from the source by construction. What must hold is that reimporting
changes nothing further. Targeting this rather than USD fidelity is what keeps the exporter small.
Verification — three complementary layers
1.5e-04Neither layer subsumes the others: the rollout cannot detect mass in a gravity pendulum (the motion
is mass-independent), and attribute comparison cannot detect behavioral errors that merely permute
array indices.
Bugs found by validation, each with a regression test
/WorlddiagonalInertiaauthored; off-diagonals lostjoint_X_p/joint_X_ccollapsed to the body origin — robots reassembled wrongKnown gaps — tracked, not silently ignored
ArticulationCfg-level override plumbing is untested. Override capture is verified at themechanism level (direct model writes, which is what the cfg layer ultimately calls); the cfg path
itself needs
AppLauncherand has to run in the Docker+Kit lane.joint_X_p/joint_X_con ~5 assets where the jointprim is not a direct child of its body, plus derived arrays (
bvh_*, AABBs,body_inv_inertia)that are rebuilt acceleration structures.
NotImplementedErrorrather than exporting something wrong.import (Franka: 64 vertices) and discards the original, so it cannot be recovered on export.
Idempotence holds; fidelity to the artist's mesh does not.
sim.export_usd()wrapper or CLI tool yet.Test plan
Rebased on latest
upstream/develop.