act-onnx

Export a trained LeRobot ACT policy to ONNX and run it autonomously on an SO-10x follower arm — without the
PyTorch policy/processor stack at inference time.

Two Python commands, plus a native C++ rollout:

Command	What it does	Heavy deps
act-onnx-export	Trace an ACT checkpoint into a self-contained .onnx graph (normalization baked in).	torch, lerobot, onnx
act-onnx-rollout	Drive the robot from the .onnx model via onnxruntime (no torch/policy at runtime).	onnxruntime, lerobot (hardware drivers)
cpp/build/act-onnx-rollout	Same robot loop in C++ (onnxruntime + OpenCV + Feetech serial; no Python at runtime).	OpenCV, onnxruntime C++

The exported graph takes raw robot state + images and returns actions in real robot units — input/output normalization is folded into the graph as affine ops, so parity with training is exact.

What broke

• A policy that had been reliably completing the task started failing. The root cause was SO-101 base rotation by a few degrees. It turned out the base was not bolted firmly enough to the table — the policy was fine, the world had moved.
• A servo was destroyed by loading the arm past spec during teleoperation, and had to be replaced. Re-ran lerobot-calibrate, with the calibration JSON shared between the Python and C++ rollouts.

Performance

Measured over 60-second rollouts on the same task, same machine, same 30 Hz target (--fps 30 --duration 60). Hardware: MacBook Air 2020, 2 cameras at 640x480.

Rollout	Ticks	Duration	Achieved
LeRobot + PyTorch (baseline, no ONNX)	1632	60.0 s	27.2 Hz
act-onnx-rollout (Python + onnxruntime)	1645	60.2 s	27.3 Hz
cpp/build/act-onnx-rollout (C++)	1656	60.0 s	27.6 Hz

All three sit ~9% below the 30 Hz target and within ~1.5% of each other — close enough that the differences are run-to-run noise. The value of the ONNX export and the C++ rollout is not speedup but deployment footprint.

Requirements

• Python ≥ 3.12
• [uv](https://docs.astral.sh/uv/)
• lerobot >= 0.6.0 (installed automatically from PyPI). The rollout imports LeRobot's SO-follower and OpenCV camera drivers, so the installed lerobot must expose lerobot.robots.so_follower.SO101FollowerConfig.

Setup

git clone https://github.com/vladitov/act-onnx.git
cd act-onnx
uv sync

This creates .venv/ and installs lerobot, torch, onnx, and onnxruntime.

Export a checkpoint

Point --policy-path at a pretrained_model directory (the one containing config.json + model.safetensors):

uv run act-onnx-export \
  --policy-path "/path/to/outputs/act_007/checkpoints/last/pretrained_model" \
  --output act_007_last.onnx \
  --check

Convenience wrapper

scripts/export.sh resolves a checkpoint from a runs directory (defaults to Google Drive, override with CHECKPOINTS_ROOT):

CHECKPOINTS_ROOT="/path/to/outputs" scripts/export.sh -p act_007 -c last --check

Run a rollout on the robot

uv run act-onnx-rollout \
  --onnx act_007_last.onnx \
  --port /dev/tty.usbmodem5B140328401 --id SO101 \
  --fps 30 --duration 60

Convenience wrapper

scripts/rollout.sh wires up the two default cameras (front, top) and the follower port/id (override with FOLLOWER_PORT / FOLLOWER_ID):

scripts/rollout.sh -m act_007_last.onnx -d 60

C++ rollout

cpp/ is a standalone CMake project that mirrors act-onnx-rollout without Python: ONNX Runtime for inference, OpenCV for cameras, and a small Feetech STS3215 bus client for the SO-10x follower (same calibration JSON as LeRobot).

./cpp/build/act-onnx-rollout \
  --onnx act_007_last.onnx \
  --port /dev/tty.usbmodem5B140328401 --id SO101 \
  --fps 30 --duration 60

Convenience wrapper

scripts/rollout_cpp.sh wires up the two default cameras (front, top) and the follower port/id (override with FOLLOWER_PORT / FOLLOWER_ID):

scripts/rollout_cpp.sh -m act_007_last.onnx -d 60

Build instructions and full dependency list: [cpp/README.md](cpp/README.md).

Model I/O

Inputs (names mirror the policy feature keys, order matters):

• observation.state — float32 (B, state_dim), raw units.
• observation.images.<cam> — float32 (B, C, H, W), [0, 1], CHW, per camera.
• observation.environment_state — float32 (B, env_dim) (only if the policy uses it).

Output:

• action — float32 (B, chunk_size, action_dim) in real robot units.

Not baked into the graph (handled by act-onnx-rollout, mirroring ACTPolicy.select_action): image uint8 -> float32 /255 + HWC→CHW, chunk slicing to n_action_steps, and the action queue.

Project layout

act-onnx/
├── pyproject.toml          # Python deps + console scripts
├── src/act_onnx/
│   ├── export.py           # act-onnx-export
│   └── rollout.py          # act-onnx-rollout (Python)
├── cpp/                    # C++ ONNX rollout (see cpp/README.md)
│   ├── CMakeLists.txt
│   ├── include/act_onnx/
│   └── src/
└── scripts/
    ├── export.sh           # checkpoint-resolving export wrapper
    └── rollout.sh          # Python robot rollout wrapper