Skip to content

Commit 7956941

Browse files
authored
Update torch-export guide (pytorch#17555)
1 parent d04ab5d commit 7956941

1 file changed

Lines changed: 117 additions & 20 deletions

File tree

‎.claude/skills/export/guides/torch-export.md‎

Lines changed: 117 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# torch.export.export()
22

3+
> This guide may lag behind upstream PyTorch. When in doubt, consult the
4+
> official docs:
5+
> - [torch.export](https://docs.pytorch.org/docs/stable/user_guide/torch_compiler/export.html)
6+
> - [torch.compiler overview](https://docs.pytorch.org/docs/stable/user_guide/torch_compiler/torch.compiler.html)
7+
38
Captures a PyTorch `nn.Module` into an `ExportedProgram` — a fully traced
49
ATen-dialect graph with no Python runtime dependency.
510

@@ -294,13 +299,43 @@ avoid the branch, or use `torch.cond`.
294299
### Data-dependent control flow
295300

296301
When a condition involves **unbacked** dynamic shapes (from `.item()`,
297-
`.nonzero()`, etc.), the tracer has no hint to evaluate it →
302+
`.nonzero()`, `mark_unbacked`, etc.), the tracer has no hint to evaluate it →
298303
`GuardOnDataDependentSymNode`.
299304

300-
Two solutions:
305+
Framework code should no longer throw data-dependent errors (DDEs). PyTorch now
306+
handles unbacked shapes in operations like `view`, `narrow`, `select`, and
307+
shape checks by automatically selecting general code paths. If you see a DDE
308+
pointing to files under `torch/`, it's likely a framework bug. DDEs you
309+
encounter will typically originate from user code.
310+
311+
Diagnose in order:
312+
313+
**1. Rewrite to avoid branching.** If there's a general code path that works
314+
for all shapes, use it instead of branching on unbacked symbols. Utilities for
315+
this:
316+
317+
- `statically_known_true(expr)` — returns `True` only if provable without
318+
adding guards; returns `False` otherwise. Never fails on data dependency.
319+
Use for optimizations that don't need recompilation.
320+
- `guard_or_false(expr)` / `guard_or_true(expr)` — may add guards for backed
321+
symbols, but returns `False`/`True` instead of failing for unbacked ones.
322+
- `hint_int(expr, fallback=None)` — extracts the hint value from the traced
323+
input without guarding. Use only for optimization paths, not correctness.
324+
325+
```python
326+
from torch.fx.experimental.symbolic_shapes import statically_known_true
327+
328+
if statically_known_true(x.numel() > 10):
329+
# optimization path
330+
...
331+
else:
332+
# general path (taken when unknown)
333+
...
334+
```
301335

302-
**`torch._check`** — assert a fact about the unbacked symbol so the tracer can
303-
continue on one branch. Creates a runtime assertion in the graph.
336+
**2. Assert one path with `torch._check`.** When you know one branch is always
337+
taken for your model, assert it. Creates a deferred runtime assertion — the
338+
condition is assumed true at compile time, checked at runtime.
304339

305340
```python
306341
nz = x.nonzero()
@@ -309,8 +344,29 @@ if nz.shape[0] > 0: # tracer now picks True
309344
return x.sin()
310345
```
311346

312-
**`torch.cond`** — trace both branches. The graph contains both subgraphs and
313-
selects at runtime. Use when you genuinely need both paths.
347+
`torch._check` semantics:
348+
- Equality `torch._check(u0 == expr)` replaces all occurrences of `u0` with
349+
`expr`, eliminating the unbacked symbol entirely.
350+
- Boolean conditions refine value ranges and are remembered. E.g.,
351+
`torch._check(u0 < 4)` also makes `u0 >= 4` evaluate to `False`.
352+
- Use `sym_or` / `sym_and` instead of Python `or` / `and` to avoid eager
353+
evaluation of unbacked expressions.
354+
355+
**`torch._check_is_size`** — like `torch._check`, but additionally marks the
356+
unbacked symbol as "size-like." Size-like symbols are assumed to never be 0 or
357+
1 under `guard_size_oblivious` checks, which eliminates many framework-internal
358+
guards. Use when passing an unbacked symbol to a function that expects a tensor
359+
dimension.
360+
361+
**3. If both branches are genuinely needed**, the code is unfixable with
362+
`torch._check`. Use `torch.cond` or restructure with padding.
363+
364+
### Higher-order control flow operators
365+
366+
These are structured operators that preserve control flow in the exported
367+
graph instead of specializing it away. All are prototype features.
368+
369+
**`torch.cond`** — if-else. Traces both branches; selects at runtime.
314370

315371
```python
316372
return torch.cond(
@@ -321,25 +377,66 @@ return torch.cond(
321377
)
322378
```
323379

324-
`torch.cond` lowers to `torch.ops.higher_order.cond` — a special node with two
325-
`GraphModule` attributes for the branches. No closures; all captured values
326-
become explicit operands. No mutations allowed in branches.
380+
Lowers to `torch.ops.higher_order.cond` — a node with two `GraphModule`
381+
attributes for the branches. No closures; all captured values become explicit
382+
operands. No mutations allowed in branches.
327383

328-
**`torch.map`** — trace a loop body over a dynamic number of iterations. Use for
329-
data-dependent loops where unrolling isn't possible:
384+
**`torch.while_loop`** — data-dependent loop. Runs `body_fn` while `cond_fn`
385+
returns `True`.
330386

331387
```python
332-
return torch.map(
333-
f=lambda x_i: x_i.sin(),
334-
xs=(x,), # map over first dim of each tensor
335-
)
388+
from torch._higher_order_ops import while_loop
389+
390+
def cond_fn(iter_count, x):
391+
return iter_count.sum() > 0
392+
393+
def body_fn(iter_count, x):
394+
return iter_count - 1, x * 2
395+
396+
final_iter, final_x = while_loop(cond_fn, body_fn, (init_iter, init_x))
336397
```
337398

338-
**`torch._check_is_size`** — like `torch._check`, but additionally marks the
339-
unbacked symbol as "size-like." Size-like symbols are assumed to never be 0 or
340-
1 under `guard_size_oblivious` checks, which eliminates many framework-internal
341-
guards. Use when passing an unbacked symbol to a function that expects a tensor
342-
dimension.
399+
`body_fn` must return tensors with the same metadata (shape, dtype) as inputs.
400+
No in-place mutations; clone before mutating. No closures.
401+
402+
**`torch.map`** — apply a function over the leading dimension of input tensors.
403+
404+
```python
405+
from torch._higher_order_ops import map
406+
407+
result = map(lambda x_i: x_i.sin(), xs) # xs: [N, ...] → result: [N, ...]
408+
```
409+
410+
Additional non-mapped arguments can be passed as `*args`. Leading dimensions
411+
of all tensors in `xs` must be consistent and non-zero. No input mutations.
412+
413+
**`torch.scan`** — cumulative operation with carried state. Like a fold that
414+
also collects intermediate outputs.
415+
416+
```python
417+
from torch._higher_order_ops import scan
418+
419+
def combine_fn(carry, x):
420+
next_carry = carry + x
421+
return next_carry, next_carry.clone() # clone to avoid aliasing
422+
423+
final_carry, cumsum = scan(combine_fn, init=torch.zeros(1), xs=xs)
424+
```
425+
426+
`combine_fn` must return `(next_carry, output)` where `next_carry` matches
427+
`init` metadata. No in-place mutations; output cannot alias inputs.
428+
429+
**`torch.associative_scan`** — like `scan` but requires an associative
430+
`combine_fn`, enabling parallel tree-reduction instead of sequential execution.
431+
432+
```python
433+
from torch._higher_order_ops.associative_scan import associative_scan
434+
435+
cumsum = associative_scan(lambda x, y: x + y, xs, dim=0, combine_mode="generic")
436+
```
437+
438+
`combine_fn` must be associative: `f(f(a, b), c) == f(a, f(b, c))`. No
439+
closures, no mutations, output cannot alias inputs.
343440

344441
## Module state
345442

0 commit comments

Comments
 (0)