Skip to content

Commit 9e00dbb

Browse files
committed
[native] Add an ExecuTorch Native docs page
Add docs/source/native-overview.md under the Advanced section. It mirrors backends/native/README.md, adds a pipeline diagram, and links to the source on GitHub. Authored with Claude Code.
1 parent 8344b97 commit 9e00dbb

3 files changed

Lines changed: 85 additions & 0 deletions

File tree

333 KB
Loading

‎docs/source/advanced-topics-section.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,18 @@ Key topics:
102102
- PTD file format specification
103103
- Custom file format handling
104104

105+
## ExecuTorch Native
106+
107+
Experimental: export a model to a single `.ptn` file and run it on one of
108+
several engines.
109+
110+
**→ {doc}`native-overview` — ExecuTorch Native**
111+
112+
Key topics:
113+
114+
- Export with `to_native` and the `.ptn` file
115+
- Runtime engines
116+
105117
## Next Steps
106118

107119
After exploring advanced topics:
@@ -122,4 +134,5 @@ backend-delegate-advanced
122134
runtime-integration-advanced
123135
compiler-ir-advanced
124136
file-formats-advanced
137+
native-overview
125138
```

‎docs/source/native-overview.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
(native-overview)=
2+
3+
# ExecuTorch Native
4+
5+
::::{note}
6+
ExecuTorch Native is experimental. The Python API, the `.ptn` format, and
7+
operator coverage may change.
8+
::::
9+
10+
ExecuTorch Native lowers a PyTorch model to a single `.ptn` file and runs it on
11+
one of several engines. In a `.pte`, delegates lower the graph ahead of time for
12+
specific backends. A `.ptn` keeps the whole graph without backend-specific
13+
information, and each engine lowers it when compiling a method.
14+
15+
```text
16+
.pte: export -> [quantize] -> to_edge -> to_backend -> to_executorch -> .pte (+ .ptd)
17+
.ptn: export -> [quantize] -> to_native -> .ptn
18+
```
19+
20+
<img src="_static/img/native-pipeline.png" alt="ExecuTorch Native pipeline: a PyTorch model goes through torch.export and to_native into a .ptn file, which the Native runtime hands to an engine on the device" style="width:600px;"/>
21+
22+
## Export
23+
24+
```python
25+
import torch
26+
from executorch.exir.native import to_native
27+
28+
ep = torch.export.export(model, example_inputs)
29+
to_native(ep).save("model.ptn")
30+
31+
# Several methods in one file.
32+
to_native({"forward": ep_forward, "prefill": ep_prefill}).save("model.ptn")
33+
```
34+
35+
The partitioner accepts core ATen ops plus `linear`, `matmul`, `addmm`,
36+
`scaled_dot_product_attention`, `rms_norm`, `native::rope`, and TorchAO affine
37+
quantize/dequantize ops. TorchAO int4 group-wise weights of `linear` and
38+
`embedding`, and GGUF k-quant weights from `executorch.extension.llm.export.gguf`,
39+
are stored packed with their scheme in tensor metadata. Each method must lower
40+
wholly to one delegate; `to_native` raises otherwise.
41+
42+
A `.ptn` is an uncompressed zip with fixed entry names:
43+
44+
| Entry | Contents |
45+
| --- | --- |
46+
| `program.ptg` | Methods as graphs; FlatBuffer defined by [`native_graph.fbs`](https://github.com/pytorch/executorch/blob/main/backends/native/serialization/native_graph.fbs). |
47+
| `program.safetensors` | Constants. Byte-identical immutable tensors are stored once. |
48+
| `aliases.json` | Deduplicated key to owner key; present only when needed. |
49+
50+
## Runtime
51+
52+
The C++ runtime reads a `.ptn` into an in-memory graph and hands each method to
53+
an engine through the interfaces in
54+
[`Engine.h`](https://github.com/pytorch/executorch/blob/main/backends/native/runtime/engine/Engine.h).
55+
56+
- `EngineHost`: process-wide device resources.
57+
- `EngineContext`: state for one loaded program, including mutable buffers.
58+
- `EngineExecutable`: one compiled method (`set_input`, `execute`, `get_output`).
59+
60+
An engine rejects a method whose semantics it cannot fully honor.
61+
62+
| Engine | Notes |
63+
| --- | --- |
64+
| Vulkan | Builds on the ET-VK `ComputeGraph` and shaders. |
65+
| ggml | CPU, CUDA, HIP, and Metal variants. |
66+
67+
The runtime currently builds only with Meta's internal Buck setup; there is no
68+
open-source build yet.
69+
70+
The source is in
71+
[`exir/native/`](https://github.com/pytorch/executorch/tree/main/exir/native) and
72+
[`backends/native/`](https://github.com/pytorch/executorch/tree/main/backends/native).

0 commit comments

Comments
 (0)