Repository navigation
Commit 52bbb30
authored
Fix export_llama never passing enable_bf16 to the XNNPACK partitioner (pytorch#21864)
Fixes pytorch#21859
## Problem
pytorch#21784 added `enable_bf16` and threaded it through
`_to_edge_and_lower_llama_xnnpack` and `get_xnnpack_partitioner`,
defaulting `False` at each layer, but the sole call site never set it:
```python
builder = _to_edge_and_lower_llama_xnnpack(
...
xnnpack_extended_ops=llm_config.backend.xnnpack.extended_ops,
generate_etrecord=llm_config.debug.generate_etrecord,
verbose=llm_config.debug.verbose,
gen_tag_fn=gen_tag_fn,
)
```
So the parameter stayed `False` on every path, and `--dtype-override
bf16 -X` exports silently lost all XNNPACK delegation of their linears:
exit 0, valid `.pte`, `Total delegated subgraphs: 0` visible only under
`-v`. Full measurements in pytorch#21859.
## Fix
Derive the flag from the dtype override at the call site:
```python
enable_bf16=llm_config.model.dtype_override.value == "bf16",
```
This is the condition the parameter was added for, and it cannot change
behavior for fp32 or fp16 exports. With it, the bf16 demo export
delegates the same 21 subgraphs as the fp32 control (per-op table in
pytorch#21859).
## Testing
New `test_bf16_xnnpack_delegates_linears` in test_export_llama_lib.py
exports the no-checkpoint demo model with `dtype_override="bf16"` and
XNNPACK on, and asserts every `aten_linear_default` is delegated. The
assertion is on linears specifically, not a bare delegation count:
measuring both states showed 12 non-gemm subgraphs delegate even without
the flag, so a count-only assertion passes either way and cannot guard
this wiring. With only the source change reverted and the test kept, the
test fails (`linear.delegated=0, non_delegated=8`); with the fix it
passes (`8, 0`).
lintrunner reports no issues on both changed files.
cc @JakeStevens as requested in pytorch#218591 parent 52f9431 commit 52bbb30
2 files changed
Lines changed: 24 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1696 | 1696 | | |
1697 | 1697 | | |
1698 | 1698 | | |
| 1699 | + | |
1699 | 1700 | | |
1700 | 1701 | | |
1701 | 1702 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
73 | 96 | | |
74 | 97 | | |
75 | 98 | | |
| |||
0 commit comments