Repository navigation
Commit c3bce5d
authored
Add fused rms norm pass (pytorch#22849)
Summary:
FuseRMSNormPass folds the HuggingFace RMSNorm decomposition into a single
aten.rms_norm.
HF *RMSNorm lowers to `weight * (x * rsqrt(mean(x**2, -1, keepdim=True) + eps))`.
The pass matches that chain (a mul over rsqrt / add / mean / pow), reads eps from
the lifted constant added to the variance, and rewrites the whole ~6-op sequence
to `aten.rms_norm(x, [D], weight, eps)` -- torch's F.rms_norm -- so the native
runtime executes RMSNorm as one op.
The matcher also sees through the optional float32 up-cast HF wraps around the
variance computation and the matching down-cast after the normalize. When the
up-cast is present it must be shared by both the variance and the scale path;
otherwise the chain is not a RMSNorm apply and is left alone. Note that the fused
aten.rms_norm computes in the input dtype, so for bf16/fp16 models this folds
away HF's fp32 variance accumulation.
Every intermediate node must be single-use so the decomposition DCEs away after
the rewrite.
The pass is added to get_default_passes(), so it runs pre-partition on the
edge-dialect graph, and aten.rms_norm.default is added to the partitioner's
_SUPPORTED_NON_CORE_OPS so it is both claimed by the delegate and preserved (not
decomposed back) via ops_to_not_decompose.
Reviewed By: digantdesai
Differential Revision: D111753289
Pull Request resolved: pytorch#228491 parent cfc96cc commit c3bce5d
6 files changed
Lines changed: 1873 additions & 0 deletions
File tree
- backends
- native
- passes
- transforms
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
| 36 | + | |
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
41 | 42 | | |
42 | 43 | | |
43 | 44 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
| 43 | + | |
43 | 44 | | |
44 | 45 | | |
45 | 46 | | |
| |||
52 | 53 | | |
53 | 54 | | |
54 | 55 | | |
| 56 | + | |
55 | 57 | | |
56 | 58 | | |
57 | 59 | | |
| |||
69 | 71 | | |
70 | 72 | | |
71 | 73 | | |
| 74 | + | |
72 | 75 | | |
73 | 76 | | |
74 | 77 | | |
| |||
0 commit comments