You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4891009
Browse filesBrowse the repository at this point in the historyBrowse files
[ET-VK] Synthesize the layer_norm affine parameters that are absent (pytorch#22506)
Fixespytorch#22504.
`add_native_layer_norm_node` already synthesizes a zero bias for
`nn.LayerNorm(bias=False)`, but still throws on a missing weight.
`F.layer_norm` called with no affine parameters at all is an ordinary
call, used whenever a module applies its own scale and shift afterwards,
and it currently makes the model abort at prepack.
This synthesizes a unit weight the same way, sizing it from
`normalized_shape` when there is no weight tensor to take a shape from.
A unit weight and a zero bias reproduce `out = (x - mean) * rstd`
exactly. The two paths now share one helper.
Verified on device: kokoro's duration predictor, whose `AdaLayerNorm`
normalizes with no affine parameters, aborts on `main` and runs to
completion with this change on an Adreno 840.
`cases.py` gains three `native_layer_norm` cases covering weight-only,
bias-only and neither.
cc @SS-JIA@manuelcandales@digantdesai@cbilgin
0 commit comments