Repository navigation
Commit 48690b1
[ET-VK] Fix fp16 inference: clamp constants and sync view_convert shader dtype combos
Two fixes for force_fp16 inference mode:
1. Clamp constant tensors to the representable range of the effective dtype
after conversion in VkGraphBuilder. When force_fp16 converts fp32 constants
to fp16, values like -100000 (used as attention mask fill values) overflow to
-inf. A subsequent `0 * (-inf)` produces NaN per IEEE 754, which propagates
through the entire model. Clamping to finfo(fp16).min/max (-65504/65504)
prevents the overflow.
2. Sync dtype combos across view_convert_buffer.yaml and
view_convert_texture.yaml so both define the same set of conversion
variants. Added float<->half, half<->int32 to the buffer YAML, and
float<->half, half<->float, uint8<->float/half/int32 to the texture YAML.
Both files now cover all 9 dtype combos: int32<->float, int32<->half,
uint8<->float/half/int32, float<->half, half<->float, half<->int32.
The float<->half variants are needed by the _to_dim_order_copy operator
when running fp16 models (e.g. SceneX).
Verified on edgeTAM first frame: mask IoU improved from 0.18 (NaN-corrupted)
to 0.9991, matching the fp32 baseline range of 0.9926-0.9995. Also verified
SceneX fp16 model (scenex_v9_512_vulkan_fp16_v3.et) no longer crashes with
missing view_convert_buffer_float_half shader.
Differential Revision: [D96036512](https://our.internmc.facebook.com/intern/diff/D96036512/)
ghstack-source-id: 350221317
Pull Request resolved: pytorch#180761 parent 76dfb19 commit 48690b1
3 files changed
Lines changed: 18 additions & 0 deletions
File tree
- backends/vulkan
- runtime/graph/ops/glsl
- serialization
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
22 | 25 | | |
23 | 26 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
17 | 21 | | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
18 | 25 | | |
19 | 26 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
121 | 129 | | |
122 | 130 | | |
123 | 131 | | |
| |||
0 commit comments