Skip to content

Commit eaa2dc6

Browse files
authored
Qualcomm: make --use_fp16 and the fp16 compiler spec agree (pytorch#21867)
--use_fp16 in the custom-op examples cleared the quantizer but still passed quant_dtype=use_8a8w. So build_executorch_binary took use_fp16=False and, because quant_dtype was set, built a default quantizer with no annotation for the custom op — producing a quantized graph containing an unquantized custom-op node. Neither of the two things the flag can mean, and its help text says "run in fp16 precision and discard ptq setting". Passing quant_dtype=None takes the fp16 branch and skips quantization, which is the signal the rest of the file already uses (scripts/mobilebert_fine_tune.py, oss_scripts/t5/t5.py). Scoped down from the original version, which also aligned the precision predicate in export_utils.py. That change flips 15 example scripts from kHtpFp16 to kHtpQuantized, six of them 16-bit, so it needs its own PR with a --compile_only sweep behind it. This PR is the examples-only half, which needs no export_utils change: with quant_dtype=None the existing predicate already yields use_fp16=True and skips quantization. Follow-up PR will carry the predicate fix with the is_quantized variable, the full 15-script list, the op-validation concern on the 16-bit ones, and the --use_fp16 test variants in test_qnn_delegate.py. Test plan: verified the fp16 branch is selected and quantization skipped for the post-fix argument shape. Running --use_fp16 end to end on these examples needs the Hexagon SDK to rebuild ExampleOpPackage, and that package's XML declares only QNN_DATATYPE_FLOAT_32 / QNN_DATATYPE_UFIXED_POINT_8 — so a Float16 path there may be a separate follow-up.
1 parent 3dcc374 commit eaa2dc6

2 files changed

Lines changed: 8 additions & 4 deletions

File tree

‎examples/qualcomm/custom_op/custom_ops_1.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -142,10 +142,12 @@ def main(args):
142142
]
143143

144144
# Quantization
145-
quant_dtype = QuantDtype.use_8a8w
145+
# quant_dtype must be None for fp16, otherwise build_executorch_binary
146+
# still lowers as a quantized graph and applies a default quantizer.
146147
if args.use_fp16:
147-
quantizer = None
148+
quant_dtype, quantizer = None, None
148149
else:
150+
quant_dtype = QuantDtype.use_8a8w
149151
quant_cfg = get_ptq_per_channel_quant_config()
150152
custom_quant_annotator = CustomOpsQuantAnnotator()
151153
custom_quant_annotator.register_annotation(

‎examples/qualcomm/custom_op/custom_ops_2.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -150,10 +150,12 @@ def main(args):
150150
]
151151

152152
# Quantization
153-
quant_dtype = QuantDtype.use_8a8w
153+
# quant_dtype must be None for fp16, otherwise build_executorch_binary
154+
# still lowers as a quantized graph and applies a default quantizer.
154155
if args.use_fp16:
155-
quantizer = None
156+
quant_dtype, quantizer = None, None
156157
else:
158+
quant_dtype = QuantDtype.use_8a8w
157159
quant_cfg = get_ptq_per_channel_quant_config()
158160
custom_quant_annotator = CustomOpsQuantAnnotator()
159161
custom_quant_annotator.register_annotation(

0 commit comments

Comments
 (0)