Skip to content

Commit e1e551b

Browse files
Arm backend: Relax DeepSeek VGF BF16 abs tolerance (pytorch#22644)
The xlarge DeepSeek VGF BF16 test is numerically stable in its mean error but exceeds the existing 0.1 absolute tolerance across platforms and random seeds. | Platform | Seed | Result | Max |Δ| | Mean |Δ| | |-----------|------|--------|----------|----------| | Ubuntu x86| 0 | Fail | 0.140625 | 0.026189 | | Ubuntu x86| 1 | Fail | 0.150391 | 0.026114 | | Ubuntu x86| 2 | Fail | 0.134766 | 0.025801 | | macOS | 0 | Fail | 0.156250 | 0.025820 | | macOS | 1 | Fail | 0.153320 | 0.025708 | | macOS | 2 | Pass | — | — | Raise atol from 0.1 to 0.17, providing modest headroom above the largest observed difference while retaining rtol=0.1. The exact test passes on Ubuntu x86 with seeds 0, 1, and 2 after this change. cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani Signed-off-by: Sangwon Ha <sangwon.ha@arm.com>
1 parent dc8c578 commit e1e551b

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

‎backends/arm/test/models/DeepSeek_R1_Distill_Qwen/test_deepseek_r1_distill_qwen_model.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -197,7 +197,7 @@ class DeepSeekR1DistillQwenModelTestCase:
197197
"base_model": DeepSeekR1DistillQwenModelTestCase(
198198
model_cls=BaseModelWrapper,
199199
config_factory=_make_deepseek_r1_distill_qwen_1_5b_model_config,
200-
atol=0.1,
200+
atol=0.17,
201201
rtol=0.1,
202202
tosa_spec="TOSA-1.0+FP+bf16",
203203
),

0 commit comments

Comments
 (0)