Skip to content

Commit 02e2d7a

Browse files
committed
[TRTLLM-17033][test] Run most qwen4_exp accuracy gates post-merge
Every gate builds the full Qwen3.8-Flash-Next-NVFP4 engine (a gsm8k gate is roughly 8-13 min on one GB300), so the 26 pre-merge gates cost several GPU hours per CI run. Pre-merge keeps main's three gates (gsm8k; gsm8k with one MTP draft and the acceptance-length floor; mmmu with MTP) and adds the one gate that runs the sparse attention path: 3k-6k-token pass keys with three drafts and IndexShare over bf16 KV. Post-merge runs one gate per opt-in variant: MTP3 alone and over CuteDSL experts, MTP7 over long pass keys, FP8 and NVFP4 KV over the pass keys, the bf16 recurrent state at MTP3, MTP7 all rejected by forced acceptance, video, and the breakable prefill graph against eager. The other gates stay in test_qwen4_exp.py for on-demand runs. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>
1 parent c8b05ad commit 02e2d7a

1 file changed

Lines changed: 23 additions & 37 deletions

File tree

‎tests/integration/test_lists/test-db/l0_b300.yml‎

Lines changed: 23 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -136,46 +136,15 @@ l0_b300:
136136
# stage instead), so 30 is 2.5x headroom.
137137
- unittest/_torch/modeling_v2 --ignore=unittest/_torch/modeling_v2/comm TIMEOUT (30)
138138
- accuracy/modeling_v2/test_gpt_oss.py::TestModelingV2GptOss120bSm103Tp1::test_gsm8k
139-
# qwen4_exp target: gsm8k, and gsm8k with the trunk's routed experts on CuteDSL;
140-
# gsm8k with MTP plus the acceptance-length floor, the only gate that sees a
141-
# miscomputed draft layer, at one draft over bf16 and fp8 KV, at three drafts
142-
# alone and over CuteDSL experts, and at five and seven drafts (the verify step's
143-
# six and eight rows per request); 3k-6k-token pass key prompts, the only cases that
144-
# run the sparse attention path (IndexShare over bf16 KV, fp8 KV), and 3k-12k-token ones
145-
# at seven drafts; seven drafts all rejected and all accepted by forced acceptance at
146-
# batch 32 over long prompts; the recurrent-state verify path that
147-
# TRTLLM_USE_GDN_REPLAY=0 selects; the recurrent state stored in bf16 without speculation,
148-
# at three drafts and at seven drafts over long prompts; mmmu with MTP and synthetic
149-
# videos, the only gates that reach the vision tower; and the opt-in breakable prefill
150-
# CUDA graph: gsm8k, gsm8k with three drafts and the length floor, 3k-6k-token pass keys
151-
# prefilled in replayed 2048-token chunks, mmmu, and replayed against eager greedy
152-
# continuations at the bucket edges and with context and generation rows in one batch.
139+
# qwen4_exp target, pre-merge: gsm8k; gsm8k with one MTP draft plus the
140+
# acceptance-length floor, the only gate that sees a miscomputed draft layer;
141+
# mmmu with MTP, which reaches the vision tower; and 3k-6k-token pass key prompts
142+
# with three drafts and IndexShare over bf16 KV, the only prompts long enough to run
143+
# the sparse attention path. The opt-in variants run post-merge (below).
153144
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_gsm8k
154-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_gsm8k_cutedsl_moe
155145
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp1_gsm8k_and_acceptance
156-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp1_fp8_kv_gsm8k_and_acceptance
157-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp1_nvfp4_kv_gsm8k_and_acceptance
158-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_gsm8k_and_acceptance
159-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_cutedsl_gsm8k_and_acceptance
160-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp5_gsm8k_and_acceptance
161-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp7_gsm8k_and_acceptance
162-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_long_context_passkey[index_share]
163-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_long_context_passkey[fp8_kv]
164-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_long_context_passkey[nvfp4_kv]
165-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp7_long_context_passkey
166-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp7_forced_acceptance_extremes[all_rejected]
167-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp7_forced_acceptance_extremes[all_accepted]
168-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_gsm8k_bf16_ssm_state
169-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_bf16_ssm_state_gsm8k_and_acceptance
170-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp7_bf16_ssm_state_long_context_passkey
171-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp1_gdn_legacy_verify
172146
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp1_mmmu
173-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp1_video
174-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_gsm8k_breakable_prefill_cuda_graph
175-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_breakable_prefill_gsm8k_and_acceptance
176-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_long_context_passkey[breakable_prefill]
177-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp1_mmmu_breakable_prefill
178-
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_breakable_prefill_matches_eager
147+
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_long_context_passkey[index_share]
179148
- condition:
180149
ranges:
181150
system_gpu_count:
@@ -196,3 +165,20 @@ l0_b300:
196165
- unittest/_torch/moe/test_moe_backend.py::test_moe_backend -k "TRTLLM"
197166
- unittest/_torch/moe/test_moe_backend.py::test_moe_backend -k "CUTEDSL"
198167
- unittest/_torch/moe/test_moe_backend.py::test_moe_backend -k "DEEPGEMM"
168+
# qwen4_exp target, post-merge: one gate per opt-in variant -- three drafts, alone and
169+
# over CuteDSL experts; seven drafts over 3k-12k-token pass keys (the verify step's
170+
# eight rows per request); fp8 and NVFP4 KV over the pass keys (their sparse attention
171+
# path); the recurrent state in bf16 at three drafts; seven drafts all rejected by
172+
# forced acceptance (the speculative rollback end to end); synthetic videos; and the
173+
# breakable prefill CUDA graph replayed against eager greedy continuations at the bucket
174+
# edges and with context and generation rows in one batch. The other gates in
175+
# test_qwen4_exp.py run on demand.
176+
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_gsm8k_and_acceptance
177+
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_cutedsl_gsm8k_and_acceptance
178+
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp7_long_context_passkey
179+
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_long_context_passkey[fp8_kv]
180+
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_long_context_passkey[nvfp4_kv]
181+
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp3_bf16_ssm_state_gsm8k_and_acceptance
182+
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp7_forced_acceptance_extremes[all_rejected]
183+
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_mtp1_video
184+
- accuracy/modeling_v2/test_qwen4_exp.py::TestModelingV2Qwen38FlashNextNvfp4Sm103Tp1::test_breakable_prefill_matches_eager

0 commit comments

Comments
 (0)