Skip to content

Commit 08e033f

Browse files
authored
[executorch][muse] Fix the DFlash final-block draft counts test (#24050)
Stacked on #24047. `CudaDFlashNative.FinalWholeBlockFitsExactlyAndPreservesAllPrefillRows` has failed in `unittest-cuda-muse-glimmer-batching` since #24043. It failed on that PR's own CI and on `main` at 8d9eec3: ``` cuda_dflash_native_test.cpp:612: Failure Expected: (drained.draft_forwards - drain.draft_forwards) > (drained.static_draft_forwards - drain.static_draft_forwards), actual: 1 vs 1 ``` The test expected the decode at position 78 to drain the 79 hidden rows through the dynamic draft before running the static draft. But `run_target` drains every owner it touches right after it appends hidden rows (added in #24042). So the 78-row prefill already drains down to the fresh context (32 in this fixture). The decode then holds at most one block (16) of rows and runs a single static draft. The dynamic forwards happen during the prefill, before the stats snapshot the test took. This is a bug in the test's measurement window, not in the executor. The fix takes the first snapshot before the prefill and checks two things: - the prefill runs only dynamic draft forwards (at least one); - the decode runs exactly one draft forward, and it is static. Test plan: - `clang-format` is clean on the changed lines. - No local GPU, so the test relies on CI: `unittest-cuda-muse-glimmer-batching` on this PR should pass.
1 parent 82fbe45 commit 08e033f

1 file changed

Lines changed: 10 additions & 7 deletions

File tree

‎examples/models/muse-glimmer/tests/cuda_dflash_native_test.cpp‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -598,20 +598,23 @@ TEST_F(CudaDFlashNative, FinalWholeBlockFitsExactlyAndPreservesAllPrefillRows) {
598598
ASSERT_TRUE(executor_);
599599
const auto sid = open();
600600
b::BatchOutput out;
601+
const auto start = executor_->stats();
601602
ASSERT_TRUE(executor_->execute({{input(sid, 0, Tokens(78, 1), false)}}, out));
602603
Peer::check(*executor_, sid, 78);
603604
EXPECT_EQ(executor_->stats().hidden_rows_d2h, 78);
605+
// 78 prefill rows exceed the fresh context, so the prefill itself drains
606+
// them through the dynamic draft.
607+
const auto prefilled = executor_->stats();
608+
EXPECT_GT(prefilled.draft_forwards - start.draft_forwards, 0);
609+
EXPECT_EQ(prefilled.static_draft_forwards, start.static_draft_forwards);
604610
fixture_->rejected.fill(0);
605-
const auto drain = executor_->stats();
606611
ASSERT_TRUE(executor_->execute({{input(sid, 78, {1})}}, out));
607612
EXPECT_EQ(out.outputs[0]->tokens, predictions(78, 2));
608613
Peer::check(*executor_, sid, 80);
609-
// 79 rows exceed one static block: drain dynamically, then draft statically.
610-
const auto drained = executor_->stats();
611-
EXPECT_EQ(drained.static_draft_forwards - drain.static_draft_forwards, 1);
612-
EXPECT_GT(
613-
drained.draft_forwards - drain.draft_forwards,
614-
drained.static_draft_forwards - drain.static_draft_forwards);
614+
// The drained remainder fits one block: the decode drafts statically.
615+
const auto drafted = executor_->stats();
616+
EXPECT_EQ(drafted.draft_forwards - prefilled.draft_forwards, 1);
617+
EXPECT_EQ(drafted.static_draft_forwards - prefilled.static_draft_forwards, 1);
615618
fixture_->rejected.fill(15);
616619
const auto before = executor_->stats();
617620
ASSERT_TRUE(executor_->execute({{input(sid, 80, {prediction(79)})}}, out));

0 commit comments

Comments
 (0)