Commit 9b9db07
committed
[bugfix] matrixgame2 causal denoise: store KV-cache end indices as Python int, not GPU tensor
Storing kv_cache["global_end_index"] / ["local_end_index"] as
torch.tensor([0], device=device) forces the consumer sites in
causal_model.py and action_module.py through their isinstance(Tensor)
branch, which does int(tensor.item()) on every access. Each .item()
is a GPU->CPU sync that serializes the autoregressive denoise loop
and prevents kernel-queue lookahead.
Both call sites already handle the Python int case (their isinstance
branch goes to else and uses the int directly). The fix is to
initialize as plain int in _initialize_kv_cache /
_initialize_action_kv_cache and let the producer ('fill_' vs
assignment) and consumer (item vs int cast) branches resolve.
Measured on Modal H100x1, num_frames=117 (10 AR blocks * 3 DMD steps
= 30 DiT forwards):
Denoise stage syncs: 10,244 -> 644 (-94%)
Denoise stage sync_ms: 264.1 -> 11.2 (-96%)
avg_queue_delay_us: 42,919 -> 20,465 (-52%)
Inference wall: 18.3s -> 17.2s (-6%)
Note the modest wall improvement: the remaining ~77% GPU-idle is
Python-interpreter throughput bound (5,455 kernels/forward at
14.8us avg), not sync-bound. This fix is a prerequisite for any
subsequent CUDA Graph / persistent-kernel work but not the full
answer.1 parent d6119c1 commit 9b9db07
1 file changed
Lines changed: 6 additions & 12 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
278 | 278 | | |
279 | 279 | | |
280 | 280 | | |
281 | | - | |
282 | | - | |
283 | | - | |
284 | | - | |
| 281 | + | |
| 282 | + | |
285 | 283 | | |
286 | 284 | | |
287 | 285 | | |
| |||
302 | 300 | | |
303 | 301 | | |
304 | 302 | | |
305 | | - | |
306 | | - | |
307 | | - | |
308 | | - | |
| 303 | + | |
| 304 | + | |
309 | 305 | | |
310 | 306 | | |
311 | 307 | | |
| |||
316 | 312 | | |
317 | 313 | | |
318 | 314 | | |
319 | | - | |
320 | | - | |
321 | | - | |
322 | | - | |
| 315 | + | |
| 316 | + | |
323 | 317 | | |
324 | 318 | | |
325 | 319 | | |
| |||
0 commit comments