Vulkan backend hangs indefinitely at a precise grid-size threshold on AMD RX 7900 XTX (not a VRAM limit)

[c-goodin]

Title: Vulkan backend hangs indefinitely at a precise grid-size threshold on AMD RX 7900 XTX (not a VRAM limit)

System

• GPU: AMD Radeon RX 7900 XTX, 24GB VRAM
• OS: Windows 11
• Python: 3.12.0
• Taichi version: 1.7.4
• Backend: Vulkan (confirmed via a real executed kernel, not just successful ti.init())

Summary

Running an MPM-style granular simulation (particles + background grid, standard substep()/reset() pattern) works correctly and performantly across a huge range of particle counts - then hangs indefinitely at one precise, reproducible threshold, with no error, crash, or timeout message. This does not appear to be a memory capacity issue - VRAM usage at the failure point is a small fraction of what's available.

Reproduction

Using a standard 2D MPM kernel (particle count = 9000 * quality**2, grid = 128 * quality), I benchmarked an escalating series of quality values, then binary-searched the boundary once a failure was found:

Quality	Particles	Result
1	9,000	OK - 271.7 fps
4	144,000	OK - 271.8 fps
16	2,304,000	OK - 279.1 fps
32	9,216,000	OK - 289.5 fps
64	36,864,000	OK - 295.4 fps
72	46,656,000	OK - 283.5 fps
74	49,284,000	OK - 298.3 fps
75	50,625,000	Hangs indefinitely (>90s, no error, no crash, no completion)
76	-	Hangs indefinitely
80	-	Hangs indefinitely
96	-	Hangs indefinitely

The boundary is exact and repeatable: quality=74 (grid 9472x9472) completes in ~0.07s at ~298fps every time. quality=75 (grid 9600x9600) hangs every time, with no exception, no driver crash message, and no eventual completion (tested up to 90s wall time before giving up).

Why this doesn't look like a VRAM problem

At quality=75, the actual memory footprint (grid fields + particle fields) is approximately 2.73 GB - about 11% of this card's 24GB. Performance shows zero degradation as the particle/grid count approaches the failure point (298fps right up to the last successful run), which is inconsistent with approaching a memory ceiling. This looks much more like a driver-level dispatch limit, a Vulkan workgroup/buffer size cap, or a codegen issue specific to this grid dimension - not resource exhaustion.

Why I think this is worth a look

This may be related to the AMD/Vulkan instability already tracked in #8572 ("The same code works on Nvidia CUDA but doesn't work on AMD Vulkan"), though the specific symptom here (a precise, hard hang threshold rather than a general failure) may point to a different or more specific underlying cause. Happy to provide the full benchmark script, exact grid dimensions, or run additional targeted tests if useful.