Repository navigation
Commit 71a713c
authored
[executorch][cuda] Check FQN weight coverage without AOTI's per-buffer warnings (#24052)
Stacked on #24050.
Every CUDA method load that binds FQN weights next to an off-graph KV
cache printed one AOTI warning line per KV buffer:
```
[WARNING] Found constant or module state buffer or empty module state parameter __et_offgraph_kv_layer_0_k in model, but not provided by user!
```
The `unittest-cuda-muse-glimmer-batching` log contains 1672 of these
lines, out of 11650 lines total.
**Why it happens.** `CudaWeightCache::load` binds the weights with
`validate_full_update=true` and intentionally leaves the off-graph KV
buffers unbound, because the KV pool binds them later. AOTI's
full-update check (`assert_all_constants`) allows absent buffers, but it
writes a warning to stderr for each absent buffer on every load.
**Fix.**
- When the AOTI library exports
`AOTInductorModelContainerGetConstantType` and
`AOTInductorModelContainerGetConstantFromFolded`, ExecuTorch runs the
same completeness rule itself. Folded constants, buffers, tensor
constants, and empty parameters may be absent. Every other constant must
be bound, otherwise `load` fails with `InvalidProgram`.
- Binding then runs with `validate_full_update=false`. In AOTI that flag
only gates `assert_all_constants`, so nothing else about binding
changes.
- If the library does not export those two APIs, `load` keeps
`validate_full_update=true` as before.
Test plan:
- `g++ -fsyntax-only` on `cuda_weight_cache.cpp` and `cuda_backend.cpp`
against the CUDA 12.6 headers.
- CI: the CUDA unittest jobs and the muse-glimmer batching job should
pass, and their logs should no longer contain the `__et_offgraph_kv_`
warnings.1 parent 08e033f commit 71a713c
3 files changed
Lines changed: 85 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
288 | 288 | | |
289 | 289 | | |
290 | 290 | | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
291 | 296 | | |
292 | 297 | | |
293 | 298 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
59 | 68 | | |
60 | 69 | | |
61 | 70 | | |
| |||
286 | 295 | | |
287 | 296 | | |
288 | 297 | | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
289 | 303 | | |
290 | 304 | | |
291 | 305 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
140 | 140 | | |
141 | 141 | | |
142 | 142 | | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
143 | 196 | | |
144 | 197 | | |
145 | 198 | | |
| |||
650 | 703 | | |
651 | 704 | | |
652 | 705 | | |
653 | | - | |
654 | | - | |
655 | | - | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
656 | 718 | | |
657 | 719 | | |
658 | 720 | | |
659 | 721 | | |
660 | 722 | | |
661 | 723 | | |
662 | | - | |
| 724 | + | |
663 | 725 | | |
664 | 726 | | |
665 | 727 | | |
| |||
0 commit comments