Repository navigation
Commit 88f0c74
authored
Treat an empty LLM data path as absent (pytorch#21835)
`LlmModuleConfig.Builder` defaults `dataPath` to `""`, and
`create_text_llm_runner` takes whatever the optional holds — the empty
string included. So an Android caller that never asked for a separate
data file still reaches the loader with `""`:
```
[llm_runner_helper.cpp:255] Reading metadata from model
[mmap_data_loader.cpp:94] Failed to open : No such file or directory (2)
[llm_runner_helper.cpp:105] Failed reading method names
[llm_runner_helper.cpp:258] Failed to get metadata from model
```
Because `LlmModule.load()` reports failure through
`throwExecutorchException`, this comes out as `Fatal signal 6 (SIGABRT)`
with abort message `'ptr'` and the process dies, rather than an error
the app can catch. That is what makes it hard to place: nothing in the
message points at a data path, and the same `.pte` and tokenizer work
everywhere else.
Reproducing it takes only the documented builder call — no data file, no
unusual model:
```kotlin
LlmModule(
LlmModuleConfig.create()
.modulePath(model)
.tokenizerPath(tokenizer)
.modelType(LlmModuleConfig.MODEL_TYPE_TEXT)
.build())
.load()
```
Passing `.dataPath(null)` explicitly is the workaround.
## The change
The builder default becomes `null`, and both runner factories ignore an
empty path so a caller in any language whose "no value" is an empty
string lands on the no-data-file branch rather than a failed mmap. Three
files: the Kotlin default, the two `data_path.has_value()` checks, and a
test for the default.
I kept the guard in C++ as well as fixing the Kotlin default, because
the optional is public API and an empty string will keep arriving from
bindings that have no null.
## Verification
On a Pixel 8a, arm64, LFM2.5-350M exported to XNNPACK 8da4w:
- before: abort during `load()`, every time
- after: loads in ~2.0 s and generates, prefill 23.8 tok/s / decode 11.9
tok/s
Those rates are low because the phone was locked while measuring, which
parks the app on the little cluster — not a property of this change.
I also confirmed the failure is not the model or the tokenizer: the
`.pte` exposes the full metadata method set the runner expects, and the
tokenizer loads and encodes identically on the same device through a
standalone harness.
This PR was authored with Claude.1 parent 33f68d1 commit 88f0c74
3 files changed
Lines changed: 56 additions & 3 deletions
File tree
- extension
- android/executorch_android/src
- androidTest/java/org/pytorch/executorch
- main/java/org/pytorch/executorch/extension/llm
- llm/runner
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
68 | | - | |
| 68 | + | |
69 | 69 | | |
70 | 70 | | |
71 | 71 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
189 | 189 | | |
190 | 190 | | |
191 | 191 | | |
192 | | - | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
193 | 196 | | |
194 | 197 | | |
195 | 198 | | |
| |||
355 | 358 | | |
356 | 359 | | |
357 | 360 | | |
358 | | - | |
| 361 | + | |
359 | 362 | | |
360 | 363 | | |
361 | 364 | | |
| |||
0 commit comments