Repository navigation
Commit 1d280bb
authored
Import transformers only where the LLM runner needs it (pytorch#22268)
## Summary
Importing the LLM runner from a plain wheel install fails:
```
>>> import executorch.extension.llm.runner
ModuleNotFoundError: No module named 'transformers'
```
`transformers` is not a dependency of this package and the wheel does
not install it, but
`extension/llm/runner/__init__.py` imported `BatchFeature` from it at
module scope. So the
entire runner was unreachable, including the compiled `_llm_runner`
extension the wheel
ships. That extension is around 3 MB and has nothing to do with
HuggingFace.
## The change
Only two functions need that class: `generate_hf` and
`generate_text_hf`. Both exist
specifically to accept output from a HuggingFace `AutoProcessor`, so
anyone calling them
already has `transformers` installed.
The class is now looked up when one of those functions runs, rather than
when the module
loads. The annotations that mention it became strings under
`TYPE_CHECKING`, so type
checkers still resolve it normally.
Nothing changes for users who have `transformers`: the same class object
is used, so the
`isinstance` checks and the conversion behave exactly as before. For
users who do not, the
runner imports fine and only the two `hf` helpers raise, naming the
package that is
missing.
## Test plan
Ran against the installed wheel on macOS arm64, from a directory
containing no checkout.
Without `transformers`:
| | result |
| --- | --- |
| before | `ModuleNotFoundError: No module named 'transformers'` |
| after | import OK, `MultimodalRunner` present, both `hf` helpers
attached |
| after, calling `generate_hf` | still raises, naming `transformers` |
With `transformers` installed, confirming behaviour is unchanged:
```
_batch_feature_type() is transformers' BatchFeature True
isinstance(BatchFeature(...), resolved) True
_hf_to_multimodal_inputs(BatchFeature(...)) 1 MultimodalInput
```1 parent f715ad2 commit 1d280bb
1 file changed
Lines changed: 44 additions & 15 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
38 | | - | |
| 38 | + | |
39 | 39 | | |
40 | | - | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
41 | 68 | | |
42 | 69 | | |
43 | 70 | | |
| |||
68 | 95 | | |
69 | 96 | | |
70 | 97 | | |
71 | | - | |
| 98 | + | |
72 | 99 | | |
73 | 100 | | |
74 | 101 | | |
| |||
171 | 198 | | |
172 | 199 | | |
173 | 200 | | |
174 | | - | |
| 201 | + | |
175 | 202 | | |
176 | 203 | | |
177 | 204 | | |
178 | 205 | | |
179 | 206 | | |
180 | 207 | | |
181 | | - | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
182 | 214 | | |
183 | 215 | | |
184 | 216 | | |
185 | 217 | | |
186 | | - | |
187 | | - | |
188 | | - | |
189 | | - | |
190 | 218 | | |
191 | 219 | | |
192 | 220 | | |
| |||
197 | 225 | | |
198 | 226 | | |
199 | 227 | | |
200 | | - | |
| 228 | + | |
201 | 229 | | |
202 | 230 | | |
203 | 231 | | |
204 | 232 | | |
205 | | - | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
206 | 239 | | |
207 | 240 | | |
208 | 241 | | |
209 | 242 | | |
210 | | - | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | 243 | | |
215 | 244 | | |
216 | 245 | | |
| |||
0 commit comments