Repository navigation
Commit 5e46a63
authored
Stop two shipped files from deleting directories they do not own (pytorch#22271)
## Summary
Two files that ship in the wheel call `shutil.rmtree` on a path derived
from the working
directory or the import path, so either could remove a user's files.
**1. `examples/models/llama/install_requirement_helper.py`** called
`rmtree` at module scope,
inside a bare `try/except: pass`. Any *import* of it removed whatever
`examples` directory was
first on `sys.path`, and a `pkgutil` walk over the installed package is
enough to trigger
that. Reproduced with a decoy package:
```
$ python -c "import executorch.examples.models.llama.install_requirement_helper"
Failed to import examples.models due to lm_eval conflict. Removing lm_eval examples module
$ find examples -type f # (empty), exit 0
```
It now runs only as a script, which is how its single caller has always
invoked it. It also
identifies lm_eval's copy specifically, by requiring a regular package
with no `models`
submodule and a single path entry, and no longer swallows errors.
**2. `backends/nxp/tests/conftest.py`** calls `rmtree` on
`Path(os.getcwd()) / ".outputs"` in
`pytest_sessionstart`, with `ignore_errors=True` hiding every failure.
That is the user's
working directory, and pytest collects a `conftest.py` from anywhere
under an installed
package, so a session started in the wrong place removed a `.outputs`
directory these tests
never created.
It now refuses unless the directory is one a previous run made,
recognised by a marker file it
drops, or is empty, which is what a run producing no artifacts leaves.
Where the directory is
its own, behaviour is unchanged.
## Note on scope
This PR originally also pruned test packages from the wheel, which is
where I first found
these two files. CI showed that does not work: ExecuTorch installs
itself non-editable and
then runs its own test suites, which import
`executorch.backends.arm.test`,
`executorch.exir.backend.test` and others from the installed package.
Two build-time fixture
generators do the same through `python -m
test.models.export_delegated_program`. Pruning
therefore breaks the project's own build and test flow, so I dropped it
and kept only the two
safety fixes, which are the actual user-facing problem.
The ~14 MB of test payload in every wheel is still worth addressing, but
it needs a decision
about that boundary from whoever owns the CI contract, so I have left it
alone here.
## Test plan
The helper, four ways: a plain import no longer deletes anything; as a
script against an
lm_eval shaped package it still removes it; an `examples` package with a
`models` submodule,
which is what ours looks like, is left alone; and a namespace package
with no `__init__.py` is
left alone.
The conftest, three ways, driving the real `pytest_sessionstart`:
| starting state | result |
| --- | --- |
| directory holding a user's file | refuses with `RuntimeError`, file
still present |
| directory it made itself | stale artifact removed, recreated, marker
written |
| nothing there at all | created, marker written |
The ownership check itself: a marked directory and an empty one both
count as its own; a
directory with unrelated content does not.1 parent 8b76a3e commit 5e46a63
2 files changed
Lines changed: 112 additions & 22 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
36 | | - | |
37 | | - | |
38 | | - | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
7 | 17 | | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | 18 | | |
12 | | - | |
13 | | - | |
14 | | - | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
15 | 36 | | |
16 | 37 | | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
0 commit comments