Skip to content

Commit 5e46a63

Browse files
authored
Stop two shipped files from deleting directories they do not own (pytorch#22271)
## Summary Two files that ship in the wheel call `shutil.rmtree` on a path derived from the working directory or the import path, so either could remove a user's files. **1. `examples/models/llama/install_requirement_helper.py`** called `rmtree` at module scope, inside a bare `try/except: pass`. Any *import* of it removed whatever `examples` directory was first on `sys.path`, and a `pkgutil` walk over the installed package is enough to trigger that. Reproduced with a decoy package: ``` $ python -c "import executorch.examples.models.llama.install_requirement_helper" Failed to import examples.models due to lm_eval conflict. Removing lm_eval examples module $ find examples -type f # (empty), exit 0 ``` It now runs only as a script, which is how its single caller has always invoked it. It also identifies lm_eval's copy specifically, by requiring a regular package with no `models` submodule and a single path entry, and no longer swallows errors. **2. `backends/nxp/tests/conftest.py`** calls `rmtree` on `Path(os.getcwd()) / ".outputs"` in `pytest_sessionstart`, with `ignore_errors=True` hiding every failure. That is the user's working directory, and pytest collects a `conftest.py` from anywhere under an installed package, so a session started in the wrong place removed a `.outputs` directory these tests never created. It now refuses unless the directory is one a previous run made, recognised by a marker file it drops, or is empty, which is what a run producing no artifacts leaves. Where the directory is its own, behaviour is unchanged. ## Note on scope This PR originally also pruned test packages from the wheel, which is where I first found these two files. CI showed that does not work: ExecuTorch installs itself non-editable and then runs its own test suites, which import `executorch.backends.arm.test`, `executorch.exir.backend.test` and others from the installed package. Two build-time fixture generators do the same through `python -m test.models.export_delegated_program`. Pruning therefore breaks the project's own build and test flow, so I dropped it and kept only the two safety fixes, which are the actual user-facing problem. The ~14 MB of test payload in every wheel is still worth addressing, but it needs a decision about that boundary from whoever owns the CI contract, so I have left it alone here. ## Test plan The helper, four ways: a plain import no longer deletes anything; as a script against an lm_eval shaped package it still removes it; an `examples` package with a `models` submodule, which is what ours looks like, is left alone; and a namespace package with no `__init__.py` is left alone. The conftest, three ways, driving the real `pytest_sessionstart`: | starting state | result | | --- | --- | | directory holding a user's file | refuses with `RuntimeError`, file still present | | directory it made itself | stale artifact removed, recreated, marker written | | nothing there at all | created, marker written | The ownership check itself: a marked directory and an empty one both count as its own; a directory with unrelated content does not.
1 parent 8b76a3e commit 5e46a63

2 files changed

Lines changed: 112 additions & 22 deletions

File tree

‎backends/nxp/tests/conftest.py‎

Lines changed: 52 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,55 @@ def pytest_sessionstart(session):
3333
import executorch.extension.pybindings.portable_lib
3434
import executorch.kernels.quantized # noqa F401
3535

36-
# Remove all cached test files
37-
shutil.rmtree(outputs_dir.OUTPUTS_DIR, ignore_errors=True)
38-
os.makedirs(outputs_dir.OUTPUTS_DIR, exist_ok=True)
36+
# Remove all cached test files.
37+
#
38+
# Guarded because OUTPUTS_DIR is derived from the working directory, so a session started
39+
# somewhere unexpected would point this at a directory these tests never created. This
40+
# conftest is reachable from an installed package, where pytest collects it for any session
41+
# run under site-packages, so the guard decides whether a delete happens at all rather than
42+
# merely tidying up.
43+
#
44+
# Only the xdist controller clears it. Every worker also runs this hook, and they run
45+
# concurrently, so letting all of them delete and recreate one directory means a worker can
46+
# remove what another just made. The controller has no workerinput attribute; a worker does.
47+
if hasattr(session.config, "workerinput"):
48+
return
49+
50+
outputs = outputs_dir.OUTPUTS_DIR
51+
if not _is_created_by_these_tests(outputs):
52+
raise RuntimeError(
53+
f"{outputs} exists but was not created by these tests, so it is not being "
54+
f"removed. Run the NXP tests from a directory that does not already contain a "
55+
f"'.outputs' directory, or delete it yourself if it is stale."
56+
)
57+
shutil.rmtree(outputs, ignore_errors=True)
58+
os.makedirs(outputs, exist_ok=True)
59+
(outputs / _MARKER_NAME).touch(exist_ok=True)
60+
61+
62+
_MARKER_NAME = ".created-by-nxp-tests"
63+
64+
65+
def _is_created_by_these_tests(directory: pathlib.Path) -> bool:
66+
"""Whether this directory is one these tests made, or does not exist yet.
67+
68+
Three ways it can be ours, in decreasing confidence:
69+
70+
- the marker file, written by every run since this check was added
71+
- empty, which is what a run producing no artifacts leaves
72+
- nothing in it but directories, which is the only shape these tests write: one
73+
subdirectory per test name, and never a file at the top level
74+
75+
The third case exists so that a directory left by a run from before the marker, or by a
76+
checkout that does not write one, is still recognised rather than stopping the session. A
77+
directory holding a file at the top level is not something these tests produce, so it is
78+
left alone.
79+
"""
80+
if not directory.is_dir():
81+
return True
82+
if (directory / _MARKER_NAME).exists():
83+
return True
84+
entries = list(directory.iterdir())
85+
if not entries:
86+
return True
87+
return all(entry.is_dir() for entry in entries)

‎examples/models/llama/install_requirement_helper.py‎

Lines changed: 60 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -4,26 +4,67 @@
44
# This source code is licensed under the BSD-style license found in the
55
# LICENSE file in the root directory of this source tree.
66

7+
"""Removes lm_eval's top level `examples` package, which collides with ours.
8+
9+
lm_eval installs a top level package literally named `examples`. ExecuTorch's own examples
10+
live at `executorch.examples`, but a bare `examples` on the path shadows nothing of ours
11+
while breaking `import examples.models` for code that expects ours, so the colliding copy
12+
is removed after installing lm_eval.
13+
14+
Run as a script, deliberately. The deletion used to happen at import, so anything that
15+
imported this module, including a plugin scan that walks the installed package, deleted
16+
whatever `examples` directory happened to be first on the path.
717
"""
8-
This removes the imported [examples] module from pip installing lm_eval due to
9-
colliding module name with ET package
10-
"""
1118

12-
try:
13-
# If the import fails, this means there's nothing to remove
14-
import examples
19+
import shutil
20+
import sys
21+
from pathlib import Path
22+
23+
24+
def _lm_eval_examples_directory() -> Path | None:
25+
"""The directory to remove, or None when there is nothing to remove.
26+
27+
Returns a path only for a top level `examples` package that has no `models` submodule,
28+
which is what identifies lm_eval's copy rather than a directory that merely shares the
29+
name. ExecuTorch's own examples are never a candidate: they are reached as
30+
`executorch.examples`, not as a top level `examples`.
31+
"""
32+
try:
33+
import examples
34+
except ImportError:
35+
return None
1536

1637
try:
17-
# If the import succeeds, this means it isn't using lm_eval's module
18-
import examples.models
19-
except:
20-
print(
21-
"Failed to import examples.models due to lm_eval conflict. Removing lm_eval examples module"
22-
)
23-
import shutil
24-
25-
examples_path = examples.__path__[0]
26-
shutil.rmtree(examples_path)
27-
28-
except:
29-
pass
38+
import examples.models # noqa: F401
39+
except ImportError:
40+
pass
41+
else:
42+
# Has a models submodule, so this is not the colliding copy.
43+
return None
44+
45+
locations = list(getattr(examples, "__path__", []))
46+
if len(locations) != 1:
47+
# A namespace package spread over several directories. Which one to remove is
48+
# ambiguous, so remove none of them.
49+
return None
50+
51+
directory = Path(locations[0]).resolve()
52+
if not (directory / "__init__.py").exists():
53+
# Only a regular package is removed. A namespace package's directory can be shared
54+
# with unrelated content.
55+
return None
56+
return directory
57+
58+
59+
def main() -> int:
60+
directory = _lm_eval_examples_directory()
61+
if directory is None:
62+
return 0
63+
64+
print(f"Removing lm_eval's colliding examples package at {directory}", flush=True)
65+
shutil.rmtree(directory)
66+
return 0
67+
68+
69+
if __name__ == "__main__":
70+
sys.exit(main())

0 commit comments

Comments
 (0)