Repository navigation
Commit dbddba4
authored
Publish only the package name in the wheel's top_level.txt (pytorch#22267)
## Summary
The pip wheel's metadata advertises twenty top level import names that
do not exist.
`top_level.txt` inside `executorch-*.dist-info/` is the list of names a
package claims to
put at the top level of your Python environment. Ours had twenty-one
lines, and only the
last one was real:
```
@EXECUTORCH_BuiltFile_%CMAKE_CACHE_DIR%/libexecutorch
@EXECUTORCH_BuiltFile_%CMAKE_CACHE_DIR%/backends/cuda/%BUILD_TYPE%/libaoti_cuda_shims
@EXECUTORCH_BuiltFile_%CMAKE_CACHE_DIR%/backends/mlx/mlx/mlx/backend/metal/kernels/mlx
... 17 more like this ...
executorch
```
Those are build recipes. The `%CMAKE_CACHE_DIR%` and `%BUILD_TYPE%`
parts are
placeholders, substituted much later when the file is actually copied
into the wheel.
## Why it happened
The wheel ships prebuilt files it only copies, such as the runtime
library and `flatc`.
Each one is declared to setuptools as an `Extension`. That is
deliberate: a non-empty
`ext_modules` is how setuptools decides a wheel is platform specific
rather than pure
Python. Since these entries are not importable modules, each carries a
synthetic name
beginning with `@`, chosen so it cannot be mistaken for a module path.
setuptools has no way to know that. It builds `top_level.txt` from the
name of every
`ext_modules` entry, so each recipe string was published as written.
Nothing breaks when you import ExecuTorch, because imports resolve
through `RECORD` and
the real directory layout. But any tool that reads `top_level.txt`, and
some packaging,
vendoring and license scanners do, sees twenty names that resolve to
nothing.
## The change
A small `Distribution` subclass drops the synthetic names from the
metadata setuptools
derives. The `@`-prefix is now a named constant, so the filter and the
name being filtered
cannot drift apart later.
## Test plan
Ran `setup.py egg_info` and read the result, in both wheel modes:
| | `top_level.txt` |
| --- | --- |
| before, full wheel | 21 lines, 20 of them recipes |
| after, full wheel | `executorch` |
| after, minimal wheel | `executorch` |
Added `.ci/scripts/tests/test_wheel_top_level_names.py`. It is a unit
test rather than a
wheel check because the behaviour is a pure function of the names on the
entries, so it
needs no built wheel. It asserts the published set is exactly
`{executorch}`, and carries
a second test that fails if the filter ever stops being necessary, so
the pair cannot
quietly stop checking anything.
I confirmed the first test fails when the filter body is removed:
```
AssertionError: assert ['@EXECUTORCH... 'executorch'] == ['executorch']
```1 parent 5e46a63 commit dbddba4
2 files changed
Lines changed: 182 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
79 | | - | |
| 79 | + | |
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
| |||
863 | 863 | | |
864 | 864 | | |
865 | 865 | | |
| 866 | + | |
| 867 | + | |
| 868 | + | |
| 869 | + | |
| 870 | + | |
| 871 | + | |
866 | 872 | | |
867 | 873 | | |
868 | 874 | | |
| |||
1018 | 1024 | | |
1019 | 1025 | | |
1020 | 1026 | | |
1021 | | - | |
| 1027 | + | |
1022 | 1028 | | |
1023 | 1029 | | |
1024 | 1030 | | |
| |||
2218 | 2224 | | |
2219 | 2225 | | |
2220 | 2226 | | |
| 2227 | + | |
| 2228 | + | |
| 2229 | + | |
| 2230 | + | |
| 2231 | + | |
| 2232 | + | |
| 2233 | + | |
| 2234 | + | |
| 2235 | + | |
| 2236 | + | |
| 2237 | + | |
| 2238 | + | |
| 2239 | + | |
| 2240 | + | |
| 2241 | + | |
| 2242 | + | |
| 2243 | + | |
| 2244 | + | |
2221 | 2245 | | |
2222 | 2246 | | |
2223 | 2247 | | |
| |||
2230 | 2254 | | |
2231 | 2255 | | |
2232 | 2256 | | |
| 2257 | + | |
2233 | 2258 | | |
2234 | 2259 | | |
2235 | 2260 | | |
| |||
0 commit comments