Skip to content

Commit f715ad2

Browse files
authored
Strip local symbols from the macOS wheel's binaries (pytorch#22272)
## Summary The macOS wheel carries about **7 MB of local symbols** that the Linux wheel does not. Both are Release builds. The difference is the linker: the GNU one discards these, Apple's keeps them. The Linux libraries have no `.symtab` section at all, while every macOS one carries a local symbol per internal function: | file | external | local | | --- | --- | --- | | `libexecutorch.dylib` | 333 | 448 | | `libexecutorch_kernels_optimized.dylib` | 1463 | 4190 | | `data/bin/flatc` | 3737 | **35496** | `flatc` is most of it. At 11,629,400 bytes it is 5.3 MB *larger* than the Linux copy, and stripped it comes to 5,571,232, which is **smaller** than Linux's 6,313,152. So this one file explains nearly the entire size gap between the two wheels. ## The change `strip -x` removes local symbols and keeps every external one, so nothing a consumer can link against changes. That flag matters: anything broader would remove exported symbols and break linking. It runs on the wheel copy only, right after the runtime path rewrite that already runs there, and deliberately not on the editable copy, where a developer's debugger and profiler want those symbols. A failure warns and ships the file as built, because a wheel that is larger than intended beats a build that stops. ## Test plan Measured on the published macOS arm64 wheel: native payload **32.8 MB to 25.7 MB, saving 7.2 MB (21%)**. Stripped a copy of an installed wheel and ran from a neutral directory: | check | result | | --- | --- | | exported symbol list, before vs after | identical | | registered backends | CoreML, MLX, Vgf, Xnnpack | | XNNPACK export + run | `[3.0, 3.0, 3.0, 3.0]` | | MLX export + run | `[3.0, 3.0, 3.0, 3.0]` | | C++ app via `find_package(executorch)` | builds, `load_method 0x0`, out `3.0` x4 | The helper itself, four ways: on a Linux file it does nothing; with no `strip` on PATH it returns quietly; on a real failure, tested with a read-only file, it warns and does not raise; and on a Mach-O library it removes 11 percent while leaving the exported set byte identical.
1 parent f2d148b commit f715ad2

3 files changed

Lines changed: 81 additions & 2 deletions

File tree

‎backends/cadence/aot/graph_builder.py‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,25 @@
88

99
# This module has moved to executorch.backends.test.graph_builder.
1010
# This re-export exists for backward compatibility.
11-
from executorch.backends.test.graph_builder import GraphBuilder, single_op_builder
11+
#
12+
# Resolved on attribute access rather than at import, because the target lives in a test
13+
# package and the wheel does not ship those. Importing it here would make this module, which
14+
# does ship, fail to import in an installed wheel.
15+
16+
from typing import Any, TYPE_CHECKING
17+
18+
if TYPE_CHECKING:
19+
from executorch.backends.test.graph_builder import ( # noqa: F401
20+
GraphBuilder,
21+
single_op_builder,
22+
)
1223

1324
__all__ = ["GraphBuilder", "single_op_builder"]
25+
26+
27+
def __getattr__(name: str) -> Any:
28+
if name in __all__:
29+
from executorch.backends.test import graph_builder
30+
31+
return getattr(graph_builder, name)
32+
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

‎backends/cadence/aot/program_builder.py‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,25 @@
44

55
# This module has moved to executorch.backends.test.program_builder.
66
# This re-export exists for backward compatibility.
7-
from executorch.backends.test.program_builder import IrMode, ProgramBuilder
7+
#
8+
# Resolved on attribute access rather than at import, because the target lives in a test
9+
# package and the wheel does not ship those. Importing it here would make this module, which
10+
# does ship, fail to import in an installed wheel.
11+
12+
from typing import Any, TYPE_CHECKING
13+
14+
if TYPE_CHECKING:
15+
from executorch.backends.test.program_builder import ( # noqa: F401
16+
IrMode,
17+
ProgramBuilder,
18+
)
819

920
__all__ = ["IrMode", "ProgramBuilder"]
21+
22+
23+
def __getattr__(name: str) -> Any:
24+
if name in __all__:
25+
from executorch.backends.test import program_builder
26+
27+
return getattr(program_builder, name)
28+
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

‎setup.py‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1272,6 +1272,14 @@ def build_extension(self, ext: _BaseExtension) -> None:
12721272
self.get_finalized_command("build"), "cmake_cache_dir", None
12731273
)
12741274
_strip_absolute_runtime_paths(dst_file, _cuda_libraries_built(cmake_cache_dir))
1275+
# After the rewrite rather than before it, because each tool re-signs the file ad hoc when
1276+
# it finishes, so whichever runs last owns the signature. Running strip last keeps the
1277+
# signature over the bytes that ship. Functionally either order works: measured, the two
1278+
# produce the same size, the same exported symbols, the same rpaths, and both verify.
1279+
#
1280+
# Only the wheel copy is stripped; the editable copy above is a developer's own build
1281+
# output, where the local symbols are what a debugger and a profiler read.
1282+
_strip_local_symbols(dst_file)
12751283

12761284

12771285
def _append_relative_search_paths(entries: List[str], depth: int = 1) -> None:
@@ -1354,6 +1362,39 @@ def _run_install_name_tool(command: List[str], library: Path) -> None:
13541362
)
13551363

13561364

1365+
def _strip_local_symbols(library: Path) -> None:
1366+
"""Discard the local symbol table from a Mach-O file the wheel ships.
1367+
1368+
Only on macOS, and only because its linker keeps these where the GNU one does not. The Linux
1369+
shared libraries ship with no .symtab section, while the macOS ones carried a local symbol for
1370+
every internal function: 448 in the runtime, 4190 in the merged kernels, and 35496 in flatc,
1371+
which alone was 6 MB of the 7 MB this removes.
1372+
1373+
`-x` removes local symbols and keeps every external one, so what a consumer can link against
1374+
does not change. Anything beyond that would strip exported symbols and break linking, so the
1375+
flag matters.
1376+
1377+
A failure here is not fatal. The file is correct either way, and a wheel that is larger than
1378+
intended is better than a build that stops. That also covers the files this runs on that are
1379+
not Mach-O at all, such as the Metal library and a template: strip declines them, and on BSD
1380+
it declines them with a zero exit, leaving the file byte for byte as it was.
1381+
"""
1382+
if not _is_macos():
1383+
return
1384+
strip = shutil.which("strip")
1385+
if strip is None:
1386+
return
1387+
result = subprocess.run(
1388+
[strip, "-x", os.fspath(library)], capture_output=True, check=False
1389+
)
1390+
if result.returncode != 0:
1391+
print(
1392+
f"warning: could not strip local symbols from {library.name}, shipping it as "
1393+
f"built: {(result.stderr or result.stdout).decode(errors='replace').strip()}",
1394+
flush=True,
1395+
)
1396+
1397+
13571398
def _parse_runtime_paths(original: str, is_mach_o: bool) -> List[str]:
13581399
"""The runtime search path entries a tool reported, as a list.
13591400

0 commit comments

Comments
 (0)