Split the runtime out of the Python bindings extension (pytorch#21610)

`pip install executorch` gives you the Python half of ExecuTorch and
nothing a C++ program can
link. Everything is fused into one large Python extension file, so a C++
developer has to clone
the repository, sync submodules, and build from source.

There is a correctness problem underneath the packaging one. Because the
runtime is fused into the
extension, anything else that needs it gets its own private copy, and
two copies mean two
registries. A backend registered in one is invisible to the other.

Build the runtime and the pieces around it as separate shared libraries,
and make the Python
extension link them instead of embedding them. A shared library is a
file a program loads at run
time, so several programs can use one copy instead of each carrying its
own.

```
executorch/
  lib/libexecutorch.so                        the runtime
  lib/libexecutorch_kernels_optimized.so      CPU operator kernels
  lib/libexecutorch_backend_xnnpack.so        the XNNPACK delegate
  lib/libexecutorch_threadpool.so             one thread pool per process
  lib/libexecutorch_etdump.so                 the profiler
  extension/pybindings/_portable_lib.so       now under a megabyte, links the above
```

The extension is much smaller than before, because it no longer contains
what it now links.

Linux only, and only when the CUDA backend is off. macOS and Windows
keep the fused extension
because the split relies on ELF sonames, the `$ORIGIN` search-path token
and GNU linker options,
none of which apply there, and enabling it elsewhere now fails while
configuring rather than much
later. A CUDA build also keeps the fused extension for now, because the
CUDA libraries are not yet
shipped alongside the others.

The CMake package a source install writes follows the same line. It
hands out the shared runtime
on Linux, and on Apple it keeps handing out the static libraries it
always did. The shared runtime
does not carry the kernels, so a consumer needs the kernels archive
alongside it, and that archive
pulls a second copy of the core in behind it. ELF resolves both copies
to the executable's
definition and the program is correct; Mach-O binds each image to its
own, so the two never
converge and a registration would not cross the boundary. Removing that
second copy on Apple means
dropping every archive already bundled into the shared runtime from the
list this file hands out,
which is a wider change than this one.

The wheel now ships the profiler as a library and documents it as
usable, so the event tracer is
turned on for the Linux and macOS builds, where it was off and every
recording hook was
preprocessed away. Windows keeps it off. Nothing here touches the
Windows branch of the preset, and
no Windows wheel has been exercised yet, so turning the hooks on there
is left for when it can be
tested.

Turning the tracer on also creates XNNPACK with basic profiling, so the
delegate read per-operator
timings after every execution whether or not a caller had attached a
tracer to read them. That
retrieval now happens only when there is a tracer to receive it. It does
not remove the cost of
profiling, since the timers around each operator remain, and the
standalone profiling build still
retrieves unconditionally through its own gate.

Built the wheel from source, installed it into a clean environment, and
checked:

- exactly one library defines each component, and it is the library that
should own it. Counting
owners alone would also pass on the old fused layout, which has exactly
one too.
- the Python extension defines none of them and resolves all of them
from outside.
- every shipped library loads with no unresolved dependency, and none of
them searches a directory
from the machine that built the wheel. The build-directory patterns are
matched as whole path
components, because a bare substring also matched an unrelated directory
a user could really
have, such as `/home/user/cmake-outputs/torchlibs`, and stripping that
breaks a dependency the
  library legitimately resolves there.
- a read-only build output is no longer archived world-writable. The
previous code granted write
to the group and to everyone via an absolute chmod mode; the fix only
grants owner-write, so a
  `0555` file lands as `0755` instead of `0777`.
- a custom operator library compiles and links against the shipped
Python extension, which is the
existing contract this must not break. Linking these libraries from a
standalone C++ application
additionally needs an installed CMake package, which the wheel does not
carry yet.
- the ahead-of-time quantized library records its route to the runtime
through the same helper the
other targets use. It was written by hand in two blocks that between
them covered only the wheel
layout and only when the wheel flag was set, so a plain
`-DEXECUTORCH_BUILD_SHARED=ON` build left
it with no route at all. Checked all four combinations of the shared
build and the presence of the
  Python extension.

Ran on Linux x86_64 and aarch64, including a Jetson device.

Two runtime requirements are new, `py-cpuinfo` and `requests`. Rewiring
the Qualcomm backend onto
the shared runtime showed that `backends/qualcomm/__init__.py` cannot be
imported from a clean
install at all: it reads the CPU vendor through `cpuinfo` to disable an
mkldnn path on AMD, and the
module it imports first does a module-scope `import requests`. Neither
was declared, so declaring
only one of them moves the failure one line rather than fixing it. They
are declared here, in the
commit that moves that backend, rather than as a separate change.

Not fixed here: these libraries bundle third-party code that torch also
links, and both keep it
visible, so a process holds two definitions of symbols like
`pthreadpool_create`. A caller reaches
whichever the loader found first. Fixing it means hiding or dropping the
bundled copies, which is a
larger change.

Author: shoumikhin

.ci/scripts/tests/test_wheel_platform_tag.py

@@ -0,0 +1,77 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Unit tests for the wheel platform tag comparison.
+
+Here rather than in the wheel checks, because the decision under test is a pure
+function of two strings. Running it as part of the wheel checks meant it needed eleven
+built wheels to exercise one comparison, and it still could not run on a machine that
+had not built one.
+
+The comparison had a real defect that this covers. The release pipeline builds in a
+manylinux image and rewrites the wheel's file name, so the tag on the file and the tag
+auditwheel reports never agree in spelling, and comparing them as text rejected every
+correct wheel. No local build reproduces that rewrite, so nothing short of a unit test
+catches it before CI.
+"""
+
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "wheel"))
+
+from test_shared_libraries import (  # noqa: E402
+    _tag_architectures_match,
+    _wheel_architecture,
+)
+
+
+@pytest.mark.parametrize(
+    "claimed,supported",
+    [
+        # What the release pipeline actually produces: it builds in a manylinux image
+        # and rewrites the file name, while auditwheel reports a plain linux tag
+        # because the wheel depends on torch without vendoring torch's libraries.
+        ("manylinux_2_28_x86_64", "linux_x86_64"),
+        ("manylinux_2_28_aarch64", "linux_aarch64"),
+        # The legacy spelling, which has no underscore before its version.
+        ("manylinux2014_x86_64", "linux_x86_64"),
+        ("manylinux2014_aarch64", "linux_aarch64"),
+        # A local build, where nothing rewrites the name.
+        ("linux_x86_64", "linux_x86_64"),
+        ("linux_aarch64", "linux_aarch64"),
+    ],
+)
+def test_accepts_tags_the_release_pipeline_produces(claimed, supported):
+    assert _tag_architectures_match(claimed, supported) is True
+
+
+@pytest.mark.parametrize(
+    "claimed,supported",
+    [
+        ("manylinux_2_28_aarch64", "linux_x86_64"),
+        ("manylinux_2_28_x86_64", "linux_aarch64"),
+        ("linux_aarch64", "linux_x86_64"),
+    ],
+)
+def test_rejects_an_architecture_mismatch(claimed, supported):
+    """A wheel labelled for the wrong architecture installs where it cannot run."""
+    assert _tag_architectures_match(claimed, supported) is False
+
+
+@pytest.mark.parametrize("tag", ["win_amd64", "macosx_11_0_arm64", "any", "", "linux"])
+def test_reports_a_tag_it_cannot_read(tag):
+    """None, not False, so an unreadable tag is not mistaken for a mismatch."""
+    assert _wheel_architecture(tag) is None
+    assert _tag_architectures_match(tag, "linux_x86_64") is None
+
+
+def test_reads_every_architecture_the_project_builds_for():
+    for architecture in ("x86_64", "aarch64", "i686", "ppc64le", "s390x", "armv7l"):
+        assert _wheel_architecture(f"linux_{architecture}") == architecture
+        assert _wheel_architecture(f"manylinux_2_28_{architecture}") == architecture

.ci/scripts/wheel/pre_build_script.sh

@@ -71,6 +71,11 @@ fi
 # Enable VGF in pybind wheel builds when the platform-specific build input is
 # available from pip.
 if [[ "$UNAME_S" == "Linux" || "$UNAME_S" == "Darwin" ]]; then
+  # The wheel rewrites recorded runtime paths after linking, which needs patchelf. It is declared
+  # under build-system requires, but that is honoured only by a frontend with build isolation and
+  # this path deliberately installs requirements itself, so without this the rewrite silently
+  # does nothing and a CUDA wheel loses its route to the NVIDIA libraries beside it.
+  python3 -m pip install patchelf || true
   if python3 -m pip install -r \
     "${GITHUB_WORKSPACE}/${REPOSITORY}/backends/arm/requirements-arm-vgf-runtime.txt"; then
     export EXECUTORCH_PYBIND_ENABLE_VGF=ON

.ci/scripts/wheel/test_linux.py

@@ -8,8 +8,11 @@
 
 import platform
 import sys
+import tempfile
+from pathlib import Path
 
 import test_base
+import test_shared_libraries
 from examples.models import Backend, Model
 
 if __name__ == "__main__":
@@ -42,6 +45,12 @@
 
         test_base.test_cmsis_nn_install()
 
+        # The wheel ships the runtime, the kernels, the delegate, the thread
+        # pool and the profiler as separate shared libraries now, so check that
+        # each has exactly one owner and that all of them are loadable.
+        with tempfile.TemporaryDirectory() as work_dir:
+            test_shared_libraries.run_tests(Path(work_dir))
+
     model_tests = [
         test_base.ModelTest(
             model=Model.Mv3,

.ci/scripts/wheel/test_linux_aarch64.py

@@ -5,7 +5,11 @@
 # This source code is licensed under the BSD-style license found in the
 # LICENSE file in the root directory of this source tree.
 
+import tempfile
+from pathlib import Path
+
 import test_base
+import test_shared_libraries
 from examples.models import Backend, Model
 
 if __name__ == "__main__":
@@ -26,6 +30,12 @@
     ), f"OpenvinoBackend not found in registered backends: {registered}"
     print("✓ OpenvinoBackend is registered")
 
+    # The wheel ships the runtime, the kernels, the delegate, the thread pool and
+    # the profiler as separate shared libraries now, so check that each has
+    # exactly one owner and that all of them are loadable.
+    with tempfile.TemporaryDirectory() as work_dir:
+        test_shared_libraries.run_tests(Path(work_dir))
+
     test_base.run_tests(
         model_tests=[
             test_base.ModelTest(