Skip to content

Commit 9377ded

Browse files
Ship the Core ML delegate as a linkable library in the macOS wheel (pytorch#22202)
## Summary On macOS, the ExecuTorch pip wheel builds the Core ML delegate as a static library and folds it into the Python extension `_C`. That has two downsides: 1. A C++ application that installs the wheel **cannot link the Core ML delegate**, because there is no standalone library to link against (XNNPACK and MLX already ship one; Core ML is the odd one out). 2. `_C` carries the delegate's code even though `_C` is otherwise just the Python bindings. This change makes Core ML match how XNNPACK and MLX already ship: when the wheel is built (`EXECUTORCH_BUILD_SHARED`), the delegate is built as a shared library named `executorch_backend_coreml` and installed into the wheel's `lib/`. `_C` then links it dynamically instead of absorbing it, and a C++ app can link it through `find_package(executorch)` with the new `backend_coreml` component. Nothing changes for a source build: off the wheel path the delegate stays static and links exactly as before. ## What changed - `backends/apple/coreml/CMakeLists.txt`: the delegate builds `SHARED` under `EXECUTORCH_BUILD_SHARED`, named `executorch_backend_coreml`, with the shared-object soname policy and an `@loader_path` runtime path, and bundles its `coreml_util` / `coreml_inmemoryfs` helper archives. The static (source-build) path is unchanged. - `setup.py`: ships `lib/libexecutorch_backend_coreml.dylib`. - `tools/cmake/executorch-wheel-config.cmake`: adds the `backend_coreml` `find_package` component. - `.ci/scripts/wheel/test_shared_libraries.py`: adds the library to the name allowlist and a one-owner row for the delegate. ## Test plan Built with `EXECUTORCH_BUILD_SHARED=ON EXECUTORCH_BUILD_COREML=ON` on macOS and confirmed with `nm`/`otool`: - `libexecutorch_backend_coreml.dylib` is produced with install name `@rpath/libexecutorch_backend_coreml.1.dylib` and an `@loader_path` rpath; - it exports the delegate entry point exactly once; - it links the shared runtime `@rpath/libexecutorch.1.dylib` (not a static copy), plus the CoreML/Accelerate/Foundation frameworks and sqlite3; - the referenced `coreml_util` / `coreml_inmemoryfs` helpers are bundled in. The wheel release check that asserts one owner per component passes, with the delegate's `get_registered_delegate()` symbol defined only by this library. --------- Co-authored-by: PyTorch Bot <pytorchbot@users.noreply.github.com>
1 parent 758122a commit 9377ded

6 files changed

Lines changed: 105 additions & 11 deletions

File tree

‎.ci/scripts/wheel/test_shared_libraries.py‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,12 @@
155155
# does: a second copy means two MLX runtimes in one process.
156156
_MLX_SYMBOLS = ("executorch::backends::mlx::mutable_state_note_handle",)
157157
_BUNDLED_MLX_SYMBOLS = ("mlx::core::allocator::free",)
158+
# A representative symbol from the Core ML delegate, exported from the shipped
159+
# library. get_registered_delegate() rather than the constructor, which demangles
160+
# to two entries (complete and base object), while this is a single definition.
161+
_COREML_SYMBOLS = (
162+
"executorch::backends::coreml::CoreMLBackendDelegate::get_registered_delegate()",
163+
)
158164

159165
# A representative symbol from the TorchAO kernels. These are Apple Silicon only, so
160166
# most wheels ship no such library and the row below is not required.
@@ -872,6 +878,11 @@ def _wheel_cuda_train() -> str:
872878

873879
# Marker for a row whose owner every Linux wheel carries and no macOS wheel does.
874880
_REQUIRED_ON_LINUX = "linux-only"
881+
# Marker for a row whose owner every macOS wheel carries and no other wheel does.
882+
# The Core ML delegate is built on every macOS wheel, so its presence is decidable
883+
# from the installed wheel and a macOS wheel that dropped it should fail rather than
884+
# quietly regress to folding it into the Python extension.
885+
_REQUIRED_ON_MACOS = "macos-only"
875886
# Narrower than the marker above on purpose. The Qualcomm SDK the delegate builds
876887
# against is published for Linux x86_64 only, so the aarch64 Linux wheels ship no such
877888
# library and demanding it there would fail a wheel that is correct. Reusing the
@@ -885,6 +896,8 @@ def _resolve_required(required):
885896
return bool(_wheel_cuda_train())
886897
if required == _REQUIRED_ON_LINUX:
887898
return sys.platform == "linux"
899+
if required == _REQUIRED_ON_MACOS:
900+
return sys.platform == "darwin"
888901
if required == _REQUIRED_ON_LINUX_X86:
889902
return sys.platform == "linux" and platform.machine() in ("x86_64", "amd64")
890903
return required
@@ -946,6 +959,15 @@ def _resolve_required(required):
946959
_library_file_name("libexecutorch_backend_xnnpack"),
947960
True,
948961
),
962+
# Required on macOS, where the delegate is built for every wheel, so a macOS
963+
# wheel that folds it back into the Python extension fails here. Not built on
964+
# any other platform, where the row resolves to not-required and skips.
965+
(
966+
"Core ML delegate",
967+
_COREML_SYMBOLS,
968+
_library_file_name("libexecutorch_backend_coreml"),
969+
_REQUIRED_ON_MACOS,
970+
),
949971
# Not required: the delegate is built only on an Apple Silicon macOS wheel whose
950972
# build found the Metal compiler, so every other wheel legitimately ships no MLX
951973
# library and the row skips.
@@ -2337,6 +2359,7 @@ def test_shipped_library_names_are_expected() -> None:
23372359
# dependency, so both have to ship and both are expected here.
23382360
"libextension_cuda",
23392361
"libexecutorch_backend_xnnpack",
2362+
"libexecutorch_backend_coreml",
23402363
"libexecutorch_backend_mlx",
23412364
"libexecutorch_backend_openvino",
23422365
"libexecutorch_backend_qnn",

‎CMakeLists.txt‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1419,7 +1419,11 @@ if(EXECUTORCH_BUILD_PYBIND)
14191419
# Only the libraries that do not already retain themselves are named. A
14201420
# delegate that calls executorch_target_link_options_shared_lib exports
14211421
# --no-as-needed around its own file to every consumer, so the CUDA and
1422-
# OpenVINO backends are covered by that and are deliberately absent here.
1422+
# OpenVINO backends are covered by that and are deliberately absent here. That
1423+
# export is emitted only for NOT APPLE, so it does nothing for an Apple-only
1424+
# delegate like Core ML; those are covered instead by Mach-O, which keeps a
1425+
# named dylib's namespace-scope initializers without an --as-needed
1426+
# equivalent.
14231427
foreach(_retained_component optimized_native_cpu_ops_lib xnnpack_backend
14241428
extension_threadpool etdump
14251429
)

‎backends/apple/coreml/CMakeLists.txt‎

Lines changed: 55 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,15 @@ if(APPLE)
1212
find_library(SQLITE_LIBRARY sqlite3 REQUIRED)
1313
endif()
1414

15+
# Signals to packaging whether the standalone shared delegate library was built.
16+
# Reset to OFF on every configure so a reused build directory that switches
17+
# platform or library type does not carry a stale value; the shared branch below
18+
# turns it on only when it actually produces the library.
19+
set(EXECUTORCH_COREML_DELEGATE_LIBRARY_BUILT
20+
OFF
21+
CACHE INTERNAL "The standalone Core ML delegate library was built"
22+
)
23+
1524
if(EXECUTORCH_BUILD_DEVTOOLS)
1625
# protobuf requires frtti
1726
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -frtti")
@@ -186,7 +195,17 @@ endif()
186195
# coremldelegate
187196

188197
if(APPLE)
189-
add_library(coremldelegate)
198+
# Build the delegate as a shared library for the wheel so a C++ consumer can
199+
# link it, and so the Python extension links it dynamically instead of
200+
# absorbing its code. Keyed on EXECUTORCH_BUILD_SHARED rather than
201+
# BUILD_SHARED_LIBS, the same way the other shipped delegates decide this, so
202+
# the wheel controls it independently of the generic CMake switch.
203+
if(EXECUTORCH_BUILD_SHARED)
204+
set(_coreml_backend_library_type SHARED)
205+
else()
206+
set(_coreml_backend_library_type STATIC)
207+
endif()
208+
add_library(coremldelegate ${_coreml_backend_library_type})
190209
target_sources(coremldelegate PRIVATE ${KVSTORE_SOURCES} ${DELEGATE_SOURCES})
191210

192211
target_include_directories(
@@ -237,11 +256,42 @@ if(APPLE)
237256
endif()
238257

239258
target_link_libraries(
240-
coremldelegate
241-
PUBLIC coreml_util coreml_inmemoryfs
242-
PRIVATE executorch_core ${ACCELERATE_FRAMEWORK} ${COREML_FRAMEWORK}
243-
${FOUNDATION_FRAMEWORK} ${SQLITE_LIBRARY}
259+
coremldelegate PRIVATE ${ACCELERATE_FRAMEWORK} ${COREML_FRAMEWORK}
260+
${FOUNDATION_FRAMEWORK} ${SQLITE_LIBRARY}
244261
)
262+
if(EXECUTORCH_BUILD_SHARED)
263+
set_target_properties(
264+
coremldelegate PROPERTIES OUTPUT_NAME executorch_backend_coreml
265+
)
266+
executorch_target_soname_policy(coremldelegate)
267+
# coreml_util and coreml_inmemoryfs are static archives this delegate owns,
268+
# so bundle them inside this one shipped library instead of making every
269+
# consumer supply them. A plain private link is enough, not whole-archive:
270+
# the delegate references the helpers it uses, so ordinary resolution pulls
271+
# exactly those objects. Neither helper registers anything from an
272+
# unreferenced namespace-scope initializer, which is the only thing
273+
# force_load would add here.
274+
target_link_libraries(
275+
coremldelegate PRIVATE coreml_util coreml_inmemoryfs executorch_shared
276+
)
277+
# Ships beside the runtime in the wheel's lib/ directory.
278+
executorch_target_shipped_runtime_path(coremldelegate)
279+
# A signal for packaging that the standalone delegate library exists. The
280+
# EXECUTORCH_BUILD_COREML flag alone is not enough because it is also on for
281+
# the Python extension on non-Apple platforms, where this shared library is
282+
# not built, so keying the wheel's install on it would look for a file that
283+
# was never produced.
284+
set(EXECUTORCH_COREML_DELEGATE_LIBRARY_BUILT
285+
ON
286+
CACHE INTERNAL "The standalone Core ML delegate library was built"
287+
)
288+
else()
289+
target_link_libraries(
290+
coremldelegate
291+
PUBLIC coreml_util coreml_inmemoryfs
292+
PRIVATE executorch_core
293+
)
294+
endif()
245295

246296
executorch_target_link_options_shared_lib(coremldelegate)
247297

‎docs/source/using-executorch-cpp.md‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -182,6 +182,7 @@ These are the components the package provides:
182182
| `backend_cuda` | The CUDA delegate | Linux |
183183
| `extension_cuda` | The CUDA stream extension | Linux |
184184
| `backend_openvino` | The OpenVINO delegate | Linux |
185+
| `backend_coreml` | The Core ML delegate, for Apple GPU and Neural Engine execution | macOS |
185186
| `backend_mlx` | The MLX delegate, for Apple GPU execution | macOS, Apple Silicon |
186187

187188
To see what your own install offers, ask CMake:
@@ -190,18 +191,14 @@ To see what your own install offers, ask CMake:
190191
find_package(executorch REQUIRED)
191192
foreach(_component
192193
runtime kernels_optimized kernels_quantized kernels_torchao
193-
backend_xnnpack backend_mlx backend_cuda extension_cuda
194+
backend_xnnpack backend_coreml backend_mlx backend_cuda extension_cuda
194195
backend_openvino threadpool etdump)
195196
if(TARGET executorch::${_component})
196197
message(STATUS "have ${_component}")
197198
endif()
198199
endforeach()
199200
```
200201

201-
On macOS the Core ML delegate is registered inside the Python extension rather than shipped as a
202-
separate C++ library, so a C++ application there cannot link it as a component; use it from
203-
Python, or build from source if you need it in C++.
204-
205202
Profiling a Core ML model records `DELEGATE_CALL`, which tells you how long the delegate ran in
206203
total. It does not record the individual operators inside the delegate, because that detail comes
207204
from the Core ML developer tools sources, and the wheel does not build them. An XNNPACK model

‎setup.py‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2170,6 +2170,8 @@ def run(self): # noqa C901
21702170
if cmake_cache.is_enabled("EXECUTORCH_BUILD_SHARED"):
21712171
cmake_build_args += ["--target", "executorch_shared"]
21722172
cmake_build_args += ["--target", "etdump"]
2173+
if cmake_cache.is_enabled("EXECUTORCH_COREML_DELEGATE_LIBRARY_BUILT"):
2174+
cmake_build_args += ["--target", "coremldelegate"]
21732175
if cmake_cache.is_enabled(
21742176
"EXECUTORCH_BUILD_PTHREADPOOL"
21752177
) and cmake_cache.is_enabled("EXECUTORCH_BUILD_CPUINFO"):
@@ -2402,6 +2404,20 @@ def run(self): # noqa C901
24022404
"EXECUTORCH_BUILD_MLX",
24032405
],
24042406
),
2407+
# Install the Core ML delegate beside them, so a C++ consumer can
2408+
# link it out of the wheel rather than only reaching it from Python.
2409+
BuiltFile(
2410+
src_dir="%CMAKE_CACHE_DIR%/backends/apple/coreml/",
2411+
src_name=get_dynamic_lib_name("executorch_backend_coreml"),
2412+
dst="executorch/lib/"
2413+
+ get_dynamic_lib_name("executorch_backend_coreml"),
2414+
dependent_cmake_flags=[
2415+
# Not EXECUTORCH_BUILD_COREML: that is also on for the Python
2416+
# extension on non-Apple platforms, where this shared library
2417+
# is not built. This flag is set only when it actually is.
2418+
"EXECUTORCH_COREML_DELEGATE_LIBRARY_BUILT",
2419+
],
2420+
),
24052421
# Install the prebuilt pybindings extension wrapper for the runtime,
24062422
# portable kernels, and a selection of backends. This lets users
24072423
# load and execute .pte files from python.

‎tools/cmake/executorch-wheel-config.cmake‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,7 @@
7171
# model. Not part of EXECUTORCH_LIBRARIES, see
7272
# below.
7373
# executorch::backend_xnnpack The XNNPACK delegate.
74+
# executorch::backend_coreml The Core ML delegate. macOS only.
7475
# executorch::backend_mlx The MLX delegate. macOS on Apple Silicon only.
7576
# Its Metal kernel archive is published as
7677
# MLX_METALLIB_PATH, see below.
@@ -336,6 +337,7 @@ if(_executorch_runtime_library AND NOT _executorch_targets_supported)
336337
ITEMS libexecutorch_kernels_optimized
337338
libexecutorch_kernels_torchao
338339
libexecutorch_backend_xnnpack
340+
libexecutorch_backend_coreml
339341
libexecutorch_backend_mlx
340342
libexecutorch_backend_cuda
341343
libexecutorch_extension_cuda
@@ -658,6 +660,8 @@ if(TARGET executorch::runtime AND TARGET executorch::threadpool)
658660
endif()
659661

660662
_executorch_define_component(backend_xnnpack executorch_backend_xnnpack)
663+
# The Core ML delegate, present only in a wheel built on macOS.
664+
_executorch_define_component(backend_coreml executorch_backend_coreml)
661665
# The MLX delegate, present only in a wheel built on Apple Silicon with the
662666
# Metal compiler available.
663667
_executorch_define_component(backend_mlx executorch_backend_mlx)

0 commit comments

Comments
 (0)