Build and ship the OpenVINO backend in the Windows wheels

The OpenVINO delegate lets ExecuTorch run models on Intel CPUs, GPUs and
NPUs. Until now the prebuilt delegate shipped only in the Linux wheels. The
Windows wheels left it out, even though Intel publishes an OpenVINO runtime
for Windows x86-64, so Windows users had to build ExecuTorch from source.

This change builds the delegate into the Windows wheels, CPU and CUDA, the
same way the Linux wheels carry it. setup.py enables EXECUTORCH_BUILD_OPENVINO
on Windows and packages the delegate's import library (.lib) beside the other
Windows DLLs. The build file passes /EHsc and /GR when CMake reports an MSVC
style compiler, which includes the clang-cl the wheels use, and -frtti and
-fexceptions otherwise.

The delegate loads the OpenVINO C library at run time, so the build needs no
OpenVINO SDK. That loader used dlopen, with libopenvino_c.so as the default
name. It is now split by platform. On Windows it reads OPENVINO_LIB_PATH with
GetEnvironmentVariableW and loads with LoadLibraryExW, so a path outside the
active code page still works, and it reports load errors as the system's
message text. A bare name such as openvino_c.dll is searched with
LOAD_LIBRARY_SEARCH_DEFAULT_DIRS, which covers the program's own folder,
System32 and folders added with AddDllDirectory, but not the current folder
or PATH. So an openvino_c.dll planted in the current folder is not loaded. A
path is made absolute and also searches its own folder, so the runtime's
dependencies, openvino.dll and tbb12.dll, load from beside it. The Python
package already finds the pip installed runtime and passes its full path; it
now looks for openvino_c.dll on Windows. The delegate is a global, so on
Windows its destructor keeps the runtime loaded rather than calling
FreeLibrary inside DllMain. On Linux the load error now also names the path
it tried.

pip install "executorch[openvino]" now installs the runtime on Windows x86-64
too. The OpenVINO versions the extra allows publish no build for Python 3.15,
or for Linux aarch64 on Python 3.14, so there the extra now installs nothing.
Before, the command failed on those Linux setups. The docs say so.

The Windows wheel smoke tests, CPU and CUDA, check that the delegate
registers. They then install the openvino extra and the quantizer the
partitioner needs. The extra is pinned to the installed version, so pip
cannot swap the wheel under test. In a new process, they require that the
delegate loads the runtime and that a model it runs matches eager. From a folder holding a copy
of the runtime, they also check that the delegate loads the copy by its full
path but not by its bare name. On Python 3.15 they only check that the extra
installs without a runtime. The shared model check now asserts that the
program is really delegated, and loads it from memory, because Windows does
not delete a file the loaded program keeps open. The shared library check
requires the delegate on Linux and Windows wheels, and finds it by its
is_available member rather than its class name, because the reader of
Windows exports matches decorated names, which spell a member and never a
class alone. The docs, the README and the PyPI description list Windows. The
backend README explains where Windows looks for the runtime's dependencies,
and its Linux OPENVINO_LIB_PATH example now finds the versioned library the
pip package ships.

Test plan:

- Built the loader for Windows with a MinGW cross compiler and ran it under
  Wine with the OpenVINO 2025.4.1 Windows DLLs. Full, forward slash, relative
  and non-ASCII paths load the runtime and find the CPU device, and so does a
  path set with SetEnvironmentVariableW after startup, or changed while it is
  read. A bare name does not load a copy planted in the current folder.
  Errors read as text with their code. Unloading the delegate's DLL leaves
  the runtime loaded.
- Ran the bare name check against that build under Wine. It passes with the
  real DLLs. It fails when the copied runtime cannot load, and when the
  loader searches the current folder.
- On Python 3.15 with pip, the extra installs without a runtime, and the
  check fails if the python_version bound is removed, also when an older
  release is available to pip. A resolver run shows the extra resolves on
  Linux aarch64 with Python 3.14 only with the new bound.
- On Linux, built the delegate against the wheel's C++ SDK and ran the
  Windows OpenVINO check with OpenVINO 2025.4.1 and nncf 3.1.0. The runtime
  loads and the model matches eager exactly, and the check fails when the
  runtime cannot load.
- Ran the repository's linters (black, usort, flake8, clang-format and
  others) on the changed files.
- On native Windows, the CPU and CUDA wheel jobs (Python 3.11) run the new
  checks. The Python 3.15 branch first runs in the nightly build.

Author: shoumikhin

.ci/scripts/wheel/test_base.py

@@ -8,10 +8,8 @@
 import os
 import subprocess
 import sys
-import tempfile
 from dataclasses import dataclass
 from functools import cache
-from pathlib import Path
 from typing import List
 
 
@@ -43,7 +41,7 @@ class ModelTest:
     backend: Backend
 
 
-def test_a_model_runs_through_the_openvino_delegate() -> None:
+def test_a_model_runs_through_the_openvino_delegate(required: bool = False) -> None:
     """Export a model to the OpenVINO delegate and run it, comparing against eager.
 
     The checks elsewhere prove the delegate library ships, owns its symbols and registers
@@ -53,7 +51,8 @@ def test_a_model_runs_through_the_openvino_delegate() -> None:
     The OpenVINO runtime is not part of the wheel. It comes from the openvino extra, so a
     default install cannot run this and says so rather than passing quietly. Where the runtime
     is present the export and the comparison are the point of the check, and a failure there
-    is a real one.
+    is a real one. A caller that installed the export stack passes required=True, so a missing
+    piece fails instead of skipping.
     """
     import torch
     from executorch.exir import to_edge_transform_and_lower
@@ -67,6 +66,8 @@ def test_a_model_runs_through_the_openvino_delegate() -> None:
     try:
         from executorch.backends.openvino.partitioner import OpenvinoPartitioner
     except ImportError as error:
+        if required:
+            raise
         print(
             f"SKIP: the OpenVINO export stack is not installed, so the delegate cannot "
             f"execute here ({error}). The wheel ships the adapter only; the runtime and the "
@@ -88,13 +89,16 @@ def forward(self, x, y):
         partitioner=[OpenvinoPartitioner([CompileSpec("device", b"CPU")])],
     )
     program = lowered.to_executorch()
+    delegates = [d.id for d in program.executorch_program.execution_plan[0].delegates]
+    assert "OpenvinoBackend" in delegates, (
+        f"the partitioner did not hand the model to OpenVINO (delegates: {delegates}), so "
+        f"running it would test the portable kernels instead of the delegate"
+    )
 
-    with tempfile.TemporaryDirectory() as work_dir:
-        pte = Path(work_dir) / "add_openvino.pte"
-        pte.write_bytes(program.buffer)
-
-        method = Runtime.get().load_program(pte).load_method("forward")
-        actual = method.execute(list(example))[0]
+    # From memory rather than a temporary file, which the loaded program keeps open and
+    # Windows then refuses to delete.
+    method = Runtime.get().load_program(program.buffer).load_method("forward")
+    actual = method.execute(list(example))[0]
 
     difference = (actual - eager).abs().max().item()
     assert difference == 0.0, (

.ci/scripts/wheel/test_cuda_windows.py

@@ -640,6 +640,15 @@ def export(destination: str) -> None:
     with tempfile.TemporaryDirectory() as work_dir:
         test_cpp_sdk.run_tests(Path(work_dir))
     import test_base
+    from executorch.extension.pybindings.portable_lib import (
+        _get_registered_backend_names,
+    )
+
+    registered = _get_registered_backend_names()
+    assert (
+        "OpenvinoBackend" in registered
+    ), f"OpenvinoBackend not found in registered backends: {registered}"
+    print("✓ OpenvinoBackend is registered")
 
     test_base.test_cmsis_nn_install()
     # The same model run the CPU row ends with, through the Python bindings.
@@ -650,3 +659,6 @@ def export(destination: str) -> None:
     test_windows.run_tests(
         model_tests=[ModelTest(model=Model.Mv3, backend=Backend.Xnnpack)]
     )
+
+    # Last, because it installs packages the wheel does not declare.
+    test_windows.test_the_openvino_delegate_runs()

.ci/scripts/wheel/test_shared_libraries.py

@@ -149,7 +149,11 @@
 # The delegate's own entry points. A second definer means the delegate is compiled
 # into the Python extension as well, which would register it twice in one process.
 _QNN_BACKEND_SYMBOLS = ("QnnExecuTorchBackendRegister",)
-_OPENVINO_BACKEND_SYMBOLS = ("executorch::backends::openvino::OpenvinoBackend",)
+# A member rather than the bare class name, because the PE reader matches MSVC decorated
+# names, which spell a member as ?is_available@OpenvinoBackend@... and never the class alone.
+_OPENVINO_BACKEND_SYMBOLS = (
+    "executorch::backends::openvino::OpenvinoBackend::is_available",
+)
 
 _BUNDLED_XNNPACK_SYMBOLS = ("xnn_create_runtime_v4",)
 
@@ -1097,31 +1101,32 @@ def _wheel_cuda_train() -> str:
 # wheel is inspected, and a row cannot call that at import time.
 _REQUIRED_ON_A_CUDA_WHEEL = "cuda-wheel-only"
 
-# Marker for a row whose owner every Linux wheel carries and no macOS wheel does.
-_REQUIRED_ON_LINUX = "linux-only"
+# Marker for a row whose owner every Linux and Windows wheel carries and no macOS wheel
+# does.
+_REQUIRED_ON_LINUX_AND_WINDOWS = "linux-and-windows-only"
 # Marker for a row whose owner every macOS wheel carries and no other wheel does.
 # The Core ML delegate is built on every macOS wheel, so its presence is decidable
 # from the installed wheel and a macOS wheel that dropped it should fail rather than
 # quietly regress to folding it into the Python extension.
 _REQUIRED_ON_MACOS = "macos-only"
-# Narrower than the marker above on purpose. The Qualcomm SDK the delegate builds
-# against is published for Linux x86_64 only, so the aarch64 Linux wheels ship no such
-# library and demanding it there would fail a wheel that is correct. Reusing the
-# Linux-wide marker would do exactly that.
+# Narrower than the Linux and Windows marker on purpose. The Qualcomm SDK the delegate
+# builds against is published for Linux x86_64 only, so the aarch64 Linux wheels ship no
+# such library and demanding it there would fail a wheel that is correct. Reusing that
+# marker would do exactly that.
 _REQUIRED_ON_LINUX_X86 = "linux-x86-only"
 # Marker for a row whose owner every aarch64 wheel carries, macOS and Linux alike. The
 # TorchAO kernels are selected by the processor rather than the operating system, so
-# neither the macOS nor the Linux marker describes them and a fixed False would let a
-# wheel that silently dropped the library pass.
+# neither the macOS nor the Linux and Windows marker describes them and a fixed False
+# would let a wheel that silently dropped the library pass.
 _REQUIRED_ON_AARCH64 = "aarch64-only"
 
 
 def _resolve_required(required):
     """Turn a row's requirement marker into the answer for the installed wheel."""
     if required == _REQUIRED_ON_A_CUDA_WHEEL:
         return bool(_wheel_cuda_train())
-    if required == _REQUIRED_ON_LINUX:
-        return sys.platform == "linux"
+    if required == _REQUIRED_ON_LINUX_AND_WINDOWS:
+        return sys.platform in ("linux", "win32")
     if required == _REQUIRED_ON_MACOS:
         return sys.platform == "darwin"
     if required == _REQUIRED_ON_LINUX_X86:
@@ -1279,15 +1284,15 @@ def _resolve_required(required):
         _library_file_name("libexecutorch_backend_mlx"),
         False,
     ),
-    # Required on Linux, where packaging turns the backend on for every non-minimal
-    # build. A fixed False passed a wheel that had compiled the delegate back into the
-    # extension: no library ships, so the row skips, and one definer inside the
+    # Required on Linux and Windows, where packaging turns the backend on for every
+    # non-minimal build. A fixed False passed a wheel that had compiled the delegate back
+    # into the extension: no library ships, so the row skips, and one definer inside the
     # extension is exactly the monolithic layout a definer count cannot distinguish.
     (
         "OpenVINO delegate",
         _OPENVINO_BACKEND_SYMBOLS,
         _library_file_name("libexecutorch_backend_openvino"),
-        _REQUIRED_ON_LINUX,
+        _REQUIRED_ON_LINUX_AND_WINDOWS,
     ),
     # Required on Linux x86_64 only, where the Qualcomm SDK the delegate builds against
     # is published. A fixed False would pass a wheel that compiled the delegate back

.ci/scripts/wheel/test_windows.py

@@ -6,7 +6,13 @@
 # This source code is licensed under the BSD-style license found in the
 # LICENSE file in the root directory of this source tree.
 
+import importlib.metadata
+import importlib.util
+import os
 import platform
+import shutil
+import subprocess
+import sys
 import tempfile
 from pathlib import Path
 from typing import List
@@ -24,6 +30,7 @@
 from executorch.exir import EdgeCompileConfig, to_edge_transform_and_lower
 from executorch.extension.pybindings.portable_lib import (
     _get_registered_backend_names,
+    _is_available,
     _load_for_executorch_from_buffer,
 )
 from test_base import ModelTest
@@ -71,14 +78,116 @@ def run_tests(model_tests: List[ModelTest]) -> None:
             raise RuntimeError(f"Unsupported backend {model_test.backend}.")
 
 
+def test_the_openvino_delegate_runs() -> None:
+    """Install the OpenVINO runtime the way a user does, then run a model through it.
+
+    The wheel ships the delegate but not the runtime, and registering the delegate does not
+    load the runtime, so nothing else here proves the delegate can open openvino_c.dll and
+    compute. The check runs in a new process because the package looks for the runtime when
+    it is imported, which in this process happened before the install.
+    """
+    # Pinned, so pip cannot swap the wheel under test for another release.
+    install = [
+        sys.executable,
+        "-m",
+        "pip",
+        "install",
+        f"executorch[openvino]=={importlib.metadata.version('executorch')}",
+    ]
+    # The same bound as the python_version marker on the openvino extra.
+    if sys.version_info >= (3, 15):
+        subprocess.run(install, check=True)
+        importlib.invalidate_caches()
+        assert (
+            importlib.util.find_spec("openvino") is None
+        ), "the openvino extra installed a runtime on a Python its marker leaves out"
+        print(
+            "SKIP: OpenVINO publishes no build for this Python, so the openvino extra "
+            "installs no runtime and the delegate cannot load here."
+        )
+        return
+    subprocess.run(
+        install
+        + [
+            "-r",
+            str(
+                Path(test_base._repository_root_dir())
+                / "backends/openvino/requirements.txt"
+            ),
+        ],
+        check=True,
+    )
+    subprocess.run(
+        [sys.executable, str(Path(__file__).resolve()), "--openvino"], check=True
+    )
+    importlib.invalidate_caches()
+    openvino = importlib.util.find_spec("openvino")
+    test_a_bare_name_does_not_load_from_the_current_folder(
+        Path(openvino.submodule_search_locations[0]) / "libs"
+    )
+
+
+def test_a_bare_name_does_not_load_from_the_current_folder(libs: Path) -> None:
+    """Run from a folder holding the real runtime and ask for it by bare name.
+
+    The standard Windows search order includes the current folder, where a planted
+    openvino_c.dll would run, so the delegate has to refuse the copy found there. The
+    same copy asked for by its full path has to load, so the refusal comes from the
+    search order and not from a copy that cannot load at all.
+    """
+    with tempfile.TemporaryDirectory() as work_dir:
+        for library in libs.glob("*.dll"):
+            shutil.copy2(library, work_dir)
+        for lib_path, expected in (
+            (str(Path(work_dir) / "openvino_c.dll"), "True"),
+            ("openvino_c.dll", "False"),
+        ):
+            probe = subprocess.run(
+                [
+                    sys.executable,
+                    "-c",
+                    "from executorch.extension.pybindings.portable_lib import "
+                    "_is_available; print(_is_available('OpenvinoBackend'))",
+                ],
+                cwd=work_dir,
+                env={**os.environ, "OPENVINO_LIB_PATH": lib_path},
+                capture_output=True,
+                text=True,
+                check=False,
+            )
+            assert probe.returncode == 0 and probe.stdout.split()[-1:] == [expected], (
+                f"OPENVINO_LIB_PATH={lib_path}, run from a folder holding the runtime, "
+                f"should make _is_available return {expected}: the full path has to "
+                "load the copy there and the bare name must not, since a planted copy "
+                f"would run:\n{probe.stdout}{probe.stderr}"
+            )
+    print(
+        "✓ a bare openvino_c.dll name does not load from the current folder, while "
+        "its full path does"
+    )
+
+
 if __name__ == "__main__":
+    if sys.argv[1:] == ["--openvino"]:
+        assert _is_available(
+            "OpenvinoBackend"
+        ), "OpenvinoBackend is registered but cannot load the OpenVINO runtime"
+        print("✓ OpenvinoBackend loads the OpenVINO runtime")
+        test_base.test_a_model_runs_through_the_openvino_delegate(required=True)
+        sys.exit(0)
+
     # Before anything else, because this is the check that fails the way a user
     # fails: with only the dependencies the wheel declares.
     with tempfile.TemporaryDirectory() as work_dir:
         test_clean_install.run_tests(Path(work_dir))
 
     if platform.system() == "Windows":
         registered = _get_registered_backend_names()
+        assert (
+            "OpenvinoBackend" in registered
+        ), f"OpenvinoBackend not found in registered backends: {registered}"
+        print("✓ OpenvinoBackend is registered")
+
         # Vulkan backend is optional: only present when the wheel was built with
         # EXECUTORCH_BUILD_VULKAN=1 and the Vulkan SDK (glslc) was available.
         if "VulkanBackend" in registered:
@@ -108,3 +217,7 @@ def run_tests(model_tests: List[ModelTest]) -> None:
             ),
         ]
     )
+
+    # Last, because it installs packages the wheel does not declare.
+    if platform.system() == "Windows":
+        test_the_openvino_delegate_runs()

README-wheel.md

@@ -34,9 +34,11 @@ to run ExecuTorch `.pte` files, with some restrictions:
 * \[macOS only] [Core ML](docs/source/backends/coreml/coreml-overview.md) and MLX backends are
   also linked into the prebuilt module.
 * \[Linux x86_64] [QNN](docs/source/backends-qualcomm.md) backend is linked into the prebuilt module.
-* \[Linux] [OpenVINO](docs/source/build-run-openvino.md) backend is also linked into the
+* \[Linux and Windows] [OpenVINO](docs/source/build-run-openvino.md) backend is also linked into the
   prebuilt module. OpenVINO requires the runtime to be installed separately:
-  `pip install executorch[openvino]`
+  `pip install executorch[openvino]`. For now the extra installs no runtime on Python 3.15,
+  or on Linux aarch64 with Python 3.14, because the OpenVINO versions it supports publish no
+  build there.
 
 On Linux, macOS and Windows the runtime, kernels and backends also ship as shared libraries
 with headers and a CMake package, so a C++ application can link them without building

README.md

@@ -200,7 +200,7 @@ is authoritative. See also the [Desktop guide](desktop/README.md).
 | iOS / iPadOS | [XNNPACK](https://docs.pytorch.org/executorch/main/backends/xnnpack/xnnpack-overview.html) CPU; [Core ML](https://docs.pytorch.org/executorch/main/backends/coreml/coreml-overview.html); [MLX](https://docs.pytorch.org/executorch/main/backends/mlx/mlx-overview.html) on physical devices *(experimental)* |
 | macOS | [XNNPACK](https://docs.pytorch.org/executorch/main/backends/xnnpack/xnnpack-overview.html); [Core ML](https://docs.pytorch.org/executorch/main/backends/coreml/coreml-overview.html); experimental [MLX](https://docs.pytorch.org/executorch/main/backends/mlx/mlx-overview.html), [Metal/AOTInductor](backends/apple/metal/README.md), and [WebGPU](https://docs.pytorch.org/executorch/main/backends/webgpu/webgpu-overview.html) paths |
 | Linux | [XNNPACK](https://docs.pytorch.org/executorch/main/backends/xnnpack/xnnpack-overview.html); [OpenVINO](https://docs.pytorch.org/executorch/main/build-run-openvino.html); experimental [CUDA/AOTInductor](https://docs.pytorch.org/executorch/main/backends/cuda/cuda-overview.html), [Vulkan](https://docs.pytorch.org/executorch/main/backends/vulkan/vulkan-overview.html), and [WebGPU](https://docs.pytorch.org/executorch/main/backends/webgpu/webgpu-overview.html) desktop paths |
-| Windows | [XNNPACK](https://docs.pytorch.org/executorch/main/backends/xnnpack/xnnpack-overview.html); experimental [CUDA/AOTInductor](https://docs.pytorch.org/executorch/main/backends/cuda/cuda-overview.html) and [Vulkan](https://docs.pytorch.org/executorch/main/backends/vulkan/vulkan-overview.html) desktop paths |
+| Windows | [XNNPACK](https://docs.pytorch.org/executorch/main/backends/xnnpack/xnnpack-overview.html); [OpenVINO](https://docs.pytorch.org/executorch/main/build-run-openvino.html); experimental [CUDA/AOTInductor](https://docs.pytorch.org/executorch/main/backends/cuda/cuda-overview.html) and [Vulkan](https://docs.pytorch.org/executorch/main/backends/vulkan/vulkan-overview.html) desktop paths |
 | Browser / WebAssembly | [Portable WebAssembly runtime](extension/wasm/README.md) and [WebGPU](https://docs.pytorch.org/executorch/main/backends/webgpu/webgpu-overview.html) *(both experimental)* |
 | Embedded / MCU | [Arm Cortex-M with CMSIS-NN](https://docs.pytorch.org/executorch/main/backends/arm-cortex-m/arm-cortex-m-overview.html) *(beta)*; [Arm Ethos-U](https://docs.pytorch.org/executorch/main/backends/arm-ethos-u/arm-ethos-u-overview.html); [NXP eIQ Neutron](https://docs.pytorch.org/executorch/main/backends/nxp/nxp-overview.html); [Cadence DSP](https://docs.pytorch.org/executorch/main/backends-cadence.html); [Zephyr](zephyr/README.md) and [Arduino](examples/arduino/README.md) integrations |
 

backends/openvino/CMakeLists.txt

@@ -28,8 +28,9 @@ set(COMMON_INCLUDE_DIRS ${EXECUTORCH_ROOT}/..)
 # Include utility CMake scripts from ExecuteTorch
 include(${EXECUTORCH_ROOT}/tools/cmake/Utils.cmake)
 
-# The backend resolves OpenVINO C API symbols via dlopen/dlsym at runtime, so
-# there is no build-time dependency on the OpenVINO SDK.
+# The backend resolves OpenVINO C API symbols at runtime (dlopen on Linux,
+# LoadLibrary on Windows), so there is no build-time dependency on the OpenVINO
+# SDK.
 
 # Shared when the wheel build asks for it, static otherwise.
 if(EXECUTORCH_BUILD_SHARED)
@@ -39,8 +40,13 @@ else()
 endif()
 add_library(openvino_backend ${_openvino_backend_library_type})
 
-# Enable exceptions and RTTI for OpenVINO backend
-target_compile_options(openvino_backend PRIVATE -frtti -fexceptions)
+# Enable exceptions and RTTI for OpenVINO backend. MSVC is also set for
+# clang-cl, which takes the cl spellings.
+if(MSVC)
+  target_compile_options(openvino_backend PRIVATE /EHsc /GR)
+else()
+  target_compile_options(openvino_backend PRIVATE -frtti -fexceptions)
+endif()
 
 # Add source files for OpenVINO backend
 target_sources(