Build and ship the OpenVINO backend in the Windows wheels

The OpenVINO delegate lets ExecuTorch run models on Intel CPUs, GPUs and
NPUs. Until now the prebuilt delegate shipped only in the Linux wheels. The
Windows wheels left it out, even though Intel publishes an OpenVINO runtime
for Windows x86-64, so Windows users had to build ExecuTorch from source.

This change builds the delegate into the Windows wheels, CPU and CUDA, the
same way the Linux wheels carry it. setup.py enables EXECUTORCH_BUILD_OPENVINO
on Windows and packages the delegate's import library (.lib) beside the other
Windows DLLs. The build file passes /EHsc and /GR when CMake reports an MSVC
style compiler, which includes the clang-cl the wheels use, and -frtti and
-fexceptions otherwise.

The delegate loads the OpenVINO C library at run time, so the build needs no
OpenVINO SDK. That loader used dlopen and a fixed libopenvino_c.so name. It is
now split by platform. On Windows it reads OPENVINO_LIB_PATH with _wgetenv and
loads with LoadLibraryExW, so a path outside the active code page still works,
and it reports load errors as text. A bare name such as openvino_c.dll is
searched with LOAD_LIBRARY_SEARCH_DEFAULT_DIRS, which leaves out the current
folder and PATH, so a DLL planted in the folder a program runs from is never
loaded. A path is made absolute and also searches its own folder, so the
runtime's dependencies, openvino.dll and tbb12.dll, load from beside it. The
Python package already finds the pip installed runtime and passes its full
path; it now looks for openvino_c.dll on Windows. The delegate is a global, so
on Windows its destructor keeps the runtime loaded rather than calling
FreeLibrary inside DllMain.

pip install "executorch[openvino]" now installs the runtime on Windows x86-64
too. The extra installs nothing on Python 3.15, where OpenVINO publishes no
build yet, so the command no longer fails there on Linux or Windows.

The Windows wheel smoke tests, CPU and CUDA, check that the delegate
registers. They then install the openvino extra and the quantizer the
partitioner needs, and in a new process require that the delegate loads the
runtime and that a model it runs matches eager. They also check that a bare
name does not load the runtime from the current folder. The shared model check
now asserts that the program is really delegated, and loads it from memory,
because Windows does not delete a file the loaded program keeps open. The
shared library check requires the delegate on Linux and Windows wheels. The
docs, the README and the PyPI description list Windows, and the backend README
explains where Windows looks for the runtime's dependencies.

Test plan:

- Built the loader for Windows with a MinGW cross compiler and ran it under
  Wine with the OpenVINO 2025.4.1 Windows DLLs. A full path, a path with
  forward slashes and a relative path load the runtime and find the CPU
  device. A bare name ignores openvino_c.dll in the current folder, which the
  previous loader loaded. Unloading the delegate's DLL leaves the runtime
  loaded, where the previous loader freed it.
- On Linux, built the delegate against the wheel's C++ SDK and ran the new
  OpenVINO check with OpenVINO 2025.4.1 and nncf 3.1.0: the runtime loads and
  the model matches eager exactly. The check fails when the runtime cannot
  load, when nncf is missing, or when nothing is delegated.
- Ran black, usort, flake8, clang-format and cmake-format on the changed files.
- On native Windows, the new checks run in this pull request's Windows wheel
  jobs.

Author: shoumikhin

.ci/scripts/wheel/test_base.py

@@ -8,10 +8,8 @@
 import os
 import subprocess
 import sys
-import tempfile
 from dataclasses import dataclass
 from functools import cache
-from pathlib import Path
 from typing import List
 
 
@@ -43,7 +41,7 @@ class ModelTest:
     backend: Backend
 
 
-def test_a_model_runs_through_the_openvino_delegate() -> None:
+def test_a_model_runs_through_the_openvino_delegate(required: bool = False) -> None:
     """Export a model to the OpenVINO delegate and run it, comparing against eager.
 
     The checks elsewhere prove the delegate library ships, owns its symbols and registers
@@ -53,7 +51,8 @@ def test_a_model_runs_through_the_openvino_delegate() -> None:
     The OpenVINO runtime is not part of the wheel. It comes from the openvino extra, so a
     default install cannot run this and says so rather than passing quietly. Where the runtime
     is present the export and the comparison are the point of the check, and a failure there
-    is a real one.
+    is a real one. A caller that installed the export stack passes required=True, so a missing
+    piece fails instead of skipping.
     """
     import torch
     from executorch.exir import to_edge_transform_and_lower
@@ -67,6 +66,8 @@ def test_a_model_runs_through_the_openvino_delegate() -> None:
     try:
         from executorch.backends.openvino.partitioner import OpenvinoPartitioner
     except ImportError as error:
+        if required:
+            raise
         print(
             f"SKIP: the OpenVINO export stack is not installed, so the delegate cannot "
             f"execute here ({error}). The wheel ships the adapter only; the runtime and the "
@@ -88,13 +89,16 @@ def forward(self, x, y):
         partitioner=[OpenvinoPartitioner([CompileSpec("device", b"CPU")])],
     )
     program = lowered.to_executorch()
+    delegates = [d.id for d in program.executorch_program.execution_plan[0].delegates]
+    assert "OpenvinoBackend" in delegates, (
+        f"the partitioner did not hand the model to OpenVINO (delegates: {delegates}), so "
+        f"running it would test the portable kernels instead of the delegate"
+    )
 
-    with tempfile.TemporaryDirectory() as work_dir:
-        pte = Path(work_dir) / "add_openvino.pte"
-        pte.write_bytes(program.buffer)
-
-        method = Runtime.get().load_program(pte).load_method("forward")
-        actual = method.execute(list(example))[0]
+    # From memory rather than a temporary file, which the loaded program keeps open and
+    # Windows then refuses to delete.
+    method = Runtime.get().load_program(program.buffer).load_method("forward")
+    actual = method.execute(list(example))[0]
 
     difference = (actual - eager).abs().max().item()
     assert difference == 0.0, (

.ci/scripts/wheel/test_cuda_windows.py

@@ -640,6 +640,15 @@ def export(destination: str) -> None:
     with tempfile.TemporaryDirectory() as work_dir:
         test_cpp_sdk.run_tests(Path(work_dir))
     import test_base
+    from executorch.extension.pybindings.portable_lib import (
+        _get_registered_backend_names,
+    )
+
+    registered = _get_registered_backend_names()
+    assert (
+        "OpenvinoBackend" in registered
+    ), f"OpenvinoBackend not found in registered backends: {registered}"
+    print("✓ OpenvinoBackend is registered")
 
     test_base.test_cmsis_nn_install()
     # The same model run the CPU row ends with, through the Python bindings.
@@ -650,3 +659,6 @@ def export(destination: str) -> None:
     test_windows.run_tests(
         model_tests=[ModelTest(model=Model.Mv3, backend=Backend.Xnnpack)]
     )
+
+    # Last, because it installs packages the wheel does not declare.
+    test_windows.test_the_openvino_delegate_runs()

.ci/scripts/wheel/test_shared_libraries.py

@@ -149,7 +149,11 @@
 # The delegate's own entry points. A second definer means the delegate is compiled
 # into the Python extension as well, which would register it twice in one process.
 _QNN_BACKEND_SYMBOLS = ("QnnExecuTorchBackendRegister",)
-_OPENVINO_BACKEND_SYMBOLS = ("executorch::backends::openvino::OpenvinoBackend",)
+# A member rather than the bare class name, because the PE reader matches MSVC decorated
+# names, which spell a member as ?is_available@OpenvinoBackend@... and never the class alone.
+_OPENVINO_BACKEND_SYMBOLS = (
+    "executorch::backends::openvino::OpenvinoBackend::is_available",
+)
 
 _BUNDLED_XNNPACK_SYMBOLS = ("xnn_create_runtime_v4",)
 
@@ -1097,17 +1101,18 @@ def _wheel_cuda_train() -> str:
 # wheel is inspected, and a row cannot call that at import time.
 _REQUIRED_ON_A_CUDA_WHEEL = "cuda-wheel-only"
 
-# Marker for a row whose owner every Linux wheel carries and no macOS wheel does.
-_REQUIRED_ON_LINUX = "linux-only"
+# Marker for a row whose owner every Linux and Windows wheel carries and no macOS wheel
+# does.
+_REQUIRED_ON_LINUX_AND_WINDOWS = "linux-and-windows-only"
 # Marker for a row whose owner every macOS wheel carries and no other wheel does.
 # The Core ML delegate is built on every macOS wheel, so its presence is decidable
 # from the installed wheel and a macOS wheel that dropped it should fail rather than
 # quietly regress to folding it into the Python extension.
 _REQUIRED_ON_MACOS = "macos-only"
-# Narrower than the marker above on purpose. The Qualcomm SDK the delegate builds
-# against is published for Linux x86_64 only, so the aarch64 Linux wheels ship no such
-# library and demanding it there would fail a wheel that is correct. Reusing the
-# Linux-wide marker would do exactly that.
+# Narrower than the Linux and Windows marker on purpose. The Qualcomm SDK the delegate
+# builds against is published for Linux x86_64 only, so the aarch64 Linux wheels ship no
+# such library and demanding it there would fail a wheel that is correct. Reusing that
+# marker would do exactly that.
 _REQUIRED_ON_LINUX_X86 = "linux-x86-only"
 # Marker for a row whose owner every aarch64 wheel carries, macOS and Linux alike. The
 # TorchAO kernels are selected by the processor rather than the operating system, so
@@ -1120,8 +1125,8 @@ def _resolve_required(required):
     """Turn a row's requirement marker into the answer for the installed wheel."""
     if required == _REQUIRED_ON_A_CUDA_WHEEL:
         return bool(_wheel_cuda_train())
-    if required == _REQUIRED_ON_LINUX:
-        return sys.platform == "linux"
+    if required == _REQUIRED_ON_LINUX_AND_WINDOWS:
+        return sys.platform in ("linux", "win32")
     if required == _REQUIRED_ON_MACOS:
         return sys.platform == "darwin"
     if required == _REQUIRED_ON_LINUX_X86:
@@ -1279,15 +1284,15 @@ def _resolve_required(required):
         _library_file_name("libexecutorch_backend_mlx"),
         False,
     ),
-    # Required on Linux, where packaging turns the backend on for every non-minimal
-    # build. A fixed False passed a wheel that had compiled the delegate back into the
-    # extension: no library ships, so the row skips, and one definer inside the
+    # Required on Linux and Windows, where packaging turns the backend on for every
+    # non-minimal build. A fixed False passed a wheel that had compiled the delegate back
+    # into the extension: no library ships, so the row skips, and one definer inside the
     # extension is exactly the monolithic layout a definer count cannot distinguish.
     (
         "OpenVINO delegate",
         _OPENVINO_BACKEND_SYMBOLS,
         _library_file_name("libexecutorch_backend_openvino"),
-        _REQUIRED_ON_LINUX,
+        _REQUIRED_ON_LINUX_AND_WINDOWS,
     ),
     # Required on Linux x86_64 only, where the Qualcomm SDK the delegate builds against
     # is published. A fixed False would pass a wheel that compiled the delegate back

.ci/scripts/wheel/test_windows.py

@@ -6,7 +6,12 @@
 # This source code is licensed under the BSD-style license found in the
 # LICENSE file in the root directory of this source tree.
 
+import importlib.util
+import os
 import platform
+import shutil
+import subprocess
+import sys
 import tempfile
 from pathlib import Path
 from typing import List
@@ -24,6 +29,7 @@
 from executorch.exir import EdgeCompileConfig, to_edge_transform_and_lower
 from executorch.extension.pybindings.portable_lib import (
     _get_registered_backend_names,
+    _is_available,
     _load_for_executorch_from_buffer,
 )
 from test_base import ModelTest
@@ -71,14 +77,96 @@ def run_tests(model_tests: List[ModelTest]) -> None:
             raise RuntimeError(f"Unsupported backend {model_test.backend}.")
 
 
+def test_the_openvino_delegate_runs() -> None:
+    """Install the OpenVINO runtime the way a user does, then run a model through it.
+
+    The wheel ships the delegate but not the runtime, and registering the delegate does not
+    load the runtime, so nothing else here proves the delegate can open openvino_c.dll and
+    compute. The check runs in a new process because the package looks for the runtime when
+    it is imported, which in this process happened before the install.
+    """
+    # The same bound as the python_version marker on the openvino extra.
+    if sys.version_info >= (3, 15):
+        print(
+            "SKIP: OpenVINO publishes no build for this Python, so the openvino extra "
+            "installs no runtime and the delegate cannot load here."
+        )
+        return
+    requirements = (
+        Path(test_base._repository_root_dir()) / "backends/openvino/requirements.txt"
+    )
+    subprocess.run(
+        [
+            sys.executable,
+            "-m",
+            "pip",
+            "install",
+            "executorch[openvino]",
+            "-r",
+            str(requirements),
+        ],
+        check=True,
+    )
+    subprocess.run(
+        [sys.executable, str(Path(__file__).resolve()), "--openvino"], check=True
+    )
+    importlib.invalidate_caches()
+    openvino = importlib.util.find_spec("openvino")
+    test_a_bare_name_does_not_load_from_the_current_folder(
+        Path(openvino.submodule_search_locations[0]) / "libs"
+    )
+
+
+def test_a_bare_name_does_not_load_from_the_current_folder(libs: Path) -> None:
+    """Run from a folder holding the real runtime and ask for it by bare name.
+
+    The standard Windows search order includes the current folder, where a planted
+    openvino_c.dll would run, so the delegate has to refuse the copy found there.
+    """
+    with tempfile.TemporaryDirectory() as work_dir:
+        for library in libs.glob("*.dll"):
+            shutil.copy2(library, work_dir)
+        result = subprocess.run(
+            [
+                sys.executable,
+                "-c",
+                "from executorch.extension.pybindings.portable_lib import _is_available; "
+                "print(_is_available('OpenvinoBackend'))",
+            ],
+            cwd=work_dir,
+            env={**os.environ, "OPENVINO_LIB_PATH": "openvino_c.dll"},
+            capture_output=True,
+            text=True,
+            check=True,
+        )
+    assert result.stdout.split()[-1] == "False", (
+        "the delegate loaded openvino_c.dll by its bare name from the current folder, "
+        f"where a planted copy would run:\n{result.stdout}{result.stderr}"
+    )
+    print("✓ a bare openvino_c.dll name does not load from the current folder")
+
+
 if __name__ == "__main__":
+    if sys.argv[1:] == ["--openvino"]:
+        assert _is_available(
+            "OpenvinoBackend"
+        ), "OpenvinoBackend is registered but cannot load the OpenVINO runtime"
+        print("✓ OpenvinoBackend loads the OpenVINO runtime")
+        test_base.test_a_model_runs_through_the_openvino_delegate(required=True)
+        sys.exit(0)
+
     # Before anything else, because this is the check that fails the way a user
     # fails: with only the dependencies the wheel declares.
     with tempfile.TemporaryDirectory() as work_dir:
         test_clean_install.run_tests(Path(work_dir))
 
     if platform.system() == "Windows":
         registered = _get_registered_backend_names()
+        assert (
+            "OpenvinoBackend" in registered
+        ), f"OpenvinoBackend not found in registered backends: {registered}"
+        print("✓ OpenvinoBackend is registered")
+
         # Vulkan backend is optional: only present when the wheel was built with
         # EXECUTORCH_BUILD_VULKAN=1 and the Vulkan SDK (glslc) was available.
         if "VulkanBackend" in registered:
@@ -108,3 +196,7 @@ def run_tests(model_tests: List[ModelTest]) -> None:
             ),
         ]
     )
+
+    # Last, because it installs packages the wheel does not declare.
+    if platform.system() == "Windows":
+        test_the_openvino_delegate_runs()

README-wheel.md

@@ -34,7 +34,7 @@ to run ExecuTorch `.pte` files, with some restrictions:
 * \[macOS only] [Core ML](docs/source/backends/coreml/coreml-overview.md) and MLX backends are
   also linked into the prebuilt module.
 * \[Linux x86_64] [QNN](docs/source/backends-qualcomm.md) backend is linked into the prebuilt module.
-* \[Linux] [OpenVINO](docs/source/build-run-openvino.md) backend is also linked into the
+* \[Linux and Windows] [OpenVINO](docs/source/build-run-openvino.md) backend is also linked into the
   prebuilt module. OpenVINO requires the runtime to be installed separately:
   `pip install executorch[openvino]`
 

README.md

@@ -200,7 +200,7 @@ is authoritative. See also the [Desktop guide](desktop/README.md).
 | iOS / iPadOS | [XNNPACK](https://docs.pytorch.org/executorch/main/backends/xnnpack/xnnpack-overview.html) CPU; [Core ML](https://docs.pytorch.org/executorch/main/backends/coreml/coreml-overview.html); [MLX](https://docs.pytorch.org/executorch/main/backends/mlx/mlx-overview.html) on physical devices *(experimental)* |
 | macOS | [XNNPACK](https://docs.pytorch.org/executorch/main/backends/xnnpack/xnnpack-overview.html); [Core ML](https://docs.pytorch.org/executorch/main/backends/coreml/coreml-overview.html); experimental [MLX](https://docs.pytorch.org/executorch/main/backends/mlx/mlx-overview.html), [Metal/AOTInductor](backends/apple/metal/README.md), and [WebGPU](https://docs.pytorch.org/executorch/main/backends/webgpu/webgpu-overview.html) paths |
 | Linux | [XNNPACK](https://docs.pytorch.org/executorch/main/backends/xnnpack/xnnpack-overview.html); [OpenVINO](https://docs.pytorch.org/executorch/main/build-run-openvino.html); experimental [CUDA/AOTInductor](https://docs.pytorch.org/executorch/main/backends/cuda/cuda-overview.html), [Vulkan](https://docs.pytorch.org/executorch/main/backends/vulkan/vulkan-overview.html), and [WebGPU](https://docs.pytorch.org/executorch/main/backends/webgpu/webgpu-overview.html) desktop paths |
-| Windows | [XNNPACK](https://docs.pytorch.org/executorch/main/backends/xnnpack/xnnpack-overview.html); experimental [CUDA/AOTInductor](https://docs.pytorch.org/executorch/main/backends/cuda/cuda-overview.html) and [Vulkan](https://docs.pytorch.org/executorch/main/backends/vulkan/vulkan-overview.html) desktop paths |
+| Windows | [XNNPACK](https://docs.pytorch.org/executorch/main/backends/xnnpack/xnnpack-overview.html); [OpenVINO](https://docs.pytorch.org/executorch/main/build-run-openvino.html); experimental [CUDA/AOTInductor](https://docs.pytorch.org/executorch/main/backends/cuda/cuda-overview.html) and [Vulkan](https://docs.pytorch.org/executorch/main/backends/vulkan/vulkan-overview.html) desktop paths |
 | Browser / WebAssembly | [Portable WebAssembly runtime](extension/wasm/README.md) and [WebGPU](https://docs.pytorch.org/executorch/main/backends/webgpu/webgpu-overview.html) *(both experimental)* |
 | Embedded / MCU | [Arm Cortex-M with CMSIS-NN](https://docs.pytorch.org/executorch/main/backends/arm-cortex-m/arm-cortex-m-overview.html) *(beta)*; [Arm Ethos-U](https://docs.pytorch.org/executorch/main/backends/arm-ethos-u/arm-ethos-u-overview.html); [NXP eIQ Neutron](https://docs.pytorch.org/executorch/main/backends/nxp/nxp-overview.html); [Cadence DSP](https://docs.pytorch.org/executorch/main/backends-cadence.html); [Zephyr](zephyr/README.md) and [Arduino](examples/arduino/README.md) integrations |
 

backends/openvino/CMakeLists.txt

@@ -28,8 +28,9 @@ set(COMMON_INCLUDE_DIRS ${EXECUTORCH_ROOT}/..)
 # Include utility CMake scripts from ExecuteTorch
 include(${EXECUTORCH_ROOT}/tools/cmake/Utils.cmake)
 
-# The backend resolves OpenVINO C API symbols via dlopen/dlsym at runtime, so
-# there is no build-time dependency on the OpenVINO SDK.
+# The backend resolves OpenVINO C API symbols at runtime (dlopen on Linux,
+# LoadLibrary on Windows), so there is no build-time dependency on the OpenVINO
+# SDK.
 
 # Shared when the wheel build asks for it, static otherwise.
 if(EXECUTORCH_BUILD_SHARED)
@@ -39,8 +40,13 @@ else()
 endif()
 add_library(openvino_backend ${_openvino_backend_library_type})
 
-# Enable exceptions and RTTI for OpenVINO backend
-target_compile_options(openvino_backend PRIVATE -frtti -fexceptions)
+# Enable exceptions and RTTI for OpenVINO backend. MSVC is also set for
+# clang-cl, which takes the cl spellings.
+if(MSVC)
+  target_compile_options(openvino_backend PRIVATE /EHsc /GR)
+else()
+  target_compile_options(openvino_backend PRIVATE -frtti -fexceptions)
+endif()
 
 # Add source files for OpenVINO backend
 target_sources(

backends/openvino/README.md

@@ -14,20 +14,28 @@ For more information on the supported hardware, please refer to [OpenVINO System
 
 ## Quick Start (pip wheel)
 
-On Linux, the OpenVINO backend is included in the ExecuTorch pip wheel. Install the OpenVINO runtime to activate it:
+On Linux and Windows, the OpenVINO backend is included in the ExecuTorch pip wheel. Install the OpenVINO runtime to activate it:
 
 ```bash
 pip install executorch[openvino]
 ```
 
-The backend automatically discovers the OpenVINO C library from the pip-installed package — no `LD_LIBRARY_PATH` setup is needed.
+The backend automatically discovers the OpenVINO C library from the pip-installed package, so no library search path setup is needed.
 
-If auto-discovery fails (e.g. non-standard install), you can point to the library explicitly:
+If auto-discovery fails (e.g. non-standard install), set `OPENVINO_LIB_PATH` to the full path of the OpenVINO C library: a `libopenvino_c.so` file on Linux, or `openvino_c.dll` on Windows. For the pip package on Linux:
 
 ```bash
-export OPENVINO_LIB_PATH=$(python3 -c "import openvino, os; print(os.path.join(os.path.dirname(openvino.__file__), 'libs', 'libopenvino_c.so'))")
+export OPENVINO_LIB_PATH="$(python3 -c 'import glob, openvino, os; print(sorted(glob.glob(os.path.join(os.path.dirname(openvino.__file__), "libs", "libopenvino_c.so*")))[0])')"
 ```
 
+On Windows, in PowerShell:
+
+```powershell
+$env:OPENVINO_LIB_PATH = python -c "import openvino, os; print(os.path.join(os.path.dirname(openvino.__file__), 'libs', 'openvino_c.dll'))"
+```
+
+On Windows, `PATH` and the current folder are not searched for the library or for its dependencies, such as `tbb12.dll`. Those must sit beside `openvino_c.dll`, as they do in the pip package, or in a folder added with `os.add_dll_directory` (`AddDllDirectory` in C++). Importing the `openvino` Python package adds its folders this way, including the ones Intel's `setupvars.bat` lists.
+
 Verify the backend is available:
 
 ```python

backends/openvino/runtime/OpenvinoApi.h

@@ -8,7 +8,6 @@
 
 #pragma once
 
-#include <dlfcn.h>
 #include <cstddef>
 #include <cstdint>
 #include <memory>
@@ -96,12 +95,9 @@ using ov_shape_create_fn =
     ov_status_e (*)(int64_t, const int64_t*, ov_shape_t*);
 using ov_shape_free_fn = ov_status_e (*)(ov_shape_t*);
 
+// Defined in OpenvinoBackend.cpp, where the platform's loader API is included.
 struct DlCloser {
-  void operator()(void* handle) {
-    if (handle) {
-      dlclose(handle);
-    }
-  }
+  void operator()(void* handle);
 };
 using DlHandle = std::unique_ptr<void, DlCloser>;