Skip to content

Commit 7a6c1c4

Browse files
authored
Stop the Qualcomm backend from doing setup work at import time (pytorch#22270)
## Summary `import executorch.backends.qualcomm` did real work as a side effect of the import itself. On Linux x86_64 it downloaded the Qualcomm SDK and a copy of libc++ over the network, could re-execute the interpreter under a staged loader, rewrote `QNN_SDK_ROOT` and `LD_LIBRARY_PATH`, and raised when the machine had no route to the download: ``` [QNN] Downloading libc++ (LLVM 14.0.0)... [QNN] Failed to stage/load libc++: <urlopen error ...> RuntimeError: Failed to set up QNN SDK. ``` On every platform it also read the CPU vendor and, on an AMD host, set `torch.backends.mkldnn.enabled = False`. That is a global PyTorch setting, so importing this package changed how *unrelated* models ran in the same interpreter. ## Why none of that has to happen at import The native adaptor does not link the SDK. It resolves QNN symbols with `dlopen` when a model is actually compiled, and says so plainly if they are missing: ``` QnnImplementation::Load Cannot load symbol QnnInterface_getProviders ``` So loading the package needs nothing, and the setup belongs on the paths that compile. ## The change Two functions hold what used to run at import: - `setup_qnn_sdk()` keeps the previous behaviour exactly, including honouring a preinstalled `QNN_SDK_ROOT`, skipping on a non-Linux-x86 host, and short-circuiting during a wheel build. It is idempotent, so every entry point can simply call it. - `disable_mkldnn_on_amd()` holds the MKLDNN change. Both are called by the four modules that load the adaptor. One of those is `builders/node_visitor.py`, which the 118 op builders all import, so a single call site covers them. ## Test plan Added `backends/qualcomm/tests/test_import_side_effects.py`, 8 tests. **All 8 fail against the previous `__init__`.** I checked that the two behavioural ones fail for the right reason rather than on a missing attribute: with a stubbed AMD vendor string, the old code flips `mkldnn.enabled` from `True` to `False` purely on import. Ran against the installed wheel on macOS arm64: | check | result | | --- | --- | | bare import, stubbed AMD host | `mkldnn` True before and after | | `setup_qnn_sdk()` | no-op off Linux x86, idempotent on repeat | | `QNN_SDK_ROOT` set | honoured, installer not called | | `EXECUTORCH_BUILDING_WHEEL=1` | short-circuits | | install failure | `RuntimeError` naming both ways out | | `disable_mkldnn_on_amd()` | `AuthenticAMD` to False, `GenuineIntel` stays True | | `cpuinfo` absent | same `ImportError` guidance as before | I do not have a Qualcomm device, so compiling a model for QNN end to end is not covered here and would be worth a look from someone who does. cc @cbilgin @psiddh
1 parent 1d280bb commit 7a6c1c4

9 files changed

Lines changed: 316 additions & 41 deletions

File tree

‎.github/workflows/pull.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1059,6 +1059,10 @@ jobs:
10591059
# Run QNN pass unit tests
10601060
pytest -xvs backends/qualcomm/tests/test_passes.py
10611061
1062+
# And the checks that importing the backend has no side effects. These need no SDK and
1063+
# no device, but they do need the package importable, which this job already arranges.
1064+
pytest -xvs backends/qualcomm/tests/test_import_side_effects.py
1065+
10621066
test-qnn-delegate-linux:
10631067
name: test-qnn-delegate-linux
10641068
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main

‎backends/qualcomm/__init__.py‎

Lines changed: 90 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,99 @@
11
import os
2+
import threading
23

3-
import torch
4+
# The Qualcomm SDK setup below is deferred rather than run here, so that importing this
5+
# package has no side effects. It used to run at import time, which meant that merely
6+
# importing the package downloaded the SDK and libc++ over the network, re-executed the
7+
# interpreter under a staged loader, rewrote QNN_SDK_ROOT and LD_LIBRARY_PATH, and raised
8+
# when a machine had no network. It also disabled PyTorch's MKLDNN backend for the whole
9+
# process on any AMD host, which affects every other model in that interpreter.
10+
#
11+
# Nothing about loading this package needs any of that. The native adaptor does not link
12+
# the SDK; it resolves QNN symbols with dlopen when a model is actually compiled, and says
13+
# so plainly when they are missing. So setup happens on the first call that needs it.
14+
from .scripts.download_qnn_sdk import ( # noqa: F401
15+
install_qnn_sdk,
16+
is_linux_x86,
17+
QNN_ZIP_URL,
18+
)
419

5-
from .scripts.download_qnn_sdk import install_qnn_sdk, is_linux_x86, QNN_ZIP_URL
20+
_sdk_ready = False
21+
# Guards the flag above. Python's import lock does not, because the setup is now called from
22+
# several modules rather than from one package __init__, and two threads importing two of them
23+
# concurrently would otherwise both run an installer that downloads and rewrites the
24+
# environment.
25+
_sdk_lock = threading.Lock()
626

7-
try:
8-
import cpuinfo
927

10-
info = cpuinfo.get_cpu_info()
11-
vendor = info.get("vendor_id_raw", "").lower()
12-
if "amd" in vendor:
13-
torch.backends.mkldnn.enabled = False
14-
except ImportError:
15-
raise ImportError("Please install the cpuinfo with pip install py-cpuinfo.")
28+
def setup_qnn_sdk() -> None:
29+
"""Make the Qualcomm SDK usable in this process, once.
30+
31+
Safe to call repeatedly and from more than one thread: the work happens on the first call and
32+
later calls return immediately. Called by the code paths that need the SDK, so a caller does
33+
not have to.
34+
"""
35+
if _sdk_ready:
36+
return
37+
with _sdk_lock:
38+
# Another thread may have finished while this one waited.
39+
if _sdk_ready:
40+
return
41+
_setup_qnn_sdk_locked()
42+
43+
44+
def _setup_qnn_sdk_locked() -> None:
45+
global _sdk_ready
1646

47+
# The wheel build imports this package to collect its files, and has no use for an SDK.
48+
if os.getenv("EXECUTORCH_BUILDING_WHEEL", "0").lower() in ("1", "true", "yes"):
49+
_sdk_ready = True
50+
return
1751

18-
env_flag = os.getenv("EXECUTORCH_BUILDING_WHEEL", "0").lower()
19-
# If users have preinstalled QNN_SDK_ROOT, we will use it.
20-
qnn_sdk_root_flag = os.getenv("QNN_SDK_ROOT", None)
52+
# A preinstalled SDK is used as it is.
53+
qnn_sdk_root = os.getenv("QNN_SDK_ROOT", None)
54+
if qnn_sdk_root:
55+
print(f"[QNN] Using QNN SDK at {qnn_sdk_root} (from QNN_SDK_ROOT)", flush=True)
56+
_sdk_ready = True
57+
return
2158

22-
if env_flag not in ("1", "true", "yes"):
23-
if qnn_sdk_root_flag:
24-
print(
25-
f"[QNN] Using QNN SDK at {qnn_sdk_root_flag} (from QNN_SDK_ROOT)",
26-
flush=True,
59+
# Downloading a prebuilt SDK is only possible for the platform it is published for.
60+
if not is_linux_x86():
61+
_sdk_ready = True
62+
return
63+
64+
if not install_qnn_sdk():
65+
raise RuntimeError(
66+
"Failed to set up QNN SDK.\n\n"
67+
"To resolve, try one of:\n"
68+
" 1. Download the SDK manually from:\n"
69+
f" {QNN_ZIP_URL}\n"
70+
" Or go to step 2 if QNN SDK already exists.\n"
71+
" 2. Set QNN_SDK_ROOT to an existing SDK installation:\n"
72+
" export QNN_SDK_ROOT=/path/to/qualcomm/sdk\n"
73+
" export LD_LIBRARY_PATH="
74+
"$QNN_SDK_ROOT/lib/x86_64-linux-clang/:$LD_LIBRARY_PATH"
2775
)
28-
elif is_linux_x86():
29-
ok = install_qnn_sdk()
30-
if not ok:
31-
raise RuntimeError(
32-
"Failed to set up QNN SDK.\n\n"
33-
"To resolve, try one of:\n"
34-
" 1. Download the SDK manually from:\n"
35-
f" {QNN_ZIP_URL}\n"
36-
" Or go to step 2 if QNN SDK already exists.\n"
37-
" 2. Set QNN_SDK_ROOT to an existing SDK installation:\n"
38-
" export QNN_SDK_ROOT=/path/to/qualcomm/sdk\n"
39-
" export LD_LIBRARY_PATH="
40-
"$QNN_SDK_ROOT/lib/x86_64-linux-clang/:$LD_LIBRARY_PATH"
41-
)
76+
77+
_sdk_ready = True
78+
79+
80+
def disable_mkldnn_on_amd() -> None:
81+
"""Turn off PyTorch's MKLDNN backend on an AMD host.
82+
83+
MKLDNN crashes on some AMD hosts, which is why this exists. The original comment described
84+
it as producing wrong results; what was measured is a core dump, from a plain convolution
85+
with nothing from this backend involved.
86+
87+
This changes a global PyTorch setting, so it is applied by the QNN compile paths rather than
88+
at import, where it would also change how unrelated models run in the same interpreter.
89+
"""
90+
import torch
91+
92+
try:
93+
import cpuinfo
94+
except ImportError:
95+
raise ImportError("Please install the cpuinfo with pip install py-cpuinfo.")
96+
97+
vendor = cpuinfo.get_cpu_info().get("vendor_id_raw", "") or ""
98+
if "amd" in vendor.lower():
99+
torch.backends.mkldnn.enabled = False

‎backends/qualcomm/builders/node_visitor.py‎

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,22 @@
44
# This source code is licensed under the BSD-style license found in the
55
# LICENSE file in the root directory of this source tree.
66

7+
# The SDK has to be usable before a model is compiled, and every builder in this package
8+
# imports this module, so this is where it is arranged. Not in the package's __init__, because
9+
# that made simply importing the package fetch an SDK over the network and change a global
10+
# PyTorch setting.
11+
#
12+
# The adaptor imported below does not itself need the SDK to load: it links no QNN library and
13+
# resolves those symbols with dlopen when a backend is started. Placing the call here is about
14+
# covering the compile paths, not about ordering against that import.
15+
from executorch.backends.qualcomm import disable_mkldnn_on_amd, setup_qnn_sdk
16+
17+
setup_qnn_sdk()
18+
disable_mkldnn_on_amd()
19+
720
from typing import Any, Dict, Optional, Tuple
821

922
import executorch.backends.qualcomm.python.PyQnnManagerAdaptor as PyQnnManager
10-
1123
import numpy as np
1224
import torch
1325
from executorch.backends.qualcomm.utils.constants import (
@@ -33,7 +45,6 @@
3345
QCOM_ZERO_POINT,
3446
QCOM_ZERO_POINTS,
3547
)
36-
3748
from executorch.exir.dialects._ops import ops as exir_ops
3849

3950
from .utils import (
@@ -46,7 +57,6 @@
4657
is_parameter,
4758
)
4859

49-
5060
QNN_QUANT_TYPE_MAP = {
5161
torch.int8: PyQnnManager.Qnn_DataType_t.QNN_DATATYPE_SFIXED_POINT_8,
5262
torch.int16: PyQnnManager.Qnn_DataType_t.QNN_DATATYPE_SFIXED_POINT_16,

‎backends/qualcomm/debugger/utils.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,17 @@
66
import tempfile
77
from typing import Sequence, Tuple
88

9+
# The SDK has to be usable before a model is compiled. See node_visitor.py for why this is
10+
# here rather than in the package's __init__.
11+
from executorch.backends.qualcomm import setup_qnn_sdk
12+
13+
setup_qnn_sdk()
14+
915
import executorch.backends.qualcomm.python.PyQnnManagerAdaptor as PyQnnManager
1016
import pandas as pd
1117
import torch
1218
from executorch.backends.qualcomm.serialization.qc_schema import QcomChipset
1319
from executorch.backends.qualcomm.utils.utils import dump_context_from_pte
14-
1520
from graphviz import Digraph
1621

1722

‎backends/qualcomm/quantizer/validators.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,13 @@
88
from dataclasses import dataclass, field
99
from typing import cast, Dict, List, Optional, Set, Tuple
1010

11+
# The SDK has to be usable before a model is compiled. See node_visitor.py for why this is
12+
# here rather than in the package's __init__. Called from this module too because it does not
13+
# import node_visitor before the adaptor, so it cannot leave the call to that module.
14+
from executorch.backends.qualcomm import setup_qnn_sdk
15+
16+
setup_qnn_sdk()
17+
1118
import executorch.backends.qualcomm.python.PyQnnManagerAdaptor as PyQnnManager
1219
import torch
1320
from executorch.backends.qualcomm.builders.node_visitor import (
Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
# Copyright (c) Qualcomm Innovation Center, Inc.
2+
# All rights reserved
3+
#
4+
# This source code is licensed under the BSD-style license found in the
5+
# LICENSE file in the root directory of this source tree.
6+
7+
"""Tests that importing the Qualcomm backend package has no side effects.
8+
9+
The package's __init__ used to do real work at import time. Merely importing it fetched the
10+
SDK and libc++ over the network, could re-execute the interpreter under a staged loader,
11+
rewrote QNN_SDK_ROOT and LD_LIBRARY_PATH, raised on a machine with no network, and disabled
12+
PyTorch's MKLDNN backend for the whole process on any AMD host. None of that is needed to
13+
load the package: the native adaptor resolves QNN symbols with dlopen when a model is
14+
compiled, so setup belongs on the paths that compile.
15+
16+
These are unit tests because the behaviour under test is what happens during import, which
17+
is observable without a Qualcomm SDK or a Qualcomm device.
18+
"""
19+
20+
import ast
21+
import importlib
22+
import sys
23+
import types
24+
from pathlib import Path
25+
26+
import executorch.backends.qualcomm as qnn
27+
import pytest
28+
import torch
29+
30+
31+
@pytest.fixture
32+
def fake_cpuinfo(monkeypatch):
33+
"""Replaces cpuinfo so a vendor can be chosen without an AMD host."""
34+
35+
def install(vendor):
36+
module = types.ModuleType("cpuinfo")
37+
module.get_cpu_info = lambda: {"vendor_id_raw": vendor}
38+
monkeypatch.setitem(sys.modules, "cpuinfo", module)
39+
40+
return install
41+
42+
43+
def test_importing_the_package_leaves_mkldnn_alone(monkeypatch, fake_cpuinfo):
44+
"""Import must not change a global PyTorch setting, even on an AMD host.
45+
46+
The setting affects every model in the interpreter, not just a Qualcomm one, so an import
47+
that flips it changes how unrelated code runs.
48+
49+
A reload is used deliberately: it re-executes every module level statement, which is exactly
50+
what an import does, and the stubbed cpuinfo is read inside those statements rather than
51+
bound at import, so unlike a stubbed installer it survives.
52+
"""
53+
fake_cpuinfo("AuthenticAMD")
54+
monkeypatch.setattr(torch.backends.mkldnn, "enabled", True)
55+
56+
importlib.reload(qnn)
57+
58+
assert torch.backends.mkldnn.enabled
59+
60+
61+
def test_importing_the_package_does_not_set_up_the_sdk():
62+
"""Import must not run the installer, which downloads and rewrites the environment.
63+
64+
Asserted against the module source rather than by importing with a stub in place. A reload
65+
re-executes the module, which rebinds the real installer and discards any stub, so a stubbed
66+
reload can only ever observe an empty call list and would pass even if the call came back.
67+
What actually matters is that no module level statement calls it.
68+
"""
69+
module = ast.parse(Path(qnn.__file__).read_text())
70+
called_at_module_level = {
71+
node.func.id
72+
for statement in module.body
73+
if isinstance(statement, ast.Expr)
74+
for node in ast.walk(statement)
75+
if isinstance(node, ast.Call) and isinstance(node.func, ast.Name)
76+
}
77+
78+
assert "install_qnn_sdk" not in called_at_module_level
79+
assert "setup_qnn_sdk" not in called_at_module_level
80+
# A guard on the two above, which would pass just as happily if the parse found nothing.
81+
assert any(
82+
isinstance(statement, ast.FunctionDef) and statement.name == "setup_qnn_sdk"
83+
for statement in module.body
84+
)
85+
86+
87+
def test_setup_is_idempotent(monkeypatch):
88+
"""The compile paths each call it, so only the first call may do the work."""
89+
calls = []
90+
monkeypatch.setattr(qnn, "install_qnn_sdk", lambda: calls.append(1) or True)
91+
monkeypatch.setattr(qnn, "is_linux_x86", lambda: True)
92+
monkeypatch.setattr(qnn, "_sdk_ready", False)
93+
monkeypatch.delenv("QNN_SDK_ROOT", raising=False)
94+
monkeypatch.delenv("EXECUTORCH_BUILDING_WHEEL", raising=False)
95+
96+
qnn.setup_qnn_sdk()
97+
qnn.setup_qnn_sdk()
98+
99+
assert len(calls) == 1
100+
101+
102+
def test_setup_honours_a_preinstalled_sdk(monkeypatch):
103+
calls = []
104+
monkeypatch.setattr(qnn, "install_qnn_sdk", lambda: calls.append(1) or True)
105+
monkeypatch.setattr(qnn, "_sdk_ready", False)
106+
monkeypatch.setenv("QNN_SDK_ROOT", "/opt/qcom/sdk")
107+
108+
qnn.setup_qnn_sdk()
109+
110+
assert not calls
111+
112+
113+
def test_setup_runs_once_under_concurrent_callers(monkeypatch):
114+
"""Several modules call it, so two threads can reach it at the same time.
115+
116+
Without a lock they both see the flag unset and both run the installer, which downloads an
117+
SDK and rewrites the environment. The import lock does not cover this, because the calls
118+
come from different modules rather than from one package __init__.
119+
"""
120+
import threading
121+
import time
122+
123+
calls = []
124+
125+
def slow_install():
126+
# Widen the window between the check and the flag being set, so an unlocked version
127+
# fails reliably rather than occasionally.
128+
time.sleep(0.2)
129+
calls.append(1)
130+
return True
131+
132+
monkeypatch.setattr(qnn, "install_qnn_sdk", slow_install)
133+
monkeypatch.setattr(qnn, "is_linux_x86", lambda: True)
134+
monkeypatch.setattr(qnn, "_sdk_ready", False)
135+
monkeypatch.delenv("QNN_SDK_ROOT", raising=False)
136+
monkeypatch.delenv("EXECUTORCH_BUILDING_WHEEL", raising=False)
137+
138+
threads = [threading.Thread(target=qnn.setup_qnn_sdk) for _ in range(8)]
139+
for thread in threads:
140+
thread.start()
141+
for thread in threads:
142+
thread.join(timeout=30)
143+
144+
# Reported before the count, so a hung thread says so rather than showing a confusing
145+
# number of installer calls.
146+
assert not any(thread.is_alive() for thread in threads)
147+
assert len(calls) == 1
148+
149+
150+
def test_setup_reports_a_failed_install(monkeypatch):
151+
"""A failure has to name the two ways out, since it cannot be resolved automatically."""
152+
monkeypatch.setattr(qnn, "install_qnn_sdk", lambda: False)
153+
monkeypatch.setattr(qnn, "is_linux_x86", lambda: True)
154+
monkeypatch.setattr(qnn, "_sdk_ready", False)
155+
monkeypatch.delenv("QNN_SDK_ROOT", raising=False)
156+
monkeypatch.delenv("EXECUTORCH_BUILDING_WHEEL", raising=False)
157+
158+
with pytest.raises(RuntimeError, match="QNN_SDK_ROOT"):
159+
qnn.setup_qnn_sdk()
160+
161+
162+
@pytest.mark.parametrize(
163+
"vendor,expected",
164+
[
165+
# The vendor string an AMD host reports. MKLDNN produces wrong results there.
166+
("AuthenticAMD", False),
167+
("GenuineIntel", True),
168+
# A host that reports nothing, such as Apple silicon.
169+
("", True),
170+
],
171+
)
172+
def test_mkldnn_is_disabled_only_on_amd(monkeypatch, fake_cpuinfo, vendor, expected):
173+
fake_cpuinfo(vendor)
174+
monkeypatch.setattr(torch.backends.mkldnn, "enabled", True)
175+
176+
qnn.disable_mkldnn_on_amd()
177+
178+
assert torch.backends.mkldnn.enabled is expected

0 commit comments

Comments
 (0)