Skip to content

Make FCB lowering and cleanup safe (#24159) #24159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
psiddh merged 1 commit into from
Oct 29, 2026
Merged

Make FCB lowering and cleanup safe (#24159) #24159

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/pull.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1365,9 +1365,9 @@ jobs:
export QNN_DELEGATE_TEST=1
export LD_LIBRARY_PATH="$QNN_SDK_ROOT/lib/x86_64-linux-clang/:$(realpath build-x86/lib/):${LD_LIBRARY_PATH:-}"

# Run operator test classes only.
# Run operator and FCB test classes only.
pytest -vs backends/qualcomm/tests/test_qnn_delegate.py \
-k "TestQNNFloatingPointOperator or TestQNNQuantizedOperator"
-k "TestQNNFloatingPointOperator or TestQNNQuantizedOperator or TestQNNFcb"

test-phi-3-mini-runner-linux:
needs: docker-image
Expand Down
75 changes: 51 additions & 24 deletions backends/qualcomm/qnn_preprocess.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,32 +158,50 @@ def preprocess(
) -> PreprocessResult:
option = generate_qnn_executorch_option(compile_specs)
obj_options = flatbuffer_to_option(option)
if obj_options.target_options is not None:
return QnnBackend.preprocess_multimethod(

@quic-boyuc quic-boyuc Oct 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @shewu-quic @winskuo-quic

Do we need to support to_backend and single method lowering?
Previously this was not included in planned scope.
Should we just delegate every call to preprocess_multimethod to reduce maintainance effort?

@shewu-quic shewu-quic Oct 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the interface contract, it looks like preprocess still needs to be supported by every backend, even though single-method lowering wasn't originally included in our planned scope.
https://github.com/pytorch/executorch/blob/main/exir/backend/backend_details.py#L132

That said, I think we can share as much implementation as possible between preprocess and preprocess_multimethod to minimize the maintenance effort.

@psiddh psiddh Oct 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we do need preprocess(), per the backend interface contract, and we have users lowering through to_backend() directly. Agree on delegating everything to preprocess_multimethod to remove the duplicate path. To keep this PR focused, I'd like to do that as a follow-up right after this lands, if that works for you. Wdyt ?

{DEFAULT_GRAPH_NAME: [edge_program]},
{DEFAULT_GRAPH_NAME: [compile_specs]},
)[DEFAULT_GRAPH_NAME][0]
qnn_manager = get_current_qnn_manager(compile_specs)
qnn_manager.InitContext([DEFAULT_GRAPH_NAME])
py_op_wrapper_list = QnnBackend._build_op_wrappers(
edge_program,
qnn_manager.IsTensorDump(),
obj_options.op_package_options.op_package_infos,
obj_options.use_mha2sha,
obj_options.backend_options.backend_type,
)
init_result = qnn_manager.InitContext([DEFAULT_GRAPH_NAME])
try:
if init_result.value != 0:
raise RuntimeError(
f"Failed to initialize QNN context: {init_result.value}"
)
py_op_wrapper_list = QnnBackend._build_op_wrappers(
edge_program,
qnn_manager.IsTensorDump(),
obj_options.op_package_options.op_package_infos,
obj_options.use_mha2sha,
obj_options.backend_options.backend_type,
)

qnn_context_binary = qnn_manager.Compile(
qnn_manager.GetGraphNames(),
[[py_op_wrapper.GetOpWrapper() for py_op_wrapper in py_op_wrapper_list]],
)
qnn_context_binary = qnn_manager.Compile(
qnn_manager.GetGraphNames(),
[
[
py_op_wrapper.GetOpWrapper()
for py_op_wrapper in py_op_wrapper_list
]
],
)

if obj_options.saver:
exit(
f"Record all QNN API calls from saver backend at: {obj_options.saver_output_dir}"
if obj_options.saver:
exit(
f"Record all QNN API calls from saver backend at: {obj_options.saver_output_dir}"
)
assert (
len(qnn_context_binary) != 0
), "Failed to generate Qnn context binary."
# For now, debug_handle_map is not used by QNN ExecuTorch
return PreprocessResult(
processed_bytes=qnn_context_binary,
debug_handle_map={},
)
assert len(qnn_context_binary) != 0, "Failed to generate Qnn context binary."
qnn_manager.DestroyContext()
# For now, debug_handle_map is not used by QNN ExecuTorch
return PreprocessResult(
processed_bytes=qnn_context_binary,
debug_handle_map={},
)
finally:
qnn_manager.DestroyContext()

@staticmethod
def _populate_delegate_mapping(
Expand All @@ -207,8 +225,12 @@ def _populate_delegate_mapping(
@staticmethod
def _get_compile_func(qnn_manager: PyQnnManager.QnnManager):
def compile_func(graph_names, op_wrapper_list):
qnn_manager.InitContext(graph_names)
init_result = qnn_manager.InitContext(graph_names)
try:
if init_result.value != 0:
raise RuntimeError(
f"Failed to initialize QNN context: {init_result.value}"
)
qnn_context_binary = qnn_manager.Compile(graph_names, op_wrapper_list)
finally:
qnn_manager.DestroyContext()
Expand All @@ -221,8 +243,13 @@ def _get_compile_func_fcb(qnn_managers: List[PyQnnManager.QnnManager]):
def compile_func(graph_names, op_wrapper_list):
with qnn_managers[0].CreateDlc() as dlc_handle:
for qnn_manager in qnn_managers:
qnn_manager.InitContext(graph_names)
init_result = qnn_manager.InitContext(graph_names)
try:
if init_result.value != 0:
raise RuntimeError(
"Failed to initialize QNN context: "
f"{init_result.value}"
)
qnn_manager.CompileToDlc(
graph_names, op_wrapper_list, dlc_handle
)
Expand Down
Loading
Loading