Skip to content

Commit 04d14b0

Browse files
authored
Qualcomm AI Engine Direct - Docs Update (pytorch#21971)
### Summary Community user has reported that some QNN docs are outdated. pytorch#21880 This PR updates the doc. ### Test plan NA
1 parent 5028bb4 commit 04d14b0

12 files changed

Lines changed: 28 additions & 28 deletions

File tree

‎.claude/skills/qualcomm/SKILL.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -83,12 +83,12 @@ LD_LIBRARY_PATH=/path/to/executorch/build-x86/lib:/path/to/qnn_sdk/lib/x86_64-li
8383
PYTHONPATH=$(dirname $EXECUTORCH_ROOT) \
8484
python backends/qualcomm/tests/test_qnn_delegate.py \
8585
TestQNNFloatingPointOperator.test_qnn_backend_abs \
86-
-H $HOST -s $DEVICE_SERIAL -m SM8850 -b build-android -a /path/to/artifacts
86+
--host $HOST --device $DEVICE_SERIAL --soc_model SM8850 --build_folder build-android -a /path/to/artifacts
8787
```
8888

8989
> **Note (build from source):** Set `PYTHONPATH` to the parent directory of the executorch repo root. Required because `executorch.examples.qualcomm` lives in the source tree and is not installed into site-packages.
9090
91-
Required flags: `-m` (SoC model), `-b` (Android build dir). Optional: `-s` (device serial), `-H` (host), `-a` (artifact dir), `-c` (compile only), `-x` (run on x86_64).
91+
Required flags: `--soc_model` (SoC model), `--build_folder` (Android build dir). Optional: `--device` (device serial), `--host` (host), `-a` (artifact dir), `--compile_only`, `--enable_x86_64`.
9292

9393
**Test classes:**
9494

‎.claude/skills/qualcomm/qnn_intermediate_debugger.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -137,12 +137,12 @@ Present the result as a fenced bash block, prefixed by a one-line note of what c
137137
>
138138
> ```bash
139139
> python -m examples.qualcomm.util_scripts.my_model_debug \
140-
> -H $HOST -s $DEVICE_SERIAL -m $SOC_MODEL -b build-android \
140+
> --host $HOST --device $DEVICE_SERIAL --soc_model $SOC_MODEL --build_folder build-android \
141141
> -d /path/to/dataset -a ./my_model_debug \
142142
> --dump_intermediate_outputs
143143
> ```
144144
145-
If the user did not give you a runnable original command in step 1 (e.g. they pasted only the script path), do **not** fabricate values for `-H` / `-s` / `-m` / `-b` / `-a` / `-d`. Stop and ask before printing — wrong device or SoC values waste a full export + on-device run.
145+
If the user did not give you a runnable original command in step 1 (e.g. they pasted only the script path), do **not** fabricate values for `--host` / `--device` / `--soc_model` / `--build_folder` / `-a` / `-d`. Stop and ask before printing — wrong device or SoC values waste a full export + on-device run.
146146
147147
After the command runs, the artifact dir will contain SVG / CSV reports — green nodes pass, red nodes fail the comparator threshold. That's the first place to look for the layer that introduces the gap.
148148

‎.wiki/backends/qnn/overview.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ There are two code paths for exporting LLMs to QNN. **Use `examples/qualcomm/oss
6666
```bash
6767
# CORRECT path for QNN LLM export
6868
python examples/qualcomm/oss_scripts/llama/llama.py \
69-
-b build-android -m SM8650 \
69+
--build_folder build-android --soc_model SM8650 \
7070
--decoder_model qwen3-0_6b \
7171
--model_mode hybrid --max_seq_len 1024 --prefill_ar_len 128 \
7272
--prompt "..." --tasks wikitext --limit 1 --compile_only
@@ -131,7 +131,7 @@ Use `--compile_only` to export, quantize, and compile a `.pte` without running o
131131

132132
```bash
133133
python examples/qualcomm/oss_scripts/llama/llama.py \
134-
-b build-android -m SM8650 --compile_only \
134+
--build_folder build-android --soc_model SM8650 --compile_only \
135135
--decoder_model qwen3-0_6b ...
136136
```
137137

‎.wiki/quantization/debugging.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Quantized LLMs (especially 1B parameter models with 4-bit weights) frequently pr
2222
```bash
2323
# Use the QNN-specific script with better quantization
2424
python examples/qualcomm/oss_scripts/llama/llama.py \
25-
--compile_only -m SM8750 \
25+
--compile_only --soc_model SM8750 \
2626
--model_mode hybrid \
2727
--decoder_model <model> \
2828
...

‎backends/qualcomm/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ PRs are always welcome to help improve the codebase in a comprehensive manner. B
120120
```bash
121121
cd $PATH_TO_EXECUTORCH
122122
# example usage of performing unit test
123-
python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNQuantizedOperator.test_qnn_backend_layer_norm -s $DEVICE_SERIAL -m SM8650 -b build-android/ -a $PATH_TO_TEST_ARTIFACTS
123+
python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNQuantizedOperator.test_qnn_backend_layer_norm --device $DEVICE_SERIAL --soc_model SM8650 --build_folder build-android/ -a $PATH_TO_TEST_ARTIFACTS
124124
```
125125
The test graph is expected to have 1 delegated node with only placeholders / output nodes being left. Check the execution report for more information.
126126

‎backends/qualcomm/debugger/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ pip install qairt-visualizer
1313
## Quick start
1414
This command launches an interactive GUI interface to visualize the `optrace` and `QHAS` results.
1515
```
16-
python -m examples.qualcomm.util_scripts.qairt_visualizer_demo -H ${host} -s {device} -m ${SOC_MODEL} -b build-android -a ${path_to_output_folder} --online_prepare
16+
python -m examples.qualcomm.util_scripts.qairt_visualizer_demo --host ${host} --device {device} --soc_model ${SOC_MODEL} --build_folder build-android -a ${path_to_output_folder} --online_prepare
1717
```
1818
- If online prepare mode is `enabled`, the following artifacts will be generated:
1919
- `model`.dlc
@@ -259,7 +259,7 @@ An Inception_V3 demo script is provided at [qnn_intermediate_debugger_demo.py](.
259259
Before running, ensure the dataset is downloaded. An example dataset can be retrieved [here](https://www.kaggle.com/datasets/ifigotin/imagenetmini-1000).
260260

261261
```bash
262-
python -m examples.qualcomm.util_scripts.qnn_intermediate_debugger_demo -b build-android -s $DEVICE_SERIAL -m $SOC_MODEL -d path/to/imagenet/val --dump_intermediate_outputs
262+
python -m examples.qualcomm.util_scripts.qnn_intermediate_debugger_demo --build_folder build-android --device $DEVICE_SERIAL --soc_model $SOC_MODEL -d path/to/imagenet/val --dump_intermediate_outputs
263263
```
264264

265265
## Limitations
@@ -290,7 +290,7 @@ The difference between the two values represents heap consumed during context ex
290290
```bash
291291
python backends/qualcomm/tests/test_qnn_delegate.py \
292292
TestQNNQuantizedUtils.test_qnn_backend_runtime_option_heap_profile \
293-
-b build-android -H ${HOST} -s ${SN} -m ${SOC_MODEL}
293+
--build_folder build-android --host ${HOST} --device ${SN} --soc_model ${SOC_MODEL}
294294
```
295295

296296
See [test_qnn_delegate.py](../tests/test_qnn_delegate.py) for the full test implementation.

‎docs/source/backends-qualcomm.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,7 @@ We use deeplab-v3-resnet101 as an example in this tutorial. Run below commands t
149149
```bash
150150
cd $EXECUTORCH_ROOT
151151

152-
python -m examples.qualcomm.scripts.deeplab_v3 -b build-android -m SM8550 --compile_only --download
152+
python -m examples.qualcomm.scripts.deeplab_v3 --build_folder build-android --soc_model SM8550 --compile_only --download
153153
```
154154

155155
You might see something like below:
@@ -287,9 +287,9 @@ The model is merely executed. If we want to feed real inputs and get model outpu
287287
```bash
288288
cd $EXECUTORCH_ROOT
289289
# android
290-
python -m examples.qualcomm.scripts.deeplab_v3 -b build-android -m SM8550 --download -s <device_serial>
290+
python -m examples.qualcomm.scripts.deeplab_v3 --build_folder build-android --soc_model SM8550 --download --device <device_serial>
291291
# (optional) linux embedded
292-
python -m examples.qualcomm.scripts.deeplab_v3 -b build-oe-linux -m SXR1230P --download -s <device_serial> -t aarch64-oe-linux-gcc-9.3
292+
python -m examples.qualcomm.scripts.deeplab_v3 --build_folder build-oe-linux --soc_model SXR1230P --download --device <device_serial> --target aarch64-oe-linux-gcc9.3
293293
```
294294
The `<device_serial>` can be found by `adb devices` command.
295295

‎docs/source/llm/build-run-llama3-qualcomm-ai-engine-direct-backend.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ To export with the Qualcomm AI Engine Direct Backend, ensure the following:
6767

6868
```bash
6969
# export llama
70-
python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --checkpoint consolidated.00.pth --params params.json --tokenizer_model tokenizer.model --decoder_model llama3_2-3b_instruct --model_mode kv --max_seq_len 1024 --prompt "I would like to learn python, could you teach me with a simple example?" --tasks wikitext --limit 1 --compile_only
70+
python examples/qualcomm/oss_scripts/llama/llama.py --build_folder build-android --device ${SERIAL_NUM} --soc_model ${SOC_MODEL} --checkpoint consolidated.00.pth --params params.json --tokenizer_model tokenizer.model --decoder_model llama3_2-3b_instruct --model_mode kv --max_seq_len 1024 --prompt "I would like to learn python, could you teach me with a simple example?" --tasks wikitext --limit 1 --compile_only
7171
```
7272
Note: end-to-end [instructions](https://github.com/pytorch/executorch/blob/main/examples/qualcomm/oss_scripts/llama/README.md)
7373

@@ -81,7 +81,7 @@ Note: end-to-end [instructions](https://github.com/pytorch/executorch/blob/main/
8181
**3.3 Run model**
8282
```bash
8383
# Run llama
84-
python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --checkpoint consolidated.00.pth --params params.json --tokenizer_model tokenizer.model --decoder_model llama3_2-3b_instruct --model_mode kv --max_seq_len 1024 --prompt "I would like to learn python, could you teach me with a simple example?" --tasks wikitext --limit 1 --pre_gen_pte ${PATH_TO_ARTIFACT}
84+
python examples/qualcomm/oss_scripts/llama/llama.py --build_folder build-android --device ${SERIAL_NUM} --soc_model ${SOC_MODEL} --checkpoint consolidated.00.pth --params params.json --tokenizer_model tokenizer.model --decoder_model llama3_2-3b_instruct --model_mode kv --max_seq_len 1024 --prompt "I would like to learn python, could you teach me with a simple example?" --tasks wikitext --limit 1 --pre_gen_pte ${PATH_TO_ARTIFACT}
8585
```
8686

8787
## What is coming?

‎examples/qualcomm/custom_op/README.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -292,9 +292,9 @@ Registers `torch.ops.my_ops.mul3.default` (multiply by 3) and delegates it via `
292292
```bash
293293
python3 examples/qualcomm/custom_op/custom_ops_1.py \
294294
--build_folder build-android \
295-
-s <device_serial> \
296-
-H <host> \
297-
-m SM8650 \
295+
--device <device_serial> \
296+
--host <host> \
297+
--soc_model SM8650 \
298298
--op_package_dir examples/qualcomm/custom_op/example_op_package_htp/ExampleOpPackage \
299299
--build_op_package
300300
```
@@ -303,7 +303,7 @@ python3 examples/qualcomm/custom_op/custom_ops_1.py \
303303
```bash
304304
python3 examples/qualcomm/custom_op/custom_ops_1.py \
305305
--build_folder build-x86 \
306-
-m SM8650 \
306+
--soc_model SM8650 \
307307
--op_package_dir examples/qualcomm/custom_op/example_op_package_htp/ExampleOpPackage \
308308
--build_op_package \
309309
--enable_x86_64
@@ -317,9 +317,9 @@ Registers `torch.ops.my_ops.split_custom.default` (splits a tensor into two halv
317317
```bash
318318
python3 examples/qualcomm/custom_op/custom_ops_2.py \
319319
--build_folder build-android \
320-
-s <device_serial> \
321-
-H <host> \
322-
-m SM8650 \
320+
--device <device_serial> \
321+
--host <host> \
322+
--soc_model SM8650 \
323323
--op_package_dir examples/qualcomm/custom_op/example_op_package_htp_multi_output/SplitCustomOpPackage \
324324
--build_op_package
325325
```
@@ -328,7 +328,7 @@ python3 examples/qualcomm/custom_op/custom_ops_2.py \
328328
```bash
329329
python3 examples/qualcomm/custom_op/custom_ops_2.py \
330330
--build_folder build-x86 \
331-
-m SM8650 \
331+
--soc_model SM8650 \
332332
--op_package_dir examples/qualcomm/custom_op/example_op_package_htp_multi_output/SplitCustomOpPackage \
333333
--build_op_package \
334334
--enable_x86_64

‎examples/qualcomm/qaihub_scripts/llama/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Please refer to [Check context binary version](../../README.md#check-context-bin
3030
#### Step4: Run default examples
3131
```bash
3232
# AIHUB_CONTEXT_BINARIES: ${PATH_TO_AIHUB_WORKSPACE}/build/llama_v2_7b_chat_quantized
33-
python examples/qualcomm/qaihub_scripts/llama/llama2/qaihub_llama2_7b.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --context_binaries ${AIHUB_CONTEXT_BINARIES} --tokenizer_bin tokenizer.bin --prompt "What is Python?"
33+
python examples/qualcomm/qaihub_scripts/llama/llama2/qaihub_llama2_7b.py --build_folder build-android --device ${SERIAL_NUM} --soc_model ${SOC_MODEL} --context_binaries ${AIHUB_CONTEXT_BINARIES} --tokenizer_bin tokenizer.bin --prompt "What is Python?"
3434
```
3535

3636
## Llama-3-8b-chat-hf
@@ -53,5 +53,5 @@ Please refer to [Check context binary version](../../README.md#check-context-bin
5353
#### Step4: Run default examples
5454
```bash
5555
# AIHUB_CONTEXT_BINARIES: ${PATH_TO_AIHUB_WORKSPACE}/build/llama_v3_8b_chat_quantized
56-
python examples/qualcomm/qaihub_scripts/llama/llama3/qaihub_llama3_8b.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --context_binaries ${AIHUB_CONTEXT_BINARIES} --tokenizer_model tokenizer.model --prompt "What is baseball?"
56+
python examples/qualcomm/qaihub_scripts/llama/llama3/qaihub_llama3_8b.py --build_folder build-android --device ${SERIAL_NUM} --soc_model ${SOC_MODEL} --context_binaries ${AIHUB_CONTEXT_BINARIES} --tokenizer_model tokenizer.model --prompt "What is baseball?"
5757
```

0 commit comments

Comments
 (0)