Repository navigation
Commit c27baa8
authored
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations (pytorch#22284)
## Summary
This PR implements the __compilation__ and __inference__ strategy
implementations, completing the strategy layer. Each strategy delegates
to injectable adapter interfaces for testability.
### What's included
#### Strategy implementations:
- `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via
`CompilerAdapter`
- Validates model, example_inputs, soc_model, and backend_type are
present
- Passes `example_inputs` explicitly to the adapter (not via
`extra_options`) — mirrors the PR5 fix for the quantization stage
- Delegates to adapter with example_inputs, compile specs, artifact dir,
soc_model, backend_type
- Filters `context.extra_options` to a compilation-relevant allow-list
- Returns artifact paths and optional `ETRecord`
- `ExecuTorchInferenceStrategy`: Runs on-device inference via
`DeviceRunnerAdapter`
- Validates artifact_paths and adapter are present (no default adapter —
device config is required)
- Push → execute → pull results flow
- Two-step protocol: uses `output_data` from execute if present, falls
back to pulled file paths
- Returns inference results, performance metrics, and optional `ETDump`
#### `DefaultCompilerAdapter`:
- `compile_model` signature finalised — mirrors
`to_edge_transform_and_lower_to_qnn`
argument-for-argument, so per-graph lowering inputs (`compile_specs`,
`dep_table`,
`passes_job`, `constant_methods`) are explicit parameters rather than
`extra_options` keys
- Body deliberately raises `NotImplementedError`: the version this
package needs is
the multi-graph one (graph-name-keyed dicts, single multi-method `.pte`
for weight
sharing), so it lands with the strategy-level fan-out that calls it
rather than
being written single-graph and then replaced. Inject a custom
`CompilerAdapter` for now.
#### Config addition:
- `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` —
sourced from the model via `ModelLoaderAdapter.get_example_inputs`
#### Orchestrator wiring:
- `genai_pipeline.py` `_run_compilation`: passes
`example_inputs=model_prep_output.example_inputs`
#### Unit tests:
- `test_executorch_compilation_strategy.py`
- `test_executorch_inference_strategy.py`
- `test_compilation_input_config.py`
### PR Review Checklist
- All new classes follow single responsibility (one class per file) -
Yes.
- All dependencies are injected via constructor with sensible defaults -
Yes.
- All external calls are behind injectable interfaces - Yes (adapter
pattern).
- Unit tests cover every public method - Yes (100% coverage on strategy
impls).
- Docstrings on all public classes and methods - Yes.
- Type annotations on all function signatures - Yes.
- Logging follows the strategy in the LLD - Yes (info on entry/exit,
debug per step).
### Related PRs
- PR 1: Core data model, engine routing & exceptions:
pytorch#20409
- PR 2: Strategy interfaces & stage wrappers:
pytorch#20795
- PR 3: Pipeline orchestrator:
pytorch#21149
- PR 4: Adapter interfaces, default implementations & dataset providers:
pytorch#21751
- PR 5: Model preparation & quantization strategies:
pytorch#21899
- PR 6: Compilation & inference strategy implementations: this pr.
- PR 7: Integration & E2E tests: pending.
## Test plan
### Run only tests added in this PR:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v
```
### Run only this PR's tests with coverage:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%
```
### Run all `genai_pipeline` tests:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v
```
### Run all tests with coverage:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%
```1 parent 6e8f02a commit c27baa8
11 files changed
Lines changed: 929 additions & 59 deletions
File tree
- backends/qualcomm/genai_pipeline
- configs
- strategies
- compilation
- inference
- tests
- configs
- strategies
- compilation
- inference
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
26 | 31 | | |
27 | 32 | | |
28 | 33 | | |
29 | 34 | | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
30 | 42 | | |
31 | 43 | | |
32 | 44 | | |
33 | 45 | | |
34 | 46 | | |
35 | 47 | | |
36 | 48 | | |
| 49 | + | |
37 | 50 | | |
38 | 51 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
22 | 27 | | |
23 | 28 | | |
24 | 29 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
150 | 150 | | |
151 | 151 | | |
152 | 152 | | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
153 | 181 | | |
154 | 182 | | |
155 | 183 | | |
| |||
241 | 269 | | |
242 | 270 | | |
243 | 271 | | |
| 272 | + | |
| 273 | + | |
244 | 274 | | |
245 | 275 | | |
246 | 276 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
31 | | - | |
32 | | - | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
33 | 35 | | |
34 | 36 | | |
35 | 37 | | |
| |||
47 | 49 | | |
48 | 50 | | |
49 | 51 | | |
| 52 | + | |
50 | 53 | | |
51 | 54 | | |
52 | 55 | | |
53 | 56 | | |
54 | 57 | | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
55 | 61 | | |
56 | 62 | | |
57 | 63 | | |
58 | 64 | | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
59 | 71 | | |
60 | 72 | | |
| 73 | + | |
| 74 | + | |
61 | 75 | | |
62 | 76 | | |
63 | 77 | | |
64 | 78 | | |
65 | 79 | | |
66 | | - | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
67 | 89 | | |
68 | 90 | | |
69 | 91 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | | - | |
| 11 | + | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| 17 | + | |
| 18 | + | |
16 | 19 | | |
17 | 20 | | |
18 | | - | |
| 21 | + | |
19 | 22 | | |
20 | 23 | | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
25 | 39 | | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
30 | 48 | | |
31 | 49 | | |
32 | 50 | | |
33 | 51 | | |
34 | 52 | | |
| 53 | + | |
35 | 54 | | |
36 | 55 | | |
37 | 56 | | |
38 | 57 | | |
39 | 58 | | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
40 | 62 | | |
41 | 63 | | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
| 64 | + | |
47 | 65 | | |
48 | 66 | | |
49 | 67 | | |
50 | | - | |
51 | | - | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
52 | 71 | | |
53 | 72 | | |
54 | 73 | | |
55 | 74 | | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
60 | 86 | | |
61 | 87 | | |
62 | 88 | | |
63 | 89 | | |
64 | 90 | | |
65 | | - | |
66 | | - | |
67 | | - | |
| 91 | + | |
| 92 | + | |
68 | 93 | | |
69 | 94 | | |
70 | | - | |
71 | | - | |
72 | | - | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
73 | 100 | | |
0 commit comments