Skip to content

Commit 7256479

Browse files
committed
[ET-VK][CI] Add end-to-end quantized LLM coverage
Summary: ## Rationale Add the seven LLM recipes used by internal GPU CI to the existing OSS NVIDIA workflow: Qwen3-0.6B, five Llama3.2-1B configurations, and Phi4 Mini. SwiftShader cannot exercise their required quantized compute features. Use CUDA's shared model export/build/run pattern. Select pinned public HF checkpoints, prepare them with existing model utilities, invoke the existing LLM exporter, check called Vulkan delegates and required operator families, and validate greedy native output. Quantized Llama embeddings also retain the 770 MiB artifact limit. ## Review order Read vulkan.yml and the shared export/run scripts first, then the HF/Llama preparation and PTE checker. Continue with the Release Vulkan Make/CMake path and installed-package Boost dependency, then the Vulkan CI regression suites and their CPU CI wiring. The shell orchestration suite was restored to Python and Vulkan CI tests are grouped under .ci/scripts/tests/vulkan. | Area | Before | After | |---|---|---| | OSS Vulkan GPU CI | Model/operator coverage without an LLM answer gate | Seven pinned LLM export/inference cells | | Shared scripts | CUDA/Metal/XNNPACK dispatch | Also supports the selected Vulkan recipes | | Llama preparation | Local Unsloth LoRA conversion wrapper | Full HF conversion through TorchTune and optional parameter output | | Native build | No llama-vulkan Make target | Release Vulkan installation plus the existing Llama runner build | | Installed package | Boost target references could be unresolved | Discover Boost only when the exported Vulkan interface needs it | Authored with OpenAI Codex. Test Plan: ## Observed Results Results below were recorded before updating the mainline base. The final shell refactor preserved all seven export command traces, metadata, prompts, and expected answers. The seven-device-case sweep preceded the final structural refactors; Qwen was freshly exported and run after existing-exporter reuse. | Correctness validation | Environment | Observed result | |---|---|---| | Vulkan CI Python suites, Llama converter tests, existing CUDA workflow tests | Isolated current-pin Python environment | 47 passed, 0 failed | | CMake package producer/consumer suite | macOS, real configure/build/install | 3 passed, 0 failed | | Restored orchestration suite after UFMT | macOS | 25 passed, included in the Python coverage above | | Seven before/after export recipe traces and metadata | Real shared scripts with command stubs | All seven identical | | Renamed PTE checker | Actual saved Qwen PTE | Passed; called VulkanBackend and required families found | | Seven earlier shared native LLM cases | Qwen: S24/Adreno 750; Llama/Phi: Pixel 9 Pro XL/Mali-G715 | Seven passed expected answers and native exit checks | | Fresh Qwen export plus shared native execution after exporter reuse | S24/Adreno 750 | Passed | | make llama-vulkan | Isolated macOS build, Boost enabled; CoreML/ASR disabled locally | Passed | | Lintrunner on edited files and new/moved CI tests | Local | Passed | | Shell syntax, whitespace, complete-patch reverse check | Local | Passed | | New NVIDIA workflow execution | NVIDIA A10G | Not run; no accessible NVIDIA validation host | | Performance validation | Result | |---|---| | Operator performance tests | Not run; no operator implementation change | | End-to-end latency/memory A/B against parent/public ancestor | Not run; this adds correctness coverage, with no performance claim | ## Reproduction Guide ### Checkout and development dependencies Install Git and Conda (Miniforge installers are available at https://github.com/conda-forge/miniforge/releases). On Linux install a C++ toolchain; on macOS install Xcode command-line tools with xcode-select --install. For NVIDIA inference, install a compatible NVIDIA Vulkan driver. The existing setup script selects and checks the hardware ICD. ```bash git clone --recurse-submodules https://github.com/pytorch/executorch.git cd executorch git checkout ssjia/vulkan-llm-ci git submodule update --init --recursive conda create -y -n etvk-ci python=3.11 conda activate etvk-ci conda install -y -c conda-forge cmake ninja glslc rust CMAKE_ARGS="-DEXECUTORCH_BUILD_VULKAN=ON -DEXECUTORCH_BUILD_KERNELS_LLM_AOT=ON" \ PYTHON_EXECUTABLE=python ./install_executorch.sh python -m pip install pytest pytest-xdist ``` On Linux, run the Vulkan dependency setup before installation. Select real-gpu for NVIDIA runtime testing, or the default setup for software-driver host preparation/export-only testing: ```bash source .ci/scripts/setup-vulkan-linux-deps.sh real-gpu ``` ### CPU regression tests These create their own fixtures; no model downloads are needed. The Python orchestration tests use command stubs, the conversion tests use real tensors, and the CMake suite builds real small package producers/consumers. ```bash python -m pytest -q .ci/scripts/tests/vulkan \ .ci/scripts/tests/test_cuda_workflow.py \ examples/models/llama/tests/test_convert_weights.py bash .ci/scripts/tests/vulkan/test_vulkan_cmake_package.sh ``` ### Generate all model artifacts The shared script downloads pinned public checkpoints, copies their tokenizers, creates exact prompt/answer files, and checks required Vulkan lowering and size. This generation stage does not require NVIDIA inference. ```bash ET_ARTIFACT_DIR="$PWD/ci-repro/models" mkdir -p "$ET_ARTIFACT_DIR" while read -r model quant; do name=$(printf '%s' "$model" | tr / _) bash .ci/scripts/export_model_artifact.sh vulkan "$model" "$quant" \ "$ET_ARTIFACT_DIR/$name/$quant" done <<'CASES' Qwen/Qwen3-0.6B quantized-8da4w-fp16-embedding unsloth/Llama-3.2-1B-Instruct quantized-8da4w unsloth/Llama-3.2-1B-Instruct quantized-8da4w-fp16 unsloth/Llama-3.2-1B-Instruct quantized-8da4w-embedding unsloth/Llama-3.2-1B-Instruct quantized-8da4w-fp16-embedding unsloth/Llama-3.2-1B-Instruct quantized-int4-weight-only microsoft/Phi-4-mini-instruct quantized-8da4w-embedding CASES ``` ### Linux/NVIDIA shared end-to-end path Use the same generated directories with RUN_EXPORT=0, or regenerate and run in one step with RUN_EXPORT=1. The helper builds make llama-vulkan and checks native exit status and the expected answer in stdout. Repeat for the other pairs above. The GitHub matrix runs these pairs as separate jobs. ```bash source .ci/scripts/setup-vulkan-linux-deps.sh real-gpu RUN_EXPORT=1 bash .ci/scripts/test_model_e2e.sh vulkan \ Qwen/Qwen3-0.6B quantized-8da4w-fp16-embedding \ "$ET_ARTIFACT_DIR/Qwen_Qwen3-0.6B/quantized-8da4w-fp16-embedding" ``` ### Android native correctness reproduction Install Android Studio from https://developer.android.com/studio. In SDK Manager install Android SDK Platform-Tools and NDK 29.0.13846066. Set ET_ANDROID_SDK to the SDK Location shown by SDK Manager and ET_DEVICE to the serial reported by adb devices. Enable USB debugging and authorize the host. Use S24 for Qwen and Pixel 9 Pro XL for the larger Llama/Phi cases, matching the recorded device sweep. Build a static Android runtime/runner from this checkout: ```bash ET_NDK="$ET_ANDROID_SDK/ndk/29.0.13846066" ET_ANDROID_BUILD="$PWD/ci-repro/android-core" ET_ANDROID_INSTALL="$PWD/ci-repro/android-install" ET_ANDROID_RUNNER="$PWD/ci-repro/android-runner" cmake -S . -B "$ET_ANDROID_BUILD" -G Ninja \ -DCMAKE_TOOLCHAIN_FILE="$ET_NDK/build/cmake/android.toolchain.cmake" \ -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-26 \ -DANDROID_STL=c++_static -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX="$ET_ANDROID_INSTALL" -DPYTHON_EXECUTABLE="$(command -v python)" \ -DEXECUTORCH_BUILD_VULKAN=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \ -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \ -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_LLM=ON \ -DEXECUTORCH_BUILD_EXTENSION_LLM_RUNNER=ON \ -DEXECUTORCH_BUILD_KERNELS_LLM=ON -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \ -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON -DEXECUTORCH_BUILD_XNNPACK=ON cmake --build "$ET_ANDROID_BUILD" --target install --parallel 4 cmake -S examples/models/llama -B "$ET_ANDROID_RUNNER" -G Ninja \ -DCMAKE_TOOLCHAIN_FILE="$ET_NDK/build/cmake/android.toolchain.cmake" \ -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-26 \ -DANDROID_STL=c++_static -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_FIND_ROOT_PATH="$ET_ANDROID_INSTALL" -DPYTHON_EXECUTABLE="$(command -v python)" cmake --build "$ET_ANDROID_RUNNER" --target llama_main --parallel 4 ``` Stage and run one generated artifact; repeat with the other directories/device as appropriate. Modern adb preserves the remote native exit code. Keep stdout and stderr separate so diagnostic text cannot satisfy the answer check. ```bash ET_MODEL_DIR="$ET_ARTIFACT_DIR/Qwen_Qwen3-0.6B/quantized-8da4w-fp16-embedding" ET_REMOTE=/data/local/tmp/etvk_llm_ci adb -s "$ET_DEVICE" shell mkdir -p "$ET_REMOTE" adb -s "$ET_DEVICE" push "$ET_ANDROID_RUNNER/llama_main" "$ET_REMOTE/llama_main" adb -s "$ET_DEVICE" shell chmod +x "$ET_REMOTE/llama_main" for file in model.pte tokenizer.json prompt.txt; do adb -s "$ET_DEVICE" push "$ET_MODEL_DIR/$file" "$ET_REMOTE/$file" done adb -s "$ET_DEVICE" shell "$ET_REMOTE/llama_main" \ --model_path "$ET_REMOTE/model.pte" --tokenizer_path "$ET_REMOTE/tokenizer.json" \ --prompt_file "$ET_REMOTE/prompt.txt" --temperature 0 --max_new_tokens 50 \ > "$ET_MODEL_DIR/runner.stdout" 2> "$ET_MODEL_DIR/runner.stderr" grep -Fiw -- "$(cat "$ET_MODEL_DIR/expected_output.txt")" "$ET_MODEL_DIR/runner.stdout" ``` No performance tests were run. There are no latency regression thresholds to reproduce in this change. Reviewers:
1 parent ad1c24d commit 7256479

15 files changed

Lines changed: 1301 additions & 28 deletions

‎.ci/scripts/export_model_artifact.sh‎

Lines changed: 150 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,16 @@
55
# This source code is licensed under the BSD-style license found in the
66
# LICENSE file in the root directory of this source tree.
77

8-
# Export model to CUDA/Metal/XNNPACK format with optional quantization
8+
# Export model to CUDA/Metal/XNNPACK/Vulkan format with optional quantization
99

1010
show_help() {
1111
cat << EOF
1212
Usage: export_model_artifact.sh <device> <hf_model> [quant_name] [output_dir] [mode]
1313
14-
Export a HuggingFace model to CUDA/Metal/XNNPACK format with optional quantization.
14+
Export a HuggingFace model to CUDA/Metal/XNNPACK/Vulkan format with optional quantization.
1515
1616
Arguments:
17-
device cuda, metal, or xnnpack (required)
17+
device cuda, metal, xnnpack, or vulkan (required)
1818
1919
hf_model HuggingFace model ID (required)
2020
Supported models:
@@ -26,14 +26,20 @@ Arguments:
2626
- nvidia/parakeet-tdt
2727
- facebook/dinov2-small-imagenet1k-1-layer
2828
- meta-models/Muse-Glimmer-30B-GGUF
29+
- Qwen/Qwen3-0.6B
30+
- unsloth/Llama-3.2-1B-Instruct
31+
- microsoft/Phi-4-mini-instruct
2932
3033
quant_name Quantization type (optional, default: non-quantized)
3134
Options:
3235
- non-quantized
3336
- quantized-int4-tile-packed (CUDA only)
34-
- quantized-int4-weight-only (CUDA only)
37+
- quantized-int4-weight-only (CUDA/Vulkan)
3538
- quantized-int4-metal (Metal only)
36-
- quantized-8da4w (XNNPACK only)
39+
- quantized-8da4w (XNNPACK/Vulkan)
40+
- quantized-8da4w-fp16 (Vulkan only)
41+
- quantized-8da4w-embedding (Vulkan only)
42+
- quantized-8da4w-fp16-embedding (Vulkan only)
3743
- kquant-17gb (Muse Glimmer only)
3844
- kquant-dynamic (Muse Glimmer only)
3945
@@ -85,6 +91,9 @@ QUANT_NAME="${3:-non-quantized}"
8591
OUTPUT_DIR="${4:-.}"
8692
MODE="${5:-}"
8793

94+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
95+
EXECUTORCH_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
96+
8897
# Validate mode if specified
8998
if [ -n "$MODE" ]; then
9099
case "$MODE" in
@@ -118,15 +127,18 @@ case "$DEVICE" in
118127
;;
119128
metal)
120129
;;
121-
xnnpack)
130+
xnnpack|vulkan)
122131
;;
123132
*)
124133
echo "Error: Unsupported device '$DEVICE'"
125-
echo "Supported devices: cuda, cuda-windows, metal, xnnpack"
134+
echo "Supported devices: cuda, cuda-windows, metal, xnnpack, vulkan"
126135
exit 1
127136
;;
128137
esac
129138

139+
CHECKPOINT_SHA256=""
140+
EXPORT_METADATA=""
141+
130142
# Determine model configuration based on HF model ID
131143
case "$HF_MODEL" in
132144
mistralai/Voxtral-Mini-3B-2507)
@@ -168,6 +180,41 @@ case "$HF_MODEL" in
168180
EXTRA_PIP=""
169181
PREPROCESSOR_FEATURE_SIZE=""
170182
PREPROCESSOR_OUTPUT=""
183+
if [ "$DEVICE" = "vulkan" ]; then
184+
test "$QUANT_NAME" = quantized-8da4w-fp16-embedding
185+
MODEL_NAME=qwen3_0_6b
186+
REVISION=c1899de289a04d12100db370d81485cdf75e47ca
187+
CONVERTER=executorch.examples.models.qwen3.convert_weights
188+
PARAMS="$EXECUTORCH_ROOT/examples/models/qwen3/config/0_6b_config.json"
189+
PROMPT=$'<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n'
190+
EXPECTED_OUTPUT=Paris
191+
fi
192+
;;
193+
unsloth/Llama-3.2-1B-Instruct)
194+
if [ "$DEVICE" != "vulkan" ]; then
195+
echo "Error: $HF_MODEL is only supported with Vulkan"
196+
exit 1
197+
fi
198+
MODEL_NAME=llama3_2
199+
REVISION=5a8abab4a5d6f164389b1079fb721cfab8d7126c
200+
CONVERTER=executorch.examples.models.llama.convert_weights
201+
CHECKPOINT_SHA256=1ff795ff6a07e6a68085d206fb84417da2f083f68391c2843cd2b8ac6df8538f
202+
EXPORT_METADATA='{"append_eos_to_prompt":0,"get_bos_id":128000,"get_eos_ids":[128009,128001]}'
203+
PROMPT=$'<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nWhat is the capital of France?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n'
204+
EXPECTED_OUTPUT="The capital of France is Paris"
205+
;;
206+
microsoft/Phi-4-mini-instruct)
207+
if [ "$DEVICE" != "vulkan" ]; then
208+
echo "Error: $HF_MODEL is only supported with Vulkan"
209+
exit 1
210+
fi
211+
test "$QUANT_NAME" = quantized-8da4w-embedding
212+
MODEL_NAME=phi_4_mini
213+
REVISION=cfbefacb99257ffa30c83adab238a50856ac3083
214+
CONVERTER=executorch.examples.models.phi_4_mini.convert_weights
215+
PARAMS="$EXECUTORCH_ROOT/examples/models/phi_4_mini/config/config.json"
216+
PROMPT=$'<|user|>\nWhat is the capital of France?<|end|>\n<|assistant|>\n'
217+
EXPECTED_OUTPUT="The capital of France is Paris"
171218
;;
172219
nvidia/parakeet-tdt)
173220
MODEL_NAME="parakeet"
@@ -232,6 +279,10 @@ case "$HF_MODEL" in
232279
;;
233280
esac
234281

282+
QMODE=8da4w
283+
FORCE_FP16=false
284+
EMBEDDING_QUANTIZE=""
285+
235286
# Determine quantization args based on quant name
236287
case "$QUANT_NAME" in
237288
non-quantized)
@@ -250,6 +301,7 @@ case "$QUANT_NAME" in
250301
exit 1
251302
fi
252303
EXTRA_ARGS="--qlinear_encoder 4w"
304+
QMODE=4w
253305
;;
254306
quantized-int4-metal)
255307
if [ "$DEVICE" != "metal" ]; then
@@ -259,12 +311,25 @@ case "$QUANT_NAME" in
259311
EXTRA_ARGS="--qlinear fpa4w --qlinear_encoder fpa4w"
260312
;;
261313
quantized-8da4w)
262-
if [ "$DEVICE" != "xnnpack" ]; then
263-
echo "Error: quantized-8da4w is only supported with xnnpack device"
314+
if [ "$DEVICE" != "xnnpack" ] && [ "$DEVICE" != "vulkan" ]; then
315+
echo "Error: quantized-8da4w is only supported with xnnpack or vulkan device"
264316
exit 1
265317
fi
266318
EXTRA_ARGS="--qlinear 8da4w --qlinear_group_size 32 --qlinear_encoder 8da4w --qlinear_encoder_group_size 32"
267319
;;
320+
quantized-8da4w-fp16|quantized-8da4w-embedding|quantized-8da4w-fp16-embedding)
321+
if [ "$DEVICE" != "vulkan" ]; then
322+
echo "Error: Quantization '$QUANT_NAME' is only supported with Vulkan"
323+
exit 1
324+
fi
325+
case "$QUANT_NAME" in
326+
*-fp16*) FORCE_FP16=true ;;
327+
esac
328+
case "$QUANT_NAME" in
329+
*-embedding) EMBEDDING_QUANTIZE=4,64 ;;
330+
esac
331+
EXTRA_ARGS=""
332+
;;
268333
kquant-17gb|kquant-dynamic)
269334
if [ "$HF_MODEL" != "meta-models/Muse-Glimmer-30B-GGUF" ]; then
270335
echo "Error: Quantization '$QUANT_NAME' can only be used with Muse Glimmer model"
@@ -295,6 +360,82 @@ if [ "$MODEL_NAME" = "muse_glimmer" ]; then
295360
fi
296361
fi
297362

363+
if [ "$DEVICE" = "vulkan" ]; then
364+
case "$MODEL_NAME" in
365+
qwen3_0_6b|llama3_2|phi_4_mini) ;;
366+
*) echo "Error: Unsupported Vulkan model: $HF_MODEL"; exit 1 ;;
367+
esac
368+
case "$QUANT_NAME" in
369+
quantized-8da4w|quantized-8da4w-fp16|quantized-8da4w-embedding|quantized-8da4w-fp16-embedding|quantized-int4-weight-only) ;;
370+
*) echo "Error: Unsupported Vulkan quantization: $QUANT_NAME"; exit 1 ;;
371+
esac
372+
mkdir -p "$OUTPUT_DIR"
373+
OUTPUT_DIR="$(cd "$OUTPUT_DIR" && pwd)"
374+
(
375+
cd "$EXECUTORCH_ROOT"
376+
MAX_SIZE_MIB=""
377+
if [ "$MODEL_NAME" = "llama3_2" ] && [ -n "$EMBEDDING_QUANTIZE" ]; then MAX_SIZE_MIB=770; fi
378+
HASH_ARGS=(--metadata-path "$OUTPUT_DIR/metadata.json")
379+
if [ -n "${CHECKPOINT_SHA256:-}" ]; then HASH_ARGS+=(--expected-sha256 "$CHECKPOINT_SHA256"); fi
380+
python - "$OUTPUT_DIR" "$HF_MODEL" "$REVISION" "$QUANT_NAME" "$QMODE" "$FORCE_FP16" \
381+
"$EMBEDDING_QUANTIZE" "$MAX_SIZE_MIB" "$PROMPT" "$EXPECTED_OUTPUT" <<'PY'
382+
import json
383+
import sys
384+
from pathlib import Path
385+
386+
output, model, revision, variant, qmode, fp16, embedding, size, prompt, expected = sys.argv[1:]
387+
metadata = {
388+
"model": model, "revision": revision, "variant": variant, "quantization": qmode,
389+
"group_size": 64, "embedding_quantize": embedding, "dtype": "fp32",
390+
"vulkan_force_fp16": fp16 == "true", "max_seq_length": 2048, "max_context_length": 2048,
391+
"use_kv_cache": True, "use_sdpa_with_kv_cache": True, "max_size_mib": int(size) if size else None,
392+
"prompt": prompt, "expected": expected, "temperature": 0, "max_new_tokens": 50,
393+
}
394+
if model == "unsloth/Llama-3.2-1B-Instruct":
395+
metadata["original_model"] = "meta-llama/Llama-3.2-1B-Instruct"
396+
output = Path(output)
397+
(output / "metadata.json").write_text(json.dumps(metadata, indent=2) + "\n")
398+
(output / "prompt.txt").write_text(prompt)
399+
(output / "expected_output.txt").write_text(expected + "\n")
400+
PY
401+
SNAPSHOT=$(python -m executorch.examples.models.llama.hf_download "$HF_MODEL" \
402+
--revision "$REVISION" "${HASH_ARGS[@]}" \
403+
2> "$OUTPUT_DIR/download.log")
404+
cp "$SNAPSHOT/tokenizer.json" "$SNAPSHOT/tokenizer_config.json" "$OUTPUT_DIR/"
405+
VULKAN_CHECKPOINT_DIR=$(mktemp -d "${RUNNER_TEMP:-/tmp}/vulkan_llm_XXXXXX")
406+
trap 'rm -rf "$VULKAN_CHECKPOINT_DIR"' EXIT
407+
CONVERT_ARGS=("$SNAPSHOT" "$VULKAN_CHECKPOINT_DIR/checkpoint.pth")
408+
if [ "$MODEL_NAME" = "llama3_2" ]; then
409+
PARAMS="$OUTPUT_DIR/params.json"
410+
CONVERT_ARGS+=(--params-output "$PARAMS")
411+
fi
412+
python -m "$CONVERTER" "${CONVERT_ARGS[@]}" > "$OUTPUT_DIR/convert.log" 2>&1
413+
VULKAN_EXPORT_ARGS=(
414+
"base.model_class=$MODEL_NAME" "base.checkpoint=\"$VULKAN_CHECKPOINT_DIR/checkpoint.pth\""
415+
"base.params=\"$PARAMS\"" "base.tokenizer_path=\"$OUTPUT_DIR/tokenizer.json\""
416+
model.dtype_override=fp32 model.use_kv_cache=true model.use_sdpa_with_kv_cache=true
417+
model.enable_dynamic_shape=true backend.vulkan.enabled=true "backend.vulkan.force_fp16=$FORCE_FP16"
418+
"quantization.qmode=$QMODE" quantization.group_size=64
419+
export.max_seq_length=2048 export.max_context_length=2048
420+
"export.output_dir=\"$OUTPUT_DIR\"" "export.output_name=\"$OUTPUT_DIR/model.pte\"" debug.verbose=true
421+
)
422+
if [ -n "$EMBEDDING_QUANTIZE" ]; then
423+
VULKAN_EXPORT_ARGS+=("quantization.embedding_quantize=\"$EMBEDDING_QUANTIZE\"")
424+
fi
425+
if [ -n "${EXPORT_METADATA:-}" ]; then VULKAN_EXPORT_ARGS+=("base.metadata='$EXPORT_METADATA'"); fi
426+
rm -f "$OUTPUT_DIR/model.pte"
427+
python -m extension.llm.export.export_llm "${VULKAN_EXPORT_ARGS[@]}" > "$OUTPUT_DIR/export.log" 2>&1
428+
test -f "$OUTPUT_DIR/model.pte"
429+
python - "$OUTPUT_DIR/model.pte" "$QMODE" "$EMBEDDING_QUANTIZE" "$MAX_SIZE_MIB" <<'PY'
430+
import sys
431+
from executorch.backends.vulkan.test.utils import verify_vulkan_llm_pte
432+
433+
verify_vulkan_llm_pte(sys.argv[1], sys.argv[2], bool(sys.argv[3]), int(sys.argv[4]) if sys.argv[4] else None)
434+
PY
435+
)
436+
exit 0
437+
fi
438+
298439
# Downloads and compiler caches go in scratch dirs outside OUTPUT_DIR because the CI job
299440
# templates upload OUTPUT_DIR even when the job fails. A failed export also empties
300441
# OUTPUT_DIR, but only if it started out empty, so a local run with output_dir=. cannot

0 commit comments

Comments
 (0)