@@ -99,6 +99,18 @@ python export_voxtral_rt.py \
9999 --qlinear fpa4w
100100```
101101
102+ ** Note:** Metal 4-bit quantization requires torchao built with experimental MPS (Metal) ops.
103+
104+ You can install torchao with Metal support from the ` ao ` repo (in third-party/ao/)
105+ ``` bash
106+ USE_CPP=1 TORCHAO_BUILD_EXPERIMENTAL_MPS=1 pip install . --no-build-isolation
107+ ```
108+
109+ Alternatively, you can build torchao with Metal support while installing ExecuTorch from source
110+ ``` bash
111+ EXECUTORCH_BUILD_KERNELS_TORCHAO=1 TORCHAO_BUILD_EXPERIMENTAL_MPS=1 ./install_executorch.sh
112+ ```
113+
102114### Options
103115
104116| Flag | Default | Description |
@@ -210,3 +222,19 @@ Ctrl+C stops recording and flushes remaining text.
210222 ` --streaming ` flag. Both ` --streaming ` and ` --mic ` runner modes
211223 require a streaming-exported model.
212224- ** ` fpa4w ` error** : This quantization requires ` --backend metal ` .
225+ - ** Metal runner fails with ` Library not loaded: @rpath/libc++.1.dylib ` ** :
226+ The AOTInductor-compiled ` .so ` inside the ` .pte ` references ` libc++ ` via
227+ ` @rpath ` , which can't be resolved when extracted to a temp directory.
228+ Add ` /usr/lib ` to ` DYLD_LIBRARY_PATH ` so dyld finds it in the shared cache:
229+ ``` bash
230+ DYLD_LIBRARY_PATH=/usr/lib \
231+ cmake-out/examples/models/voxtral_realtime/voxtral_realtime_runner ...
232+ ```
233+ - ** Metal runner fails with ` Library not loaded: libomp.dylib ` ** :
234+ The AOTInductor-compiled ` .so ` links against OpenMP. Install it via
235+ Homebrew and add it to ` DYLD_LIBRARY_PATH ` :
236+ ``` bash
237+ brew install libomp
238+ DYLD_LIBRARY_PATH=/usr/lib:$( brew --prefix libomp) /lib \
239+ cmake-out/examples/models/voxtral_realtime/voxtral_realtime_runner ...
240+ ```
0 commit comments