Skip to content

Commit 6e8b206

Browse files
Update documentation with new ecosystem integrations, demos, and resource links (pytorch#16371)
docs/source/success-stories.md 1. Added Arm ML Embedded Evaluation Kit to Ecosystem Integrations (Cortex-M, Ethos-U NPUs) 2. Added Whisper to Featured Demos (CUDA and Metal backends) 3. Added Audio Generation demo (Stable Audio Open Small on Arm CPUs with KleidiAI) 4. Added YouTube video link to Audio Generation demo 5. Added Alif Semiconductor Ensemble to Ecosystem Integrations (E4/E6/E8 MCUs with Ethos-U85) 6. Added blog link to Ultralytics entry 7. Added Digica AI SDK to Ecosystem Integrations (ExecuTorch-powered SDK for iOS/Android/edge) README.md (top-level) 1. Updated Whisper link from external repo to local examples/models/whisper/README.md 2. Added Unsloth to Resources line for fine-tuned LLM deployment 3. Added Desktop Guide link for CUDA/Metal GPU inference 4. Added Zephyr Guide link for Zephyr RTOS integration examples/models/whisper/README.md 1. Added Mobile Demo section with link to Android WhisperApp website/index.html 1. Updated Whisper link to in-tree whisper README 2. Added Reality Labs blog link to Production Deployments 3. Added LiquidAI blog link to Ecosystem Integration 4. Added Software Mansion link to Ecosystem Integration examples/qualcomm/oss_scripts/llama/README.md 1. Added YouTube video tutorial "Build Along: Run LLMs Locally on Qualcomm Hardware Using ExecuTorch" docs/source/backends-cadence.md 1. Added Cadence blog link about running optimized PyTorch models on Cadence DSPs --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 73ae813 commit 6e8b206

6 files changed

Lines changed: 48 additions & 9 deletions

File tree

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -192,7 +192,7 @@ See [examples/models/llama](examples/models/llama/README.md) for complete workfl
192192
| macOS | XNNPACK, MPS, Metal *(experimental)* |
193193
| Embedded / MCU | XNNPACK, ARM Ethos-U, NXP, Cadence DSP |
194194

195-
See [Backend Documentation](https://docs.pytorch.org/executorch/main/backends-overview.html) for detailed hardware requirements and optimization guides.
195+
See [Backend Documentation](https://docs.pytorch.org/executorch/main/backends-overview.html) for detailed hardware requirements and optimization guides. For desktop/laptop GPU inference with CUDA and Metal, see the [Desktop Guide](desktop/README.md). For Zephyr RTOS integration, see the [Zephyr Guide](zephyr/README.md).
196196

197197
## Production Deployments
198198

@@ -204,9 +204,9 @@ ExecuTorch powers on-device AI at scale across Meta's family of apps, VR/AR devi
204204

205205
**Multimodal:** [Llava](examples/models/llava/README.md) (vision-language), [Voxtral](examples/models/voxtral/README.md) (audio-language), [Gemma](examples/models/gemma3) (vision-language)
206206

207-
**Vision/Speech:** [MobileNetV2](https://github.com/meta-pytorch/executorch-examples/tree/main/mv2), [DeepLabV3](https://github.com/meta-pytorch/executorch-examples/tree/main/dl3), [Whisper](https://github.com/meta-pytorch/executorch-examples/tree/main/whisper/android/WhisperApp)
207+
**Vision/Speech:** [MobileNetV2](https://github.com/meta-pytorch/executorch-examples/tree/main/mv2), [DeepLabV3](https://github.com/meta-pytorch/executorch-examples/tree/main/dl3), [Whisper](examples/models/whisper/README.md) <!-- @lint-ignore -->
208208

209-
**Resources:** [`examples/`](examples/) directory • [executorch-examples](https://github.com/meta-pytorch/executorch-examples) out-of-tree demos • [Optimum-ExecuTorch](https://github.com/huggingface/optimum-executorch) for HuggingFace models
209+
**Resources:** [`examples/`](examples/) directory • [executorch-examples](https://github.com/meta-pytorch/executorch-examples) out-of-tree demos • [Optimum-ExecuTorch](https://github.com/huggingface/optimum-executorch) for HuggingFace models • [Unsloth](https://docs.unsloth.ai/new/deploy-llms-phone) for fine-tuned LLM deployment <!-- @lint-ignore -->
210210

211211
## Key Features
212212

‎docs/source/backends-cadence.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ In this tutorial we will walk you through the process of getting setup to build
1010

1111
In addition to the chip, the HiFi4 Neural Network Library ([nnlib](https://github.com/foss-xtensa/nnlib-hifi4)) offers an optimized set of library functions commonly used in NN processing that we utilize in this example to demonstrate how common operations can be accelerated.
1212

13+
For an overview of the Cadence ExecuTorch integration with performance benchmarks, see the blog post: [Running Optimized PyTorch Models on Cadence DSPs with ExecuTorch](https://community.cadence.com/cadence_blogs_8/b/ip/posts/running-optimized-pytorch-models-on-cadence-dsps-with-executorch).
14+
1315
On top of being able to run on the Xtensa HiFi4 DSP, another goal of this tutorial is to demonstrate how portable ExecuTorch is and its ability to run on a low-power embedded device such as the Xtensa HiFi4 DSP. This workflow does not require any delegates, it uses custom operators and compiler passes to enhance the model and make it more suitable to running on Xtensa HiFi4 DSPs. A custom [quantizer](https://pytorch.org/tutorials/prototype/quantization_in_pytorch_2_0_export_tutorial.html) is used to represent activations and weights as `uint8` instead of `float`, and call appropriate operators. Finally, custom kernels optimized with Xtensa intrinsics provide runtime acceleration.
1416

1517
::::{grid} 2

‎docs/source/success-stories.md‎

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ Powers Instagram, WhatsApp, Facebook, and Messenger with real-time on-device AI
3030
**Hardware:** Quest 3, Ray-Ban Meta Smart Glasses, Meta Ray-Ban Display
3131

3232
Enables real-time computer vision, hand tracking, voice commands, and translation on power-constrained wearable devices.
33+
[Read Blog →](https://ai.meta.com/blog/executorch-reality-labs-on-device-ai/)
3334
:::
3435

3536
:::{grid-item-card} **Liquid AI: Efficient, Flexible On-Device Intelligence**
@@ -106,14 +107,39 @@ PyTorch-native quantization and optimization library for preparing efficient mod
106107

107108
Optimize LLM fine-tuning with faster training and reduced VRAM usage, then deploy efficiently with ExecuTorch.
108109

109-
[Example Model →](https://huggingface.co/metascroy/Qwen3-4B-int8-int4-unsloth) • [Blog →](https://docs.unsloth.ai/new/quantization-aware-training-qat)
110+
[Example Model →](https://huggingface.co/metascroy/Qwen3-4B-int8-int4-unsloth) • [Blog →](https://docs.unsloth.ai/new/quantization-aware-training-qat) • [Doc →](https://docs.unsloth.ai/new/deploy-llms-phone)
110111
:::
111112

112113
:::{grid-item-card} **Ultralytics**
113114
:class-header: bg-secondary text-white
114115

115116
Deploy on-device inference for Ultralytics YOLO models using ExecuTorch.
116-
[Explore →](https://docs.ultralytics.com/integrations/executorch/)
117+
118+
[Explore →](https://docs.ultralytics.com/integrations/executorch/) • [Blog →](https://www.ultralytics.com/blog/deploy-ultralytics-yolo-models-using-the-executorch-integration)
119+
:::
120+
121+
:::{grid-item-card} **Arm ML Embedded Evaluation Kit**
122+
:class-header: bg-secondary text-white
123+
124+
Build and deploy ML applications on Arm Cortex-M (M55, M85) and Ethos-U NPUs (U55, U65, U85) using ExecuTorch.
125+
126+
[Explore →](https://gitlab.arm.com/artificial-intelligence/ethos-u/ml-embedded-evaluation-kit)
127+
:::
128+
129+
:::{grid-item-card} **Alif Semiconductor Ensemble**
130+
:class-header: bg-secondary text-white
131+
132+
Run generative AI on Ensemble E4/E6/E8 MCUs with Arm Ethos-U85 NPU acceleration.
133+
134+
[Learn More →](https://alifsemi.com/press-release/alif-semiconductor-elevates-generative-ai-with-support-for-executorch-runtime/)
135+
:::
136+
137+
:::{grid-item-card} **Digica AI SDK**
138+
:class-header: bg-secondary text-white
139+
140+
Automate PyTorch model deployment to iOS, Android, and edge devices with ExecuTorch-powered SDK.
141+
142+
[Blog →](https://www.digica.com/blog/effortless-edge-deployment-of-ai-models-with-digicas-ai-sdk-feat-executorch.html)
117143
:::
118144

119145
::::
@@ -126,8 +152,12 @@ Deploy on-device inference for Ultralytics YOLO models using ExecuTorch.
126152

127153
- **Voxtral** - Deploy audio-text-input LLM on CPU (via XNNPACK) and on CUDA. [Try →](https://github.com/pytorch/executorch/blob/main/examples/models/voxtral/README.md)
128154

155+
- **Whisper** - Deploy OpenAI's Whisper speech recognition model on CUDA and Metal backends. [Try →](https://github.com/pytorch/executorch/blob/main/examples/models/whisper/README.md) <!-- @lint-ignore -->
156+
129157
- **LoRA adapter** - Export two LoRA adapters that share a single foundation weight file, saving memory and disk space. [Try →](https://github.com/meta-pytorch/executorch-examples/tree/main/program-data-separation/cpp/lora_example)
130158

131159
- **OpenVINO from Intel** - Deploy [Yolo12](https://github.com/pytorch/executorch/tree/main/examples/models/yolo12), [Llama](https://github.com/pytorch/executorch/tree/main/examples/openvino/llama), and [Stable Diffusion](https://github.com/pytorch/executorch/tree/main/examples/openvino/stable_diffusion) on [OpenVINO from Intel](https://www.intel.com/content/www/us/en/developer/articles/community/optimizing-executorch-on-ai-pcs.html).
132160

161+
- **Audio Generation** - Generate audio from text prompts using Stable Audio Open Small on Arm CPUs with XNNPACK and KleidiAI. [Try →](https://github.com/Arm-Examples/ML-examples/tree/main/kleidiai-examples/audiogen-et) • [Video →](https://www.youtube.com/watch?v=q2P0ESVxhAY) <!-- @lint-ignore -->
162+
133163
*Want to showcase your demo? [Submit here →](https://github.com/pytorch/executorch/issues)*

‎examples/models/whisper/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -166,3 +166,7 @@ cmake-out/examples/models/whisper/whisper_runner \
166166
--processor_path whisper_preprocessor.pte \
167167
--temperature 0
168168
```
169+
170+
## Mobile Demo
171+
172+
For an Android demo app, see the [Whisper Android App](https://github.com/meta-pytorch/executorch-examples/tree/main/whisper/android/WhisperApp) in the executorch-examples repository. <!-- @lint-ignore -->

‎examples/qualcomm/oss_scripts/llama/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
# Summary
22

33
## Overview
4+
5+
**Video Tutorial:** [Build Along: Run LLMs Locally on Qualcomm Hardware Using ExecuTorch](https://www.youtube.com/watch?v=41PKDlGM3oU)
6+
47
This file provides you the instructions to run LLM Decoder model with different parameters via Qualcomm HTP backend. We currently support the following models:
58
<!-- numbered list will be automatically generated -->
69
1. LLAMA2 Stories 110M

‎website/index.html‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -918,16 +918,16 @@ <h2 class="section-title">Success <span class="highlight">Stories</span></h2>
918918
<h3 style="font-size: 1.5rem; margin-bottom: 1.5rem; color: var(--text-dark);">Production Deployments</h3>
919919
<ul style="color: var(--text-gray); line-height: 2; list-style: none; padding-left: 0;">
920920
<li><strong class="highlight"><a href="https://engineering.fb.com/2025/07/28/android/executorch-on-device-ml-meta-family-of-apps/" style="color: var(--primary); text-decoration: none;">Meta Family of Apps</a>:</strong> Production deployment across Instagram, Facebook, and WhatsApp serving billions of users</li>
921-
<li><strong class="highlight">Meta Reality Labs:</strong> Powers Quest 3 VR and Ray-Ban Meta Smart Glasses AI experiences</li>
921+
<li><strong class="highlight"><a href="https://ai.meta.com/blog/executorch-reality-labs-on-device-ai/" style="color: var(--primary); text-decoration: none;">Meta Reality Labs</a>:</strong> Powers Quest 3 VR and Ray-Ban Meta Smart Glasses AI experiences</li>
922922
</ul>
923923
</div>
924924

925925
<div style="margin-bottom: 3rem;">
926926
<h3 style="font-size: 1.5rem; margin-bottom: 1.5rem; color: var(--text-dark);">Ecosystem Integration</h3>
927927
<ul style="color: var(--text-gray); line-height: 2; list-style: none; padding-left: 0;">
928928
<li><strong class="highlight"><a href="https://github.com/huggingface/optimum-executorch" style="color: var(--primary); text-decoration: none;">Hugging Face</a>:</strong> Optimum-ExecuTorch for direct transformer model deployment</li>
929-
<li><strong class="highlight">LiquidAI:</strong> Next-generation Liquid Foundation Models optimized for edge deployment</li>
930-
<li><strong class="highlight">Software Mansion:</strong> React Native ExecuTorch bringing edge AI to mobile apps</li>
929+
<li><strong class="highlight"><a href="https://www.liquid.ai/blog/how-liquid-ai-uses-executorch-to-power-efficient-flexible-on-device-intelligence" style="color: var(--primary); text-decoration: none;">LiquidAI</a>:</strong> Next-generation Liquid Foundation Models optimized for edge deployment</li>
930+
<li><strong class="highlight"><a href="https://docs.swmansion.com/react-native-executorch/" style="color: var(--primary); text-decoration: none;">Software Mansion</a>:</strong> React Native ExecuTorch bringing edge AI to mobile apps</li>
931931
</ul>
932932
</div>
933933

@@ -936,7 +936,7 @@ <h3 style="font-size: 1.5rem; margin-bottom: 1.5rem; color: var(--text-dark);">E
936936
<ul style="color: var(--text-gray); line-height: 2; list-style: none; padding-left: 0;">
937937
<li><strong class="highlight">LLMs:</strong> <a href="https://github.com/pytorch/executorch/blob/main/examples/models/llama/README.md" style="color: var(--primary); text-decoration: none;">Llama 3.2/3.1/3</a>, <a href="https://github.com/pytorch/executorch/blob/main/examples/models/qwen3/README.md" style="color: var(--primary); text-decoration: none;">Qwen 3</a>, <a href="https://github.com/pytorch/executorch/blob/main/examples/models/phi_4_mini/README.md" style="color: var(--primary); text-decoration: none;">Phi-4-mini</a>, <a href="https://github.com/pytorch/executorch/blob/main/examples/models/lfm2/README.md" style="color: var(--primary); text-decoration: none;">LiquidAI LFM2</a></li>
938938
<li><strong class="highlight">Multimodal:</strong> <a href="https://github.com/pytorch/executorch/blob/main/examples/models/llava/README.md" style="color: var(--primary); text-decoration: none;">Llava</a> (vision-language), <a href="https://github.com/pytorch/executorch/blob/main/examples/models/voxtral/README.md" style="color: var(--primary); text-decoration: none;">Voxtral</a> (audio-language)</li>
939-
<li><strong class="highlight">Vision/Speech:</strong> <a href="https://github.com/meta-pytorch/executorch-examples/tree/main/mv2" style="color: var(--primary); text-decoration: none;">MobileNetV2</a>, <a href="https://github.com/meta-pytorch/executorch-examples/tree/main/dl3" style="color: var(--primary); text-decoration: none;">DeepLabV3</a>, <a href="https://github.com/meta-pytorch/executorch-examples/tree/main/whisper/android/WhisperApp" style="color: var(--primary); text-decoration: none;">Whisper</a></li>
939+
<li><strong class="highlight">Vision/Speech:</strong> <a href="https://github.com/meta-pytorch/executorch-examples/tree/main/mv2" style="color: var(--primary); text-decoration: none;">MobileNetV2</a>, <a href="https://github.com/meta-pytorch/executorch-examples/tree/main/dl3" style="color: var(--primary); text-decoration: none;">DeepLabV3</a>, <a href="https://github.com/pytorch/executorch/blob/main/examples/models/whisper/README.md" style="color: var(--primary); text-decoration: none;">Whisper</a></li> <!-- @lint-ignore -->
940940
</ul>
941941
</div>
942942

0 commit comments

Comments
 (0)