⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
-
Updated
Oct 22, 2026 - Swift
#
apple-sili
Here are 2 public repositories matching this topic...
Optimize LLM inference with near-optimal 4-bit weight quantization and on-the-fly dequantization for lower memory use and faster matmul
desktop-app python compression ai metal cuda pytorch transformer attention multi-model mlx rocm inference-optimization huggingface on-device-ai kv-cache llm vllm quansloth apple-sili
-
Updated
Oct 14, 2026 - Python
Add this topic to your repo
To associate your repository with the apple-sili topic, visit your repo's landing page and select "manage topics."