weight-streaming

Here are 4 public repositories matching this topic...

matt-k-wong / mlx-flash

Flash weight streaming for MLX: run massive models larger than your RAM on Apple Silicon.

macos machine-learning metal optimization mlx memory-optimization apple-silicon large-language-models llm llm-inference lm-studio weight-streaming

Updated Jun 13, 2026
Python

infinition / vortex

Sponsor

Star

An LLM inference engine written in pure Rust, designed to run large models on hardware that would normally refuse them.

rust deep-learning mmap io-uring large-language-models llm-inference sparse-routing weight-streaming low-memory-inference activation-bloom-filters

Updated Aug 26, 2026
Rust

miguel2180 / mlx-flash

Star

Run large MLX models on Apple Silicon with flash weight streaming, using native precision beyond RAM limits

menubar clustering optimization transformer quantization speculative-execution swiftui bpe memory-optimization apple-silicon bpe-encoder llm-training ssd-streaming weight-streaming expert-caching

Updated Sep 1, 2026
Python

i-mrDed / weight-streaming

Star

Run LLMs larger than your RAM — out-of-core local inference on consumer hardware (MoE, GGUF, llama.cpp). NVMe-as-memory with honest telemetry: real tok/s, page faults, disk traffic.

python benchmarking inference self-hosted moe nvme out-of-core memory-mapping llm llama-cpp local-llm qwen gguf deepseek weight-streaming

Updated Aug 20, 2026
Python

Add this topic to your repo

To associate your repository with the weight-streaming topic, visit your repo's landing page and select "manage topics."

Learn more