prefill

LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.

A from-scratch Prefill/Decode disaggregation inference engine for LLMs

Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parallel architecture, suitable for MOE model hybrid inference.

Prefills factories with faker method suggestions to increase productivity

LANCache Manager - Self-hosted web dashboard for monitoring Lancache data, Real-time download tracking, bandwidth analytics, cache management, and client monitoring. Docker deployment with Prometheus metrics and Grafana integration.

Multi-GPU prefill acceleration for llama.cpp

Generate pre-filled PO files from POT file, using source PO files in gulp.

NanoPD, a minimalistic implementation of the disaggregated PD serve

Generate pre-filled PO files from POT file, using source PO files in NodeJS.

Measure and visualize why LLM inference is slow: bottleneck analysis, model dissection, KV-cache, GEMM/GEMV, quantization, and memory-bound decoding.

React component to create a button/link to prefill Codepen.io with supplied code.

(WORKS) NSFW/Coding refusals removing local proxy. If ENI is not enough.

Automatically pre-fill course section CRNs or select a template for instant RPI class registration when your time ticket is activated.

Event-driven benchmark of adaptive batch composition policies for LLM serving, measuring how prefill and decode interference affects TTFT, TPOT, and throughput under different memory pressure regimes.

Analyse approfondie de Dual Path (DeepSeek) : architecture double chemin pour charger le cache KV depuis le stockage, mutualisation de la bande passante, +100% de débit d'inférence et -56% sur le premier token sans GPU supplémentaire. Fiche de lecture, mindmap textuelle et Q/R détaillées sur l'optimisation agentique.

Save VLM prefill for VQA tasks by letting a policy select which tokens are important at test-time!

A maintained continuation of celeste's Loom — a recursively-branching LLM interface for Obsidian (AGPL-3.0)

LLM 推理性能决策基线：TTFT/TPOT、KV Cache、吞吐与解码策略对照

Simulates system prompt KV cache reuse across 540 configurations — measuring prefill savings, TTFT reduction, and eviction policy tradeoffs under variable prompt sizes, output lengths, and competing prompt workloads.

Production-ready LiteLLM Custom Callback to fix Claude 4.x prefill compatibility issue on Amazon Bedrock (resolves LiteLLM #22930)