Skip to content

@llm-d llm-d

All

    Repositories list

    22 repositories

    • llm-d-go-template

      Public template
      Go microservice template for llm-d repos. Use 'Use this template' to create a new Go project with standard CI, linting, Prow, and governance.
      Makefile
      •
      Apache License 2.0
      •8•0•1•6•Updated Sep 2, 2026Sep 2, 2026

    • llm-d-prism

      Public
      Performance analysis for distributed inference systems
      JavaScript
      •
      Apache License 2.0
      •22•14•12•0•Updated Sep 2, 2026Sep 2, 2026

    • llm-d-inference-sim

      Public
      A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.
      incubating
      incubating
      Go
      •
      Apache License 2.0
      •134•193•8•3•Updated Sep 2, 2026Sep 2, 2026

    • llm-d-router

      Public
      llm-d Router: The intelligent entry point for inference requests
      kubernetesainetworking
      kubernetesainetworkinginferencegateway-api
      Go
      •
      Apache License 2.0
      •351•315•234•128•Updated Sep 2, 2026Sep 2, 2026

    • llm-d-kv-cache

      Public
      Distributed KV cache scheduling & offloading libraries
      aiincubating
      aiincubating
      Go
      •
      Apache License 2.0
      •156•175•26•24•Updated Sep 2, 2026Sep 2, 2026

    • llm-d-batch-gateway

      Public
      llm-d-batch-gateway is a standalone, backend-agnostic, OpenAI-compatible batch API and processing engine
      Go
      •
      Apache License 2.0
      •47•18•19•7•Updated Sep 2, 2026Sep 2, 2026

    • llm-d

      Public
      Achieve state of the art inference performance with modern accelerators on Kubernetes
      kubernetesaigpu
      kubernetesaigpucncfinferencemodel-serverllmdistributed-inferenceintelligent-routing
      Shell
      •
      Apache License 2.0
      •740•4.4k•124•97•Updated Sep 2, 2026Sep 2, 2026

    • llm-d-async

      Public
      Asynchronous Processor for Inference Gateway. Orchestrator of queues
      Go
      •
      Apache License 2.0
      •34•12•20•6•Updated Sep 1, 2026Sep 1, 2026

    • llm-d.github.io

      Public
      Website for llm-d: This repository builds the website seen at llm-d.ai
      documentationopensourceai
      documentationopensourceaidocusaurus
      MDX
      •
      Apache License 2.0
      •67•11•9•8•Updated Sep 1, 2026Sep 1, 2026

    • llm-d-benchmark

      Public
      llm-d benchmark scripts and tooling
      incubating
      incubating
      Python
      •
      Apache License 2.0
      •134•66•46•12•Updated Sep 1, 2026Sep 1, 2026

    • llm-d-infra

      Public
      repo for CI and infrastructure required to maintain llm-d org member repos
      Shell
      •
      Apache License 2.0
      •33•2•25•8•Updated Aug 31, 2026Aug 31, 2026

    • llm-d-api-extensions

      Public
      Apache License 2.0
      •1•0•1•1•Updated Aug 31, 2026Aug 31, 2026

    • llm-d-workload-variant-autoscaler

      Public
      Variant optimization autoscaler for distributed inference workloads
      Go
      •
      Apache License 2.0
      •91•57•127•42•Updated Aug 31, 2026Aug 31, 2026

    • llm-d-inference-payload-processor

      Public
      Inference payload processor for llm-d
      Go
      •
      Apache License 2.0
      •44•19•44•14•Updated Aug 28, 2026Aug 28, 2026

    • llm-d-latency-predictor

      Public
      Latency prediction service for ML-model based scoring with llm-d-inference-scheduler
      Python
      •
      Apache License 2.0
      •27•8•11•25•Updated Aug 24, 2026Aug 24, 2026

    • llm-d-pd-utils

      Public
      Python
      •16•5•11•8•Updated Aug 13, 2026Aug 13, 2026

    • .project

      Public
      Project metadata for llm-d - CNCF .project automation
      0•0•1•2•Updated Aug 13, 2026Aug 13, 2026

    • llm-d-python-template

      Public template
      Python project template for llm-d repos. Use 'Use this template' to create a new Python project with standard CI, linting, Prow, and governance.
      Makefile
      •
      Apache License 2.0
      •7•1•1•3•Updated Jul 22, 2026Jul 22, 2026

    • .github

      Public
      8•0•1•2•Updated May 18, 2026May 18, 2026

    • llm-d-routing-sidecar

      Public archive
      Incubating P/D sidecar for llm-d
      incubating
      incubating
      Go
      •
      Apache License 2.0
      •33•17•0•0•Updated Nov 13, 2025Nov 13, 2025

    • llm-d-deployer

      Public archive
      Helm charts for llm-d
      helmhelm-chartllm-d
      helmhelm-chartllm-d
      Shell
      •
      Apache License 2.0
      •57•52•34•17•Updated Jul 22, 2025Jul 22, 2025

    • llm-d-model-service

      Public archive
      Simplified model deployment on llm-d
      incubating
      incubating
      Go
      •
      Apache License 2.0
      •18•28•29•6•Updated Jul 2, 2025Jul 2, 2025
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.