README.md

Demos {#ovms_docs_demos}

---
maxdepth: 1
hidden:
---

Text generation <ovms_text_generation>
Image generation <ovms_demos_image_generation>
Audio <ovms_demos_audio>
Text Embeddings <ovms_demos_embeddings>
Text Reranking <ovms_demos_rerank>
Classic models <ovms_demos_classic_models>
MediaPipe <ovms_demos_mediapipe>
Python Node <ovms_demos_python_node>
Integrations <ovms_demos_integrations>

Text Generation

Demo	Description
LLM Text Generation	Generate text with LLM models and continuous batching pipeline.
VLM Text Generation	Generate text with VLM models and continuous batching pipeline.
AI Agents with MCP servers	OpenAI agents with MCP servers and serving LLM models.
RAG with OpenAI API endpoint and langchain	Example how to use RAG with model server endpoints.
Long context LLMs	Recommendations for handling very long context in LLM models.
Structured output	Generate structured (JSON) output from LLM models.
Speculative decoding	Speed up LLM inference with speculative decoding.
LLM on NPU	Generate text with LLM models and NPU acceleration.
Scaling on multi CPU and GPU	Scale LLM serving across multiple CPUs and GPUs.
Loading models in GGUF	Serve GGUF models with OVMS.

Image Generation

Demo	Description
Image Generation	Generate images with diffusion models.

Audio

Demo	Description
Audio demos	Text-to-speech and automatic speech recognition demos.

Text Embeddings

Demo	Description
OpenAI API text embeddings	Get text embeddings via endpoint compatible with OpenAI API.

Text Reranking

Demo	Description
Reranking with Cohere API	Rerank documents via endpoint compatible with Cohere.

Classic Models

Demo	Description
Image Classification	Run prediction on a JPEG image using image classification model via gRPC API.
Using ONNX Model	Run prediction on a JPEG image using image classification ONNX model via gRPC API in two preprocessing variants. This demo uses pipeline with image_transformation custom node.
Using TensorFlow Model	Run image classification using directly imported TensorFlow model.
Classification with PaddlePaddle	Perform classification on an image with a PaddlePaddle model.
Age gender recognition	Run prediction on a JPEG image using age gender recognition model via gRPC API.
Face Detection	Run prediction on a JPEG image using face detection model via gRPC API.
Person, Vehicle, Bike Detection	Run prediction on a video file or camera stream using person, vehicle, bike detection model via gRPC API.
Using input strings	Handling AI model with text as the model input.
Using output strings	Handling AI model with string output.
Natural Language Processing with BERT	Provide a knowledge source and a query and use BERT model for question answering via gRPC API. This demo uses dynamic shape feature.
Benchmark App	Generate traffic and measure performance of the model served in OpenVINO Model Server.

MediaPipe

Demo	Description
Object Detection	A pipeline implementing object detection.
Iris	A pipeline implementing iris detection.
Holistic	A complex pipeline linking several image analytical models and image transformations.
Realtime Stream Analysis	Analyze RTSP video stream in real time with generic application template for custom pre and post processing routines.
Image classification	Basic example with a single inference node.
Chain of models	A chain of models in a graph.
CLIP image classification	Classify image according to provided labels using CLIP model embedded in a multi-node MediaPipe graph.

Python Node

Demo	Description
OpenClip with python execution	A pipeline implementing OpenClip classification in Python Node.

Integrations

Demo	Description
Integration with Open WebUI	Using Open WebUI with OVMS as inference provider. Shows text and image generation as well as usage with RAG and tools.
Visual Studio Code assistant	Use Continue or Cline extension to Visual Studio Code with local OVMS serving.