README.md

Demos {#ovms_docs_demos}

---
maxdepth: 1
hidden:
---
ovms_demos_continuous_batching
ovms_demos_integration_with_open_webui
ovms_demos_code_completion_vsc
ovms_demos_audio
ovms_demos_rerank
ovms_demos_embeddings
ovms_demos_continuous_batching_vlm
ovms_demos_image_generation
ovms_demo_clip_image_classification
ovms_demo_age_gender_guide
ovms_demo_horizontal_text_detection
ovms_demo_optical_character_recognition
ovms_demo_face_detection
ovms_demo_face_blur_pipeline
ovms_demo_capi_inference_demo
ovms_demo_single_face_analysis_pipeline
ovms_demo_multi_faces_analysis_pipeline
ovms_docs_demo_ensemble
ovms_docs_demo_mediapipe_image_classification
ovms_docs_demo_mediapipe_multi_model
ovms_docs_demo_mediapipe_object_detection
ovms_docs_demo_mediapipe_holistic
ovms_docs_demo_mediapipe_iris
ovms_docs_image_classification
ovms_demo_using_onnx_model
ovms_demo_tf_classification
ovms_demo_person_vehicle_bike_detection
ovms_demo_vehicle_analysis_pipeline
ovms_demo_real_time_stream_analysis
ovms_demo_using_paddlepaddle_model
ovms_demo_bert
ovms_demo_universal-sentence-encoder
ovms_string_output_model_demo
ovms_demos_gguf

OpenVINO Model Server demos have been created to showcase the usage of the model server as well as demonstrate it’s capabilities.

Check Out New Generative AI Demos

Demo	Description
AI Agents with MCP servers and serving language models	OpenAI agents with MCP servers and serving LLM models
Integration with Open WebUI	Using OpenWeb UI with OVMS as inference provider. Shows text and image generation as well as usage with RAG and tools
LLM Text Generation with continuous batching	Generate text with LLM models and continuous batching pipeline
VLM Text Generation with continuous batching	Generate text with VLM models and continuous batching pipeline
OpenAI API text embeddings	Get text embeddings via endpoint compatible with OpenAI API
Reranking with Cohere API	Rerank documents via endpoint compatible with Cohere
RAG with OpenAI API endpoint and langchain	Example how to use RAG with model server endpoints
LLM on NPU	Generate text with LLM models and NPU acceleration
VLM on NPU	Generate text with VLM models and NPU acceleration
Long context LLMs	Recommendations for handling very long context in LLM models
Visual Studio Code assistant	Use Continue extension to Visual Studio Code with local OVMS serving
Image Generation	Generate images
GGUF models support	Serve GGUF models with OVMS

Check out the list below to see complete step-by-step examples of using OpenVINO Model Server with real world use cases:

With Traditional Models

Demo	Description
Image Classification	Run prediction on a JPEG image using image classification model via gRPC API.
Using ONNX Model	Run prediction on a JPEG image using image classification ONNX model via gRPC API in two preprocessing variants. This demo uses pipeline with image_transformation custom node.
Using TensorFlow Model	Run image classification using directly imported TensorFlow model.
Age gender recognition	Run prediction on a JPEG image using age gender recognition model via gRPC API.
Face Detection	Run prediction on a JPEG image using face detection model via gRPC API.
Classification with PaddlePaddle	Perform classification on an image with a PaddlePaddle model.
Natural Language Processing with BERT	Provide a knowledge source and a query and use BERT model for question answering use case via gRPC API. This demo uses dynamic shape feature.
Using inputs data in string format with universal-sentence-encoder model	Handling AI model with text as the model input.
Person, Vehicle, Bike Detection	Run prediction on a video file or camera stream using person, vehicle, bike detection model via gRPC API.
Benchmark App	Generate traffic and measure performance of the model served in OpenVINO Model Server.

With Python Nodes

Demo	Description
CLIP image classification	Classify image according to provided labels using CLIP model embedded in a multi-node MediaPipe graph.

With MediaPipe Graphs

Demo	Description
Real Time Stream Analysis	Analyze RTSP video stream in real time with generic application template for custom pre and post processing routines as well as simple results visualizer for displaying predictions in the browser.
Image classification	Basic example with a single inference node.
Chain of models	A chain of models in a graph.
Object detection	A pipeline implementing object detection
Iris demo	A pipeline implementing iris detection
Holistic demo	A complex pipeline linking several image analytical models and image transformations

With DAG Pipelines

Demo	Description
Horizontal Text Detection in Real-Time	Run prediction on camera stream using a horizontal text detection model via gRPC API. This demo uses pipeline with horizontal_ocr custom node and demultiplexer.
Optical Character Recognition Pipeline	Run prediction on a JPEG image using a pipeline of text recognition and text detection models with a custom node for intermediate results processing via gRPC API. This demo uses pipeline with east_ocr custom node and demultiplexer.
Single Face Analysis Pipeline	Run prediction on a JPEG image using a simple pipeline of age-gender recognition and emotion recognition models via gRPC API to analyze image with a single face. This demo uses pipeline
Multi Faces Analysis Pipeline	Run prediction on a JPEG image using a pipeline of age-gender recognition and emotion recognition models via gRPC API to extract multiple faces from the image and analyze all of them. This demo uses pipeline with model_zoo_intel_object_detection custom node and demultiplexer
Model Ensemble Pipeline	Combine multiple image classification models into one pipeline and aggregate results to improve classification accuracy.
Face Blur Pipeline	Detect faces and blur image using a pipeline of object detection models with a custom node for intermediate results processing via gRPC API. This demo uses pipeline with face_blur custom node.
Vehicle Analysis Pipeline	Detect vehicles and recognize their attributes using a pipeline of vehicle detection and vehicle attributes recognition models with a custom node for intermediate results processing via gRPC API. This demo uses pipeline with model_zoo_intel_object_detection custom node.

With C++ Client

Demo	Description
C API applications	How to use C API from the OpenVINO Model Server to create C and C++ application.

With Go Client

Demo	Description
Image Classification	Run prediction on a JPEG image using image classification model via gRPC API.