From d8903307a872b482c9b21ff495fb272fae20095d Mon Sep 17 00:00:00 2001 From: Misha Chornyi Date: Mon, 25 Aug 2025 12:16:25 -0700 Subject: [PATCH 1/4] TPRD-1606: Update versions in README.md --- README.md | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 731557cb2b..4ec498bc64 100644 --- a/README.md +++ b/README.md @@ -27,17 +27,12 @@ --> [![License](https://img.shields.io/badge/License-BSD3-lightgrey.svg)](https://opensource.org/licenses/BSD-3-Clause) ->[!WARNING] ->You are currently on the `main` branch which tracks under-development progress ->towards the next release. The current release is version [2.59.1](https://github.com/triton-inference-server/server/releases/latest) ->and corresponds to the 25.07 container release on NVIDIA GPU Cloud (NGC). - # Triton Inference Server Triton Inference Server is an open source inference serving software that streamlines AI inferencing. Triton enables teams to deploy any AI model from multiple deep learning and machine learning frameworks, including TensorRT, -PyTorch, ONNX, OpenVINO, Python, RAPIDS FIL, and more. Triton +TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS FIL, and more. Triton Inference Server supports inference across cloud, data center, edge and embedded devices on NVIDIA GPUs, x86 and ARM CPU, or AWS Inferentia. Triton Inference Server delivers optimized performance for many query types, including real time, @@ -61,7 +56,7 @@ Major features include: - Provides [Backend API](https://github.com/triton-inference-server/backend) that allows adding custom backends and pre/post processing operations - Supports writing custom backends in python, a.k.a. - [Python-based backends.](https://github.com/triton-inference-server/backend/blob/main/docs/python_based_backends.md#python-based-backends) + [Python-based backends.](https://github.com/triton-inference-server/backend/blob/r25.08/docs/python_based_backends.md#python-based-backends) - Model pipelines using [Ensembling](docs/user_guide/architecture.md#ensemble-models) or [Business Logic Scripting @@ -166,16 +161,17 @@ configuration](docs/user_guide/model_configuration.md) for the model. - Triton supports multiple execution engines, called [backends](https://github.com/triton-inference-server/backend#where-can-i-find-all-the-backends-that-are-available-for-triton), including [TensorRT](https://github.com/triton-inference-server/tensorrt_backend), + [TensorFlow](https://github.com/triton-inference-server/tensorflow_backend), [PyTorch](https://github.com/triton-inference-server/pytorch_backend), [ONNX](https://github.com/triton-inference-server/onnxruntime_backend), [OpenVINO](https://github.com/triton-inference-server/openvino_backend), [Python](https://github.com/triton-inference-server/python_backend), and more - Not all the above backends are supported on every platform supported by Triton. Look at the - [Backend-Platform Support Matrix](https://github.com/triton-inference-server/backend/blob/main/docs/backend_platform_support_matrix.md) + [Backend-Platform Support Matrix](https://github.com/triton-inference-server/backend/blob/r25.08/docs/backend_platform_support_matrix.md) to learn which backends are supported on your target platform. - Learn how to [optimize performance](docs/user_guide/optimization.md) using the - [Performance Analyzer](https://github.com/triton-inference-server/perf_analyzer/blob/main/README.md) + [Performance Analyzer](https://github.com/triton-inference-server/perf_analyzer/blob/r25.08/README.md) and [Model Analyzer](https://github.com/triton-inference-server/model_analyzer) - Learn how to [manage loading and unloading models](docs/user_guide/model_management.md) in @@ -189,14 +185,14 @@ A Triton *client* application sends inference and other requests to Triton. The [Python and C++ client libraries](https://github.com/triton-inference-server/client) provide APIs to simplify this communication. -- Review client examples for [C++](https://github.com/triton-inference-server/client/blob/main/src/c%2B%2B/examples), - [Python](https://github.com/triton-inference-server/client/blob/main/src/python/examples), - and [Java](https://github.com/triton-inference-server/client/blob/main/src/java/src/main/java/triton/client/examples) +- Review client examples for [C++](https://github.com/triton-inference-server/client/blob/r25.08/src/c%2B%2B/examples), + [Python](https://github.com/triton-inference-server/client/blob/r25.08/src/python/examples), + and [Java](https://github.com/triton-inference-server/client/blob/r25.08/src/java/src/main/java/triton/client/examples) - Configure [HTTP](https://github.com/triton-inference-server/client#http-options) and [gRPC](https://github.com/triton-inference-server/client#grpc-options) client options - Send input data (e.g. a jpeg image) directly to Triton in the [body of an HTTP - request without any additional metadata](https://github.com/triton-inference-server/server/blob/main/docs/protocol/extension_binary_data.md#raw-binary-request) + request without any additional metadata](https://github.com/triton-inference-server/server/blob/r25.08/docs/protocol/extension_binary_data.md#raw-binary-request) ### Extend Triton @@ -205,7 +201,7 @@ designed for modularity and flexibility - [Customize Triton Inference Server container](docs/customization_guide/compose.md) for your use case - [Create custom backends](https://github.com/triton-inference-server/backend) - in either [C/C++](https://github.com/triton-inference-server/backend/blob/main/README.md#triton-backend-api) + in either [C/C++](https://github.com/triton-inference-server/backend/blob/r25.08/README.md#triton-backend-api) or [Python](https://github.com/triton-inference-server/python_backend) - Create [decoupled backends and models](docs/user_guide/decoupled_models.md) that can send multiple responses for a request or not send any responses for a request @@ -214,7 +210,7 @@ designed for modularity and flexibility decryption, or conversion - Deploy Triton on [Jetson and JetPack](docs/user_guide/jetson.md) - [Use Triton on AWS - Inferentia](https://github.com/triton-inference-server/python_backend/tree/main/inferentia) + Inferentia](https://github.com/triton-inference-server/python_backend/tree/r25.08/inferentia) ### Additional Documentation From b531e6b31984b19f469e8d7b1a4714ebb5dc3cd7 Mon Sep 17 00:00:00 2001 From: Misha Chornyi Date: Mon, 25 Aug 2025 15:17:22 -0700 Subject: [PATCH 2/4] TPRD-1609: Update compatibility.md --- docs/introduction/compatibility.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/introduction/compatibility.md b/docs/introduction/compatibility.md index 450944756d..9ada3d122e 100644 --- a/docs/introduction/compatibility.md +++ b/docs/introduction/compatibility.md @@ -38,6 +38,7 @@ | Triton release version | NGC Tag | Python version | Torch version | TensorRT version | TensorRT-LLM version | CUDA version | CUDA Driver version | Size | | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 25.08 | nvcr.io/nvidia/tritonserver:25.08-trtllm-python-py3 | Python 3.12.3 | 2.8.0a0+5228986c39.nv25.5 | 10.11.0.33 | 0.21.0 | 12.9.0.043 | 575.51.03 | 20.49 GB | | 25.07 | nvcr.io/nvidia/tritonserver:25.07-trtllm-python-py3 | Python 3.12.3 | 2.7.0a0+79aa17489c.nv25.4 | 10.10.0.31 | 0.20.0 | 12.9.0.036 | 575.51.03 | 18.3G | | 25.06 | nvcr.io/nvidia/tritonserver:25.06-trtllm-python-py3 | Python 3.12.3 | 2.7.0a0+79aa17489c.nv25.4 | 10.10.0.31 | 0.20.0 | 12.9.0.036 | 575.51.03 | 18.3G | | 25.05 | nvcr.io/nvidia/tritonserver:25.05-trtllm-python-py3 | Python 3.12.3 | 2.7.0a0+7c8ec84dab.nv25.3 | 10.9.0.34 | 0.19.0 | 12.8.1.012 | 570.124.06 | 17G | @@ -59,7 +60,10 @@ | Triton release version | NGC Tag | Python version | vLLM version | CUDA version | CUDA Driver version | Size | | --- | --- | --- | --- | --- | --- | --- | -| 25.05 | nvcr.io/nvidia/tritonserver:25.04-vllm-python-py3 | Python 3.12.3 | 0.8.4+c4369543.nv25.5.cu129 | 12.9.0.043 | 575.51.03 | 10G | +| 25.08 | nvcr.io/nvidia/tritonserver:25.08-vllm-python-py3 | Python 3.12.3 | 0.9.2+4ef1e343.nv25.8.post1.cu130 | 12.9.0.043 | 575.51.03 | 8.1G | +| 25.07 | nvcr.io/nvidia/tritonserver:25.07-vllm-python-py3 | Python 3.12.3 | 0.9.0rc1+1958ee56.nv25.6.cu129 | 12.9.0.043 | 575.51.03 | 10G | +| 25.06 | nvcr.io/nvidia/tritonserver:25.06-vllm-python-py3 | Python 3.12.3 | 0.9.0rc1+1958ee56.nv25.6.cu129 | 12.9.0.043 | 575.51.03 | 10G | +| 25.05 | nvcr.io/nvidia/tritonserver:25.05-vllm-python-py3 | Python 3.12.3 | 0.8.4+dc1a3e10.nv25.5.cu129 | 12.9.0.043 | 575.51.03 | 10G | | 25.04 | nvcr.io/nvidia/tritonserver:25.04-vllm-python-py3 | Python 3.12.3 | 0.8.1+5f4af9e0.nv25.4.cu129 | 12.9.0.036 | 575.51.02 | 10G | | 25.03 | nvcr.io/nvidia/tritonserver:25.03-vllm-python-py3 | Python 3.12.3 | 0.7.3+04de634a.nv25.3.cu128 | 12.8.1.012 | 570.124.06 | 22G | | 25.02 | nvcr.io/nvidia/tritonserver:25.02-vllm-python-py3 | Python 3.12.3 | 0.7.0+5e800e3d.nv25.2.cu128 | 12.8.0.038 | 570.86.10 | 22G | @@ -78,6 +82,9 @@ | Triton release version | ONNX Runtime | | --- | --- | +| 25.08 | 1.23.0+1d1712fdaf | +| 25.07 | 1.22.0 | +| 25.06 | 1.22.0 | | 25.05 | 1.22.0 | | 25.04 | 1.21.0 | | 25.03 | 1.21.0 | From 9282d7d5509364ee416f3c8a8e55a3d67155bedc Mon Sep 17 00:00:00 2001 From: Misha Chornyi Date: Mon, 25 Aug 2025 15:46:19 -0700 Subject: [PATCH 3/4] TPRD-1606: Update versions in README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 4ec498bc64..751c19a191 100644 --- a/README.md +++ b/README.md @@ -210,7 +210,7 @@ designed for modularity and flexibility decryption, or conversion - Deploy Triton on [Jetson and JetPack](docs/user_guide/jetson.md) - [Use Triton on AWS - Inferentia](https://github.com/triton-inference-server/python_backend/tree/r25.08/inferentia) + Inferentia](https://github.com/triton-inference-server/python_backend/tree/main/inferentia) ### Additional Documentation From 2ce1e1656590de61cc79e0644d6b607b9f349de0 Mon Sep 17 00:00:00 2001 From: Misha Chornyi Date: Mon, 25 Aug 2025 15:47:47 -0700 Subject: [PATCH 4/4] TPRD-1606: README.md Remove TensorFlow --- README.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 751c19a191..c25849bf60 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ Triton Inference Server is an open source inference serving software that streamlines AI inferencing. Triton enables teams to deploy any AI model from multiple deep learning and machine learning frameworks, including TensorRT, -TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS FIL, and more. Triton +PyTorch, ONNX, OpenVINO, Python, RAPIDS FIL, and more. Triton Inference Server supports inference across cloud, data center, edge and embedded devices on NVIDIA GPUs, x86 and ARM CPU, or AWS Inferentia. Triton Inference Server delivers optimized performance for many query types, including real time, @@ -161,7 +161,6 @@ configuration](docs/user_guide/model_configuration.md) for the model. - Triton supports multiple execution engines, called [backends](https://github.com/triton-inference-server/backend#where-can-i-find-all-the-backends-that-are-available-for-triton), including [TensorRT](https://github.com/triton-inference-server/tensorrt_backend), - [TensorFlow](https://github.com/triton-inference-server/tensorflow_backend), [PyTorch](https://github.com/triton-inference-server/pytorch_backend), [ONNX](https://github.com/triton-inference-server/onnxruntime_backend), [OpenVINO](https://github.com/triton-inference-server/openvino_backend), @@ -210,7 +209,7 @@ designed for modularity and flexibility decryption, or conversion - Deploy Triton on [Jetson and JetPack](docs/user_guide/jetson.md) - [Use Triton on AWS - Inferentia](https://github.com/triton-inference-server/python_backend/tree/main/inferentia) + Inferentia](https://github.com/triton-inference-server/python_backend/tree/r25.08/inferentia) ### Additional Documentation