diff --git a/README.md b/README.md index 731557cb2b..c25849bf60 100644 --- a/README.md +++ b/README.md @@ -27,11 +27,6 @@ --> [![License](https://img.shields.io/badge/License-BSD3-lightgrey.svg)](https://opensource.org/licenses/BSD-3-Clause) ->[!WARNING] ->You are currently on the `main` branch which tracks under-development progress ->towards the next release. The current release is version [2.59.1](https://github.com/triton-inference-server/server/releases/latest) ->and corresponds to the 25.07 container release on NVIDIA GPU Cloud (NGC). - # Triton Inference Server Triton Inference Server is an open source inference serving software that @@ -61,7 +56,7 @@ Major features include: - Provides [Backend API](https://github.com/triton-inference-server/backend) that allows adding custom backends and pre/post processing operations - Supports writing custom backends in python, a.k.a. - [Python-based backends.](https://github.com/triton-inference-server/backend/blob/main/docs/python_based_backends.md#python-based-backends) + [Python-based backends.](https://github.com/triton-inference-server/backend/blob/r25.08/docs/python_based_backends.md#python-based-backends) - Model pipelines using [Ensembling](docs/user_guide/architecture.md#ensemble-models) or [Business Logic Scripting @@ -172,10 +167,10 @@ configuration](docs/user_guide/model_configuration.md) for the model. [Python](https://github.com/triton-inference-server/python_backend), and more - Not all the above backends are supported on every platform supported by Triton. Look at the - [Backend-Platform Support Matrix](https://github.com/triton-inference-server/backend/blob/main/docs/backend_platform_support_matrix.md) + [Backend-Platform Support Matrix](https://github.com/triton-inference-server/backend/blob/r25.08/docs/backend_platform_support_matrix.md) to learn which backends are supported on your target platform. - Learn how to [optimize performance](docs/user_guide/optimization.md) using the - [Performance Analyzer](https://github.com/triton-inference-server/perf_analyzer/blob/main/README.md) + [Performance Analyzer](https://github.com/triton-inference-server/perf_analyzer/blob/r25.08/README.md) and [Model Analyzer](https://github.com/triton-inference-server/model_analyzer) - Learn how to [manage loading and unloading models](docs/user_guide/model_management.md) in @@ -189,14 +184,14 @@ A Triton *client* application sends inference and other requests to Triton. The [Python and C++ client libraries](https://github.com/triton-inference-server/client) provide APIs to simplify this communication. -- Review client examples for [C++](https://github.com/triton-inference-server/client/blob/main/src/c%2B%2B/examples), - [Python](https://github.com/triton-inference-server/client/blob/main/src/python/examples), - and [Java](https://github.com/triton-inference-server/client/blob/main/src/java/src/main/java/triton/client/examples) +- Review client examples for [C++](https://github.com/triton-inference-server/client/blob/r25.08/src/c%2B%2B/examples), + [Python](https://github.com/triton-inference-server/client/blob/r25.08/src/python/examples), + and [Java](https://github.com/triton-inference-server/client/blob/r25.08/src/java/src/main/java/triton/client/examples) - Configure [HTTP](https://github.com/triton-inference-server/client#http-options) and [gRPC](https://github.com/triton-inference-server/client#grpc-options) client options - Send input data (e.g. a jpeg image) directly to Triton in the [body of an HTTP - request without any additional metadata](https://github.com/triton-inference-server/server/blob/main/docs/protocol/extension_binary_data.md#raw-binary-request) + request without any additional metadata](https://github.com/triton-inference-server/server/blob/r25.08/docs/protocol/extension_binary_data.md#raw-binary-request) ### Extend Triton @@ -205,7 +200,7 @@ designed for modularity and flexibility - [Customize Triton Inference Server container](docs/customization_guide/compose.md) for your use case - [Create custom backends](https://github.com/triton-inference-server/backend) - in either [C/C++](https://github.com/triton-inference-server/backend/blob/main/README.md#triton-backend-api) + in either [C/C++](https://github.com/triton-inference-server/backend/blob/r25.08/README.md#triton-backend-api) or [Python](https://github.com/triton-inference-server/python_backend) - Create [decoupled backends and models](docs/user_guide/decoupled_models.md) that can send multiple responses for a request or not send any responses for a request @@ -214,7 +209,7 @@ designed for modularity and flexibility decryption, or conversion - Deploy Triton on [Jetson and JetPack](docs/user_guide/jetson.md) - [Use Triton on AWS - Inferentia](https://github.com/triton-inference-server/python_backend/tree/main/inferentia) + Inferentia](https://github.com/triton-inference-server/python_backend/tree/r25.08/inferentia) ### Additional Documentation diff --git a/docs/introduction/compatibility.md b/docs/introduction/compatibility.md index 450944756d..9ada3d122e 100644 --- a/docs/introduction/compatibility.md +++ b/docs/introduction/compatibility.md @@ -38,6 +38,7 @@ | Triton release version | NGC Tag | Python version | Torch version | TensorRT version | TensorRT-LLM version | CUDA version | CUDA Driver version | Size | | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 25.08 | nvcr.io/nvidia/tritonserver:25.08-trtllm-python-py3 | Python 3.12.3 | 2.8.0a0+5228986c39.nv25.5 | 10.11.0.33 | 0.21.0 | 12.9.0.043 | 575.51.03 | 20.49 GB | | 25.07 | nvcr.io/nvidia/tritonserver:25.07-trtllm-python-py3 | Python 3.12.3 | 2.7.0a0+79aa17489c.nv25.4 | 10.10.0.31 | 0.20.0 | 12.9.0.036 | 575.51.03 | 18.3G | | 25.06 | nvcr.io/nvidia/tritonserver:25.06-trtllm-python-py3 | Python 3.12.3 | 2.7.0a0+79aa17489c.nv25.4 | 10.10.0.31 | 0.20.0 | 12.9.0.036 | 575.51.03 | 18.3G | | 25.05 | nvcr.io/nvidia/tritonserver:25.05-trtllm-python-py3 | Python 3.12.3 | 2.7.0a0+7c8ec84dab.nv25.3 | 10.9.0.34 | 0.19.0 | 12.8.1.012 | 570.124.06 | 17G | @@ -59,7 +60,10 @@ | Triton release version | NGC Tag | Python version | vLLM version | CUDA version | CUDA Driver version | Size | | --- | --- | --- | --- | --- | --- | --- | -| 25.05 | nvcr.io/nvidia/tritonserver:25.04-vllm-python-py3 | Python 3.12.3 | 0.8.4+c4369543.nv25.5.cu129 | 12.9.0.043 | 575.51.03 | 10G | +| 25.08 | nvcr.io/nvidia/tritonserver:25.08-vllm-python-py3 | Python 3.12.3 | 0.9.2+4ef1e343.nv25.8.post1.cu130 | 12.9.0.043 | 575.51.03 | 8.1G | +| 25.07 | nvcr.io/nvidia/tritonserver:25.07-vllm-python-py3 | Python 3.12.3 | 0.9.0rc1+1958ee56.nv25.6.cu129 | 12.9.0.043 | 575.51.03 | 10G | +| 25.06 | nvcr.io/nvidia/tritonserver:25.06-vllm-python-py3 | Python 3.12.3 | 0.9.0rc1+1958ee56.nv25.6.cu129 | 12.9.0.043 | 575.51.03 | 10G | +| 25.05 | nvcr.io/nvidia/tritonserver:25.05-vllm-python-py3 | Python 3.12.3 | 0.8.4+dc1a3e10.nv25.5.cu129 | 12.9.0.043 | 575.51.03 | 10G | | 25.04 | nvcr.io/nvidia/tritonserver:25.04-vllm-python-py3 | Python 3.12.3 | 0.8.1+5f4af9e0.nv25.4.cu129 | 12.9.0.036 | 575.51.02 | 10G | | 25.03 | nvcr.io/nvidia/tritonserver:25.03-vllm-python-py3 | Python 3.12.3 | 0.7.3+04de634a.nv25.3.cu128 | 12.8.1.012 | 570.124.06 | 22G | | 25.02 | nvcr.io/nvidia/tritonserver:25.02-vllm-python-py3 | Python 3.12.3 | 0.7.0+5e800e3d.nv25.2.cu128 | 12.8.0.038 | 570.86.10 | 22G | @@ -78,6 +82,9 @@ | Triton release version | ONNX Runtime | | --- | --- | +| 25.08 | 1.23.0+1d1712fdaf | +| 25.07 | 1.22.0 | +| 25.06 | 1.22.0 | | 25.05 | 1.22.0 | | 25.04 | 1.21.0 | | 25.03 | 1.21.0 |