From 31cd28d16f61f324d9bb2e48a878486dc8e3b7e8 Mon Sep 17 00:00:00 2001 From: Yingge He <157551214+yinggeh@users.noreply.github.com> Date: Tue, 13 Jan 2026 10:22:35 -0800 Subject: [PATCH 1/7] docs: Fix rst code block rendering (#8599) --- docs/client_guide/python_readme.rst | 36 +++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/docs/client_guide/python_readme.rst b/docs/client_guide/python_readme.rst index 51951fad51..cbcf5c9b25 100644 --- a/docs/client_guide/python_readme.rst +++ b/docs/client_guide/python_readme.rst @@ -1,5 +1,5 @@ .. -.. Copyright 2024-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +.. Copyright 2024-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. .. .. Redistribution and use in source and binary forms, with or without .. modification, are permitted provided that the following conditions @@ -72,13 +72,17 @@ Clone Repository ^^^^^^^^^^^^^^^^ .. code:: bash + git clone https://github.com/triton-inference-server/tutorials.git cd tutorials/Triton_Inference_Server_Python_API + Build ``triton-python-api:r24.01`` Image ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code:: bash + ./build.sh + Supported Backends ^^^^^^^^^^^^^^^^^^ @@ -110,19 +114,25 @@ The following command starts a container and volume mounts the current directory as ``workspace``. .. code:: bash + ./run.sh + Enter Python Shell ~~~~~~~~~~~~~~~~~~ .. code:: bash + python3 + Create and Start a Server Instance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: python + import tritonserver server = tritonserver.Server(model_repository="/workspace/identity-models") server.start() + List Models ~~~~~~~~~~~ @@ -137,13 +147,17 @@ Example Output their current state. .. code:: python + {('identity', 1): {'name': 'identity', 'version': 1, 'state': 'READY'}} + Send an Inference Request ~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: python + model = server.model("identity") responses = model.infer(inputs={"string_input":[["hello world!"]]}) + Iterate through Responses ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -151,15 +165,19 @@ Iterate through Responses results of an inference request. .. code:: python + for response in responses: print(response.outputs["string_output"].to_string_array()) + .. _example-output-1: Example Output ^^^^^^^^^^^^^^ .. code:: python + [['hello world!']] + Stable Diffusion ---------------- @@ -174,7 +192,9 @@ Please note the following command will take many minutes depending on your hardware configuration and network connection. .. code:: bash - ./build.sh --framework diffusion --build-models + + ./build.sh --framework diffusion --build-models + .. _supported-backends-1: Supported Backends @@ -205,25 +225,31 @@ The following command starts a container and volume mounts the current directory as ``workspace``. .. code:: bash + ./run.sh --framework diffusion + .. _enter-python-shell-1: Enter Python Shell ~~~~~~~~~~~~~~~~~~ .. code:: bash + python3 + .. _create-and-start-a-server-instance-1: Create and Start a Server Instance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: python + import tritonserver import numpy from PIL import Image server = tritonserver.Server(model_repository="/workspace/diffusion-models") server.start() + .. _list-models-1: List Models @@ -239,24 +265,30 @@ Example Output ^^^^^^^^^^^^^^ .. code:: python + {('stable_diffusion', 1): {'name': 'stable_diffusion', 'version': 1, 'state': 'READY'}, ('text_encoder', 1): {'name': 'text_encoder', 'version': 1, 'state': 'READY'}, ('vae', 1): {'name': 'vae', 'version': 1, 'state': 'READY'}} + .. _send-an-inference-request-1: Send an Inference Request ~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: python + model = server.model("stable_diffusion") responses = model.infer(inputs={"prompt":[["butterfly in new york, realistic, 4k, photograph"]]}) + Iterate through Responses and save image ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: python + for response in responses: generated_image = numpy.from_dlpack(response.outputs["generated_image"]) generated_image = generated_image.squeeze().astype(numpy.uint8) image_ = Image.fromarray(generated_image) image_.save("sample_generated_image.jpg") + .. _example-output-3: Example Output From d31b72f756b48a9527da8848c1771f6976d0544e Mon Sep 17 00:00:00 2001 From: Misha Chornyi <99709299+mc-nv@users.noreply.github.com> Date: Wed, 14 Jan 2026 12:03:11 -0800 Subject: [PATCH 2/7] Update build.py versions (#8601) --- build.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/build.py b/build.py index 71791f5745..4fc77e03aa 100755 --- a/build.py +++ b/build.py @@ -73,12 +73,12 @@ DEFAULT_TRITON_VERSION_MAP = { "release_version": "2.66.0dev", "triton_container_version": "26.02dev", - "upstream_container_version": "25.12", + "upstream_container_version": "26.01", "ort_version": "1.23.2", "ort_openvino_version": "2025.4.1", "standalone_openvino_version": "2025.4.1", "dcgm_version": "4.4.2-1", - "vllm_version": "0.11.1", + "vllm_version": "0.13.0", "rhel_py_version": "3.12.3", } From 17b43538a2662c18809d96bb4f91a8947d2cf6bb Mon Sep 17 00:00:00 2001 From: Misha Chornyi <99709299+mc-nv@users.noreply.github.com> Date: Thu, 15 Jan 2026 08:24:43 -0800 Subject: [PATCH 3/7] Update README and versions for 26.01 (#8605) --- Dockerfile.sdk | 4 +-- README.md | 29 ++++++++----------- deploy/aws/values.yaml | 4 +-- deploy/fleetcommand/Chart.yaml | 4 +-- deploy/fleetcommand/values.yaml | 8 ++--- deploy/gcp/values.yaml | 4 +-- .../perf-analyzer-script/triton_client.yaml | 4 +-- .../server-deployer/build_and_push.sh | 8 ++--- .../server-deployer/chart/triton/Chart.yaml | 6 ++-- .../server-deployer/chart/triton/values.yaml | 8 ++--- .../server-deployer/data-test/schema.yaml | 4 +-- .../server-deployer/schema.yaml | 6 ++-- .../gke-marketplace-app/trt-engine/README.md | 8 ++--- deploy/k8s-onprem/values.yaml | 4 +-- deploy/oci/values.yaml | 4 +-- docs/introduction/release_notes.md | 6 ++-- python/openai/README.md | 8 ++--- qa/common/gen_jetson_trt_models | 2 +- qa/common/gen_qa_model_repository | 2 +- 19 files changed, 59 insertions(+), 64 deletions(-) diff --git a/Dockerfile.sdk b/Dockerfile.sdk index 59bdaea509..8041286307 100644 --- a/Dockerfile.sdk +++ b/Dockerfile.sdk @@ -1,4 +1,4 @@ -# Copyright 2019-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# Copyright 2019-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -29,7 +29,7 @@ # # Base image on the minimum Triton container -ARG BASE_IMAGE=nvcr.io/nvidia/tritonserver:25.12-py3-min +ARG BASE_IMAGE=nvcr.io/nvidia/tritonserver:26.01-py3-min ARG TRITON_CLIENT_REPO_SUBDIR=clientrepo ARG TRITON_REPO_ORGANIZATION=http://github.com/triton-inference-server diff --git a/README.md b/README.md index 522927495c..9a3c48ff39 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ [![License](https://img.shields.io/badge/License-BSD3-lightgrey.svg)](https://opensource.org/licenses/BSD-3-Clause) ->[!WARNING] ->You are currently on the `main` branch which tracks under-development progress ->towards the next release. The current release is version [2.64.0](https://github.com/triton-inference-server/server/releases/latest) ->and corresponds to the 25.12 container release on NVIDIA GPU Cloud (NGC). - # Triton Inference Server Triton Inference Server is an open source inference serving software that @@ -61,7 +56,7 @@ Major features include: - Provides [Backend API](https://github.com/triton-inference-server/backend) that allows adding custom backends and pre/post processing operations - Supports writing custom backends in python, a.k.a. - [Python-based backends.](https://github.com/triton-inference-server/backend/blob/main/docs/python_based_backends.md#python-based-backends) + [Python-based backends.](https://github.com/triton-inference-server/backend/blob/r26.01/docs/python_based_backends.md#python-based-backends) - Model pipelines using [Ensembling](docs/user_guide/architecture.md#ensemble-models) or [Business Logic Scripting @@ -90,16 +85,16 @@ Inference Server with the ```bash # Step 1: Create the example model repository -git clone -b r25.12 https://github.com/triton-inference-server/server.git +git clone -b r26.01 https://github.com/triton-inference-server/server.git cd server/docs/examples ./fetch_models.sh # Step 2: Launch triton from the NGC Triton container -docker run --gpus=1 --rm --net=host -v ${PWD}/model_repository:/models nvcr.io/nvidia/tritonserver:25.12-py3 tritonserver --model-repository=/models --model-control-mode explicit --load-model densenet_onnx +docker run --gpus=1 --rm --net=host -v ${PWD}/model_repository:/models nvcr.io/nvidia/tritonserver:26.01-py3 tritonserver --model-repository=/models --model-control-mode explicit --load-model densenet_onnx # Step 3: Sending an Inference Request # In a separate console, launch the image_client example from the NGC Triton SDK container -docker run -it --rm --net=host nvcr.io/nvidia/tritonserver:25.12-py3-sdk /workspace/install/bin/image_client -m densenet_onnx -c 3 -s INCEPTION /workspace/images/mug.jpg +docker run -it --rm --net=host nvcr.io/nvidia/tritonserver:26.01-py3-sdk /workspace/install/bin/image_client -m densenet_onnx -c 3 -s INCEPTION /workspace/images/mug.jpg # Inference should return the following Image '/workspace/images/mug.jpg': @@ -172,10 +167,10 @@ configuration](docs/user_guide/model_configuration.md) for the model. [Python](https://github.com/triton-inference-server/python_backend), and more - Not all the above backends are supported on every platform supported by Triton. Look at the - [Backend-Platform Support Matrix](https://github.com/triton-inference-server/backend/blob/main/docs/backend_platform_support_matrix.md) + [Backend-Platform Support Matrix](https://github.com/triton-inference-server/backend/blob/r26.01/docs/backend_platform_support_matrix.md) to learn which backends are supported on your target platform. - Learn how to [optimize performance](docs/user_guide/optimization.md) using the - [Performance Analyzer](https://github.com/triton-inference-server/perf_analyzer/blob/main/README.md) + [Performance Analyzer](https://github.com/triton-inference-server/perf_analyzer/blob/r26.01/README.md) and [Model Analyzer](https://github.com/triton-inference-server/model_analyzer) - Learn how to [manage loading and unloading models](docs/user_guide/model_management.md) in @@ -189,14 +184,14 @@ A Triton *client* application sends inference and other requests to Triton. The [Python and C++ client libraries](https://github.com/triton-inference-server/client) provide APIs to simplify this communication. -- Review client examples for [C++](https://github.com/triton-inference-server/client/blob/main/src/c%2B%2B/examples), - [Python](https://github.com/triton-inference-server/client/blob/main/src/python/examples), - and [Java](https://github.com/triton-inference-server/client/blob/main/src/java/src/main/java/triton/client/examples) +- Review client examples for [C++](https://github.com/triton-inference-server/client/blob/r26.01/src/c%2B%2B/examples), + [Python](https://github.com/triton-inference-server/client/blob/r26.01/src/python/examples), + and [Java](https://github.com/triton-inference-server/client/blob/r26.01/src/java/src/main/java/triton/client/examples) - Configure [HTTP](https://github.com/triton-inference-server/client#http-options) and [gRPC](https://github.com/triton-inference-server/client#grpc-options) client options - Send input data (e.g. a jpeg image) directly to Triton in the [body of an HTTP - request without any additional metadata](https://github.com/triton-inference-server/server/blob/main/docs/protocol/extension_binary_data.md#raw-binary-request) + request without any additional metadata](https://github.com/triton-inference-server/server/blob/r26.01/docs/protocol/extension_binary_data.md#raw-binary-request) ### Extend Triton @@ -205,7 +200,7 @@ designed for modularity and flexibility - [Customize Triton Inference Server container](docs/customization_guide/compose.md) for your use case - [Create custom backends](https://github.com/triton-inference-server/backend) - in either [C/C++](https://github.com/triton-inference-server/backend/blob/main/README.md#triton-backend-api) + in either [C/C++](https://github.com/triton-inference-server/backend/blob/r26.01/README.md#triton-backend-api) or [Python](https://github.com/triton-inference-server/python_backend) - Create [decoupled backends and models](docs/user_guide/decoupled_models.md) that can send multiple responses for a request or not send any responses for a request diff --git a/deploy/aws/values.yaml b/deploy/aws/values.yaml index 6ccd3f5612..45cda89942 100644 --- a/deploy/aws/values.yaml +++ b/deploy/aws/values.yaml @@ -1,4 +1,4 @@ -# Copyright (c) 2019-2025, NVIDIA CORPORATION. All rights reserved. +# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -27,7 +27,7 @@ replicaCount: 1 image: - imageName: nvcr.io/nvidia/tritonserver:25.12-py3 + imageName: nvcr.io/nvidia/tritonserver:26.01-py3 pullPolicy: IfNotPresent modelRepositoryPath: s3://triton-inference-server-repository/model_repository numGpus: 1 diff --git a/deploy/fleetcommand/Chart.yaml b/deploy/fleetcommand/Chart.yaml index b923491e4d..8760515775 100644 --- a/deploy/fleetcommand/Chart.yaml +++ b/deploy/fleetcommand/Chart.yaml @@ -1,4 +1,4 @@ -# Copyright (c) 2019-2025, NVIDIA CORPORATION. All rights reserved. +# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -26,7 +26,7 @@ apiVersion: v1 # appVersion is the Triton version; update when changing release -appVersion: 2.64.0 +appVersion: 2.65.0 description: Triton Inference Server (Fleet Command) name: triton-inference-server # version is the Chart version; update when changing anything in the chart diff --git a/deploy/fleetcommand/values.yaml b/deploy/fleetcommand/values.yaml index 6cbbb50553..bc6fb0b759 100644 --- a/deploy/fleetcommand/values.yaml +++ b/deploy/fleetcommand/values.yaml @@ -1,4 +1,4 @@ -# Copyright (c) 2019-2025, NVIDIA CORPORATION. All rights reserved. +# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -27,7 +27,7 @@ replicaCount: 1 image: - imageName: nvcr.io/nvidia/tritonserver:25.12-py3 + imageName: nvcr.io/nvidia/tritonserver:26.01-py3 pullPolicy: IfNotPresent numGpus: 1 serverCommand: tritonserver @@ -47,13 +47,13 @@ image: # # To set model control mode, uncomment and configure below # TODO: Fix the following url, it is invalid - # See https://github.com/triton-inference-server/server/blob/r25.12/docs/user_guide/model_management.md + # See https://github.com/triton-inference-server/server/blob/r26.01/docs/user_guide/model_management.md # for more details #- --model-control-mode=explicit|poll|none # # Additional server args # - # see https://github.com/triton-inference-server/server/blob/r25.12/README.md + # see https://github.com/triton-inference-server/server/blob/r26.01/README.md # for more details service: diff --git a/deploy/gcp/values.yaml b/deploy/gcp/values.yaml index 97ceba2606..4612dd4ae6 100644 --- a/deploy/gcp/values.yaml +++ b/deploy/gcp/values.yaml @@ -1,4 +1,4 @@ -# Copyright (c) 2019-2025, NVIDIA CORPORATION. All rights reserved. +# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -27,7 +27,7 @@ replicaCount: 1 image: - imageName: nvcr.io/nvidia/tritonserver:25.12-py3 + imageName: nvcr.io/nvidia/tritonserver:26.01-py3 pullPolicy: IfNotPresent modelRepositoryPath: gs://triton-inference-server-repository/model_repository numGpus: 1 diff --git a/deploy/gke-marketplace-app/benchmark/perf-analyzer-script/triton_client.yaml b/deploy/gke-marketplace-app/benchmark/perf-analyzer-script/triton_client.yaml index 181005d382..3d5abafa3b 100644 --- a/deploy/gke-marketplace-app/benchmark/perf-analyzer-script/triton_client.yaml +++ b/deploy/gke-marketplace-app/benchmark/perf-analyzer-script/triton_client.yaml @@ -1,4 +1,4 @@ -# Copyright 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# Copyright 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -33,7 +33,7 @@ metadata: namespace: default spec: containers: - - image: nvcr.io/nvidia/tritonserver:25.12-py3-sdk + - image: nvcr.io/nvidia/tritonserver:26.01-py3-sdk imagePullPolicy: Always name: nv-triton-client securityContext: diff --git a/deploy/gke-marketplace-app/server-deployer/build_and_push.sh b/deploy/gke-marketplace-app/server-deployer/build_and_push.sh index 22c57b09db..0121d266fb 100755 --- a/deploy/gke-marketplace-app/server-deployer/build_and_push.sh +++ b/deploy/gke-marketplace-app/server-deployer/build_and_push.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Copyright 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# Copyright 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -27,9 +27,9 @@ export REGISTRY=gcr.io/$(gcloud config get-value project | tr ':' '/') export APP_NAME=tritonserver -export MAJOR_VERSION=2.64 -export MINOR_VERSION=2.64.0 -export NGC_VERSION=25.12-py3 +export MAJOR_VERSION=2.65 +export MINOR_VERSION=2.65.0 +export NGC_VERSION=26.01-py3 docker pull nvcr.io/nvidia/$APP_NAME:$NGC_VERSION diff --git a/deploy/gke-marketplace-app/server-deployer/chart/triton/Chart.yaml b/deploy/gke-marketplace-app/server-deployer/chart/triton/Chart.yaml index 0bec3222a7..ddbc69c63c 100644 --- a/deploy/gke-marketplace-app/server-deployer/chart/triton/Chart.yaml +++ b/deploy/gke-marketplace-app/server-deployer/chart/triton/Chart.yaml @@ -1,4 +1,4 @@ -# Copyright (c) 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# Copyright (c) 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -25,7 +25,7 @@ # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. apiVersion: v1 -appVersion: "2.64" +appVersion: "2.65" description: Triton Inference Server name: triton-inference-server -version: 2.64.0 +version: 2.65.0 diff --git a/deploy/gke-marketplace-app/server-deployer/chart/triton/values.yaml b/deploy/gke-marketplace-app/server-deployer/chart/triton/values.yaml index d0d0b87493..15387d650d 100644 --- a/deploy/gke-marketplace-app/server-deployer/chart/triton/values.yaml +++ b/deploy/gke-marketplace-app/server-deployer/chart/triton/values.yaml @@ -1,4 +1,4 @@ -# Copyright (c) 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# Copyright (c) 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -31,14 +31,14 @@ maxReplicaCount: 3 tritonProtocol: HTTP # HPA GPU utilization autoscaling target HPATargetAverageValue: 85 -modelRepositoryPath: gs://triton_sample_models/25.12 -publishedVersion: '2.64.0' +modelRepositoryPath: gs://triton_sample_models/26.01 +publishedVersion: '2.65.0' gcpMarketplace: true image: registry: gcr.io repository: nvidia-ngc-public/tritonserver - tag: 25.12-py3 + tag: 26.01-py3 pullPolicy: IfNotPresent # modify the model repository here to match your GCP storage bucket numGpus: 1 diff --git a/deploy/gke-marketplace-app/server-deployer/data-test/schema.yaml b/deploy/gke-marketplace-app/server-deployer/data-test/schema.yaml index add506981a..c63bcbc2d5 100644 --- a/deploy/gke-marketplace-app/server-deployer/data-test/schema.yaml +++ b/deploy/gke-marketplace-app/server-deployer/data-test/schema.yaml @@ -1,4 +1,4 @@ -# Copyright (c) 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# Copyright (c) 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -27,7 +27,7 @@ x-google-marketplace: schemaVersion: v2 applicationApiVersion: v1beta1 - publishedVersion: '2.64.0' + publishedVersion: '2.65.0' publishedVersionMetadata: releaseNote: >- Initial release. diff --git a/deploy/gke-marketplace-app/server-deployer/schema.yaml b/deploy/gke-marketplace-app/server-deployer/schema.yaml index dc1f75ae7c..f9c31d268a 100644 --- a/deploy/gke-marketplace-app/server-deployer/schema.yaml +++ b/deploy/gke-marketplace-app/server-deployer/schema.yaml @@ -1,4 +1,4 @@ -# Copyright (c) 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# Copyright (c) 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -27,7 +27,7 @@ x-google-marketplace: schemaVersion: v2 applicationApiVersion: v1beta1 - publishedVersion: '2.64.0' + publishedVersion: '2.65.0' publishedVersionMetadata: releaseNote: >- Initial release. @@ -89,7 +89,7 @@ properties: modelRepositoryPath: type: string title: Bucket where models are stored. Please make sure the user/service account to create the GKE app has permission to this GCS bucket. Read Triton documentation on configs and formatting details, supporting TensorRT, TensorFlow, Pytorch, Onnx ... etc. - default: gs://triton_sample_models/25.12 + default: gs://triton_sample_models/26.01 image.ldPreloadPath: type: string title: Leave this empty by default. Triton allows users to create custom layers for backend such as TensorRT plugin, the compiled shared library must be provided via LD_PRELOAD environment variable. diff --git a/deploy/gke-marketplace-app/trt-engine/README.md b/deploy/gke-marketplace-app/trt-engine/README.md index 20cdb977d9..6f33ac0a1c 100644 --- a/deploy/gke-marketplace-app/trt-engine/README.md +++ b/deploy/gke-marketplace-app/trt-engine/README.md @@ -1,5 +1,5 @@ -# [Triton Inference Server Release 25.12](https://docs.nvidia.com/deeplearning/triton-inference-server/release-notes/rel-25-12.html#rel-25-12) +# [Triton Inference Server Release 26.01](https://docs.nvidia.com/deeplearning/triton-inference-server/release-notes/rel-26-01.html#rel-26-01) -The Triton Inference Server container image, release 25.12, is available +The Triton Inference Server container image, release 26.01, is available on [NGC](https://ngc.nvidia.com/catalog/containers/nvidia:tritonserver) and is open source on [GitHub](https://github.com/triton-inference-server/server). Release notes can diff --git a/python/openai/README.md b/python/openai/README.md index ac0ec5e53b..ed770b4e05 100644 --- a/python/openai/README.md +++ b/python/openai/README.md @@ -1,5 +1,5 @@ [![License](https://img.shields.io/badge/License-BSD3-lightgrey.svg)](https://opensource.org/licenses/BSD-3-Clause) +>[!WARNING] +>You are currently on the `main` branch which tracks under-development progress +>towards the next release. The current release is version [2.65.0](https://github.com/triton-inference-server/server/releases/latest) +>and corresponds to the 26.01 container release on NVIDIA GPU Cloud (NGC). + # Triton Inference Server Triton Inference Server is an open source inference serving software that @@ -56,7 +61,7 @@ Major features include: - Provides [Backend API](https://github.com/triton-inference-server/backend) that allows adding custom backends and pre/post processing operations - Supports writing custom backends in python, a.k.a. - [Python-based backends.](https://github.com/triton-inference-server/backend/blob/r26.01/docs/python_based_backends.md#python-based-backends) + [Python-based backends.](https://github.com/triton-inference-server/backend/blob/main/docs/python_based_backends.md#python-based-backends) - Model pipelines using [Ensembling](docs/user_guide/architecture.md#ensemble-models) or [Business Logic Scripting @@ -167,10 +172,10 @@ configuration](docs/user_guide/model_configuration.md) for the model. [Python](https://github.com/triton-inference-server/python_backend), and more - Not all the above backends are supported on every platform supported by Triton. Look at the - [Backend-Platform Support Matrix](https://github.com/triton-inference-server/backend/blob/r26.01/docs/backend_platform_support_matrix.md) + [Backend-Platform Support Matrix](https://github.com/triton-inference-server/backend/blob/main/docs/backend_platform_support_matrix.md) to learn which backends are supported on your target platform. - Learn how to [optimize performance](docs/user_guide/optimization.md) using the - [Performance Analyzer](https://github.com/triton-inference-server/perf_analyzer/blob/r26.01/README.md) + [Performance Analyzer](https://github.com/triton-inference-server/perf_analyzer/blob/main/README.md) and [Model Analyzer](https://github.com/triton-inference-server/model_analyzer) - Learn how to [manage loading and unloading models](docs/user_guide/model_management.md) in @@ -184,14 +189,14 @@ A Triton *client* application sends inference and other requests to Triton. The [Python and C++ client libraries](https://github.com/triton-inference-server/client) provide APIs to simplify this communication. -- Review client examples for [C++](https://github.com/triton-inference-server/client/blob/r26.01/src/c%2B%2B/examples), - [Python](https://github.com/triton-inference-server/client/blob/r26.01/src/python/examples), - and [Java](https://github.com/triton-inference-server/client/blob/r26.01/src/java/src/main/java/triton/client/examples) +- Review client examples for [C++](https://github.com/triton-inference-server/client/blob/main/src/c%2B%2B/examples), + [Python](https://github.com/triton-inference-server/client/blob/main/src/python/examples), + and [Java](https://github.com/triton-inference-server/client/blob/main/src/java/src/main/java/triton/client/examples) - Configure [HTTP](https://github.com/triton-inference-server/client#http-options) and [gRPC](https://github.com/triton-inference-server/client#grpc-options) client options - Send input data (e.g. a jpeg image) directly to Triton in the [body of an HTTP - request without any additional metadata](https://github.com/triton-inference-server/server/blob/r26.01/docs/protocol/extension_binary_data.md#raw-binary-request) + request without any additional metadata](https://github.com/triton-inference-server/server/blob/main/docs/protocol/extension_binary_data.md#raw-binary-request) ### Extend Triton @@ -200,7 +205,7 @@ designed for modularity and flexibility - [Customize Triton Inference Server container](docs/customization_guide/compose.md) for your use case - [Create custom backends](https://github.com/triton-inference-server/backend) - in either [C/C++](https://github.com/triton-inference-server/backend/blob/r26.01/README.md#triton-backend-api) + in either [C/C++](https://github.com/triton-inference-server/backend/blob/main/README.md#triton-backend-api) or [Python](https://github.com/triton-inference-server/python_backend) - Create [decoupled backends and models](docs/user_guide/decoupled_models.md) that can send multiple responses for a request or not send any responses for a request