Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Dockerfile.sdk
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2019-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright 2019-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -29,7 +29,7 @@
#

# Base image on the minimum Triton container
ARG BASE_IMAGE=nvcr.io/nvidia/tritonserver:25.12-py3-min
ARG BASE_IMAGE=nvcr.io/nvidia/tritonserver:26.01-py3-min

ARG TRITON_CLIENT_REPO_SUBDIR=clientrepo
ARG TRITON_REPO_ORGANIZATION=http://github.com/triton-inference-server
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!--
# Copyright 2018-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright 2018-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -29,8 +29,8 @@

>[!WARNING]
>You are currently on the `main` branch which tracks under-development progress
>towards the next release. The current release is version [2.64.0](https://github.com/triton-inference-server/server/releases/latest)
>and corresponds to the 25.12 container release on NVIDIA GPU Cloud (NGC).
>towards the next release. The current release is version [2.65.0](https://github.com/triton-inference-server/server/releases/latest)
>and corresponds to the 26.01 container release on NVIDIA GPU Cloud (NGC).

# Triton Inference Server

Expand Down Expand Up @@ -90,16 +90,16 @@ Inference Server with the

```bash
# Step 1: Create the example model repository
git clone -b r25.12 https://github.com/triton-inference-server/server.git
git clone -b r26.01 https://github.com/triton-inference-server/server.git
cd server/docs/examples
./fetch_models.sh

# Step 2: Launch triton from the NGC Triton container
docker run --gpus=1 --rm --net=host -v ${PWD}/model_repository:/models nvcr.io/nvidia/tritonserver:25.12-py3 tritonserver --model-repository=/models --model-control-mode explicit --load-model densenet_onnx
docker run --gpus=1 --rm --net=host -v ${PWD}/model_repository:/models nvcr.io/nvidia/tritonserver:26.01-py3 tritonserver --model-repository=/models --model-control-mode explicit --load-model densenet_onnx

# Step 3: Sending an Inference Request
# In a separate console, launch the image_client example from the NGC Triton SDK container
docker run -it --rm --net=host nvcr.io/nvidia/tritonserver:25.12-py3-sdk /workspace/install/bin/image_client -m densenet_onnx -c 3 -s INCEPTION /workspace/images/mug.jpg
docker run -it --rm --net=host nvcr.io/nvidia/tritonserver:26.01-py3-sdk /workspace/install/bin/image_client -m densenet_onnx -c 3 -s INCEPTION /workspace/images/mug.jpg

# Inference should return the following
Image '/workspace/images/mug.jpg':
Expand Down
9 changes: 2 additions & 7 deletions build.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,12 +73,12 @@
DEFAULT_TRITON_VERSION_MAP = {
"release_version": "2.66.0dev",
"triton_container_version": "26.02dev",
"upstream_container_version": "25.12",
"upstream_container_version": "26.01",
"ort_version": "1.23.2",
"ort_openvino_version": "2025.4.1",
"standalone_openvino_version": "2025.4.1",
"dcgm_version": "4.4.2-1",
"vllm_version": "0.11.1",
"vllm_version": "0.13.0",
"rhel_py_version": "3.12.3",
}

Expand Down Expand Up @@ -2093,11 +2093,6 @@ def backend_build(
cmake_script.mkdir(build_dir)
cmake_script.cwd(build_dir)
if be == "tensorrtllm":
github_organization = (
"https://github.com/NVIDIA"
if "triton-inference-server" in FLAGS.github_organization
else FLAGS.github_organization
)
repository_name = "TensorRT-LLM"
cmake_script.gitclone(repository_name, tag, be, github_organization)
else:
Expand Down
4 changes: 2 additions & 2 deletions deploy/aws/values.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2019-2025, NVIDIA CORPORATION. All rights reserved.
# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -27,7 +27,7 @@
replicaCount: 1

image:
imageName: nvcr.io/nvidia/tritonserver:25.12-py3
imageName: nvcr.io/nvidia/tritonserver:26.01-py3
pullPolicy: IfNotPresent
modelRepositoryPath: s3://triton-inference-server-repository/model_repository
numGpus: 1
Expand Down
4 changes: 2 additions & 2 deletions deploy/fleetcommand/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2019-2025, NVIDIA CORPORATION. All rights reserved.
# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -26,7 +26,7 @@

apiVersion: v1
# appVersion is the Triton version; update when changing release
appVersion: 2.64.0
appVersion: 2.65.0
description: Triton Inference Server (Fleet Command)
name: triton-inference-server
# version is the Chart version; update when changing anything in the chart
Expand Down
8 changes: 4 additions & 4 deletions deploy/fleetcommand/values.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2019-2025, NVIDIA CORPORATION. All rights reserved.
# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -27,7 +27,7 @@
replicaCount: 1

image:
imageName: nvcr.io/nvidia/tritonserver:25.12-py3
imageName: nvcr.io/nvidia/tritonserver:26.01-py3
pullPolicy: IfNotPresent
numGpus: 1
serverCommand: tritonserver
Expand All @@ -47,13 +47,13 @@ image:
#
# To set model control mode, uncomment and configure below
# TODO: Fix the following url, it is invalid
# See https://github.com/triton-inference-server/server/blob/r25.12/docs/user_guide/model_management.md
# See https://github.com/triton-inference-server/server/blob/r26.01/docs/user_guide/model_management.md
# for more details
#- --model-control-mode=explicit|poll|none
#
# Additional server args
#
# see https://github.com/triton-inference-server/server/blob/r25.12/README.md
# see https://github.com/triton-inference-server/server/blob/r26.01/README.md
# for more details

service:
Expand Down
4 changes: 2 additions & 2 deletions deploy/gcp/values.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2019-2025, NVIDIA CORPORATION. All rights reserved.
# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -27,7 +27,7 @@
replicaCount: 1

image:
imageName: nvcr.io/nvidia/tritonserver:25.12-py3
imageName: nvcr.io/nvidia/tritonserver:26.01-py3
pullPolicy: IfNotPresent
modelRepositoryPath: gs://triton-inference-server-repository/model_repository
numGpus: 1
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -33,7 +33,7 @@ metadata:
namespace: default
spec:
containers:
- image: nvcr.io/nvidia/tritonserver:25.12-py3-sdk
- image: nvcr.io/nvidia/tritonserver:26.01-py3-sdk
imagePullPolicy: Always
name: nv-triton-client
securityContext:
Expand Down
8 changes: 4 additions & 4 deletions deploy/gke-marketplace-app/server-deployer/build_and_push.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# Copyright 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -27,9 +27,9 @@

export REGISTRY=gcr.io/$(gcloud config get-value project | tr ':' '/')
export APP_NAME=tritonserver
export MAJOR_VERSION=2.64
export MINOR_VERSION=2.64.0
export NGC_VERSION=25.12-py3
export MAJOR_VERSION=2.65
export MINOR_VERSION=2.65.0
export NGC_VERSION=26.01-py3

docker pull nvcr.io/nvidia/$APP_NAME:$NGC_VERSION

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright (c) 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand All @@ -25,7 +25,7 @@
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

apiVersion: v1
appVersion: "2.64"
appVersion: "2.65"
description: Triton Inference Server
name: triton-inference-server
version: 2.64.0
version: 2.65.0
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright (c) 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -31,14 +31,14 @@ maxReplicaCount: 3
tritonProtocol: HTTP
# HPA GPU utilization autoscaling target
HPATargetAverageValue: 85
modelRepositoryPath: gs://triton_sample_models/25.12
publishedVersion: '2.64.0'
modelRepositoryPath: gs://triton_sample_models/26.01
publishedVersion: '2.65.0'
gcpMarketplace: true

image:
registry: gcr.io
repository: nvidia-ngc-public/tritonserver
tag: 25.12-py3
tag: 26.01-py3
pullPolicy: IfNotPresent
# modify the model repository here to match your GCP storage bucket
numGpus: 1
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright (c) 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -27,7 +27,7 @@
x-google-marketplace:
schemaVersion: v2
applicationApiVersion: v1beta1
publishedVersion: '2.64.0'
publishedVersion: '2.65.0'
publishedVersionMetadata:
releaseNote: >-
Initial release.
Expand Down
6 changes: 3 additions & 3 deletions deploy/gke-marketplace-app/server-deployer/schema.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright (c) 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -27,7 +27,7 @@
x-google-marketplace:
schemaVersion: v2
applicationApiVersion: v1beta1
publishedVersion: '2.64.0'
publishedVersion: '2.65.0'
publishedVersionMetadata:
releaseNote: >-
Initial release.
Expand Down Expand Up @@ -89,7 +89,7 @@ properties:
modelRepositoryPath:
type: string
title: Bucket where models are stored. Please make sure the user/service account to create the GKE app has permission to this GCS bucket. Read Triton documentation on configs and formatting details, supporting TensorRT, TensorFlow, Pytorch, Onnx ... etc.
default: gs://triton_sample_models/25.12
default: gs://triton_sample_models/26.01
image.ldPreloadPath:
type: string
title: Leave this empty by default. Triton allows users to create custom layers for backend such as TensorRT plugin, the compiled shared library must be provided via LD_PRELOAD environment variable.
Expand Down
8 changes: 4 additions & 4 deletions deploy/gke-marketplace-app/trt-engine/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!--
# Copyright (c) 2021-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright (c) 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -33,7 +33,7 @@
```
docker run --gpus all -it --network host \
--shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \
-v ~:/scripts nvcr.io/nvidia/tensorrt:25.12-py3
-v ~:/scripts nvcr.io/nvidia/tensorrt:26.01-py3

pip install onnx six torch tf2onnx tensorflow

Expand All @@ -57,7 +57,7 @@ mkdir -p engines

python3 builder.py -m models/fine-tuned/bert_tf_ckpt_large_qa_squad2_amp_128_v19.03.1/model.ckpt -o engines/bert_large_int8_bs1_s128.engine -b 1 -s 128 -c models/fine-tuned/bert_tf_ckpt_large_qa_squad2_amp_128_v19.03.1/ -v models/fine-tuned/bert_tf_ckpt_large_qa_squad2_amp_128_v19.03.1/vocab.txt --int8 --fp16 --strict --calib-num 1 -iln -imh

gsutil cp bert_large_int8_bs1_s128.engine gs://triton_sample_models/25.12/bert/1/model.plan
gsutil cp bert_large_int8_bs1_s128.engine gs://triton_sample_models/26.01/bert/1/model.plan
```

For each Triton upgrade, container version used to generate the model, and the model path in GCS `gs://triton_sample_models/25.12/` should be updated accordingly with the correct version.
For each Triton upgrade, container version used to generate the model, and the model path in GCS `gs://triton_sample_models/26.01/` should be updated accordingly with the correct version.
4 changes: 2 additions & 2 deletions deploy/k8s-onprem/values.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2019-2025, NVIDIA CORPORATION. All rights reserved.
# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -30,7 +30,7 @@ tags:
openshift: false

image:
imageName: nvcr.io/nvidia/tritonserver:25.12-py3
imageName: nvcr.io/nvidia/tritonserver:26.01-py3
pullPolicy: IfNotPresent
modelRepositoryServer: < Replace with the IP Address of your file server >
modelRepositoryPath: /srv/models
Expand Down
4 changes: 2 additions & 2 deletions deploy/oci/values.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2019-2025, NVIDIA CORPORATION. All rights reserved.
# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
Expand Down Expand Up @@ -27,7 +27,7 @@
replicaCount: 1

image:
imageName: nvcr.io/nvidia/tritonserver:25.12-py3
imageName: nvcr.io/nvidia/tritonserver:26.01-py3
pullPolicy: IfNotPresent
modelRepositoryPath: s3://https://<OCI_NAMESPACE>.compat.objectstorage.<OCI_REGION>.oraclecloud.com:443/triton-inference-server-repository
numGpus: 1
Expand Down
Loading
Loading