Merge branch 'main' into mwittwer/sagemaker_input_size_limit

Author: mattwittwer

Dockerfile.sdk

@@ -41,7 +41,7 @@ ARG TRITON_ENABLE_GPU=ON
 ARG JAVA_BINDINGS_MAVEN_VERSION=3.8.4
 ARG JAVA_BINDINGS_JAVACPP_PRESETS_TAG=1.5.8
 # DCGM version to install for Model Analyzer
-ARG DCGM_VERSION=4.4.2-1
+ARG DCGM_VERSION=4.5.2-1
 
 ARG NVIDIA_TRITON_SERVER_SDK_VERSION=unknown
 ARG NVIDIA_BUILD_ID=unknown

README.md

@@ -54,8 +54,8 @@ Major features include:
   frameworks](https://github.com/triton-inference-server/fil_backend)
 - [Concurrent model
   execution](docs/user_guide/architecture.md#concurrent-model-execution)
-- [Dynamic batching](docs/user_guide/model_configuration.md#dynamic-batcher)
-- [Sequence batching](docs/user_guide/model_configuration.md#sequence-batcher) and
+- [Dynamic batching](docs/user_guide/batcher.md#dynamic-batcher)
+- [Sequence batching](docs/user_guide/batcher.md#sequence-batcher) and
   [implicit state management](docs/user_guide/architecture.md#implicit-state-management)
   for stateful models
 - Provides [Backend API](https://github.com/triton-inference-server/backend) that
@@ -70,8 +70,8 @@ Major features include:
   protocols](docs/customization_guide/inference_protocols.md) based on the community
   developed [KServe
   protocol](https://github.com/kserve/kserve/tree/master/docs/predict-api/v2)
-- A [C API](docs/customization_guide/inference_protocols.md#in-process-triton-server-api) and
-  [Java API](docs/customization_guide/inference_protocols.md#java-bindings-for-in-process-triton-server-api)
+- A [C API](docs/customization_guide/inprocess_c_api.md) and
+  [Java API](docs/customization_guide/inprocess_java_api.md)
   allow Triton to link directly into your application for edge and other in-process use cases
 - [Metrics](docs/user_guide/metrics.md) indicating GPU utilization, server
   throughput, server latency, and more

TRITON_VERSION

@@ -1 +1 @@
-2.66.0dev
+2.67.0dev

build.py

@@ -71,13 +71,13 @@
 #
 
 DEFAULT_TRITON_VERSION_MAP = {
-    "release_version": "2.66.0dev",
-    "triton_container_version": "26.02dev",
+    "release_version": "2.67.0dev",
+    "triton_container_version": "26.03dev",
     "upstream_container_version": "26.01",
-    "ort_version": "1.23.2",
+    "ort_version": "1.24.1",
     "ort_openvino_version": "2025.4.1",
     "standalone_openvino_version": "2025.4.1",
-    "dcgm_version": "4.4.2-1",
+    "dcgm_version": "4.5.2-1",
     "vllm_version": "0.13.0",
     "rhel_py_version": "3.12.3",
 }

deploy/mlflow-triton-plugin/README.md

@@ -1,5 +1,5 @@
 <!--
-# Copyright 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright 2021-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
 # modification, are permitted provided that the following conditions
@@ -50,7 +50,7 @@ to be deployed.
 
 The plugin can be installed from source using the following commands
 
-```
+```bash
 python setup.py install
 ```
 
@@ -89,7 +89,7 @@ as additional artifact which Triton will be used to serve the model. Otherwise,
 the server should be run with auto-complete feature enabled
 (`--strict-model-config=false`) to generate the model configuration.
 
-```
+```bash
 import mlflow.onnx
 import onnx
 model = onnx.load("examples/onnx_float32_int32_int32/1/model.onnx")
@@ -105,7 +105,7 @@ containing the model files following the
 [model layout](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_repository.md#repository-layout).
 Below is an example usage:
 
-```
+```bash
 cd /scripts
 
 python publish_model_to_mlflow.py --model_name onnx_float32_int32_int32 --model_directory <path-to-the-examples-directory>/onnx_float32_int32_int32 --flavor triton
@@ -117,7 +117,7 @@ Once a model is published and tracked in MLflow, it can be deployed to Triton
 via MLflow's deployments command, the following command will download the model
 to Triton's model repository and request Triton to load the model.
 
-```
+```bash
 mlflow deployments create -t triton --flavor triton --name onnx_float32_int32_int32 -m models:/onnx_float32_int32_int32/1
 ```
 
@@ -126,7 +126,7 @@ mlflow deployments create -t triton --flavor triton --name onnx_float32_int32_in
 After the model is deployed, the following command is the CLI usage to send
 inference request to a deployment.
 
-```
+```bash
 mlflow deployments predict -t triton --name onnx_float32_int32_int32 --input-path <path-to-the-examples-directory>/input.json --output-path output.json
 ```
 
@@ -145,15 +145,15 @@ MLflow deployments create API deploys a model to the Triton target, which will
 download the model to Triton's model repository and request Triton to load the
 model.
 
-To create a MLflow deployment using CLI:
+To create a MLflow deployment using CLI
 
-```
+```bash
 mlflow deployments create -t triton --flavor triton --name model_name -m models:/model_name/1
 ```
 
-To create a MLflow deployment using Python API:
+To create a MLflow deployment using Python API
 
-```
+```bash
 from mlflow.deployments import get_deploy_client
 client = get_deploy_client('triton')
 client.create_deployment("model_name", "models:/model_name/1", flavor="triton")
@@ -167,13 +167,13 @@ Triton to unload the model.
 
 To delete a MLflow deployment using CLI
 
-```
+```bash
 mlflow deployments delete -t triton --name model_name
 ```
 
-To delete a MLflow deployment using CLI
+To delete a MLflow deployment using Python API
 
-```
+```bash
 from mlflow.deployments import get_deploy_client
 client = get_deploy_client('triton')
 client.delete_deployment("model_name")
@@ -187,13 +187,13 @@ repository and request Triton to reload the model.
 
 To update a MLflow deployment using CLI
 
-```
+```bash
 mlflow deployments update -t triton --flavor triton --name model_name -m models:/model_name/2
 ```
 
 To update a MLflow deployment using Python API
 
-```
+```bash
 from mlflow.deployments import get_deploy_client
 client = get_deploy_client('triton')
 client.update_deployment("model_name", "models:/model_name/2", flavor="triton")
@@ -205,13 +205,13 @@ MLflow deployments list API lists all existing deployments in Triton target.
 
 To list all MLflow deployments using CLI
 
-```
+```bash
 mlflow deployments list -t triton
 ```
 
 To list all MLflow deployments using Python API
 
-```
+```bash
 from mlflow.deployments import get_deploy_client
 client = get_deploy_client('triton')
 client.list_deployments()
@@ -223,12 +223,12 @@ MLflow deployments get API returns information regarding a specific deployments
 in Triton target.
 
 To list a specific MLflow deployment using CLI
-```
+```bash
 mlflow deployments get -t triton --name model_name
 ```
 
 To list a specific MLflow deployment using Python API
-```
+```bash
 from mlflow.deployments import get_deploy_client
 client = get_deploy_client('triton')
 client.get_deployment("model_name")
@@ -241,14 +241,14 @@ request to Triton and returns the Triton response.
 
 To run inference using CLI
 
-```
+```bash
 mlflow deployments predict -t triton --name model_name --input-path input_file --output-path output_file
 
 ```
 
 To run inference using Python API
 
-```
+```bash
 from mlflow.deployments import get_deploy_client
 client = get_deploy_client('triton')
 client.predict("model_name", inputs)

docs/README.md

@@ -1,5 +1,5 @@
 <!--
-# Copyright 2018-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright 2018-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
 # modification, are permitted provided that the following conditions
@@ -111,17 +111,17 @@ The Model Configuration ModelOptimizationPolicy property is used to specify opti
 
 #### Scheduling and Batching
 
-Triton supports batching individual inference requests to improve compute resource utilization. This is extremely important as individual requests typically will not saturate GPU resources thus not leveraging the parallelism provided by GPUs to its extent. Learn more about Triton's [Batcher and Scheduler](user_guide/model_configuration.md#scheduling-and-batching).
-- [Default Scheduler - Non-Batching](user_guide/model_configuration.md#default-scheduler)
-- [Dynamic Batcher](user_guide/model_configuration.md#dynamic-batcher)
+Triton supports batching individual inference requests to improve compute resource utilization. This is extremely important as individual requests typically will not saturate GPU resources thus not leveraging the parallelism provided by GPUs to its extent. Learn more about Triton's [Batcher and Scheduler](#scheduling-and-batching).
+- [Default Scheduler - Non-Batching](user_guide/scheduler.md#default-scheduler)
+- [Dynamic Batcher](user_guide/batcher.md#dynamic-batcher)
   - [How to Configure Dynamic Batcher](user_guide/model_configuration.md#recommended-configuration-process)
-    - [Delayed Batching](user_guide/model_configuration.md#delayed-batching)
+    - [Delayed Batching](user_guide/batcher.md#delayed-batching)
     - [Preferred Batch Size](user_guide/model_configuration.md#preferred-batch-sizes)
   - [Preserving Request Ordering](user_guide/model_configuration.md#preserve-ordering)
   - [Priority Levels](user_guide/model_configuration.md#priority-levels)
   - [Queuing Policies](user_guide/model_configuration.md#queue-policy)
   - [Ragged Batching](user_guide/ragged_batching.md)
-- [Sequence Batcher](user_guide/model_configuration.md#sequence-batcher)
+- [Sequence Batcher](user_guide/batcher.md#sequence-batcher)
   - [Stateful Models](user_guide/model_execution.md#stateful-models)
   - [Control Inputs](user_guide/model_execution.md#control-inputs)
   - [Implicit State - Stateful Inference Using a Stateless Model](user_guide/implicit_state_management.md#implicit-state-management)

docs/client_guide/python.rst

@@ -1,5 +1,5 @@
 ..
-.. Copyright 2024-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+.. Copyright 2024-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 ..
 .. Redistribution and use in source and binary forms, with or without
 .. modification, are permitted provided that the following conditions
@@ -29,11 +29,10 @@
 Python
 ####
 
-.. include:: python_readme.rst
-
 .. toctree::
    :maxdepth: 1
    :hidden:
 
+   Overview <../tutorials/Triton_Inference_Server_Python_API/README.md>
    Kafka I/O <../tutorials/Triton_Inference_Server_Python_API/examples/kafka-io/README.md>
    Rayserve <../tutorials/Triton_Inference_Server_Python_API/examples/rayserve/README.md>