From 6d4a519574e963158364d6924110fd72624a012f Mon Sep 17 00:00:00 2001 From: Matthew Wittwer Date: Thu, 9 Jul 2026 16:38:01 +0000 Subject: [PATCH 1/4] Update secure model deployment docs --- docs/customization_guide/deploy.md | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/docs/customization_guide/deploy.md b/docs/customization_guide/deploy.md index 2a9553bb2f..0589c93692 100644 --- a/docs/customization_guide/deploy.md +++ b/docs/customization_guide/deploy.md @@ -38,7 +38,7 @@ standalone process. This guide is intended to provide some key points and best practices that users deploying Triton based solutions should consider. -| [Deploying Behind a Secure Gateway or Proxy](#deploying-behind-a-secure-proxy-or-gateway) | [Running with Least Privilege](#running-with-least-privilege) | +| [Deploying Behind a Secure Gateway or Proxy](#deploying-behind-a-secure-proxy-or-gateway) | [Running with Least Privilege](#running-with-least-privilege) | [Securing Model and Backend Code](#securing-model-and-backend-code) | > [!IMPORTANT] > Ultimately the security of a solution based on Triton @@ -84,6 +84,29 @@ as an "Application" or "Service" within the trusted internal network. * [https://konghq.com/blog/enterprise/envoy-service-mesh] * [https://www.solo.io/topics/envoy-proxy/] +## Securing Model and Backend Code + +> [!WARNING] +> Some Triton backends execute code loaded from the model repository. +> Depending on the backend, this code may run in the Triton process or +> in a backend worker process and can inherit the deployment's +> operating-system privileges, filesystem access, available +> credentials, and network access. Triton does not provide a security +> sandbox for arbitrary model or backend code. + +Treat executable model and backend code as trusted deployment +artifacts. Restrict write access to model repositories and backend +directories, restrict access to model-control APIs, and review this +code before deployment. + +Caller authentication and authorization do not make inference inputs +safe for arbitrary use. Treat request-derived values as untrusted +unless they have been validated for their intended use. Before using +these values in network requests, file paths, subprocesses, +deserialization, or media decoding, validate them against an explicit +policy and enforce limits on size, execution time, concurrency, and +other resource consumption. + ## Running with Least Privilege The security principle of least privilege advocates that a process be From 6171488ad22fff16a5e98bb619d71d1c783cf278 Mon Sep 17 00:00:00 2001 From: Matthew Wittwer Date: Mon, 13 Jul 2026 16:34:53 +0000 Subject: [PATCH 2/4] Update wording --- docs/customization_guide/deploy.md | 38 ++++++++++++++++-------------- 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/docs/customization_guide/deploy.md b/docs/customization_guide/deploy.md index 0589c93692..a744bc22b6 100644 --- a/docs/customization_guide/deploy.md +++ b/docs/customization_guide/deploy.md @@ -38,7 +38,7 @@ standalone process. This guide is intended to provide some key points and best practices that users deploying Triton based solutions should consider. -| [Deploying Behind a Secure Gateway or Proxy](#deploying-behind-a-secure-proxy-or-gateway) | [Running with Least Privilege](#running-with-least-privilege) | [Securing Model and Backend Code](#securing-model-and-backend-code) | +| [Deploying Behind a Secure Proxy or Gateway](#deploying-behind-a-secure-proxy-or-gateway) | [Securing Model and Backend Code](#securing-model-and-backend-code) | [Running with Least Privilege](#running-with-least-privilege) | > [!IMPORTANT] > Ultimately the security of a solution based on Triton @@ -88,24 +88,26 @@ as an "Application" or "Service" within the trusted internal network. > [!WARNING] > Some Triton backends execute code loaded from the model repository. -> Depending on the backend, this code may run in the Triton process or -> in a backend worker process and can inherit the deployment's -> operating-system privileges, filesystem access, available -> credentials, and network access. Triton does not provide a security -> sandbox for arbitrary model or backend code. - -Treat executable model and backend code as trusted deployment -artifacts. Restrict write access to model repositories and backend -directories, restrict access to model-control APIs, and review this -code before deployment. - -Caller authentication and authorization do not make inference inputs -safe for arbitrary use. Treat request-derived values as untrusted -unless they have been validated for their intended use. Before using -these values in network requests, file paths, subprocesses, +> Depending on the backend, this code may run in the Triton server +> process or in a separate process managed by the backend. It can +> exercise the operating-system privileges and access available to that +> process, including filesystem access, available credentials, and +> network access. Triton does not provide a security sandbox for +> arbitrary model or backend code. + +Only deploy executable model and backend code from trusted sources. +Restrict write access to model repositories and backend directories, +restrict model control APIs to trusted operators when they are enabled, +and review executable code before deployment. + +Even when a caller is authenticated and authorized, treat +request-derived values as untrusted until they have been validated for +their intended use. Before using these values in outbound network +requests, filesystem operations, subprocess invocations, deserialization, or media decoding, validate them against an explicit -policy and enforce limits on size, execution time, concurrency, and -other resource consumption. +policy and enforce limits on input size, execution time, concurrency, +and other resource use. Use deployment-level controls, such as outbound +network restrictions, to limit the impact of validation failures. ## Running with Least Privilege From a000ac587fb92b57db3e9028f0f840f9710a05ab Mon Sep 17 00:00:00 2001 From: Matthew Wittwer Date: Mon, 13 Jul 2026 17:29:10 +0000 Subject: [PATCH 3/4] formatting --- docs/customization_guide/deploy.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/customization_guide/deploy.md b/docs/customization_guide/deploy.md index a744bc22b6..175402ccec 100644 --- a/docs/customization_guide/deploy.md +++ b/docs/customization_guide/deploy.md @@ -38,7 +38,9 @@ standalone process. This guide is intended to provide some key points and best practices that users deploying Triton based solutions should consider. -| [Deploying Behind a Secure Proxy or Gateway](#deploying-behind-a-secure-proxy-or-gateway) | [Securing Model and Backend Code](#securing-model-and-backend-code) | [Running with Least Privilege](#running-with-least-privilege) | +| [Secure Proxy or Gateway](#deploying-behind-a-secure-proxy-or-gateway) | +[Model and Backend Code](#securing-model-and-backend-code) | +[Least Privilege](#running-with-least-privilege) | > [!IMPORTANT] > Ultimately the security of a solution based on Triton From 97e7f4fcae3dfb1a6bb434fd20d7984121001cb1 Mon Sep 17 00:00:00 2001 From: Matthew Wittwer Date: Mon, 13 Jul 2026 17:41:45 +0000 Subject: [PATCH 4/4] restore section names --- docs/customization_guide/deploy.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/customization_guide/deploy.md b/docs/customization_guide/deploy.md index 175402ccec..b3a41679e7 100644 --- a/docs/customization_guide/deploy.md +++ b/docs/customization_guide/deploy.md @@ -38,9 +38,9 @@ standalone process. This guide is intended to provide some key points and best practices that users deploying Triton based solutions should consider. -| [Secure Proxy or Gateway](#deploying-behind-a-secure-proxy-or-gateway) | -[Model and Backend Code](#securing-model-and-backend-code) | -[Least Privilege](#running-with-least-privilege) | +| [Deploying Behind a Secure Proxy or Gateway](#deploying-behind-a-secure-proxy-or-gateway) | +[Securing Model and Backend Code](#securing-model-and-backend-code) | +[Running with Least Privilege](#running-with-least-privilege) | > [!IMPORTANT] > Ultimately the security of a solution based on Triton