11<!--
2- # Copyright 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+ # Copyright 2021-2026 , NVIDIA CORPORATION & AFFILIATES. All rights reserved.
33#
44# Redistribution and use in source and binary forms, with or without
55# modification, are permitted provided that the following conditions
@@ -50,7 +50,7 @@ to be deployed.
5050
5151The plugin can be installed from source using the following commands
5252
53- ```
53+ ``` bash
5454python setup.py install
5555```
5656
@@ -89,7 +89,7 @@ as additional artifact which Triton will be used to serve the model. Otherwise,
8989the server should be run with auto-complete feature enabled
9090(` --strict-model-config=false ` ) to generate the model configuration.
9191
92- ```
92+ ``` bash
9393import mlflow.onnx
9494import onnx
9595model = onnx.load(" examples/onnx_float32_int32_int32/1/model.onnx" )
@@ -105,7 +105,7 @@ containing the model files following the
105105[ model layout] ( https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_repository.md#repository-layout ) .
106106Below is an example usage:
107107
108- ```
108+ ``` bash
109109cd /scripts
110110
111111python publish_model_to_mlflow.py --model_name onnx_float32_int32_int32 --model_directory < path-to-the-examples-directory> /onnx_float32_int32_int32 --flavor triton
@@ -117,7 +117,7 @@ Once a model is published and tracked in MLflow, it can be deployed to Triton
117117via MLflow's deployments command, the following command will download the model
118118to Triton's model repository and request Triton to load the model.
119119
120- ```
120+ ``` bash
121121mlflow deployments create -t triton --flavor triton --name onnx_float32_int32_int32 -m models:/onnx_float32_int32_int32/1
122122```
123123
@@ -126,7 +126,7 @@ mlflow deployments create -t triton --flavor triton --name onnx_float32_int32_in
126126After the model is deployed, the following command is the CLI usage to send
127127inference request to a deployment.
128128
129- ```
129+ ``` bash
130130mlflow deployments predict -t triton --name onnx_float32_int32_int32 --input-path < path-to-the-examples-directory> /input.json --output-path output.json
131131```
132132
@@ -145,15 +145,15 @@ MLflow deployments create API deploys a model to the Triton target, which will
145145download the model to Triton's model repository and request Triton to load the
146146model.
147147
148- To create a MLflow deployment using CLI:
148+ To create a MLflow deployment using CLI
149149
150- ```
150+ ``` bash
151151mlflow deployments create -t triton --flavor triton --name model_name -m models:/model_name/1
152152```
153153
154- To create a MLflow deployment using Python API:
154+ To create a MLflow deployment using Python API
155155
156- ```
156+ ``` bash
157157from mlflow.deployments import get_deploy_client
158158client = get_deploy_client(' triton' )
159159client.create_deployment(" model_name" , " models:/model_name/1" , flavor=" triton" )
@@ -167,13 +167,13 @@ Triton to unload the model.
167167
168168To delete a MLflow deployment using CLI
169169
170- ```
170+ ``` bash
171171mlflow deployments delete -t triton --name model_name
172172```
173173
174- To delete a MLflow deployment using CLI
174+ To delete a MLflow deployment using Python API
175175
176- ```
176+ ``` bash
177177from mlflow.deployments import get_deploy_client
178178client = get_deploy_client(' triton' )
179179client.delete_deployment(" model_name" )
@@ -187,13 +187,13 @@ repository and request Triton to reload the model.
187187
188188To update a MLflow deployment using CLI
189189
190- ```
190+ ``` bash
191191mlflow deployments update -t triton --flavor triton --name model_name -m models:/model_name/2
192192```
193193
194194To update a MLflow deployment using Python API
195195
196- ```
196+ ``` bash
197197from mlflow.deployments import get_deploy_client
198198client = get_deploy_client(' triton' )
199199client.update_deployment(" model_name" , " models:/model_name/2" , flavor=" triton" )
@@ -205,13 +205,13 @@ MLflow deployments list API lists all existing deployments in Triton target.
205205
206206To list all MLflow deployments using CLI
207207
208- ```
208+ ``` bash
209209mlflow deployments list -t triton
210210```
211211
212212To list all MLflow deployments using Python API
213213
214- ```
214+ ``` bash
215215from mlflow.deployments import get_deploy_client
216216client = get_deploy_client(' triton' )
217217client.list_deployments ()
@@ -223,12 +223,12 @@ MLflow deployments get API returns information regarding a specific deployments
223223in Triton target.
224224
225225To list a specific MLflow deployment using CLI
226- ```
226+ ``` bash
227227mlflow deployments get -t triton --name model_name
228228```
229229
230230To list a specific MLflow deployment using Python API
231- ```
231+ ``` bash
232232from mlflow.deployments import get_deploy_client
233233client = get_deploy_client(' triton' )
234234client.get_deployment(" model_name" )
@@ -241,14 +241,14 @@ request to Triton and returns the Triton response.
241241
242242To run inference using CLI
243243
244- ```
244+ ``` bash
245245mlflow deployments predict -t triton --name model_name --input-path input_file --output-path output_file
246246
247247```
248248
249249To run inference using Python API
250250
251- ```
251+ ``` bash
252252from mlflow.deployments import get_deploy_client
253253client = get_deploy_client(' triton' )
254254client.predict(" model_name" , inputs)
0 commit comments