You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/guides/applications/big-data/ai-inferencing-with-tensorrt-and-pytorch/index.md
+37-13Lines changed: 37 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,11 +27,11 @@ The primary AI model used in this guide is a ResNet50 computer vision (CV) model
27
27
28
28
### TensorRt
29
29
30
-
30
+
[TensorRT](https://developer.nvidia.com/tensorrt) is an API and tool ecosystem by NVIDIA that includes inference compilers, runtimes, and deep learning model optimizations. TensorRT is trained on all major frameworks and is used to improve performance on NVIDIA GPUs using techniques like kernel auto-tuning, dynamic tensor memory management, and multi-stream execution. It directly integrates with PyTorch using the TensorRT Framework Integrations API to achieve up to 6x faster inferencing.
31
31
32
32
### PyTorch
33
33
34
-
34
+
[PyTorch](https://pytorch.org/) is an open-source machine learning framework based on the [Torch library](https://docs.pytorch.org/docs/stable/library.html) and developed by Meta AI for training deep learning models. PyTorch is written in Python and integrates with TensorRT through [Torch-TensorRT](https://github.com/pytorch/TensorRT), so developers can optimize PyTorch models without changing existing codebases. PyTorch integrates with [CUDA](https://en.wikipedia.org/wiki/CUDA) (Compute Unified Device Architecture) to take advantage of parallel computing architectures found in NVIDIA GPUs.
35
35
36
36
## Before You Begin
37
37
@@ -48,7 +48,7 @@ This guide is written for a non-root user. Commands that require elevated privil
48
48
49
49
## Deploy an NVIDIA RTX 4000 Ada Instance
50
50
51
-
Akamai's NVIDIA RTX 4000 Ada GPU instances can be deployed using Cloud Manager or the Linode CLI.
51
+
Akamai's NVIDIA RTX 4000 Ada GPU instances can be deployed using Cloud Manager or the Linode CLI. This guide is written for use with the Ubuntu 24.04 LTS distribution.
52
52
53
53
### Deploy Using Cloud Manager
54
54
@@ -59,14 +59,14 @@ Akamai's NVIDIA RTX 4000 Ada GPU instances can be deployed using Cloud Manager o
59
59
60
60
## Set Up Your Development Environment
61
61
62
-
Once it is fully deployed, connect to your GPU instance to update system packages and install system dependencies.
62
+
Once it is fully deployed, connect to your GPU instance to update system packages and install system dependencies. It is recommended to follow the steps in our [Set up and secure a Linode](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to configure a limited user with sudo access and secure your sever.
63
63
64
64
### Update Packages
65
65
66
-
1. Log into your instance via SSH:
66
+
1. Log into your instance via SSH. Replace {{< placeholder "user" >}} with your sudo username and {{< placeholder "IP_ADDRESS" >}} with your Linode instance's IP address:
1. Use the following command to verify successful driver installation:
@@ -121,10 +121,28 @@ Once it is fully deployed, connect to your GPU instance to update system package
121
121
nvidia-smi
122
122
```
123
123
124
-
You should see basic information about your RTX 4000 Ada instance and its driver version:
124
+
This displays basic information about your RTX 4000 Ada instance and its driver version. Your driver and software versions may vary based on release date:
@@ -140,17 +158,23 @@ Set up and use a Python Virtual Environment (venv) so that you can isolate Pytho
140
158
source~/venv/bin/activate
141
159
```
142
160
143
-
1. Upgrade pip to the latest version to completethe setup:
161
+
You can confirm you are using your virtual environment when you see `(venv)` at the beginning of your command prompt:
144
162
145
-
```command
163
+
```output
164
+
(venv) user@hostname
165
+
```
166
+
167
+
1. While in your virtual environment, upgrade pip to the latest version to complete the setup:
168
+
169
+
```command {title="(venv)"}
146
170
pip install --upgrade pip
147
171
```
148
172
149
173
### Install PyTorch and TensorRT
150
174
151
-
While using your virtual environment, install PyTorch, TensorRT, and dependencies. These are the primary AI libraries needed to run your inference workloads.
175
+
Remain in your virtual environment to install PyTorch, TensorRT, and dependencies. These are the primary AI libraries needed to run your inference workloads.
0 commit comments