Skip to content

Commit 76f058e

Browse files
committed
added descriptions for tensorrt and pytorch
1 parent 503035d commit 76f058e

1 file changed

Lines changed: 37 additions & 13 deletions

File tree

  • docs/guides/applications/big-data/ai-inferencing-with-tensorrt-and-pytorch

docs/guides/applications/big-data/ai-inferencing-with-tensorrt-and-pytorch/index.md

Lines changed: 37 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -27,11 +27,11 @@ The primary AI model used in this guide is a ResNet50 computer vision (CV) model
2727

2828
### TensorRt
2929

30-
30+
[TensorRT](https://developer.nvidia.com/tensorrt) is an API and tool ecosystem by NVIDIA that includes inference compilers, runtimes, and deep learning model optimizations. TensorRT is trained on all major frameworks and is used to improve performance on NVIDIA GPUs using techniques like kernel auto-tuning, dynamic tensor memory management, and multi-stream execution. It directly integrates with PyTorch using the TensorRT Framework Integrations API to achieve up to 6x faster inferencing.
3131

3232
### PyTorch
3333

34-
34+
[PyTorch](https://pytorch.org/) is an open-source machine learning framework based on the [Torch library](https://docs.pytorch.org/docs/stable/library.html) and developed by Meta AI for training deep learning models. PyTorch is written in Python and integrates with TensorRT through [Torch-TensorRT](https://github.com/pytorch/TensorRT), so developers can optimize PyTorch models without changing existing codebases. PyTorch integrates with [CUDA](https://en.wikipedia.org/wiki/CUDA) (Compute Unified Device Architecture) to take advantage of parallel computing architectures found in NVIDIA GPUs.
3535

3636
## Before You Begin
3737

@@ -48,7 +48,7 @@ This guide is written for a non-root user. Commands that require elevated privil
4848

4949
## Deploy an NVIDIA RTX 4000 Ada Instance
5050

51-
Akamai's NVIDIA RTX 4000 Ada GPU instances can be deployed using Cloud Manager or the Linode CLI.
51+
Akamai's NVIDIA RTX 4000 Ada GPU instances can be deployed using Cloud Manager or the Linode CLI. This guide is written for use with the Ubuntu 24.04 LTS distribution.
5252

5353
### Deploy Using Cloud Manager
5454

@@ -59,14 +59,14 @@ Akamai's NVIDIA RTX 4000 Ada GPU instances can be deployed using Cloud Manager o
5959

6060
## Set Up Your Development Environment
6161

62-
Once it is fully deployed, connect to your GPU instance to update system packages and install system dependencies.
62+
Once it is fully deployed, connect to your GPU instance to update system packages and install system dependencies. It is recommended to follow the steps in our [Set up and secure a Linode](https://techdocs.akamai.com/cloud-computing/docs/set-up-and-secure-a-compute-instance) guide to configure a limited user with sudo access and secure your sever.
6363

6464
### Update Packages
6565

66-
1. Log into your instance via SSH:
66+
1. Log into your instance via SSH. Replace {{< placeholder "user" >}} with your sudo username and {{< placeholder "IP_ADDRESS" >}} with your Linode instance's IP address:
6767

6868
```command
69-
ssh user@{{< placeholder "IP_ADDRESS" >}}
69+
ssh {{< placeholder "user" >}}@{{< placeholder "IP_ADDRESS" >}}
7070
```
7171

7272
1. Update your system and install build tools and system dependencies:
@@ -112,7 +112,7 @@ Once it is fully deployed, connect to your GPU instance to update system package
112112
1. After the reboot is complete, log back into your instance:
113113

114114
```command
115-
ssh user@{{< placeholder "IP_ADDRESS" >}}
115+
ssh {{< placeholder "user" >}}@{{< placeholder "IP_ADDRESS" >}}
116116
```
117117

118118
1. Use the following command to verify successful driver installation:
@@ -121,10 +121,28 @@ Once it is fully deployed, connect to your GPU instance to update system package
121121
nvidia-smi
122122
```
123123

124-
You should see basic information about your RTX 4000 Ada instance and its driver version:
124+
This displays basic information about your RTX 4000 Ada instance and its driver version. Your driver and software versions may vary based on release date:
125125

126126
```output
127-
127+
+-----------------------------------------------------------------------------------------+
128+
| NVIDIA-SMI 575.57.08 Driver Version: 575.57.08 CUDA Version: 12.9 |
129+
|-----------------------------------------+------------------------+----------------------+
130+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
131+
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
132+
| | | MIG M. |
133+
|=========================================+========================+======================|
134+
| 0 NVIDIA RTX 4000 Ada Gene... On | 00000000:00:02.0 Off | Off |
135+
| 30% 35C P8 4W / 130W | 2MiB / 20475MiB | 0% Default |
136+
| | | N/A |
137+
+-----------------------------------------+------------------------+----------------------+
138+
139+
+-----------------------------------------------------------------------------------------+
140+
| Processes: |
141+
| GPU GI CI PID Type Process name GPU Memory |
142+
| ID ID Usage |
143+
|=========================================================================================|
144+
| No running processes found |
145+
+-----------------------------------------------------------------------------------------+
128146
```
129147

130148
## Configure Your Python Environment
@@ -140,17 +158,23 @@ Set up and use a Python Virtual Environment (venv) so that you can isolate Pytho
140158
source ~/venv/bin/activate
141159
```
142160

143-
1. Upgrade pip to the latest version to complete the setup:
161+
You can confirm you are using your virtual environment when you see `(venv)` at the beginning of your command prompt:
144162

145-
```command
163+
```output
164+
(venv) user@hostname
165+
```
166+
167+
1. While in your virtual environment, upgrade pip to the latest version to complete the setup:
168+
169+
```command {title="(venv)"}
146170
pip install --upgrade pip
147171
```
148172

149173
### Install PyTorch and TensorRT
150174

151-
While using your virtual environment, install PyTorch, TensorRT, and dependencies. These are the primary AI libraries needed to run your inference workloads.
175+
Remain in your virtual environment to install PyTorch, TensorRT, and dependencies. These are the primary AI libraries needed to run your inference workloads.
152176

153-
```command
177+
```command {title="(venv)"}
154178
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
155179
pip install requests
156180
pip install nvidia-pyindex

0 commit comments

Comments
 (0)