Skip to content

Commit 55e175d

Browse files
dvdksndvdksn
authored
Merge pull request #24433 from doringeman/containers-gpu
docs(containers): move GPU access to dedicated page
2 parents dd59914 + 0913862 commit 55e175d

File tree

2 files changed

+116
-83
lines changed

2 files changed

+116
-83
lines changed

‎content/manuals/engine/containers/gpu.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
---
2+
title: GPU access
3+
weight: 40
4+
description: How to access NVIDIA GPUs from a container
5+
keywords: docker, GPU, NVIDIA, cuda, nvidia-smi, device, container toolkit
6+
---
7+
8+
## Access an NVIDIA GPU
9+
10+
### Prerequisites
11+
12+
Visit the official [NVIDIA drivers page](https://www.nvidia.com/Download/index.aspx)
13+
to download and install the proper drivers. Reboot your system once you have
14+
done so.
15+
16+
Verify that your GPU is running and accessible.
17+
18+
### Install NVIDIA Container Toolkit
19+
20+
Follow the official NVIDIA Container Toolkit [installation instructions](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
21+
22+
### Expose GPUs for use
23+
24+
Include the `--gpus` flag when you start a container to access GPU resources.
25+
26+
To expose all available GPUs:
27+
28+
```console
29+
$ docker run -it --rm --gpus all ubuntu nvidia-smi
30+
```
31+
32+
The output looks like the following:
33+
34+
```text
35+
+---------------------------------------------------------------------------------------+
36+
| NVIDIA-SMI 535.288.01 Driver Version: 535.288.01 CUDA Version: 12.2 |
37+
|-----------------------------------------+----------------------+----------------------+
38+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
39+
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
40+
| | | MIG M. |
41+
|=========================================+======================+======================|
42+
| 0 NVIDIA L4 Off | 00000000:31:00.0 Off | 0 |
43+
| N/A 40C P0 27W / 72W | 0MiB / 23034MiB | 4% Default |
44+
| | | N/A |
45+
+-----------------------------------------+----------------------+----------------------+
46+
47+
+---------------------------------------------------------------------------------------+
48+
| Processes: |
49+
| GPU GI CI PID Type Process name GPU Memory |
50+
| ID ID Usage |
51+
|=======================================================================================|
52+
| No running processes found |
53+
+---------------------------------------------------------------------------------------+
54+
```
55+
56+
The leftmost column in the GPU table shows the index of each GPU (`0` for the
57+
NVIDIA L4 in the previous example). Use these index numbers to target specific GPUs
58+
with the `device` option.
59+
60+
To expose a single GPU by index:
61+
62+
```console
63+
$ docker run -it --rm --gpus device=0 ubuntu nvidia-smi
64+
```
65+
66+
To expose a GPU by its UUID, first list UUIDs with `nvidia-smi -L`:
67+
68+
```console
69+
$ nvidia-smi -L
70+
GPU 0: NVIDIA L4 (UUID: GPU-3a23c669-1f69-c64e-cf85-44e9b07e7a2a)
71+
```
72+
73+
Then pass the UUID to `--gpus`:
74+
75+
```console
76+
$ docker run -it --rm --gpus device=GPU-3a23c669-1f69-c64e-cf85-44e9b07e7a2a ubuntu nvidia-smi
77+
```
78+
79+
On systems with multiple GPUs, you can expose several by index. The `device`
80+
value must be quoted because it contains a comma:
81+
82+
```console
83+
$ docker run -it --rm --gpus '"device=0,2"' ubuntu nvidia-smi
84+
```
85+
86+
This exposes the GPUs at index `0` and `2` — the first and third GPUs listed in
87+
`nvidia-smi` output.
88+
89+
> [!NOTE]
90+
>
91+
> NVIDIA GPUs can only be accessed by systems running a single engine.
92+
93+
### Set NVIDIA capabilities
94+
95+
You can set capabilities manually. For example, on Ubuntu you can run the
96+
following:
97+
98+
```console
99+
$ docker run --gpus 'all,capabilities=utility' --rm ubuntu nvidia-smi
100+
```
101+
102+
This enables the `utility` driver capability which adds the `nvidia-smi` tool to
103+
the container.
104+
105+
Capabilities and other configurations can be set in images via environment
106+
variables. For valid variables, see the
107+
[nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/docker-specialized.html)
108+
documentation. These variables can be set in a Dockerfile.
109+
110+
You can also use CUDA images, which set these variables automatically. See the
111+
official [CUDA images](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda)
112+
NGC catalog page.

‎content/manuals/engine/containers/resource_constraints.md‎

Lines changed: 4 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
---
22
title: Resource constraints
33
weight: 30
4-
description: Specify the runtime options for a container
5-
keywords: docker, daemon, configuration, runtime
4+
description: Limit container memory and CPU usage with runtime configuration flags
5+
keywords: resource constraints, memory limits, CPU limits, cgroups, OOM, swap, docker run, memory swap
66
aliases:
77
- /engine/admin/resource_constraints/
88
- /config/containers/resource_constraints/
@@ -265,84 +265,5 @@ If the kernel or Docker daemon isn't configured correctly, an error occurs.
265265

266266
## GPU
267267

268-
### Access an NVIDIA GPU
269-
270-
#### Prerequisites
271-
272-
Visit the official [NVIDIA drivers page](https://www.nvidia.com/Download/index.aspx)
273-
to download and install the proper drivers. Reboot your system once you have
274-
done so.
275-
276-
Verify that your GPU is running and accessible.
277-
278-
#### Install nvidia-container-toolkit
279-
280-
Follow the official NVIDIA Container Toolkit [installation instructions](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).
281-
282-
#### Expose GPUs for use
283-
284-
Include the `--gpus` flag when you start a container to access GPU resources.
285-
Specify how many GPUs to use. For example:
286-
287-
```console
288-
$ docker run -it --rm --gpus all ubuntu nvidia-smi
289-
```
290-
291-
Exposes all available GPUs and returns a result akin to the following:
292-
293-
```bash
294-
+-------------------------------------------------------------------------------+
295-
| NVIDIA-SMI 384.130 Driver Version: 384.130 |
296-
|-------------------------------+----------------------+------------------------+
297-
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
298-
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
299-
|===============================+======================+========================|
300-
| 0 GRID K520 Off | 00000000:00:03.0 Off | N/A |
301-
| N/A 36C P0 39W / 125W | 0MiB / 4036MiB | 0% Default |
302-
+-------------------------------+----------------------+------------------------+
303-
+-------------------------------------------------------------------------------+
304-
| Processes: GPU Memory |
305-
| GPU PID Type Process name Usage |
306-
|===============================================================================|
307-
| No running processes found |
308-
+-------------------------------------------------------------------------------+
309-
```
310-
311-
Use the `device` option to specify GPUs. For example:
312-
313-
```console
314-
$ docker run -it --rm --gpus device=GPU-3a23c669-1f69-c64e-cf85-44e9b07e7a2a ubuntu nvidia-smi
315-
```
316-
317-
Exposes that specific GPU.
318-
319-
```console
320-
$ docker run -it --rm --gpus '"device=0,2"' ubuntu nvidia-smi
321-
```
322-
323-
Exposes the first and third GPUs.
324-
325-
> [!NOTE]
326-
>
327-
> NVIDIA GPUs can only be accessed by systems running a single engine.
328-
329-
#### Set NVIDIA capabilities
330-
331-
You can set capabilities manually. For example, on Ubuntu you can run the
332-
following:
333-
334-
```console
335-
$ docker run --gpus 'all,capabilities=utility' --rm ubuntu nvidia-smi
336-
```
337-
338-
This enables the `utility` driver capability which adds the `nvidia-smi` tool to
339-
the container.
340-
341-
Capabilities as well as other configurations can be set in images via
342-
environment variables. More information on valid variables can be found in the
343-
[nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/docker-specialized.html)
344-
documentation. These variables can be set in a Dockerfile.
345-
346-
You can also use CUDA images, which set these variables automatically. See the
347-
official [CUDA images](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda)
348-
NGC catalog page.
268+
For information on how to access NVIDIA GPUs from a container, see
269+
[GPU access](gpu.md).

0 commit comments

Comments
 (0)