Skip to content

gpu: document required system-probe.yaml flag and gpu.enabled for non-containerized Linux hosts #23853

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gouri-yerra merged 2 commits into from
May 27, 2026
Merged

gpu: document required system-probe.yaml flag and gpu.enabled for non-containerized Linux hosts #23853

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cbbe3b3
Update GPU README to document required system-probe.yaml flag and gpu…
gouri-yerra May 27, 2026
8947984
Address review feedback: style and consistency fixes in GPU README
gouri-yerra May 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions gpu/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,21 +33,32 @@ The check also uses eBPF probes to assign GPU usage and performance metrics to p

#### Host

The agent needs to be configured to enable GPU-related features. Add the following parameters to the `/etc/datadog-agent/datadog.yaml` configuration file and then restart the Agent:
GPU monitoring requires configuration in both `/etc/datadog-agent/datadog.yaml` and `/etc/datadog-agent/system-probe.yaml`. Configuring only one of these files results in incomplete metrics collection.

1. Add the following parameters to `/etc/datadog-agent/datadog.yaml`:

```yaml
gpu:
enabled: true
collect_gpu_tags: true
enable_nvml_detection: true
```

Enabling the `gpu` integration requires `system-probe` to have the configuration option enabled for collecting per-process metrics. Inside the `/etc/datadog-agent/system-probe.yaml` configuration file, the following parameters must be set:
2. Add the following parameter to `/etc/datadog-agent/system-probe.yaml`. This flag loads the eBPF module responsible for per-process GPU metrics and is required even for non-containerized hosts:

```yaml
gpu_monitoring:
enabled: true
```

The check in the Agent configuration file is enabled by default whenever NVIDIA GPUs and their drivers are detected in the system, as long as the `enable_nvml_detection` parameter is set to `true`. However, it can also be configured manually following these steps:
3. Restart both the Agent and system-probe:

```shell
sudo systemctl restart datadog-agent
sudo systemctl restart datadog-agent-sysprobe
```

The check in the Agent configuration file is enabled by default whenever NVIDIA GPUs and their drivers are detected in the system, as long as the `enable_nvml_detection` parameter is set to `true`. The check can also be configured manually following these steps:

1. Edit the `gpu.d/conf.yaml` file, in the `conf.d/` folder at the root of your
Agent's configuration directory, to start collecting your GPU performance data.
Expand Down
Loading