Skip to content

Commit 271b60d

Browse files
Document Dynamic Tests for NetFlow setup (#37677)
* empty * Document NetFlow Dynamic Tests setup * Avoid unrelated Network Path link label churn * Normalize NetFlow Dynamic Tests wording * Document NetFlow self-source skip behavior * Update content/en/network_monitoring/network_path/glossary.md Co-authored-by: jeff-morgan-dd <jeff.morgan@datadoghq.com> --------- Co-authored-by: jeff-morgan-dd <jeff.morgan@datadoghq.com>
1 parent 1d63226 commit 271b60d

4 files changed

Lines changed: 111 additions & 14 deletions

File tree

content/en/network_monitoring/netflow/_index.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,9 @@ further_reading:
66
- link: "/network_monitoring/devices/profiles"
77
tag: "Documentation"
88
text: "Using Profiles with Network Device Monitoring"
9+
- link: "/network_monitoring/network_path/setup/#dynamic-tests-for-netflow-experimental"
10+
tag: "Documentation"
11+
text: "Set up Dynamic Tests for NetFlow"
912
- link: "https://www.datadoghq.com/blog/monitor-netflow-with-datadog/"
1013
tag: "Blog"
1114
text: "Monitor NetFlow traffic data with Datadog"
@@ -125,6 +128,12 @@ You can visualize the flows in NetFlow Monitoring by clicking on the {{< ui >}}F
125128

126129
{{< img src="network_device_monitoring/netflow/flows.png" alt="Hover over a flow aggregated from a device emitting netflow to access related network connections" width="100%" >}}
127130

131+
## Network Path for NetFlow
132+
133+
Dynamic Tests for NetFlow can automatically run Network Path tests from the Agent that collects NetFlow traffic to destination IPs observed in NetFlow records. Use Dynamic Tests for NetFlow to add hop-by-hop route and latency context to your NetFlow destinations.
134+
135+
Dynamic Tests for NetFlow are experimental and require Agent `v7.81+`. To set up Dynamic Tests for NetFlow, see [Network Path setup][11].
136+
128137
## NetFlow monitor
129138

130139
Click on the {{< ui >}}Create Monitor{{< /ui >}} icon from any of the views to create a [NetFlow monitor][6]. When creating the monitor, consider the following fields with respect to the source IP or destination IP from the perspective of the device. These fields provide insights into network traffic patterns and help with optimizing performance and security.
@@ -371,3 +380,4 @@ Use the `netstat -s` command to see if there are any dropped UDP packets:
371380
[8]: https://github.com/DataDog/datadog-agent/blob/f6ae461a7d22aaf398de5a94d9330694d69560d6/pkg/config/config_template.yaml#L4203-L4275
372381
[9]: /network_monitoring/devices/troubleshooting#traps-or-flows-not-being-received-at-all
373382
[10]: https://app.datadoghq.com/devices/settings/enrichment/ip
383+
[11]: /network_monitoring/network_path/setup/#dynamic-tests-for-netflow-experimental

content/en/network_monitoring/network_path/_index.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,10 +33,11 @@ The following diagram depicts the typical flow of a network path from a source (
3333

3434
## Setup methods
3535

36-
Network Path supports two Agent-based collection methods. You can use either method on its own or both together:
36+
Network Path supports multiple Agent-based collection methods. You can use one method on its own or combine multiple methods:
3737

3838
- **[Scheduled tests][6]**: Monitor specific network paths by defining source-destination pairs in the Agent configuration file. Use scheduled tests to continuously monitor a known set of endpoints, such as critical APIs or partner services.
3939
- **[Dynamic tests][7]**: Automatically discover and monitor network paths based on actual network traffic observed by [Cloud Network Monitoring][8]. Use dynamic tests for broad visibility without manually listing every destination.
40+
- **[Dynamic Tests for NetFlow][10]**: Automatically run Network Path tests from the Agent host to destination IPs observed in [NetFlow Monitoring][11]. Use Dynamic Tests for NetFlow to add hop-by-hop route visibility to NetFlow traffic without manually configuring individual destinations.
4041

4142
To create Network Path tests in Synthetic Monitoring instead, see [Network Path Testing in Synthetic Monitoring][9].
4243

@@ -62,3 +63,5 @@ Use the following views and tools to set up Network Path and investigate network
6263
[7]: /network_monitoring/network_path/setup/#dynamic-tests
6364
[8]: /network_monitoring/cloud_network_monitoring/
6465
[9]: /synthetics/network_path_tests/
66+
[10]: /network_monitoring/network_path/setup/#dynamic-tests-for-netflow-experimental
67+
[11]: /network_monitoring/netflow/

content/en/network_monitoring/network_path/glossary.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,12 +11,14 @@ Network Path provides hop-by-hop visibility into the route between a source and
1111
| Concept | Description |
1212
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
1313
| **[Network Path][6]** | Network Path provides hop-by-hop visibility into the route between a source and a destination, so you can identify where latency, packet loss, or routing changes occur. |
14+
| **[Dynamic Tests for NetFlow][8]** | An experimental Network Path collection method that runs Network Path tests from the Agent host to destination IPs observed in NetFlow traffic. |
15+
| **Origin** | The source that triggered a Network Path test, such as network traffic, NetFlow, a scheduled Agent configuration, or Synthetic Monitoring. Use the `origin` facet to filter paths by collection method. |
1416
| **Autonomous System (AS / ASN)** | A collection of IP routing prefixes managed by a single network operator. Network Path groups hop by Autonomous System (AS) or Autonomous System Number (ASN) to show routing domains along the path.
1517
| **[Path View][7]** | The Network Path visualization that displays each hop, grouped by Autonomous System Number (ASN), region, or network, along with probe status and hop metrics. |
1618
| **Hop** | A network node along a route between source and destination, identified by IP address and associated metadata (ASN, cloud region, provider). | |
17-
| **Source** | The starting point of a Network Path probe, typically an Agent-monitored host or container running the Datadog network monitoring tracer. |
19+
| **Source** | The starting point of a Network Path probe, such as an Agent-monitored host, container, Synthetic location, or the Agent that collects NetFlow traffic. |
1820
| **Destination** | The endpoint that the Network Path probe is targeting, such as a service, public endpoint, or domain. |
19-
| **Traceroute** | The mechanism that Network Path uses to determine intermediate hops and latency. CNM sends controlled probes, similar to traceroute, to discover each hop on the route. |
21+
| **Traceroute** | The mechanism that Network Path uses to determine intermediate hops and latency. Network Path sends controlled probes, similar to traceroute, to discover each hop on the route. |
2022
| **Latency per hop** | The round-trip time between the probe source and each hop. This helps identify slow or congested nodes. |
2123
| **Packet loss per hop** | The percentage of probe packets dropped before reaching or returning from a hop, useful for diagnosing reliability issues. |
2224

@@ -41,4 +43,5 @@ Network Path provides hop-by-hop visibility into the route between a source and
4143

4244

4345
[6]: /network_monitoring/network_path/
44-
[7]: /network_monitoring/network_path/path_view
46+
[7]: /network_monitoring/network_path/path_view
47+
[8]: /network_monitoring/network_path/setup/#dynamic-tests-for-netflow-experimental

content/en/network_monitoring/network_path/setup.md

Lines changed: 91 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,13 @@ Setting up Network Path involves configuring your environment to monitor and tra
2525

2626
<div class="alert alert-info">This page covers Network Path setup for Agent-based configuration in Network Monitoring. To create Network Path tests in Synthetic Monitoring, see <a href="/synthetics/network_path_tests/">Network Path Testing in Synthetic Monitoring</a>.</div>
2727

28-
Datadog provides two Agent-based collection methods. You can use either method on its own or combine both:
28+
Datadog provides three Agent-based collection methods. You can use one method on its own or combine multiple methods:
2929

3030
| Method | When to use |
3131
|--------|-------------|
3232
| **[Scheduled&nbsp;tests](#scheduled-tests)** | Monitor specific source-destination pairs that you define in the Agent configuration. Best for tracking a known set of endpoints, such as critical APIs or partner services. |
3333
| **[Dynamic&nbsp;tests](#dynamic-tests)** | Automatically discover and monitor paths based on traffic observed by [Cloud Network Monitoring][1]. Best for broad visibility without manually listing every destination. |
34+
| **[Dynamic&nbsp;Tests&nbsp;for&nbsp;NetFlow](#dynamic-tests-for-netflow-experimental)** | Automatically run Network Path tests from the Agent host to destination IPs observed in [NetFlow Monitoring][6]. Best for adding hop-by-hop route visibility to NetFlow traffic without manually configuring individual destinations. |
3435

3536
### Scheduled tests
3637

@@ -173,7 +174,7 @@ Agent `v7.72+` is required.
173174
- "tag_key2:tag_value2"
174175
```
175176

176-
3. Restart the Agent after making these configuration changes to start seeing network paths.
177+
3. Restart the Agent after making these configuration changes to start seeing network paths.
177178

178179
{{% /tab %}}
179180
{{% tab "Helm" %}}
@@ -230,6 +231,7 @@ Datadog Autodiscovery allows you to enable Network Path on a per-service basis t
230231
datadog:
231232
traceroute:
232233
enabled: true
234+
```
233235

234236
2. After the module is enabled, Datadog automatically detects Network Path annotations added to your Kubernetes pod. For more information, see [Kubernetes and Integrations][2].
235237

@@ -501,20 +503,86 @@ datadog:
501503
{{% /tab %}}
502504
{{< /tabs >}}
503505

504-
#### Filter syntax
506+
### Dynamic Tests for NetFlow (Experimental)
507+
508+
<div class="alert alert-info">Dynamic Tests for NetFlow are experimental and require Agent <code>v7.81+</code>. To enable this feature, contact Datadog Support or your account team.</div>
509+
510+
Configure Dynamic Tests for NetFlow to run Network Path tests from the Agent host to destination IPs observed in NetFlow records. Dynamic Tests for NetFlow do not require [Cloud Network Monitoring][1] or `network_path.connections_monitoring.enabled`.
511+
512+
Dynamic Tests for NetFlow run from the Datadog Agent that collects NetFlow traffic. They do not run from the NetFlow exporter, router, or original flow source. Deploy the Agent close enough to the observed flow sources for traceroutes from that Agent to represent the paths you want to investigate.
513+
514+
**Prerequisites**:
515+
516+
- [NetFlow Monitoring][6] must be configured and receiving flows.
517+
- Agent `v7.81+` is required.
518+
519+
{{< tabs >}}
520+
{{% tab "Linux" %}}
521+
522+
1. Enable the `system-probe` traceroute module in `/etc/datadog-agent/system-probe.yaml` by adding the following:
523+
524+
```yaml
525+
traceroute:
526+
enabled: true
527+
```
528+
529+
2. Enable Dynamic Tests for NetFlow in `/etc/datadog-agent/datadog.yaml`:
530+
531+
```yaml
532+
network_path:
533+
netflow_monitoring:
534+
enabled: true
535+
collector:
536+
monitor_ip_without_domain: true
537+
```
538+
539+
`monitor_ip_without_domain: true` is required because Dynamic Tests for NetFlow target observed destination IP addresses and the Network Path collector skips IP-only targets by default.
540+
541+
3. Restart the Agent after making these configuration changes.
542+
543+
{{% /tab %}}
544+
{{% tab "Windows" %}}
545+
546+
1. Enable the `system-probe` traceroute module in `%ProgramData%\Datadog\system-probe.yaml` by adding the following:
547+
548+
```yaml
549+
traceroute:
550+
enabled: true
551+
```
552+
553+
2. Enable Dynamic Tests for NetFlow in `%ProgramData%\Datadog\datadog.yaml`:
554+
555+
```yaml
556+
network_path:
557+
netflow_monitoring:
558+
enabled: true
559+
collector:
560+
monitor_ip_without_domain: true
561+
```
562+
563+
`monitor_ip_without_domain: true` is required because Dynamic Tests for NetFlow target observed destination IP addresses and the Network Path collector skips IP-only targets by default.
564+
565+
3. Restart the Agent after making these configuration changes.
566+
567+
{{% /tab %}}
568+
{{< /tabs >}}
569+
570+
After the Agent reports paths, open the [Network Path][4] UI and filter for `origin:netflow` to view paths generated from NetFlow traffic.
571+
572+
### Filter syntax
505573

506574
Configure filters to include or exclude domains and IPs, allowing you to:
507575

508576
- Reduce monitoring overhead for internal networks
509577
- Focus on external traffic patterns
510578
- Exclude known infrastructure ranges that don't require monitoring
511579

580+
The same `network_path.collector.filters` list applies to dynamic tests and Dynamic Tests for NetFlow. For Dynamic Tests for NetFlow, use `match_ip` filters because Dynamic Tests for NetFlow target observed destination IP addresses.
581+
512582
To include or exclude specific domains or IP ranges from dynamic tests, add the following to your `/etc/datadog-agent/datadog.yaml` file:
513583

514584
```yaml
515585
network_path:
516-
connections_monitoring:
517-
enabled: true
518586
collector:
519587
filters:
520588
# exclude single domain
@@ -596,8 +664,23 @@ If no data appears in the [Network Path][4] UI, the feature may not be fully ena
596664

597665
2. At least one Network Path feature must be active, such as:
598666

599-
- [Individual paths](#monitor-individual-paths) configured through the `conf.d/network_path.d` file.
600-
- Experimental [network traffic paths](#network-traffic-paths-experimental) configured by enabling both `network_path.connections_monitoring` and [Cloud Network Monitoring][1](CNM).
667+
- [Scheduled tests](#scheduled-tests) configured through the `conf.d/network_path.d` file.
668+
- [Dynamic tests](#dynamic-tests) configured by enabling both `network_path.connections_monitoring.enabled` and [Cloud Network Monitoring][1].
669+
- [Dynamic Tests for NetFlow](#dynamic-tests-for-netflow-experimental) configured by enabling `network_path.netflow_monitoring.enabled` and [NetFlow Monitoring][6].
670+
671+
### No Dynamic Tests for NetFlow data in the UI
672+
673+
If no paths with `origin:netflow` appear in the [Network Path][4] UI, verify the following:
674+
675+
1. The Agent is version `7.81+`.
676+
2. [NetFlow Monitoring][6] is enabled and receiving flows.
677+
3. The traceroute module is enabled in `system-probe.yaml`.
678+
4. `network_path.netflow_monitoring.enabled` and `network_path.collector.monitor_ip_without_domain` are set to `true` in `datadog.yaml`.
679+
5. Your `network_path.collector.filters` configuration does not exclude the destination IPs you expect to monitor.
680+
681+
Dynamic Tests for NetFlow automatically skip NetFlow records whose source IP is assigned to the Agent host before destination filters are evaluated. This prevents self-scheduling loops and is expected behavior. For NAT or alias cases where Agent-sourced traffic appears from another source IP, use `network_path.collector.source_excludes` to exclude those source IPs.
682+
683+
Then filter the Network Path UI for `origin:netflow`.
601684

602685
### Error: status code: 404
603686

@@ -620,7 +703,5 @@ If you encounter an error like the following:
620703
[3]: /help
621704
[4]: https://app.datadoghq.com/network/path
622705
[5]: https://github.com/DataDog/datadog-agent/blob/main/cmd/agent/dist/conf.d/network_path.d/conf.yaml.example
706+
[6]: /network_monitoring/netflow/
623707
[15]: /synthetics/network_path_tests/
624-
625-
626-

0 commit comments

Comments
 (0)