Skip to content

Commit 9fe93b9

Browse files
apatruddogmaycmlee
andauthored
updates docs with metrics otel information (#35645)
* updates docs with metrics otel information * Small updates to documentation * revert otlp/http -> otlphttp * Update content/en/observability_pipelines/sources/opentelemetry.md Co-authored-by: May Lee <may.lee@datadoghq.com> * Update content/en/observability_pipelines/sources/datadog_agent.md Co-authored-by: May Lee <may.lee@datadoghq.com> --------- Co-authored-by: May Lee <may.lee@datadoghq.com>
1 parent 9e7218d commit 9fe93b9

3 files changed

Lines changed: 116 additions & 15 deletions

File tree

content/en/observability_pipelines/sources/datadog_agent.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ further_reading:
2020

2121
Use Observability Pipelines' Datadog Agent source to receive logs or metrics ({{< tooltip glossary="preview" case="title" >}}) from the Datadog Agent.
2222

23-
**Note**: If you are using the Datadog Distribution of OpenTelemetry (DDOT) Collector for logs, you must [use the OpenTelemetry source to send logs to Observability Pipelines][4].
23+
**Note**: If you are using the Datadog Distribution of OpenTelemetry (DDOT) Collector to collect logs or metrics, you must [use the OpenTelemetry source to send that data to Observability Pipelines][4].
2424

2525
## Prerequisites
2626

@@ -163,7 +163,7 @@ datadog:
163163
{{< partial name="whats-next/whats-next.html" >}}
164164

165165
[1]: /observability_pipelines/configuration/set_up_pipelines/
166-
[4]: /observability_pipelines/sources/opentelemetry/#send-logs-from-the-datadog-distribution-of-opentelemetry-collector-to-observability-pipelines
166+
[4]: /observability_pipelines/sources/opentelemetry/#send-data-from-the-datadog-distribution-of-opentelemetry-collector-to-observability-pipelines
167167
[5]: /observability_pipelines/configuration/install_the_worker/advanced_worker_configurations/
168168
[6]: https://app.datadoghq.com/observability-pipelines
169169
[7]: /api/latest/observability-pipelines/

content/en/observability_pipelines/sources/opentelemetry.md

Lines changed: 110 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -9,16 +9,19 @@ products:
99
- name: Logs
1010
icon: logs
1111
url: /observability_pipelines/configuration/?tab=logs#pipeline-types
12+
- name: Metrics
13+
icon: metrics
14+
url: /observability_pipelines/configuration/?tab=metrics#pipeline-types
1215
---
1316

1417
{{< product-availability >}}
1518

1619
## Overview
1720

18-
Use Observability Pipelines' OpenTelemetry (OTel) source to collect logs from your OTel Collector through HTTP or gRPC. Select and set up this source when you set up a pipeline. The information below is configured in the pipelines UI.
21+
Use Observability Pipelines' OpenTelemetry (OTel) source to collect logs or metrics ({{< tooltip glossary="preview" case="title" >}}) from your OTel Collector through HTTP or gRPC. Select and set up this source when you set up a pipeline.
1922

2023
**Notes**:
21-
- If you are using the Datadog Distribution of OpenTelemetry (DDOT) Collector, use the OpenTelemetry source to [send logs to Observability Pipelines](#send-logs-from-the-datadog-distribution-of-opentelemetry-collector-to-observability-pipelines).
24+
- If you are using the Datadog Distribution of OpenTelemetry (DDOT) Collector, use the OpenTelemetry source to [send data to Observability Pipelines](#send-data-from-the-datadog-distribution-of-opentelemetry-collector-to-observability-pipelines).
2225
- If you are using the Splunk HEC Distribution of the OpenTelemetry Collector, use the [Splunk HEC source][4] to send logs to Observability Pipelines.
2326

2427
### When to use this source
@@ -60,10 +63,10 @@ Toggle the switch to enable TLS. The following certificate and key files are req
6063
{{% tab "Secrets Management" %}}
6164

6265
- HTTP address identifier:
63-
- References the HTTP socket address on which the Observability Pipelines Worker listens for logs from the OTel collector.
66+
- References the HTTP socket address on which the Observability Pipelines Worker listens for data from the OTel collector.
6467
- The default identifier is `SOURCE_OTEL_HTTP_ADDRESS`.
6568
- gRPC address identifier:
66-
- References the gRPC socket address on which the Observability Pipelines Worker listens for logs from the OTel collector.
69+
- References the gRPC socket address on which the Observability Pipelines Worker listens for data from the OTel collector.
6770
- The default identifier is `SOURCE_OTEL_GRPC_ADDRESS`.
6871
- TLS passphrase identifier (when TLS is enabled):
6972
- The default identifier is `SOURCE_OTEL_KEY_PASS`.
@@ -77,17 +80,20 @@ Toggle the switch to enable TLS. The following certificate and key files are req
7780
{{% /tab %}}
7881
{{< /tabs >}}
7982

80-
## Send logs to the Observability Pipelines Worker
83+
## Send data to the Observability Pipelines Worker
8184

8285
Configure your OTel exporters to point to HTTP or gRPC. The Worker exposes configurable listener ports for each protocol.
8386

8487
<div class="alert alert-info">The ports 4318 (HTTP) and 4317 (gRPC) shown below are examples only. You can configure the port value for either protocol in the Worker. Ensure your OTel exporters match the port value you choose.</a></div>
8588

89+
{{< tabs >}}
90+
{{% tab "Logs" %}}
91+
8692
### HTTP configuration example
8793

8894
The Worker exposes the HTTP endpoint on port 4318, which is the default port. You can configure the port value in the Worker.
8995

90-
For example, to configure an OTel exporter over HTTP in Python:
96+
For example, to configure an OTel log exporter over HTTP in Python:
9197

9298
```python
9399
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter
@@ -100,7 +106,7 @@ For example, to configure an OTel exporter over HTTP in Python:
100106

101107
The Worker exposes the gRPC endpoint on port 4317, which is the default port. You can configure the port value in the Worker.
102108

103-
For example, to configure an OTel exporter over gRPC in Python:
109+
For example, to configure an OTel log exporter over gRPC in Python:
104110

105111
```python
106112
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
@@ -114,7 +120,48 @@ Set the listener address environment variables to the following default values.
114120
- HTTP listener address: `worker:4318`
115121
- gRPC listener address: `worker:4317`
116122

117-
## Send logs from the Datadog Distribution of OpenTelemetry Collector to Observability Pipelines
123+
{{% /tab %}}
124+
125+
{{% tab "Metrics" %}}
126+
127+
### HTTP configuration example
128+
129+
The Worker exposes the HTTP endpoint on port 4318, which is the default port. You can configure the port value in the Worker.
130+
131+
For example, to configure an OTel metric exporter over HTTP in Python:
132+
133+
```python
134+
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
135+
http_exporter = OTLPMetricExporter(
136+
endpoint="http://worker:4318/v1/metrics"
137+
)
138+
```
139+
140+
### gRPC configuration example
141+
142+
The Worker exposes the gRPC endpoint on port 4317, which is the default port. You can configure the port value in the Worker.
143+
144+
For example, to configure an OTel metric exporter over gRPC in Python:
145+
146+
```python
147+
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
148+
grpc_exporter = OTLPMetricExporter(
149+
endpoint="grpc://worker:4317"
150+
)
151+
```
152+
153+
Set the listener address environment variables to the following default values. If you configured different port values in the Worker, use those instead.
154+
155+
- HTTP listener address: `worker:4318`
156+
- gRPC listener address: `worker:4317`
157+
158+
{{% /tab %}}
159+
{{< /tabs >}}
160+
161+
## Send data from the Datadog Distribution of OpenTelemetry Collector to Observability Pipelines
162+
163+
{{< tabs >}}
164+
{{% tab "Logs" %}}
118165

119166
To send logs from the Datadog Distribution of the OpenTelemetry (DDOT) Collector:
120167
1. Deploy the DDOT Collector using Helm. See [Install the DDOT Collector as a Kubernetes DaemonSet][5] for instructions.
@@ -127,13 +174,14 @@ To send logs from the Datadog Distribution of the OpenTelemetry (DDOT) Collector
127174
```
128175
exporters:
129176
otlphttp:
130-
endpoint: http://opw-observability-pipelines-worker.default.svc.cluster.local:4318
177+
endpoint: http://opw-observability-pipelines-worker.<NAMESPACE>.svc.cluster.local:4318
131178
...
132179
service:
133180
pipelines:
134181
logs:
135182
exporters: [otlphttp]
136183
```
184+
Replace `<NAMESPACE>` with the Kubernetes namespace where the Observability Pipelines Worker is deployed (for example, `default`).
137185
1. Redeploy the Datadog Agent with the updated [`otel-config.yaml`][9]. For example, if the Agent is installed in Kubernetes:
138186
```
139187
helm upgrade --install datadog-agent datadog/datadog \
@@ -146,6 +194,59 @@ To send logs from the Datadog Distribution of the OpenTelemetry (DDOT) Collector
146194
- `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_ENABLED`
147195
- `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL`
148196
- Logs sent from DDOT might have nested objects that prevent Datadog from parsing the logs correctly. To resolve this, Datadog recommends using the [Custom Processor][8] to flatten the nested `resource` object.
197+
- If the DDOT Collector and the Observability Pipelines Worker are running on the same host, their default OTLP receiver ports (4317/4318) may conflict. In a typical Kubernetes deployment, the Collector and the Worker run in separate pods, so this is not an issue.
198+
199+
[5]: /opentelemetry/setup/ddot_collector/install/kubernetes_daemonset/?tab=datadogoperator
200+
[6]: /observability_pipelines/configuration/set_up_pipelines/
201+
[7]: /observability_pipelines/processors/edit_fields#add-field
202+
[8]: /observability_pipelines/processors/custom_processor
203+
[9]: https://docs.datadoghq.com/opentelemetry/setup/ddot_collector/install/kubernetes_daemonset/?tab=helm#configure-the-opentelemetry-collector
204+
205+
{{% /tab %}}
206+
207+
{{% tab "Metrics" %}}
208+
209+
To send metrics from the Datadog Distribution of the OpenTelemetry (DDOT) Collector:
210+
1. Deploy the DDOT Collector using Helm. See [Install the DDOT Collector as a Kubernetes DaemonSet][5] for instructions.
211+
1. [Set up a pipeline][6] on Observability Pipelines using the [OpenTelemetry source](#set-up-the-source-in-the-pipeline-ui).
212+
1. (Optional) Datadog recommends adding an [Edit Fields processor][7] to the pipeline that appends the field `op_otel_ddot:true`.
213+
1. When you install the Worker, for the OpenTelemetry source environment variables:
214+
1. Set your HTTP listener to `0.0.0.0:4318`.
215+
1. Set your gRPC listener to `0.0.0.0:4317`.
216+
1. After you install the Worker and deployed the pipeline, update the OpenTelemetry Collector's [`otel-config.yaml`][9] to include an exporter that sends metrics to Observability Pipelines. For example:
217+
```
218+
exporters:
219+
otlphttp:
220+
endpoint: http://opw-observability-pipelines-worker.<NAMESPACE>.svc.cluster.local:4318
221+
...
222+
service:
223+
pipelines:
224+
metrics:
225+
exporters: [otlphttp]
226+
```
227+
Replace `<NAMESPACE>` with the Kubernetes namespace where the Observability Pipelines Worker is deployed (for example, `default`).
228+
1. Redeploy the Datadog Agent with the updated [`otel-config.yaml`][9]. For example, if the Agent is installed in Kubernetes:
229+
```
230+
helm upgrade --install datadog-agent datadog/datadog \
231+
--values ./agent.yaml \
232+
--set-file datadog.otelCollector.config=./otel-config.yaml
233+
```
234+
235+
**Notes**:
236+
- Because DDOT is sending metrics to Observability Pipelines, and not the Datadog Agent, the following settings do not work for sending metrics from DDOT to Observability Pipelines:
237+
- `DD_OBSERVABILITY_PIPELINES_WORKER_METRICS_ENABLED`
238+
- `DD_OBSERVABILITY_PIPELINES_WORKER_METRICS_URL`
239+
- Metrics sent from DDOT might have nested objects that prevent Datadog from parsing the metrics correctly. To resolve this, Datadog recommends using the [Custom Processor][8] to flatten the nested `resource` object.
240+
- If the DDOT Collector and the Observability Pipelines Worker are running on the same host, their default OTLP receiver ports (4317/4318) may conflict. In a typical Kubernetes deployment, the Collector and the Worker run in separate pods, so this is not an issue.
241+
242+
[5]: /opentelemetry/setup/ddot_collector/install/kubernetes_daemonset/?tab=datadogoperator
243+
[6]: /observability_pipelines/configuration/set_up_pipelines/
244+
[7]: /observability_pipelines/processors/edit_fields#add-field
245+
[8]: /observability_pipelines/processors/custom_processor
246+
[9]: https://docs.datadoghq.com/opentelemetry/setup/ddot_collector/install/kubernetes_daemonset/?tab=helm#configure-the-opentelemetry-collector
247+
248+
{{% /tab %}}
249+
{{< /tabs >}}
149250
150251
## Further Reading
151252
Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
1-
You must provide both HTTP and gRPC endpoints. Configure your OTLP exporters to point to one of these endpoints. See [Send logs to the Observability Pipelines Worker][10230] for more information.
1+
You must provide both HTTP and gRPC endpoints. Configure your OTLP exporters to point to one of these endpoints. See [Send data to the Observability Pipelines Worker][10230] for more information.
22

33
- HTTP listener address
4-
- The Observability Pipelines Worker listens to this socket address to receive logs from the OTel collector.
4+
- The Observability Pipelines Worker listens to this socket address to receive data from the OTel collector.
55
- The default environment variable is `DD_OP_SOURCE_OTEL_HTTP_ADDRESS`.
66

77
- gRPC listener address
8-
- The Observability Pipelines Worker listens to this socket address to receive logs from the OTel collector.
8+
- The Observability Pipelines Worker listens to this socket address to receive data from the OTel collector.
99
- The default environment variable is `DD_OP_SOURCE_OTEL_GRPC_ADDRESS`.
1010

1111
If TLS is enabled:
1212

1313
- OpenTelemetry TLS passphrase
1414
- The default environment variable is `DD_OP_SOURCE_OTEL_KEY_PASS`.
1515

16-
[10230]: /observability_pipelines/sources/opentelemetry/#send-logs-to-the-observability-pipelines-worker
16+
[10230]: /observability_pipelines/sources/opentelemetry/#send-data-to-the-observability-pipelines-worker

0 commit comments

Comments
 (0)