Skip to content

Commit c04fbb9

Browse files
vjfridgeclaudejoepeeplessameerank
authored
[FFL-2449] Add server-side flag evaluation metrics documentation (#37257)
* [FFL-2449] Add server-side flag evaluation metrics setup guide Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [FFL-2449] Update existing feature flag pages to reference evaluation metrics guide Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [FFL-2449] Add Feature Flag Graphs concepts page Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [FFL-2449] Mark feature_flag.evaluations metric as experimental in server SDK pages Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [FFL-2449] Fix Vale warnings in new content Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * clean up flag_graphs page * [FFL-2449] Add flag graphs screenshots Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * clean up server eval setup page * clean up server feature flags * clean up server feature flags pt2 * clean up SDK pages: remove DD_METRICS_OTEL_ENABLED, add further_reading links Remove DD_METRICS_OTEL_ENABLED from all server SDK pages and replace with comments pointing to the setup guide, matching the _index.md pattern. Add further_reading links to server_flag_evaluation_metrics and flag_graphs on all SDK pages. Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * replace dark mode screenshots with light mode versions Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * add new Graphs page to side nav menu * [FFL-2449] Incorporate sameerank and joepeeples's feedback Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [FFL-2449] Update flag list screenshot Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [FFL-2449] Note Python tracer defaults to gRPC for OTLP --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Joe Peeples <joe.peeples@datadoghq.com> Co-authored-by: Sameeran Kunche <sameeran.kunche@datadoghq.com>
1 parent 8fa4641 commit c04fbb9

18 files changed

Lines changed: 298 additions & 12 deletions

File tree

‎config/_default/menus/main.en.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6332,6 +6332,11 @@ menu:
63326332
parent: feature_flags_concepts
63336333
identifier: feature_flags_history
63346334
weight: 307
6335+
- name: Feature Flag Graphs
6336+
url: feature_flags/concepts/flag_graphs
6337+
parent: feature_flags_concepts
6338+
identifier: feature_flags_concepts_flag_graphs
6339+
weight: 308
63356340
- name: Implementation Patterns
63366341
url: feature_flags/implementation_patterns
63376342
parent: feature_flags

‎content/en/feature_flags/concepts/_index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,5 +13,6 @@ Learn how Datadog Feature Flags work and how to configure flags, environments, t
1313
{{< nextlink href="/feature_flags/concepts/traffic_splitting" >}}Traffic Splitting and Randomization{{< /nextlink >}}
1414
{{< nextlink href="/feature_flags/concepts/distribution_channels" >}}Distribution Channels{{< /nextlink >}}
1515
{{< nextlink href="/feature_flags/concepts/flag_history" >}}Flag History{{< /nextlink >}}
16+
{{< nextlink href="/feature_flags/concepts/flag_graphs" >}}Feature Flag Graphs{{< /nextlink >}}
1617
{{< nextlink href="/feature_flags/concepts/stale_flags" >}}Stale Flags{{< /nextlink >}}
1718
{{< /whatsnext >}}

‎content/en/feature_flags/concepts/flag_graphs.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
title: Feature Flag Graphs
3+
description: Understand the graphs on the feature flags list and details pages and how to use them to monitor your rollout.
4+
further_reading:
5+
- link: "/feature_flags/server/"
6+
tag: "Documentation"
7+
text: "Server-Side Feature Flags"
8+
- link: "/feature_flags/client/"
9+
tag: "Documentation"
10+
text: "Client-Side Feature Flags"
11+
- link: "/dashboards/"
12+
tag: "Documentation"
13+
text: "Dashboards"
14+
---
15+
16+
## Overview
17+
18+
Datadog Feature Flags provides graphs at two levels: the flags list page gives a snapshot of all your flags at a glance, and the flag details page shows in-depth graphs for a single flag.
19+
20+
## Flags list page
21+
22+
Each summary row shows a total evaluation count and a mini graph, so you can assess activity across your flag inventory without opening each flag.
23+
24+
{{< img src="feature_flags/flag_graphs/flag_list.png" alt="Feature flags list page showing evaluation counts and sparklines per flag" style="width:100%;" >}}
25+
26+
Each row shows:
27+
28+
- **Evaluation count**: Total number of client and server evaluations in the past hour
29+
- **Mini graph**: Individual client and server evaluations in the past hour, grouped by variant
30+
- **Variants**: The variant colors are the key for the mini graph
31+
32+
## Flag details page
33+
34+
The flag details page includes observability insights to help you identify how a single flag is performing.
35+
36+
### Targeting rule evaluation counts
37+
38+
This shows the total client and server evaluations that fell through to each targeting rule in the given **Real-time metric overview** time range.
39+
40+
{{< img src="feature_flags/flag_graphs/targeting_rule_distribution.png" alt="Targeting rule evaluation count" style="width:100%;" >}}
41+
42+
Use this count to confirm that targeting rules are working as expected and to see how traffic is distributed across targeting rules in a given environment, including the percentage of total traffic in the time range.
43+
44+
### Client and server evaluations
45+
46+
{{< img src="feature_flags/flag_graphs/evaluations.png" alt="Client and server evaluations graph split by variant" style="width:100%;" >}}
47+
48+
Client SDK evaluations have two views, each grouped by variant:
49+
50+
- **Unique**: Count of unique users or entities that evaluated each variant (deduplicated by targeting key)
51+
- **Total**: Count of all evaluations by variant, including repeated evaluations of the same targeting key
52+
53+
Server SDK evaluations show a count of all evaluations by variant. This requires flag evaluation metrics to be enabled. See [Set Up Server-Side Flag Evaluation Metrics][1].
54+
55+
### Errors and latency
56+
57+
{{< img src="feature_flags/flag_graphs/errors_latency.png" alt="Error and latency graphs" style="width:100%;" >}}
58+
59+
Error rate and latency graphs are available for client evaluations. They are not available for server evaluations.
60+
61+
## Export to a dashboard
62+
63+
{{< img src="feature_flags/flag_graphs/dashboard_export.png" alt="Export graph to a dashboard" style="width:100%;" >}}
64+
65+
Choose **Save to dashboard** to add a graph to a Datadog dashboard. From there, you can filter and group the data for further insights.
66+
67+
## Further reading
68+
69+
{{< partial name="whats-next/whats-next.html" >}}
70+
71+
[1]: /feature_flags/guide/server_flag_evaluation_metrics/

‎content/en/feature_flags/guide/_index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ cascade:
1212

1313
{{< whatsnext desc="Feature Flags Guides:" >}}
1414
{{< nextlink href="/getting_started/feature_flags" >}}Getting started with Feature Flags{{< /nextlink >}}
15+
{{< nextlink href="/feature_flags/guide/server_flag_evaluation_metrics" >}}Set Up Server-Side Flag Evaluation Metrics{{< /nextlink >}}
1516
{{< nextlink href="/feature_flags/guide/migrate_flags_with_cli" >}}Migrate Flags with the Flag Migration CLI{{< /nextlink >}}
1617
{{< nextlink href="/feature_flags/guide/migrate_from_launchdarkly" >}}Migrate Your Feature Flags from LaunchDarkly{{< /nextlink >}}
1718
{{< nextlink href="/feature_flags/guide/migrate_from_statsig" >}}Migrate Your Feature Flags from Statsig{{< /nextlink >}}

‎content/en/feature_flags/guide/server_flag_evaluation_metrics.md‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
---
2+
title: Set Up Server-Side Flag Evaluation Metrics
3+
description: Configure the Datadog Agent and your application to emit and visualize flag evaluation metrics for server-side feature flags.
4+
further_reading:
5+
- link: "/feature_flags/server/"
6+
tag: "Documentation"
7+
text: "Server-Side Feature Flags"
8+
- link: "/feature_flags/concepts/flag_graphs/"
9+
tag: "Documentation"
10+
text: "Feature Flag Graphs"
11+
- link: "/metrics/"
12+
tag: "Documentation"
13+
text: "Metrics"
14+
- link: "/dashboards/"
15+
tag: "Documentation"
16+
text: "Dashboards"
17+
---
18+
19+
## Overview
20+
21+
Flag evaluation metrics let you measure how often each variant of a feature flag is returned by your server-side application. Use these metrics to track flag adoption over time, verify targeting rules are working as expected, and graph flag evaluation data on dashboards.
22+
23+
<div class="alert alert-warning">The <code>feature_flag.evaluations</code> metric is experimental and may change or be removed in a future release.</div>
24+
25+
## Prerequisites
26+
27+
Before setting up flag evaluation metrics, confirm the following:
28+
29+
- [Server-side feature flags][1] are already configured.
30+
- Datadog Agent 7.32.0 or later is running.
31+
- `DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true` is set on your application.
32+
- Your server-side tracer meets the minimum version for flag evaluation metrics support:
33+
34+
| Language | Minimum tracer version |
35+
| -------- | ---------------------- |
36+
| .NET | 3.44.0 |
37+
| Go | 2.8.0 |
38+
| Java | 1.62.0 |
39+
| Node.js | 5.99.0 |
40+
| PHP | 1.21.1 |
41+
| Python | 4.7.0 |
42+
| Ruby | 2.32.0 |
43+
44+
## Step 1: Enable the Agent OTLP receiver
45+
46+
Flag evaluation metrics are emitted over OpenTelemetry (OTLP). The Datadog Agent includes an OTLP receiver that is off by default. For setup instructions, see [OTLP Ingestion by the Datadog Agent][2].
47+
48+
You only need to enable the protocol your application uses (gRPC on port 4317, or HTTP on port 4318).
49+
50+
<div class="alert alert-info">If you are running Agent v7.61.0 or later in Docker, set <code>HOST_PROC=/proc</code> on the Agent container to work around a known issue with the OTLP pipeline.</div>
51+
52+
## Step 2: Configure your application
53+
54+
Set the following environment variable on your application, in addition to the standard [server-side feature flag configuration][1]:
55+
56+
{{< code-block lang="bash" >}}
57+
# Enable flag evaluation metrics
58+
DD_METRICS_OTEL_ENABLED=true
59+
{{< /code-block >}}
60+
61+
By default, most tracers send OTLP metrics to the Agent at `DD_AGENT_HOST` on port `4318` (HTTP). If your application already sets `DD_AGENT_HOST` to reach the Agent, no endpoint configuration is required.
62+
63+
Set an OTLP endpoint explicitly in any of these cases:
64+
65+
- The Agent is not reachable at `DD_AGENT_HOST` on the default OTLP port (for example, a remote Agent or a non-default port).
66+
- You use the **Java** tracer. The Java tracer does not derive the endpoint from `DD_AGENT_HOST`; it defaults to `localhost:4318`. Set the endpoint whenever the Agent is not on `localhost`.
67+
- You use the **Python** tracer. The Python tracer defaults to gRPC on port `4317`, not HTTP. Enable the gRPC OTLP receiver on the Agent, or override the protocol to use HTTP instead:
68+
69+
{{< code-block lang="bash" >}}
70+
OTEL_EXPORTER_OTLP_ENDPOINT=http://<AGENT_HOST>:4318
71+
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
72+
{{< /code-block >}}
73+
74+
To set the endpoint, use the standard OpenTelemetry variable:
75+
76+
{{< code-block lang="bash" >}}
77+
# Point OTLP data at the Datadog Agent (HTTP, port 4318)
78+
OTEL_EXPORTER_OTLP_ENDPOINT=http://<AGENT_HOST>:4318
79+
80+
# Or use gRPC (port 4317). For most tracers, the default protocol is http/protobuf,
81+
# so set the protocol explicitly when switching to gRPC:
82+
# OTEL_EXPORTER_OTLP_ENDPOINT=http://<AGENT_HOST>:4317
83+
# OTEL_EXPORTER_OTLP_PROTOCOL=grpc
84+
{{< /code-block >}}
85+
86+
Replace `<AGENT_HOST>` with the hostname or IP address of your Datadog Agent.
87+
88+
Docker Compose example:
89+
90+
{{< code-block lang="yaml" filename="docker-compose.yml" >}}
91+
services:
92+
datadog-agent:
93+
environment:
94+
- DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT=0.0.0.0:4317
95+
- DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_HTTP_ENDPOINT=0.0.0.0:4318
96+
- HOST_PROC=/proc # If running Agent v7.61.0+ in Docker
97+
98+
app-go:
99+
environment:
100+
- DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true
101+
- DD_METRICS_OTEL_ENABLED=true
102+
- OTEL_EXPORTER_OTLP_ENDPOINT=http://datadog-agent:4318
103+
depends_on:
104+
datadog-agent:
105+
condition: service_healthy
106+
{{< /code-block >}}
107+
108+
## Step 3: Verify metrics are flowing
109+
110+
After deploying, confirm metrics are reaching Datadog:
111+
112+
1. Go to [Metrics Explorer][3] and search for `feature_flag.evaluations`.
113+
2. If the metric does not appear within a few minutes of your application evaluating flags, check:
114+
- The Agent OTLP receiver is enabled and the correct port is exposed.
115+
- `OTEL_EXPORTER_OTLP_ENDPOINT` points to the Agent, not a separate collector.
116+
- Your application is actively evaluating flags with a server SDK at runtime (the code path is being executed).
117+
118+
## Step 4: Enable metric retention
119+
120+
By default, `feature_flag.evaluations` retains only one hour of data. To retain a longer history:
121+
122+
1. Go to [Metrics Summary][4] and search for `feature_flag.evaluations`.
123+
2. Select the metric and enable **Historical Metrics**.
124+
125+
This is an opt-in setting and is not enabled automatically for OTLP metrics.
126+
127+
## Graph flag evaluations on a dashboard
128+
129+
Use the following query to graph flag evaluations by flag key and variant on a [dashboard][5]:
130+
131+
{{< code-block lang="text" >}}
132+
sum:feature_flag.evaluations{*} by {feature_flag.key,feature_flag.result.variant}
133+
{{< /code-block >}}
134+
135+
The `feature_flag.evaluations` metric is a counter with the following tags:
136+
137+
| Tag | Description |
138+
| ------------------------------------ | -------------------------------------------------- |
139+
| `feature_flag.key` | The flag key being evaluated |
140+
| `feature_flag.result.variant` | The variant returned by the evaluation |
141+
| `feature_flag.result.reason` | The reason for the evaluation result |
142+
| `feature_flag.result.allocation_key` | The identifier for the evaluated targeting rules (emitted only when present) |
143+
| `error.type` | The error type (emitted only on error evaluations) |
144+
145+
## Further reading
146+
147+
{{< partial name="whats-next/whats-next.html" >}}
148+
149+
[1]: /feature_flags/server/
150+
[2]: /opentelemetry/setup/otlp_ingest_in_the_agent/
151+
[3]: https://app.datadoghq.com/metric/explorer
152+
[4]: https://app.datadoghq.com/metric/summary
153+
[5]: /dashboards/

‎content/en/feature_flags/server/_index.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@ further_reading:
88
- link: "/remote_configuration/"
99
tag: "Documentation"
1010
text: "Remote Configuration"
11+
- link: "/feature_flags/guide/server_flag_evaluation_metrics/"
12+
tag: "Guide"
13+
text: "Set Up Server-Side Flag Evaluation Metrics"
14+
- link: "/feature_flags/concepts/flag_graphs/"
15+
tag: "Concept"
16+
text: "Feature Flag Graphs"
1117
- link: "/feature_flags/implementation_patterns/serverless/"
1218
tag: "Documentation"
1319
text: "Serverless environments and Feature Flags"
@@ -75,14 +81,14 @@ DD_TRACE_AGENT_PORT=8126
7581
DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true
7682

7783
# Optional: Enable flag evaluation metrics
78-
DD_METRICS_OTEL_ENABLED=true
84+
# See "Set Up Server-Side Flag Evaluation Metrics" documentation
7985
{{< /code-block >}}
8086

8187
<div class="alert alert-warning">The <code>DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true</code> environment variable is required to enable the feature flagging provider. Java also supports the system property <code>-Ddd.experimental.flagging.provider.enabled=true</code>, and Ruby and Node.js support code-based configuration as an alternative. See the SDK-specific documentation for details.</div>
8288

8389
<div class="alert alert-info">Remote Configuration must be available for server-side Feature Flags. It is enabled by default on Agent 7.47.0 and later. Only set SDK-level Remote Configuration variables (such as <code>DD_REMOTE_CONFIG_ENABLED=true</code>) if your tracer has Remote Configuration disabled and you need to override that setting.</div>
8490

85-
<div class="alert alert-info">Set <code>DD_METRICS_OTEL_ENABLED=true</code> to enable flag evaluation metrics. Without this, the SDK does not emit metrics for flag evaluations. When enabled, each evaluation records a <code>feature_flag.evaluations</code> counter metric tagged with the flag key, result variant, and evaluation reason.</div>
91+
See <a href="/feature_flags/guide/server_flag_evaluation_metrics/">Set Up Server-Side Flag Evaluation Metrics</a> to enable the experimental <code>feature_flag.evaluations</code> metric. See <a href="/feature_flags/concepts/flag_graphs/">Feature Flag Graphs</a> for more information on available graphing.
8692

8793
## Testing with in-memory providers
8894

‎content/en/feature_flags/server/dotnet.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@ further_reading:
88
- link: "/tracing/trace_collection/dd_libraries/dotnet-core/"
99
tag: "Documentation"
1010
text: ".NET Tracing"
11+
- link: "/feature_flags/guide/server_flag_evaluation_metrics/"
12+
tag: "Guide"
13+
text: "Set Up Server-Side Flag Evaluation Metrics"
14+
- link: "/feature_flags/concepts/flag_graphs/"
15+
tag: "Concept"
16+
text: "Feature Flag Graphs"
1117
---
1218

1319
## Overview
@@ -33,7 +39,7 @@ Set the following environment variables:
3339
DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true
3440

3541
# Optional: Enable flag evaluation metrics
36-
DD_METRICS_OTEL_ENABLED=true
42+
# See "Set Up Server-Side Flag Evaluation Metrics" documentation
3743

3844
# Required: Service identification
3945
DD_SERVICE=<YOUR_SERVICE_NAME>
@@ -42,6 +48,8 @@ DD_ENV=<YOUR_ENVIRONMENT>
4248

4349
<div class="alert alert-info">The <code>EXPERIMENTAL_</code> prefix is retained for backwards compatibility; the provider itself is stable.</div>
4450

51+
See <a href="/feature_flags/guide/server_flag_evaluation_metrics/">Set Up Server-Side Flag Evaluation Metrics</a> to enable the experimental <code>feature_flag.evaluations</code> metric. See <a href="/feature_flags/concepts/flag_graphs/">Feature Flag Graphs</a> for more information on available graphing.
52+
4553
## Installation
4654

4755
Install the Datadog [.NET SDK][3] and [OpenFeature SDK][4] using NuGet:

‎content/en/feature_flags/server/go.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@ further_reading:
88
- link: "/tracing/trace_collection/dd_libraries/go/"
99
tag: "Documentation"
1010
text: "Go Tracing"
11+
- link: "/feature_flags/guide/server_flag_evaluation_metrics/"
12+
tag: "Guide"
13+
text: "Set Up Server-Side Flag Evaluation Metrics"
14+
- link: "/feature_flags/concepts/flag_graphs/"
15+
tag: "Concept"
16+
text: "Feature Flag Graphs"
1117
---
1218

1319
## Overview
@@ -31,7 +37,7 @@ Set the following environment variables:
3137
DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true
3238

3339
# Optional: Enable flag evaluation metrics
34-
DD_METRICS_OTEL_ENABLED=true
40+
# See "Set Up Server-Side Flag Evaluation Metrics" documentation
3541

3642
# Required: Service identification
3743
DD_SERVICE=<YOUR_SERVICE_NAME>
@@ -40,6 +46,8 @@ DD_ENV=<YOUR_ENVIRONMENT>
4046

4147
<div class="alert alert-info">The <code>EXPERIMENTAL_</code> prefix is retained for backwards compatibility; the provider itself is stable.</div>
4248

49+
See <a href="/feature_flags/guide/server_flag_evaluation_metrics/">Set Up Server-Side Flag Evaluation Metrics</a> to enable the experimental <code>feature_flag.evaluations</code> metric. See <a href="/feature_flags/concepts/flag_graphs/">Feature Flag Graphs</a> for more information on available graphing.
50+
4351
## Installation
4452

4553
Install the Datadog OpenFeature provider package:

0 commit comments

Comments
 (0)