[Configuration] Redact OTLP header configs in telemetry - #8763
Open
bm1549 wants to merge 6 commits into
Open
Conversation
bm1549
added
the
AI Generated
This comment has been minimized.
This comment has been minimized.
BenchmarksBenchmark execution time: 2026-07-30 00:44:36 Comparing candidate commit 4e92d47 in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 71 metrics, 0 unstable metrics, 63 known flaky benchmarks, 63 flaky benchmarks without significant changes.
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8763) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8763) - mean (71ms) : 68, 75
master - mean (73ms) : 69, 77
section Bailout
This PR (8763) - mean (78ms) : 75, 82
master - mean (79ms) : 75, 82
section CallTarget+Inlining+NGEN
This PR (8763) - mean (1,087ms) : 1035, 1140
master - mean (1,107ms) : 1045, 1168
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8763) - mean (112ms) : 107, 118
master - mean (115ms) : 109, 122
section Bailout
This PR (8763) - mean (113ms) : 108, 118
master - mean (113ms) : 109, 117
section CallTarget+Inlining+NGEN
This PR (8763) - mean (782ms) : 761, 803
master - mean (790ms) : 770, 810
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8763) - mean (102ms) : 97, 106
master - mean (100ms) : 95, 105
section Bailout
This PR (8763) - mean (99ms) : 95, 103
master - mean (100ms) : 97, 102
section CallTarget+Inlining+NGEN
This PR (8763) - mean (941ms) : 905, 978
master - mean (956ms) : 916, 996
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8763) - mean (97ms) : 93, 102
master - mean (100ms) : 94, 106
section Bailout
This PR (8763) - mean (100ms) : 97, 104
master - mean (101ms) : 97, 105
section CallTarget+Inlining+NGEN
This PR (8763) - mean (820ms) : 787, 853
master - mean (823ms) : 785, 862
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8763) - mean (192ms) : 189, 196
master - mean (214ms) : 209, 220
section Bailout
This PR (8763) - mean (196ms) : 194, 199
master - mean (218ms) : 215, 221
section CallTarget+Inlining+NGEN
This PR (8763) - mean (1,174ms) : 1132, 1215
master - mean (1,288ms) : 1235, 1341
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8763) - mean (273ms) : 270, 277
master - mean (308ms) : 302, 314
section Bailout
This PR (8763) - mean (274ms) : 271, 277
master - mean (307ms) : 300, 313
section CallTarget+Inlining+NGEN
This PR (8763) - mean (941ms) : 925, 957
master - mean (1,023ms) : 1000, 1046
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8763) - mean (268ms) : 264, 271
master - mean (303ms) : 295, 312
section Bailout
This PR (8763) - mean (267ms) : 265, 270
master - mean (304ms) : 294, 313
section CallTarget+Inlining+NGEN
This PR (8763) - mean (1,147ms) : 1110, 1184
master - mean (1,214ms) : 1178, 1251
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8763) - mean (266ms) : 262, 270
master - mean (303ms) : 295, 311
section Bailout
This PR (8763) - mean (266ms) : 264, 269
master - mean (304ms) : 297, 312
section CallTarget+Inlining+NGEN
This PR (8763) - mean (1,026ms) : 972, 1080
master - mean (1,110ms) : 1018, 1201
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
bm1549
force-pushed
the
brian.marks/omit-sensitive-config-telemetry
branch
from
June 8, 2026 21:39
d3df1fa to
ca91b50
Compare
bm1549
marked this pull request as ready for review
June 9, 2026 21:33
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ca91b501da
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
andrewlock
reviewed
Jun 10, 2026
bm1549
force-pushed
the
brian.marks/omit-sensitive-config-telemetry
branch
from
June 17, 2026 01:14
9bbc976 to
fa2afe1
Compare
…lemetry Redact the OTEL_EXPORTER_OTLP_HEADERS family (base, METRICS, TRACES, LOGS) in instrumentation configuration telemetry. The values are now recorded as <redacted> in the configuration array of app-started and app-client-configuration-change events. - Switch the OTLP header reads in ExporterSettings to AsRedactedString(). - Add AsRedactedDictionaryResult and thread a recordValue flag through the dictionary configuration sources so the OTLP metrics/logs header reads in TracerSettings are recorded as redacted. - Mark the four OTLP header variants with sensitive: true in supported-configurations.yaml and accept the attribute in the YAML reader. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… flag Capture the `sensitive` flag from supported-configurations.yaml through the source generator and emit a generated ConfigurationKeys.SensitiveKeys set (keys plus their aliases). ConfigurationTelemetry.Record gates recordValue on this set so a sensitive config's value is redacted regardless of the call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After rebasing onto master (which added scope to ConfigurationEntry), the Aliases property changed type from string[]? to EquatableArray<string>. Call .AsArray() to convert when constructing ConfigEntry. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bm1549
force-pushed
the
brian.marks/omit-sensitive-config-telemetry
branch
from
July 29, 2026 23:56
96d2e66 to
4e92d47
Compare
andrewlock
reviewed
Jul 30, 2026
andrewlock
left a comment
andrewlock
left a comment
Member
There was a problem hiding this comment.
I'm thinking out loud as to whether this is the best approach 🤔 With this PR, we're adding an IsSensitive() lookup to every configuration read, on the startup hot path... We already have a mechanism for redacting keys (record: false) so it feels like the extra lookup is somewhat redundant.
I'm wondering if we can take a slightly different approach, building on the analyzers that we already have and also ensuring that we mark all our other sensitive keys as sensitive 🤔
I'm thinking:
- Keep the
sensitive: trueannotation in supported-configurations.yaml - Add/extend the config analyzers to ensure that any keys marked as sensitive are only called with
record: false(this is the tricky bit)
That would have the advantage of shifting the check cost to compile time instead of runtime. Without digging into it, I'm not quite sure how easy it would be, but I think it's worth exploring to solve this optimally.
In the mean time, we can simply set record: false for these 4 config keys, which is a tiny PR we can merge ASAP 🙂
Comment on lines
+20
to
+32
| /// <summary> | ||
| /// The set of configuration keys (including aliases) marked <c>sensitive</c> in | ||
| /// supported-configurations.yaml. Telemetry redacts the value of any key in this set. | ||
| /// </summary> | ||
| private static readonly HashSet<string> SensitiveKeys = new() | ||
| { | ||
| "OTEL_EXPORTER_OTLP_HEADERS", | ||
| "OTEL_EXPORTER_OTLP_LOGS_HEADERS", | ||
| "OTEL_EXPORTER_OTLP_METRICS_HEADERS", | ||
| "OTEL_EXPORTER_OTLP_TRACES_HEADERS", | ||
| }; | ||
|
|
||
| public static bool IsSensitive(string key) => SensitiveKeys.Contains(key); |
Member
There was a problem hiding this comment.
Given there's only 4 keys here, and that this is on the hotpath for all configurations now, this code should probably avoid the overhead of a hashset and be written like this:
Suggested change
| /// <summary> | |
| /// The set of configuration keys (including aliases) marked <c>sensitive</c> in | |
| /// supported-configurations.yaml. Telemetry redacts the value of any key in this set. | |
| /// </summary> | |
| private static readonly HashSet<string> SensitiveKeys = new() | |
| { | |
| "OTEL_EXPORTER_OTLP_HEADERS", | |
| "OTEL_EXPORTER_OTLP_LOGS_HEADERS", | |
| "OTEL_EXPORTER_OTLP_METRICS_HEADERS", | |
| "OTEL_EXPORTER_OTLP_TRACES_HEADERS", | |
| }; | |
| public static bool IsSensitive(string key) => SensitiveKeys.Contains(key); | |
| public static bool IsSensitive(string key) => | |
| key is | |
| "OTEL_EXPORTER_OTLP_HEADERS" or | |
| "OTEL_EXPORTER_OTLP_LOGS_HEADERS" or | |
| "OTEL_EXPORTER_OTLP_METRICS_HEADERS" or | |
| "OTEL_EXPORTER_OTLP_TRACES_HEADERS"; |
Or maybe even
public static bool IsSensitive(string key) => key.Length switch
{
26 => key == "OTEL_EXPORTER_OTLP_HEADERS",
31 => key == "OTEL_EXPORTER_OTLP_LOGS_HEADERS",
34 => key == "OTEL_EXPORTER_OTLP_METRICS_HEADERS",
33 => key == "OTEL_EXPORTER_OTLP_TRACES_HEADERS",
_ => false,
};(depending on benchmarking)
Comment on lines
+125
to
+142
| [Fact] | ||
| public void PublicDictionaryApiRetainsFiveArgumentOverload() | ||
| { | ||
| var parameterTypes = new[] | ||
| { | ||
| typeof(string), | ||
| typeof(IConfigurationTelemetry), | ||
| typeof(Func<IDictionary<string, string>, bool>), | ||
| typeof(bool), | ||
| typeof(char), | ||
| }; | ||
|
|
||
| typeof(IConfigurationSource) | ||
| .GetMethod(nameof(IConfigurationSource.GetDictionary), parameterTypes) | ||
| .Should() | ||
| .NotBeNull(); | ||
| } | ||
|
|
Member
There was a problem hiding this comment.
This is unnecessary, shall we write a test for every unchanged API? 😂
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
Redacts
OTEL_EXPORTER_OTLP_HEADERS,OTEL_EXPORTER_OTLP_TRACES_HEADERS,OTEL_EXPORTER_OTLP_METRICS_HEADERS, andOTEL_EXPORTER_OTLP_LOGS_HEADERSfrom configuration telemetry.Reason for change
OTLP headers can contain credentials and should not be sent in telemetry.
Implementation details
Adds
sensitive: trueto the configuration registry and generates a private lookup that includes each key and its aliases.ConfigurationTelemetry.Recorduses that lookup as the single redaction policy. The duplicate metrics-header parsing path was removed fromTracerSettings;ExporterSettingsremains the owner. The existing five-argument publicIConfigurationSource.GetDictionarymember is unchanged.Test coverage
Datadog.Tracefor all target frameworks with no warnings.Other details
None.