[Configuration] Enforce sensitive configuration redaction at compile time

docs/development/Configuration/AddingConfigurationKeys.md

@@ -57,6 +57,7 @@ ConfigurationKeys.ProductName.cs. Without a product name, the keys will go in th
 - `product`: Groups the key into a product-specific partial class (e.g., `OpenTelemetry`)
 - `aliases`: A list of fallback environment variable names checked in order when the primary key is not found
 - `const_name`: Overrides the auto-generated constant name (useful for backward compatibility). For `managed`, the default is PascalCase.
+- `sensitive`: Marks a credential-bearing value. Use `true` only when the configuration value itself must never be recorded in telemetry. The value must be `true` or `false` (case-insensitive); any other value is a YAML parse error.
 
 These fields are mandatory to keep the configuration registry complete and to ensure consistent behavior and documentation across products.
 
@@ -181,9 +182,23 @@ The codebase includes Roslyn analyzers that enforce the use of configuration key
 
 - **`ConfigurationBuilderWithKeysAnalyzer`** - Enforces that `ConfigurationBuilder.WithKeys()` method calls only accept string constants from `ConfigurationKeys` or `PlatformKeys` classes, not hardcoded strings or variables.
 
+For keys marked `sensitive: true`, the analyzer also enforces telemetry redaction at compile time. Read a sensitive string through `AsRedactedString()` or `AsRedactedStringResult()`. Use `AsRedactedDictionaryResult()` for dictionary-valued settings, or `AsStringResult(..., recordValue: false)` when the explicit `false` is a compile-time constant. Other accessors and storing the intermediate `WithKeys()` result are rejected because they could record the value.
+
+```csharp
+var apiKey = config.WithKeys(ConfigurationKeys.ApiKey).AsRedactedString();
+var result = config.WithKeys(ConfigurationKeys.ApiKey)
+                   .AsStringResult(validator: null, converter: null, recordValue: false);
+var headers = config.WithKeys(ConfigurationKeys.OpenTelemetry.ExporterOtlpLogsHeaders)
+                    .AsRedactedDictionaryResult(separator: '=');
+```
+
+Aliases use the normal `WithKeys()` fallback chain. When a sensitive primary key falls back to an alias, the selected value remains redacted because the redacted accessor records no value in telemetry.
+
 ##### Diagnostic rules:
 - **DD0007**: Triggers when hardcoded string literals are used instead of configuration key constants
 - **DD0008**: Triggers when variables or expressions are used instead of configuration key constants
+- **DD0015**: Triggers when a sensitive configuration key is not read through `AsRedactedString`, `AsRedactedStringResult`, `AsRedactedDictionaryResult`, or `AsStringResult` with compile-time `recordValue: false`
+- **DD0016**: Triggers when the analyzer cannot load exactly one valid `supported-configurations.yaml` additional file
 
 #### 2. EnvironmentGetEnvironmentVariableAnalyzer
 

docs/superpowers/specs/2026-07-31-sensitive-configuration-analyzer-design.md

@@ -0,0 +1,89 @@
+# Sensitive Configuration Analyzer Design
+
+## Context
+
+Configuration telemetry records most configuration values to help diagnose tracer setup. Some settings, including `DD_API_KEY` and OpenTelemetry exporter headers, contain credentials and must only produce redacted telemetry entries.
+
+The existing configuration API already supports redacted string reads through `AsRedactedString()` and `AsRedactedStringResult()`. Adding a runtime sensitivity lookup to every telemetry write would protect the values, but it would also add work to the tracer startup hot path. This design moves enforcement to compile time instead.
+
+## Goals
+
+- Keep `supported-configurations.yaml` as the single source of truth for sensitive configuration keys.
+- Report a build error when tracer code reads a sensitive key through an accessor that records its value.
+- Redact the four OTLP header settings without a runtime sensitivity lookup.
+- Mark existing credential-bearing settings, currently `DD_API_KEY`, as sensitive.
+- Preserve alias fallback, parsing behavior, and public APIs.
+
+## Non-goals
+
+- Runtime enforcement for third-party callers or reflection-based configuration reads.
+- Data-flow analysis across variables that store `ConfigurationBuilder.HasKeys` values.
+- Treating every deliberately redacted value as sensitive. For example, AppSec's HTML template path is redacted because its value can be large, not because it contains a credential.
+- Adding a code fix in this change.
+
+## Architecture
+
+### YAML metadata
+
+Add an optional `sensitive: true` property to configuration entries. The shared `YamlReader` will parse and expose the property on `ConfigurationEntry`; omitted values default to `false`. Values are case-insensitive `true` or `false`. Other tokens are parse errors reported by the source generator as `DDSG0007` and by the analyzer as `DD0016`.
+
+The analyzer project will link the existing shared YAML parser and its helper types. `tracer/src/Directory.Build.props` will supply `supported-configurations.yaml` to every project that receives the analyzer. During compilation start, the analyzer will require exactly one readable additional file, parse it once, and build an immutable set of canonical sensitive keys. Missing, ambiguous, unreadable, or invalid metadata produces `DD0016` instead of disabling the rule.
+
+### Analyzer rule
+
+Extend `ConfigurationBuilderWithKeysAnalyzer` with a new error diagnostic for unsafe sensitive-key reads.
+
+For each `ConfigurationBuilder.WithKeys(...)` invocation, the analyzer will continue enforcing that the argument is a direct `ConfigurationKeys` or `PlatformKeys` constant. When the constant value is in the sensitive-key set, the analyzer will use Roslyn operations and symbols to inspect the immediately chained accessor. Parentheses and conversions around `WithKeys(...)` are transparent, but the accessor must resolve to an instance method on the returned `ConfigurationBuilder.HasKeys` type:
+
+- `AsRedactedString(...)` is allowed.
+- `AsRedactedStringResult(...)` is allowed.
+- `AsRedactedDictionaryResult(...)` is allowed.
+- `AsStringResult(..., recordValue: false)` is allowed when Roslyn can prove the argument is the constant `false`.
+- Any accessor that records values, an unrecognized accessor, or storing the intermediate `HasKeys` value is rejected because the analyzer cannot prove that telemetry is redacted.
+
+The diagnostic will be reported on the sensitive configuration-key argument and explain which redacted accessors are permitted.
+
+This deliberately targets the fluent `ConfigurationBuilder` path already governed by the analyzer. Direct `IConfigurationSource` calls are outside this rule and are already intended to be restricted separately.
+
+### Tracer configuration reads
+
+Mark these registry entries as sensitive:
+
+- `DD_API_KEY`
+- `OTEL_EXPORTER_OTLP_HEADERS`
+- `OTEL_EXPORTER_OTLP_LOGS_HEADERS`
+- `OTEL_EXPORTER_OTLP_METRICS_HEADERS`
+- `OTEL_EXPORTER_OTLP_TRACES_HEADERS`
+
+Change the OTLP string reads in `ExporterSettings` to `AsRedactedString()`. Remove the duplicate metrics-header parsing in `TracerSettings`, leaving `ExporterSettings` as its owner.
+
+`TracerSettings` still owns OTLP log headers as a parsed dictionary. It will use `AsRedactedDictionaryResult()` so `JsonConfigurationSource` keeps its object and array handling. The accessor passes an internal telemetry decorator to the existing dictionary API. The decorator forces string-value records to use `recordValue: false`, including parse and unexpected-type errors, and forwards every other operation unchanged. The existing default, filtering, and trimming flow remains in place. Alias fallback remains in `ConfigurationBuilder`, so the general header setting continues to work as the logs fallback.
+
+No runtime `IsSensitive()` method or sensitivity collection will be generated.
+
+## Data Flow
+
+1. MSBuild supplies `supported-configurations.yaml` as a Roslyn additional file to every analyzer consumer under `tracer/src`.
+2. The analyzer parses the registry once at compilation start and collects keys marked `sensitive: true`.
+3. A syntax action resolves each `WithKeys()` argument to its constant value.
+4. Sensitive constants must flow directly into an accessor that records telemetry with `recordValue: false`.
+5. At runtime, redacted string accessors pass `recordValue: false` to configuration sources. The redacted dictionary accessor wraps telemetry internally and uses the existing dictionary interface. Both paths record redacted entries without a key lookup.
+
+## Testing
+
+Follow red-green TDD for each behavior:
+
+- YAML parser tests prove mixed-case boolean values are captured, omitted values default to false, parser state resets between entries, and invalid tokens produce `DDSG0007`.
+- Analyzer tests prove ordinary keys may use normal accessors.
+- Analyzer tests prove sensitive keys may use both redacted accessors and an explicit constant `recordValue: false`.
+- Analyzer tests prove sensitive keys fail with normal string/dictionary accessors, `recordValue: true`, non-constant values, missing accessors, and stored intermediate values.
+- Analyzer tests prove missing or invalid sensitivity metadata fails closed with `DD0016`, including in a consumer assembly.
+- Configuration tests prove all OTLP header values produce non-empty redacted telemetry while a nearby non-sensitive setting still records normally. Log-header coverage includes JSON objects, primary and alias strings, and unexpected object types.
+- Existing API-key call sites compile under the new rule because they already use redacted accessors.
+- Existing OTLP header parsing and fallback tests continue to pass after ownership consolidation.
+
+Run the analyzer and source-generator test suites, affected tracer configuration tests, and a full `Datadog.Trace` build before publishing.
+
+## Documentation
+
+Update the configuration-key development guide to document `sensitive`, its compile-time enforcement, and the accepted redacted access patterns.

tracer/build/_build/NativeValidation/YamlReader.cs

@@ -32,6 +32,7 @@ public static ParsedConfigurationData ParseSupportedConfigurations(string yamlCo
             string? currentConstName = null;
             string[]? currentScope = null;
             var currentAliases = new List<string>();
+            var currentSensitive = false;
             var inDocumentation = false;
             var inAliases = false;
             var documentationBuilder = new StringBuilder();
@@ -65,7 +66,7 @@ public static ParsedConfigurationData ParseSupportedConfigurations(string yamlCo
                     if (currentConfigKey != null)
                     {
                         var doc = inDocumentation ? documentationBuilder.ToString().TrimEnd() : currentDocumentation;
-                        configurations[currentConfigKey] = new ConfigurationEntry(currentConfigKey, currentProduct ?? string.Empty, doc, currentConstName, currentScope, currentAliases.Count > 0 ? currentAliases.ToArray() : null);
+                        configurations[currentConfigKey] = new ConfigurationEntry(currentConfigKey, currentProduct ?? string.Empty, doc, currentConstName, currentScope, currentAliases.Count > 0 ? currentAliases.ToArray() : null, currentSensitive);
                     }
 
                     inSupportedConfigurations = false;
@@ -118,7 +119,7 @@ public static ParsedConfigurationData ParseSupportedConfigurations(string yamlCo
                             if (currentConfigKey != null)
                             {
                                 var doc = inDocumentation ? documentationBuilder.ToString().TrimEnd() : currentDocumentation;
-                                configurations[currentConfigKey] = new ConfigurationEntry(currentConfigKey, currentProduct ?? string.Empty, doc, currentConstName, currentScope, currentAliases.Count > 0 ? currentAliases.ToArray() : null);
+                                configurations[currentConfigKey] = new ConfigurationEntry(currentConfigKey, currentProduct ?? string.Empty, doc, currentConstName, currentScope, currentAliases.Count > 0 ? currentAliases.ToArray() : null, currentSensitive);
                             }
 
                             currentConfigKey = potentialKey;
@@ -127,6 +128,7 @@ public static ParsedConfigurationData ParseSupportedConfigurations(string yamlCo
                             currentConstName = null;
                             currentScope = null;
                             currentAliases.Clear();
+                            currentSensitive = false;
                             inDocumentation = false;
                             inAliases = false;
                             documentationBuilder.Clear();
@@ -144,7 +146,7 @@ public static ParsedConfigurationData ParseSupportedConfigurations(string yamlCo
                             if (propColonIdx > 0)
                             {
                                 var propName = trimmedLine.Substring(0, propColonIdx);
-                                if (propName is "const_name" or "product" or "implementation" or "type" or "default" or "aliases" or "deprecation_message" or "scope")
+                                if (propName is "const_name" or "product" or "implementation" or "type" or "default" or "aliases" or "deprecation_message" or "scope" or "sensitive")
                                 {
                                     // End of documentation, process this property
                                     inDocumentation = false;
@@ -259,6 +261,21 @@ public static ParsedConfigurationData ParseSupportedConfigurations(string yamlCo
                                     break;
                                 case "aliases":
                                     inAliases = true;
+                                    break;
+                                case "sensitive":
+                                    if (propValue.Equals("true", StringComparison.OrdinalIgnoreCase))
+                                    {
+                                        currentSensitive = true;
+                                    }
+                                    else if (propValue.Equals("false", StringComparison.OrdinalIgnoreCase))
+                                    {
+                                        currentSensitive = false;
+                                    }
+                                    else
+                                    {
+                                        throw new InvalidOperationException($"Invalid sensitive value on line {lineNumber}: '{propValue}'. Expected true or false.");
+                                    }
+
                                     break;
                                 case "documentation":
                                     if (propValue == "|-" || propValue == "|")
@@ -303,7 +320,7 @@ public static ParsedConfigurationData ParseSupportedConfigurations(string yamlCo
             if (currentConfigKey != null)
             {
                 var doc = inDocumentation ? documentationBuilder.ToString().TrimEnd() : currentDocumentation;
-                configurations[currentConfigKey] = new ConfigurationEntry(currentConfigKey, currentProduct ?? string.Empty, doc, currentConstName, currentScope, currentAliases.Count > 0 ? currentAliases.ToArray() : null);
+                configurations[currentConfigKey] = new ConfigurationEntry(currentConfigKey, currentProduct ?? string.Empty, doc, currentConstName, currentScope, currentAliases.Count > 0 ? currentAliases.ToArray() : null, currentSensitive);
             }
 
             return new ParsedConfigurationData(configurations, deprecations);
@@ -371,14 +388,15 @@ public bool MoveNext()
         /// </summary>
         internal readonly struct ConfigurationEntry : IEquatable<ConfigurationEntry>
         {
-            public ConfigurationEntry(string key, string? product, string? documentation, string? constName, string[]? scope, string[]? aliases = null)
+            public ConfigurationEntry(string key, string? product, string? documentation, string? constName, string[]? scope, string[]? aliases = null, bool sensitive = false)
             {
                 Key = key;
                 Product = product;
                 Documentation = documentation;
                 ConstName = constName;
                 Scope = scope is null ? default : new EquatableArray<string>(scope);
                 Aliases = aliases is null ? default : new EquatableArray<string>(aliases);
+                Sensitive = sensitive;
             }
 
             public string Key { get; }
@@ -393,17 +411,20 @@ public ConfigurationEntry(string key, string? product, string? documentation, st
 
             public EquatableArray<string> Aliases { get; }
 
+            public bool Sensitive { get; }
+
             public bool Equals(ConfigurationEntry other)
                 => Key == other.Key
                 && Product == other.Product
                 && Documentation == other.Documentation
                 && ConstName == other.ConstName
                 && Scope == other.Scope
-                && Aliases == other.Aliases;
+                && Aliases == other.Aliases
+                && Sensitive == other.Sensitive;
 
             public override bool Equals(object? obj) => obj is ConfigurationEntry other && Equals(other);
 
-            public override int GetHashCode() => HashCode.Combine(Key, Product, Documentation, ConstName, Scope, Aliases);
+            public override int GetHashCode() => HashCode.Combine(Key, Product, Documentation, ConstName, Scope, Aliases, Sensitive);
         }
 
         /// <summary>