fix(openfeature): disable the provider for unsupported sources

[leoromanovsky]

Edit: We made additional decisions during this review and in offline discussions. The code changes and validation comment below show the final decisions. The remaining PR body preserves the original proposal for the project record.

Motivation

The configuration registry currently discards an explicit, nonblank DD_FEATURE_FLAGS_CONFIGURATION_SOURCE value if it does not recognize the value.

This behavior silently selects a different delivery mode:

• Without a legacy setting, the default becomes agentless.
• With DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true, legacy precedence selects remote_config.

This behavior is not safe for an invalid value. It is also incorrect for the reserved future offline mode.

A typing error or an unimplemented mode must not start a CDN request. It must not advertise the FFE Remote Configuration capability.

Blank values and values that contain only whitespace remain semantically absent. They retain the existing default and legacy compatibility behavior.

The invalid and reserved-source system tests make a valid agentless backend available. The tests access the provider. They then require zero backend requests and no FFE Remote Configuration capability or product.

This test design separates fail-closed behavior from an unavailable test backend.

On parent #9481, both tests fail as expected.

Each explicit source causes one authenticated request to the canonical agentless path. The endpoint returns HTTP 200.

requests_total=1
last_auth_present=true
last_status_code=200

The parent result shows that source selection causes the failure. Docker connectivity does not cause the failure.

Local Docker proof on the stacked result

This PR contains one commit, 3395eccc707ff0b06ea504ae668a80017e6787b4. It is directly on #9481 head 51cae1bdd2378ecf0f3c77147e2934e33ca2ceb4.

The PARAMETRIC Docker harness mounted the stacked worktree. It ran system-tests head a5dc9b8433ee3fd2892ea2968b90acb26274313c with one worker:

PYTEST_XDIST_AUTO_NUM_WORKERS=1 TEST_LIBRARY=nodejs \
  ./run.sh PARAMETRIC --skip-parametric-build -vv -s \
  tests/parametric/test_ffe/test_configuration_sources.py

test_invalid_configuration_source_fails_closed          PASSED
test_reserved_offline_configuration_source_fails_closed PASSED
test_default_agentless_positive[source-absent]           PASSED
test_missing_auth_cold                                   PASSED
test_missing_auth_warm                                   PASSED
...
29 passed in 363.31s

The complete file also passed these behaviors:

• Remote Configuration selection and isolation.
• Stable kill-switch precedence.
• Blank and legacy compatibility.
• Malformed-response and timeout failures during cold startup.
• Recovery.
• Last-known-good state preservation.
• ETag handling.
• Poller non-overlap.

The same stacked tree passed the repository checks:

npm run test:openfeature                                      212 passing
npx mocha packages/dd-trace/test/config/index.spec.js --grep "Feature Flagging configuration source"
                                                               23 passing
npx mocha packages/dd-trace/test/exporters/common/request.spec.js
                                                               36 passing
npm run test:integration:openfeature                            5 passing
configuration-source contract                                 63/63 combinations
npm run verify:config:types                                   passed
npm run lint                                                  passed

Changes

• Preserve explicit, nonblank, unsupported source values.
• Preserve the environment-variable origin of each unsupported value.
• Treat blank values and values that contain only whitespace as unset.
• Retain the existing default and legacy precedence for unset values.
• Register offline as a reserved value.
• Reject offline during provider initialization.
• Reject other unsupported sources before selecting a delivery path.
• Update the generated configuration types.
• Update the 63-case source-precedence contract matrix.

Decisions

• Stack this one-commit PR on #9481.
• Use the working Docker and custom-endpoint implementation from fix(openfeature): allow custom agentless endpoints #9481 for CI validation.
• Keep the stable DD_FEATURE_FLAGS_ENABLED=false kill switch authoritative.
• Disable provider initialization before source validation when the kill switch is false.
• Keep custom-endpoint and missing-auth behavior in the parent PR.
• Keep only source precedence and fail-closed behavior in this PR.
• Reserve offline without implementing startup bytes.
• Do not add a network delivery path for offline.

[dd-octo-sts]

Overall package size

Self size: 7.51 MB
Deduped: 8.17 MB
No deduping: 8.17 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.2 | 124.41 kB | 440.65 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

[datadog-datadog-us1-prod]

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 1 job - 1 passed on retry

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 98.44% (-0.00%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: fc260f0 | Docs | Datadog PR Page | Give us feedback!}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.44%. Comparing base (d82bd2b) to head (fc260f0).

Additional details and impacted files

@@                                  Coverage Diff                                   @@
##           leo.romanovsky/ffl-2697-custom-agentless-endpoints    #9482      +/-   ##
======================================================================================
- Coverage                                               98.44%   98.44%   -0.01%     
======================================================================================
  Files                                                     940      943       +3     
  Lines                                                  126874   127283     +409     
  Branches                                                11093    12617    +1524     
======================================================================================
+ Hits                                                   124903   125301     +398     
- Misses                                                   1971     1982      +11     

Flag	Coverage Δ
aiguard	57.88% <91.89%> (+0.37%)	⬆️
aiguard-integration	56.24% <62.16%> (-0.10%)	⬇️
apm-bucket-0	57.73% <91.89%> (-0.02%)	⬇️
apm-bucket-1	64.03% <91.89%> (-0.01%)	⬇️
apm-bucket-2	62.79% <91.89%> (-0.03%)	⬇️
apm-bucket-3	60.15% <91.89%> (-0.02%)	⬇️
apm-capabilities-tracing	64.50% <100.00%> (-0.07%)	⬇️
apm-integrations-aerospike	56.74% <91.89%> (-0.01%)	⬇️
apm-integrations-confluentinc-kafka-javascript	61.67% <91.89%> (-0.02%)	⬇️
apm-integrations-couchbase	57.18% <91.89%> (-0.02%)	⬇️
apm-integrations-http	62.81% <91.89%> (-0.01%)	⬇️
apm-integrations-kafkajs	62.22% <91.89%> (-0.02%)	⬇️
apm-integrations-next	59.27% <91.89%> (-0.02%)	⬇️
apm-integrations-prisma	58.75% <91.89%> (-0.02%)	⬇️
appsec	72.94% <91.89%> (+<0.01%)	⬆️
appsec-express_fastify_graphql	70.44% <91.89%> (+<0.01%)	⬆️
appsec-integration	51.39% <62.16%> (-0.10%)	⬇️
appsec-kafka_ldapjs_lodash	64.02% <91.89%> (-0.01%)	⬇️
appsec-mongodb-core_mongoose_mysql	67.74% <91.89%> (+<0.01%)	⬆️
appsec-next	57.58% <91.89%> (-0.01%)	⬇️
appsec-node-serialize_passport_postgres	67.41% <91.89%> (+<0.01%)	⬆️
appsec-sourcing_stripe_template	65.75% <91.89%> (-0.01%)	⬇️
debugger	64.93% <100.00%> (-0.01%)	⬇️
instrumentations-bucket-0	51.71% <91.89%> (-0.13%)	⬇️
instrumentations-bucket-1	60.28% <91.89%> (-0.07%)	⬇️
instrumentations-bucket-10	61.85% <91.89%> (-0.30%)	⬇️
instrumentations-bucket-11	56.76% <91.89%> (+5.01%)	⬆️
instrumentations-bucket-12	52.08% <91.89%> (-0.25%)	⬇️
instrumentations-bucket-13	52.09% <91.89%> (+0.22%)	⬆️
instrumentations-bucket-2	53.40% <91.89%> (-0.36%)	⬇️
instrumentations-bucket-3	53.69% <91.89%> (-5.65%)	⬇️
instrumentations-bucket-4	59.28% <91.89%> (+6.87%)	⬆️
instrumentations-bucket-5	50.10% <91.89%> (-7.58%)	⬇️
instrumentations-bucket-6	60.99% <91.89%> (+0.13%)	⬆️
instrumentations-bucket-7	58.43% <91.89%> (-0.10%)	⬇️
instrumentations-bucket-8	59.51% <91.89%> (-0.16%)	⬇️
instrumentations-bucket-9	52.09% <91.89%> (-9.58%)	⬇️
instrumentations-instrumentation-couchbase	51.15% <91.89%> (+<0.01%)	⬆️
instrumentations-instrumentation-zlib	51.74% <91.89%> (?)
instrumentations-integration-esbuild	34.12% <48.64%> (-0.07%)	⬇️
llmobs-ai_anthropic_bedrock	62.62% <91.89%> (-0.24%)	⬇️
llmobs-bucket-1	61.59% <91.89%> (-0.07%)	⬇️
llmobs-openai	62.75% <91.89%> (-0.04%)	⬇️
llmobs-sdk	66.25% <91.89%> (-0.01%)	⬇️
llmobs-vertex-ai	59.25% <91.89%> (-0.05%)	⬇️
master-coverage	?
openfeature	56.22% <100.00%> (+0.02%)	⬆️
openfeature-unit	53.49% <92.10%> (-0.01%)	⬇️
platform-core_esbuild_instrumentations-misc	40.44% <91.89%> (-0.07%)	⬇️
platform-integration	61.51% <70.27%> (-0.02%)	⬇️
platform-shimmer_unit-guardrails_webpack	39.14% <91.89%> (-0.06%)	⬇️
plugins-bucket-0	57.14% <91.89%> (-0.02%)	⬇️
plugins-bucket-1	54.44% <62.16%> (-0.02%)	⬇️
plugins-bucket-11	62.37% <91.89%> (-0.02%)	⬇️
plugins-bucket-18	62.06% <91.89%> (-0.02%)	⬇️
plugins-bucket-19	60.16% <91.89%> (-0.02%)	⬇️
plugins-bucket-20	62.16% <91.89%> (-0.02%)	⬇️
plugins-bucket-4	58.66% <91.89%> (-0.04%)	⬇️
plugins-bullmq_cassandra_cookie	61.85% <91.89%> (-0.02%)	⬇️
plugins-cookie-parser_crypto_dd-trace-api	56.84% <91.89%> (-0.01%)	⬇️
plugins-fetch_fs_generic-pool	58.87% <91.89%> (-0.02%)	⬇️
plugins-google-cloud-pubsub_grpc_handlebars	64.79% <91.89%> (-0.01%)	⬇️
plugins-hapi_hono_ioredis	60.32% <91.89%> (-0.02%)	⬇️
plugins-jest_knex_langgraph	55.70% <91.89%> (-0.01%)	⬇️
plugins-ldapjs_light-my-request_limitd-client	58.60% <91.89%> (-0.02%)	⬇️
plugins-lodash_mariadb_memcached	58.10% <91.89%> (-0.02%)	⬇️
plugins-moleculer_mongodb_mongodb-core	62.01% <91.89%> (-0.02%)	⬇️
plugins-mongoose_multer_mysql	59.12% <91.89%> (-0.02%)	⬇️
plugins-mysql2_nats_node-serialize	60.82% <91.89%> (-0.02%)	⬇️
plugins-opensearch_passport-http_pino	59.64% <91.89%> (-0.02%)	⬇️
plugins-postgres_process_pug	58.38% <91.89%> (-0.02%)	⬇️
plugins-redis_router_sequelize	62.14% <91.89%> (-0.02%)	⬇️
plugins-test-and-upstream-rhea_undici_url	61.72% <91.89%> (-0.01%)	⬇️
plugins-valkey_vm_winston	58.11% <91.89%> (-0.02%)	⬇️
plugins-ws	59.68% <91.89%> (-0.04%)	⬇️
profiling	62.25% <100.00%> (-0.01%)	⬇️
serverless-aws-sdk-aws-sdk	55.16% <91.89%> (-0.02%)	⬇️
serverless-aws-sdk-bedrockruntime	54.86% <91.89%> (-0.01%)	⬇️
serverless-aws-sdk-client	56.52% <91.89%> (-0.02%)	⬇️
serverless-aws-sdk-dynamodb	55.77% <91.89%> (+<0.01%)	⬆️
serverless-aws-sdk-eventbridge	49.48% <91.89%> (-0.01%)	⬇️
serverless-aws-sdk-kinesis	59.45% <91.89%> (-0.01%)	⬇️
serverless-aws-sdk-lambda	57.49% <91.89%> (-0.01%)	⬇️
serverless-aws-sdk-s3	55.71% <91.89%> (-0.01%)	⬇️
serverless-aws-sdk-serverless-peer-service	59.85% <91.89%> (-0.01%)	⬇️
serverless-aws-sdk-sns	60.24% <91.89%> (-0.01%)	⬇️
serverless-aws-sdk-sqs	60.66% <91.89%> (-0.02%)	⬇️
serverless-aws-sdk-stepfunctions	55.69% <91.89%> (-0.01%)	⬇️
serverless-aws-sdk-util	51.60% <91.89%> (+<0.01%)	⬆️
serverless-bucket-0	54.34% <62.16%> (-0.02%)	⬇️
serverless-bucket-1	59.35% <100.00%> (-0.02%)	⬇️
test-optimization-cucumber	71.87% <91.89%> (-0.04%)	⬇️
test-optimization-cypress	65.81% <62.16%> (-0.01%)	⬇️
test-optimization-jest	73.31% <91.89%> (+0.04%)	⬆️
test-optimization-mocha	73.50% <91.89%> (-0.02%)	⬇️
test-optimization-playwright-playwright-atr	60.40% <62.16%> (-0.01%)	⬇️
test-optimization-playwright-playwright-efd	60.58% <62.16%> (-0.01%)	⬇️
test-optimization-playwright-playwright-final-status	60.55% <62.16%> (-0.01%)	⬇️
test-optimization-playwright-playwright-impacted-tests	60.28% <62.16%> (-0.01%)	⬇️
test-optimization-playwright-playwright-reporting	61.56% <62.16%> (-0.01%)	⬇️
test-optimization-playwright-playwright-test-management	61.09% <62.16%> (-0.01%)	⬇️
test-optimization-playwright-playwright-test-span	60.45% <62.16%> (-0.06%)	⬇️
test-optimization-selenium	60.03% <62.16%> (-0.01%)	⬇️
test-optimization-testopt	58.47% <70.27%> (-0.02%)	⬇️
test-optimization-vitest	70.30% <91.89%> (-0.08%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3395eccc70

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[pr-commenter]

Benchmarks

Benchmark execution time: 2026-07-23 04:07:27

Comparing candidate commit fc260f0 in PR branch leo.romanovsky/ffl-2697-configuration-source-grandfathering with baseline commit d82bd2b in branch leo.romanovsky/ffl-2697-custom-agentless-endpoints.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2332 metrics, 26 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-24

• unstable execution_time [-205.754ms; +218.313ms] or [-7.691%; +8.160%]

scenario:appsec-appsec-enabled-26

• unstable execution_time [-232.687ms; +229.284ms] or [-9.167%; +9.033%]

scenario:appsec-appsec-enabled-with-attacks-24

• unstable execution_time [-168.031ms; +151.593ms] or [-5.425%; +4.894%]

scenario:appsec-appsec-enabled-with-attacks-26

• unstable execution_time [-189.727ms; +192.588ms] or [-6.529%; +6.627%]

scenario:appsec-control-20

• unstable execution_time [-134.641ms; +113.157ms] or [-8.119%; +6.823%]

scenario:appsec-control-24

• unstable execution_time [-113.748ms; +117.382ms] or [-9.137%; +9.429%]

scenario:appsec-control-26

• unstable execution_time [-123.780ms; +118.309ms] or [-10.005%; +9.562%]

scenario:appsec-iast-no-vulnerability-control-20

• unstable execution_time [-17.990ms; +13.562ms] or [-6.787%; +5.117%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-20

• unstable execution_time [-15511.064µs; +17464.998µs] or [-6.079%; +6.844%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-20

• unstable execution_time [-12.915ms; +23.552ms] or [-5.013%; +9.142%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-20

• unstable execution_time [-28.089ms; +41.991ms] or [-5.123%; +7.659%]

scenario:debugger-line-probe-with-snapshot-default-26

• unstable cpu_user_time [-5233.060ms; +3752.133ms] or [-41.361%; +29.656%]
• unstable execution_time [-5239.806ms; +3745.498ms] or [-39.062%; +27.922%]
• unstable instructions [-46.8G instructions; +33.0G instructions] or [-43.575%; +30.765%]
• unstable max_rss_usage [-16.037MB; +11.734MB] or [-9.497%; +6.949%]
• unstable throughput [-732.668op/s; +1012.284op/s] or [-28.046%; +38.750%]

scenario:debugger-line-probe-without-snapshot-24

• unstable cpu_user_time [-1977.158ms; +3176.373ms] or [-23.791%; +38.221%]
• unstable execution_time [-2011.294ms; +3190.043ms] or [-22.258%; +35.303%]
• unstable instructions [-17.2G instructions; +27.2G instructions] or [-25.362%; +40.222%]
• unstable max_rss_usage [-9.201MB; +13.600MB] or [-5.856%; +8.656%]
• unstable throughput [-843.529op/s; +532.877op/s] or [-23.145%; +14.621%]

scenario:dogstatsd-with-tags-20

• unstable cpu_user_time [-393.763ms; +246.891ms] or [-8.342%; +5.230%]
• unstable execution_time [-400.863ms; +241.714ms] or [-8.355%; +5.038%]
• unstable throughput [-86076.056op/s; +142209.261op/s] or [-4.921%; +8.131%]

scenario:plugin-graphql-long-with-depth-and-collapse-off-20

• unstable max_rss_usage [-22165.531KB; +22705.531KB] or [-5.617%; +5.754%]

scenario:plugin-graphql-long-with-depth-off-26

• unstable max_rss_usage [-25.017MB; +16.705MB] or [-11.231%; +7.499%]

[BridgeAR]

It is actually not silently omitted. We automatically generate telemetry for it and log a warning, if debug mode is active.

I do not think we should change how things work right now. Throwing for invalid configs is something no tracer does and that was an agreement that even if something is not properly configured, it should not crash a customers application. As such, doing this would actively go against that rule.

[leoromanovsky]

I applied the misconfiguration policy.

I then merged the updated parent branch into this branch. The resulting head is fdf84bd76.

DataDog/system-tests#7365 contains the corresponding test-contract changes.

PYTEST_XDIST_AUTO_NUM_WORKERS=1 TEST_LIBRARY=nodejs pytest -S PARAMETRIC --skip-parametric-build --runxfail -vv -s tests/parametric/test_ffe/test_configuration_sources.py

Result: 29 passed in 362.26s.

The complete run verifies:

• Arbitrary invalid sources disable the provider.
• The reserved offline source disables the provider.
• The provider does not throw for these source values.
• The provider does not start CDN or Remote Configuration traffic.
• Custom agentless endpoints do not receive the Datadog API key.
• The test configures a sentinel DD_API_KEY to verify key removal.
• Explicit 401, malformed payload, timeout, recovery, last-known-good, ETag, and poller-concurrency behavior.

The repository checks also passed:

• 212 OpenFeature tests.
• 23 configuration-source tests.
• Generated configuration types.
• The 63/63 configuration matrix: 12 agentless, 12 Remote Configuration, and 39 disabled.