Skip to content

Enable EVP flagevaluation system tests for Node.js#7182

Draft
leoromanovsky wants to merge 5 commits into
mainfrom
leo.romanovsky/ffe-evp-flagevaluation-enable-nodejs-clean
Draft

Enable EVP flagevaluation system tests for Node.js#7182
leoromanovsky wants to merge 5 commits into
mainfrom
leo.romanovsky/ffe-evp-flagevaluation-enable-nodejs-clean

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Motivation

tests/ffe/test_flag_eval_evp.py now covers the EVP flagevaluation contract for server-side SDKs. Node.js has a passing local focused run against the implementation PR, so its manifest can move from missing_feature (FFL-2446) to the validated version.

Changes

  • Enables tests/ffe/test_flag_eval_evp.py for the validated Node.js express4 weblog in manifests/nodejs.yml at *ref_6_0_0; other Node.js weblogs remain incomplete_test_app.
  • Leaves the test implementation unchanged from Add EVP flagevaluation system tests #7146.
  • Keeps this as a sibling PR based on leo.romanovsky/ffe-evp-flagevaluation-system-tests, not stacked on other language enablement branches.

Decisions

  • This PR enables only the validated Node.js weblog; other languages are enabled by separate sibling PRs after their own local pass evidence.
  • Existing OTel metric coverage stays in tests/ffe/test_flag_eval_metrics.py.

Validation

  • SYSTEM_TEST_BUILD_TIMEOUT=1200 ./build.sh --library nodejs --weblog-variant express4 --docker-platform linux/amd64 - PASS on 2026-06-20.
  • TEST_LIBRARY=nodejs WEBLOG_VARIANT=express4 ./run.sh +l nodejs FEATURE_FLAGGING_AND_EXPERIMENTATION -k "test_flag_eval_evp" - PASS on 2026-06-20, 8 passed, 2627 deselected in 260.16s.
  • Agent 7.80.1; library nodejs@6.0.0-pre; weblog express4; Linux x86_64.
  • Agent mock-intake aggregation and degradation cases passed in the focused run.
  • Real-backend dogfooding prefix is recorded in the Node.js SDK PR: server-evp-e2e-nodejs-1781931066-user-.

@github-actions

github-actions Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

manifests/nodejs.yml                                                    @DataDog/dd-trace-js
tests/ffe/test_flag_eval_evp.py                                         @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Jun 20, 2026

Copy link
Copy Markdown

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 4 Pipeline jobs failed

Testing the test | System Tests (nodejs, dev) / End-to-end #1 / express4 1   View in Datadog   GitHub Actions

🧪 8 Tests failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[express4] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7f6faac7eca0>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7f6fdc5d3440>

    def test_ffe_evp_flagevaluation_basic(self) -> None:
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
...
tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation.test_ffe_evp_flagevaluation_burst_aggregation[express4] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-burst-aggregation-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7f6fab45ce00>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation object at 0x7f6fdc5d1220>

    def test_ffe_evp_flagevaluation_burst_aggregation(self) -> None:
        for index, response in enumerate(self.responses):
...
View all 8 test failures

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-8.0 1   View in Datadog   GitHub Actions

🧪 1 Test failed

tests.ffe.test_exposures.Test_FFE_Exposure_Events.test_ffe_multiple_remote_config_files[apache-mod-8.0] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for exposure event for flags ['test-flag-1', 'test-flag-2'] and subject 'test-user-multi'
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_exposure_event.<locals>.<lambda> at 0x7f723f2dede0>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_exposures.Test_FFE_Exposure_Events object at 0x7f726c66ef30>

    def test_ffe_multiple_remote_config_files(self):
        """Test that FFE correctly handles multiple remote config files with different flags."""
...

Testing the test | System Tests (php, prod) / End-to-end #1 / apache-mod-7.3 1   View in Datadog   GitHub Actions

🧪 1 Test failed

tests.ffe.test_exposures.Test_FFE_Exposure_Events.test_ffe_multiple_remote_config_files[apache-mod-7.3] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for exposure event for flags ['test-flag-1', 'test-flag-2'] and subject 'test-user-multi'
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_exposure_event.<locals>.<lambda> at 0x7f936f2eed40>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_exposures.Test_FFE_Exposure_Events object at 0x7f9395e97e00>

    def test_ffe_multiple_remote_config_files(self):
        """Test that FFE correctly handles multiple remote config files with different flags."""
...

View all 4 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a03c3fb | Docs | Datadog PR Page | Give us feedback!

@datadog-system-tests-org

datadog-system-tests-org Bot commented Jun 20, 2026

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

🚦 4 Pipeline jobs failed

Testing the test | System Tests (nodejs, dev) / End-to-end #1 / express4 1   View in Datadog   GitHub Actions

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-8.0 1   View in Datadog   GitHub Actions

Testing the test | System Tests (php, prod) / End-to-end #1 / apache-mod-7.3 1   View in Datadog   GitHub Actions

View all 4 failed jobs.

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a03c3fb | Docs | Give us feedback!

Base automatically changed from leo.romanovsky/ffe-evp-flagevaluation-contract-fix to main June 22, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants