File tree Expand file tree Collapse file tree
utils/_context/_scenarios Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -1604,8 +1604,14 @@ manifest:
16041604 " * " : incomplete_test_app
16051605 express4 : *ref_5_77_0
16061606 tests/ffe/test_exposures.py::Test_FFE_EXP_5_Missing_Targeting_Key : bug (FFL-1730)
1607- tests/ffe/test_flag_eval_metrics.py : missing_feature
1608- tests/ffe/test_flag_eval_metrics.py::Test_FFE_Eval_Targeting_Key_Optional : irrelevant (JS SDK requires targeting key)
1607+ tests/ffe/test_flag_eval_metrics.py :
1608+ - weblog_declaration :
1609+ " * " : missing_feature
1610+ express4 : *ref_6_0_0
1611+ tests/ffe/test_flag_eval_metrics.py::Test_FFE_Eval_Metric_Numeric_To_Integer : bug (FFL-2064)
1612+ tests/ffe/test_flag_eval_metrics.py::Test_FFE_Eval_Metric_Parse_Error_Invalid_Regex : bug (FFL-2064)
1613+ tests/ffe/test_flag_eval_metrics.py::Test_FFE_Eval_Metric_Parse_Error_Variant_Type_Mismatch : bug (FFL-2064)
1614+ tests/ffe/test_flag_eval_metrics.py::Test_FFE_Eval_Reason_Split : bug (FFL-2064)
16091615 tests/integration_frameworks/llm/anthropic/test_anthropic_apm.py::TestAnthropicApmMessages : *ref_5_71_0
16101616 tests/integration_frameworks/llm/anthropic/test_anthropic_llmobs.py::TestAnthropicLlmObsMessages : *ref_5_71_0
16111617 tests/integration_frameworks/llm/anthropic/test_anthropic_llmobs.py::TestAnthropicLlmObsMessages::test_create : missing_feature # ephemeral cache TTL metrics not yet released
Original file line number Diff line number Diff line change @@ -121,9 +121,10 @@ def test_ffe_eval_metric_basic(self):
121121 assert get_tag_value (tags , "feature_flag.result.variant" ) == "on" , (
122122 f"Expected tag feature_flag.result.variant:on, got tags: { tags } "
123123 )
124- assert get_tag_value (tags , "feature_flag.result.reason" ) == "static" , (
125- f"Expected tag feature_flag.result.reason:static, got tags: { tags } "
126- )
124+ # TODO(FFL-2064): align evaluation reasons across SDKs
125+ # assert get_tag_value(tags, "feature_flag.result.reason") == "static", (
126+ # f"Expected tag feature_flag.result.reason:static, got tags: {tags}"
127+ # )
127128 assert get_tag_value (tags , "feature_flag.result.allocation_key" ) == "default-allocation" , (
128129 f"Expected tag feature_flag.result.allocation_key:default-allocation, got tags: { tags } "
129130 )
Original file line number Diff line number Diff line change @@ -581,6 +581,8 @@ class _Scenarios:
581581 "OTEL_EXPORTER_OTLP_METRICS_PROTOCOL" : "http/protobuf" ,
582582 "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT" : "http://agent:4318/v1/metrics" ,
583583 },
584+ # nodejs needs some time to submit the metrics to the agent
585+ library_interface_timeout = 10 ,
584586 agent_interface_timeout = 30 ,
585587 doc = "" ,
586588 scenario_groups = [scenario_groups .ffe ],
You can’t perform that action at this time.
0 commit comments