Use weblog metadata to declare weblog's category, and connect to scenario matrix builder

[cbeauchesne]

Motivation

Knowning which scenario runs on which weblog is quite complex. So far, it was done by one of the crappiest piece of code I've ever wrote : hardcoded names with organic rules that are added on top of each over threw times.

SSI and libinjection also has it own recipes, with a cleaner approach, where dat is stored in json, and logic in python.

As we'll create an official workflow in gitlab, we need a unified (and clean) way of doing so.

This PR propose an unified way of tacking this, and implement it to end-to-end scenario, SSI will come later

Changes

• Add a categories field on weblog metadata data. This category describe what library is shipped inside the weblog :
  • dd_trace : the basic dd-trace HTTP instrumentation
  • dd_trace_graphql : a dd-trace GraphQL instrumentation
  • dd_trace_lambda : a dd-trace lambda instrumentation
  • dd_trace_frameworks : a dd-trace framework instrumentation (mostly AI)
  • open_telemetry : an Open Telemetry instrumentation
• Add a weblog_categories property on Scenario

The logic is quite simple : a scenario supports a given weblog if both of them has the same weblog category.

On top of that, weblog metadata has two extra field : supported_scenarios and excluded_scenarios, for any fine tune inclusion/exclusion.

Few notes :

• I've preferred yml to json to store data, as it's easier to read for human (as long as we don't use crazy yml syntax)
• weblog category is very semantic : it must describe what's inside the weblog, nothing more, nothing less.
• I've discovered few excluded point in the matrix that does not make any senses, we'll tackle this later

Workflow

1. ⚠️ Create your PR as draft ⚠️
2. Work on you PR until the CI passes
3. Mark it as ready for review
   • Test logic is modified? -> Get a review from RFC owner.
   • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

• Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
• A docker base image is modified?
  • the relevant build-XXX-image label is present
• A scenario is added, removed or renamed?
  • Get a review from R&P team

[github-actions]

CODEOWNERS have been resolved as:

utils/build/docker/cpp_httpd/weblog_metadata.yml                        @DataDog/system-tests-core
utils/build/docker/cpp_kong/weblog_metadata.yml                         @DataDog/system-tests-core
utils/build/docker/cpp_nginx/weblog_metadata.yml                        @DataDog/system-tests-core
utils/build/docker/dotnet/weblog_metadata.yml                           @DataDog/apm-dotnet @DataDog/asm-dotnet @DataDog/system-tests-core
utils/build/docker/java_lambda/weblog_metadata.yml                      @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
utils/build/docker/java_otel/weblog_metadata.yml                        @DataDog/opentelemetry @DataDog/system-tests-core
utils/build/docker/nodejs_lambda/weblog_metadata.yml                    @DataDog/dd-trace-js @DataDog/system-tests-core
utils/build/docker/nodejs_otel/weblog_metadata.yml                      @DataDog/dd-trace-js @DataDog/system-tests-core
utils/build/docker/php/weblog_metadata.yml                              @DataDog/apm-php @DataDog/system-tests-core
utils/build/docker/python_lambda/weblog_metadata.yml                    @DataDog/apm-python @DataDog/asm-python @DataDog/system-tests-core
utils/build/docker/python_otel/weblog_metadata.yml                      @DataDog/apm-python @DataDog/asm-python @DataDog/system-tests-core
utils/build/docker/ruby/weblog_metadata.yml                             @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby_lambda/weblog_metadata.yml                      @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
tests/test_the_test/test_ci_orchestrator.py                             @DataDog/system-tests-core
tests/test_the_test/test_minimal_number_of_scenarios.py                 @DataDog/system-tests-core
utils/_context/_scenarios/__init__.py                                   @DataDog/system-tests-core
utils/_context/_scenarios/_docker_fixtures.py                           @DataDog/system-tests-core
utils/_context/_scenarios/ai_guard.py                                   @DataDog/system-tests-core
utils/_context/_scenarios/appsec_low_waf_timeout.py                     @DataDog/system-tests-core
utils/_context/_scenarios/appsec_rasp.py                                @DataDog/system-tests-core
utils/_context/_scenarios/aws_lambda.py                                 @DataDog/system-tests-core
utils/_context/_scenarios/core.py                                       @DataDog/system-tests-core
utils/_context/_scenarios/debugger.py                                   @DataDog/system-tests-core
utils/_context/_scenarios/default.py                                    @DataDog/system-tests-core
utils/_context/_scenarios/endtoend.py                                   @DataDog/system-tests-core
utils/_context/_scenarios/integration_frameworks.py                     @DataDog/system-tests-core
utils/_context/_scenarios/integrations.py                               @DataDog/system-tests-core
utils/_context/_scenarios/ipv6.py                                       @DataDog/system-tests-core
utils/_context/_scenarios/parametric.py                                 @DataDog/system-tests-core
utils/_context/_scenarios/performance.py                                @DataDog/system-tests-core
utils/_context/_scenarios/profiling.py                                  @DataDog/system-tests-core
utils/_context/constants.py                                             @DataDog/system-tests-core
utils/_context/weblog_metadata.py                                       @DataDog/system-tests-core
utils/build/docker/golang/weblog_metadata.yml                           @DataDog/dd-trace-go-guild @DataDog/system-tests-core
utils/build/docker/java/weblog_metadata.yml                             @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
utils/build/docker/nodejs/weblog_metadata.yml                           @DataDog/dd-trace-js @DataDog/system-tests-core
utils/build/docker/otel_collector/weblog_metadata.yml                   @DataDog/system-tests-core
utils/build/docker/python/weblog_metadata.yml                           @DataDog/apm-python @DataDog/asm-python @DataDog/system-tests-core
utils/scripts/ci_orchestrators/workflow_data.py                         @DataDog/system-tests-core

[datadog-datadog-prod-us1-2]

 

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 2 Pipeline jobs failed

Testing the test | System Tests (nodejs, dev) / End-to-end #1 / fastify 1     

Testing the test | all-jobs-are-green     

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: da0b94e | Docs | Datadog PR Page | Give us feedback!}