Skip to content

Dudik/debugger/guardrails tests#7287

Draft
dudikeleti wants to merge 4 commits into
mainfrom
dudik/debugger/guardrails-tests
Draft

Dudik/debugger/guardrails tests#7287
dudikeleti wants to merge 4 commits into
mainfrom
dudik/debugger/guardrails-tests

Conversation

@dudikeleti

Copy link
Copy Markdown
Contributor

Motivation

Add focused system-test coverage for the debugger guardrails required for GA.

Changes

  • Add Test_Debugger_Evaluation_Timeout: expensive when evaluation (ReDoS regex, large collection filter) must be aborted and surfaced as an evaluation-error snapshot (populated evaluationErrors[], no captured data), not a normal snapshot.
  • Add Test_Debugger_Snapshot_Guardrails: completed snapshots are trimmed to ≤1 MB, and capture exceeding the time budget reports notCapturedReason=timeout.
  • Extend test_default_max_length to assert notCapturedReason=stringLength, completing default capture-limit reason coverage.
  • Extract shared captures_contain_data helper into debugger.utils (reused by the condition-error tests).
  • All new guardrail rows are marked missing_feature across every tracer; enable per-language as support lands. The regex fixture only exercises backtracking engines (linear engines like Go stay disabled via manifests).

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

…reasons and evaluation timeout behavior, with new rows disabled in manifests until tracer support is ready.
…re timeout reporting. 1MB snapshot cap and notCapturedReason=timeout behavior.
…ves the setup-time negative snapshot wait so evaluation-timeout assertions rely on collected scenario output instead of a fixed sleep.
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

tests/debugger/test_debugger_guardrails.py                              @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/probes/probe_capture_timeout_reason.json           @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/probes/probe_evaluation_timeout_collection_filter.json  @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/probes/probe_evaluation_timeout_regex.json         @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/probes/probe_snapshot_size_cap.json                @DataDog/debugger @DataDog/system-tests-core
manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet
manifests/golang.yml                                                    @DataDog/dd-trace-go-guild
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
manifests/nodejs.yml                                                    @DataDog/dd-trace-js
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
manifests/python.yml                                                    @DataDog/apm-python @DataDog/asm-python
manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby
tests/debugger/test_debugger_condition_errors.py                        @DataDog/debugger @DataDog/system-tests-core
tests/debugger/test_debugger_probe_snapshot.py                          @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils.py                                                 @DataDog/debugger @DataDog/system-tests-core

@datadog-datadog-prod-us1-2

Copy link
Copy Markdown

Pipelines

Fix all issues with BitsAI

⚠️ Warnings

🚦 3 Pipeline jobs failed

Testing the test | Test the test   View in Datadog   GitHub Actions

Testing the test | lint / lint   View in Datadog   GitHub Actions

Testing the test | all-jobs-are-green   View in Datadog   GitHub Actions

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9f01c86 | Docs | Datadog PR Page | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant