[debugger] Split method capture-expressions into parameter/local tests; enable Ruby by p-datadog · Pull Request #7309 · DataDog/system-tests

p-datadog · 2026-07-14T17:30:50Z

Motivation

Enable Ruby capture-expressions system tests, bringing Ruby DI toward parity with Java / Python / .NET / Node.js / Go. dd-trace-rb adds the feature in DataDog/dd-trace-rb#5845 (merged; ships in 2.38.0).

Background

dd-trace-rb method probes capture method arguments (and the receiver), not method-body locals. Line probes capture locals.

Changes

tests/debugger/test_debugger_capture_expressions.py: split the method-probe test into two, both running unconditionally on /debugger/expression for every language:
- test_method_parameter_capture_expressions — asserts inputValue, a method parameter (Java/.NET/PHP inject it; Python/Go capture it as a local; Ruby receives it as a passed keyword argument — see weblog change).
- test_method_local_capture_expressions — asserts localValue/testStruct, method-body locals.
- Line-probe test unchanged.
Ruby weblog (shared/rails/.../debugger_controller.rb + rails72/rails80 routes): /debugger/expression now routes to a run_expression wrapper that calls expression(inputValue:), so inputValue is a real passed argument a method probe can capture. The wrapper is placed in existing padding so all line-locked method positions (used by method_and_language_to_line_number) are preserved.
manifests/ruby.yml:
- test_method_parameter_capture_expressions: enabled on rails72/rails80/uds-rails at v2.38.0-dev.
- test_method_local_capture_expressions and test_complex_capture_expressions: missing_feature (Local variable capture not implemented for method probes).

Other-language manifests are unchanged: they declare Test_Debugger_Method_Capture_Expressions at the class level, which covers both new method names by prefix match, and both tests use the existing /debugger/expression endpoint they already support (the split is an assertion-partition of the previous combined test).

Coverage by language

The four capture-expression tests:

Parameter — Test_Debugger_Method_Capture_Expressions::test_method_parameter_capture_expressions (method probe captures inputValue)
Local — ...::test_method_local_capture_expressions (method probe captures localValue/testStruct)
Complex — ...::test_complex_capture_expressions (method probe: getmember/index over testStruct)
Line — Test_Debugger_Line_Capture_Expressions::test_log_line_capture_expressions (line probe)

Language	Parameter	Local	Complex	Line
Java (spring-boot*)	✅ v1.38.0	✅ v1.38.0	✅ v1.38.0	✅ v1.38.0
.NET	❌ missing_feature	❌ missing_feature	❌ missing_feature	❌ missing_feature
PHP (≠ laravel11x)	✅ v1.19.0	✅ v1.19.0	✅ v1.19.0	❌ missing_feature
Python	❌ missing_feature	❌ missing_feature	❌ missing_feature	❌ missing_feature
Node.js (express/fastify)	❌ missing_feature	❌ missing_feature	❌ missing_feature	✅ ~v5.87
Go	✅ v2.2.3	✅ v2.2.3	❌ index unsupported	✅ v2.2.3
Ruby (rails)	✅ v2.38.0-dev	❌ locals not captured	❌ locals not captured	✅ v2.38.0-dev

Notes:

"Parameter" asserts inputValue, genuinely a method parameter in Java/.NET/PHP and (via the run_expression wrapper) Ruby; in Go it is a method-body local that Go's method probes still capture — same assertion, different underlying kind.
Ruby is the only language that cannot pass Local/Complex: its method probes do not capture method-body locals (a dd-trace-rb limitation). Ruby's coverage is Parameter (method probe) + Line.

Testing

Verified against merged dd-trace-rb#5845 (datadog 2.38.0.dev, master 97cc6f6), rails72, DEBUGGER_PROBES_SNAPSHOT:

test_method_parameter_capture_expressions — passes (inputValue = {type: String, value: "testValue"}).
test_log_line_capture_expressions — passes.
test_method_local_capture_expressions, test_complex_capture_expressions — skipped (missing_feature).

Also ran DEBUGGER_EXPRESSION_LANGUAGE (rails72): 12 line-probe tests pass, confirming the controller's line-locked positions are intact.

Not run locally: Java/PHP/Go — the split is an assertion-partition on the same probe/endpoint they already pass, so their result is unchanged; CI confirms.

Workflow

Draft. Gated at v2.38.0-dev. Touches shared debugger test logic and the Ruby weblog, so this wants a review from the debugger RFC owner before un-drafting.

github-actions · 2026-07-14T17:31:22Z

CODEOWNERS have been resolved as:

manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby
tests/debugger/test_debugger_capture_expressions.py                     @DataDog/debugger @DataDog/system-tests-core
utils/build/docker/ruby/rails72/config/routes.rb                        @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/rails80/config/routes.rb                        @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
utils/build/docker/ruby/shared/rails/app/controllers/debugger_controller.rb  @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core

datadog-prod-us1-6 · 2026-07-14T17:31:46Z

Tests

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 3 Pipeline jobs failed

Testing the test | System Tests (python, dev) / End-to-end #1 / uwsgi-poc 1
🧪 1 Test failed
tests.appsec.iast.sink.test_unvalidated_redirect.TestUnvalidatedHeader_ExtendedLocation.test_extended_location_data[uwsgi-poc] from system_tests_suite
AssertionError: Expected a single vulnerability with the matching criteria
assert 2 == 1
 &#43;  where 2 = len([{&#39;evidence&#39;: {&#39;valueParts&#39;: [{&#39;source&#39;: 0, &#39;value&#39;: &#39;http://dummy.location.com&#39;}]}, &#39;hash&#39;: 1185812129, &#39;location&#39;: {...ated_redirect_insecure_header&#39;, &#39;path&#39;: &#39;app.py&#39;, &#39;spanId&#39;: 5141841287092376225, ...}, &#39;type&#39;: &#39;UNVALIDATED_REDIRECT&#39;}])

self = &lt;tests.appsec.iast.sink.test_unvalidated_redirect.TestUnvalidatedHeader_ExtendedLocation object at 0x7f5210472990&gt;

    def test_extended_location_data(self):
&gt;       validate_extended_location_data(self.r, self.vulnerability_type)

tests/appsec/iast/sink/test_unvalidated_redirect.py:126: 
...
Testing the test | System Tests (php, prod) / End-to-end #2 / laravel11x 2
🧪 1 Test failed

All test failures are known flaky.
❄️ Known flaky: tests.appsec.api_security.test_apisec_sampling.Test_API_Security_Sampling_Different_Endpoints.test_sampling_delay[laravel11x] from system_tests_suite
assert False
 &#43;  where False = all(&lt;generator object Test_API_Security_Sampling_Different_Endpoints.test_sampling_delay.&lt;locals&gt;.&lt;genexpr&gt; at 0x7ff29c14db60&gt;)

self = &lt;tests.appsec.api_security.test_apisec_sampling.Test_API_Security_Sampling_Different_Endpoints object at 0x7ff2b7ea64b0&gt;

    def test_sampling_delay(self):
        assert self.request1.status_code == 200
        schema1 = get_schema(self.request1, &#34;req.headers&#34;)
        assert schema1 is not None
    
...
Not introduced in this PR.
Testing the test | all-jobs-are-green

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: cd66b6b | Docs | Datadog PR Page | Give us feedback!}

…method probes) Ruby DI does not capture local variables in method probes, so Test_Debugger_Method_Capture_Expressions (which references the expression method's locals inputValue/localValue/testStruct) cannot pass. Verified against dd-trace-rb#5845: all captures come back as NilClass/isNull and the complex index expressions are not emitted. The line-probe test captures locals at the probe line and passes, so Test_Debugger_Line_Capture_Expressions stays enabled. Mirrors the existing gating of Test_Debugger_PII_Redaction::test_pii_redaction_method_full.

Method probes in dd-trace-rb capture method arguments, not method-body locals. Add a helper method that receives inputValue as a keyword argument so a method probe can capture it via a capture expression. Covers rails72, rails80, and uds-rails (built from rails80).

Ruby method probes capture method arguments but not method-body locals, so the combined method test could not pass. Split it: - test_method_argument_capture_expressions: captures inputValue. Ruby targets the new /debugger/expression/args endpoint (inputValue is a keyword argument); other languages use /debugger/expression where inputValue is already a method parameter (Java/.NET/PHP) or local. - test_method_local_capture_expressions: captures localValue/testStruct. Enable the argument test for Ruby (rails72/rails80/uds-rails) at v2.38.0-dev; keep the local and complex tests missing_feature (local capture in method probes not implemented). Verified against dd-trace-rb#5845: argument test captures inputValue={String,testValue}.

The split method tests had a per-language conditional (Ruby used a separate /debugger/expression/args endpoint). Remove it: make Ruby's probed 'expression' method receive inputValue as a passed keyword argument via a 'run_expression' wrapper action, so a method probe captures it. Now both split tests run unconditionally on /debugger/expression for every language: - test_method_parameter_capture_expressions: asserts inputValue (a method parameter in Java/.NET/PHP; a captured local in Python/Go/Node; the passed kwarg in Ruby). - test_method_local_capture_expressions: asserts localValue/testStruct (method-body locals). Ruby method probes do not capture locals, so this stays missing_feature for Ruby. Also fixes a line-shift regression from the previous commit: inserting the args helper after line 82 pushed expression_operators and the other expression-language methods off their hardcoded lines (102/122/162/192/233) in method_and_language_to_line_number, breaking the Ruby expression-language line-probe tests. run_expression is placed in existing padding so all line-locked positions are preserved. Removes the /debugger/expression/args route and probe_capture_expressions_method_args.json. Verified against merged dd-trace-rb#5845 (datadog 2.38.0.dev, master 97cc6f6), rails72: parameter + line capture-expression tests pass; local + complex skip (missing_feature); 12 expression-language line tests pass.

[ruby] Enable capture expressions system tests

5ba56db

p-ddsign added 5 commits July 14, 2026 13:36

[ruby] Gate capture expressions system tests at v2.38.0-dev

c66e549

p-datadog changed the title ~~[ruby] Enable capture expressions system tests~~ [debugger] Split method capture-expressions into parameter/local tests; enable Ruby Jul 14, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[debugger] Split method capture-expressions into parameter/local tests; enable Ruby#7309

[debugger] Split method capture-expressions into parameter/local tests; enable Ruby#7309
p-datadog wants to merge 6 commits into
mainfrom
ddsign/ruby-capture-expressions

p-datadog commented Jul 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 14, 2026 •

edited

Loading

Uh oh!

datadog-prod-us1-6 Bot commented Jul 14, 2026 •

edited by datadog-prod-us1-3 Bot

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

p-datadog commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Background

Changes

Coverage by language

Testing

Workflow

Uh oh!

github-actions Bot commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datadog-prod-us1-6 Bot commented Jul 14, 2026 • edited by datadog-prod-us1-3 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

p-datadog commented Jul 14, 2026 •

edited

Loading

github-actions Bot commented Jul 14, 2026 •

edited

Loading

datadog-prod-us1-6 Bot commented Jul 14, 2026 •

edited by datadog-prod-us1-3 Bot

Loading