Skip to content

Add decoded span assert rules - #12088

Open
PerfectSlayer wants to merge 2 commits into
bbujon/smoke-tests-decoded-spansfrom
bbujon/smoke-tests-decoded-span-assertions
Open

Add decoded span assert rules#12088
PerfectSlayer wants to merge 2 commits into
bbujon/smoke-tests-decoded-spansfrom
bbujon/smoke-tests-decoded-span-assertions

Conversation

@PerfectSlayer

@PerfectSlayer PerfectSlayer commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

This PR introduces decoded span assert rules.

EDIT: This PR unit tests need test agent JSON  span deserialization first. I’ll introduce an intermediate PR to the stack to add it. It is still good to review in the meantime.

Motivation

The goal is to provide capabilities for smoke tests to evaluate trace and span structures and values.

Additional Notes

This is mainly inspired from my prior work with instrumentation tests but decoded spans are not like DDSpan. They don’t hold the same values / types so new rules needed to be put in place for smoke tests.

Contributor Checklist

Jira ticket: APMLP-1247

@PerfectSlayer PerfectSlayer added the type: feature Enhancements and improvements label Jul 28, 2026
@PerfectSlayer
PerfectSlayer requested a review from a team as a code owner July 28, 2026 09:15
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Bits has a CI fix ready

🟢 Investigated · 🟢 Fix prepared · ⚪ Validation skipped · 🟠 Ready

SmokeMatcherTest imports the nonexistent datadog.smoketest.backend.TestAgentTraceDecoder, so the smoke-test compile job cannot resolve its synthetic trace fixtures. The tests now create lightweight in-test DecodedTrace and DecodedSpan fixtures, preserving the matcher coverage without depending on an unavailable backend decoder.

Commit fix to this PR


View in Datadog | Reviewed commit 5134699 · Any feedback? Reach out in #deveng-pr-agent

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Autotest was unable to complete this review. View session

Please try again by commenting @autotest review.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 513469965b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread dd-smoke-tests/src/main/java/datadog/smoketest/trace/SmokeTraceAssertions.java Outdated
Comment thread dd-smoke-tests/src/main/java/datadog/smoketest/trace/SpanMatcher.java Outdated
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
Patch Coverage: 2.44%
Overall Coverage: 57.51% (-0.03%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 798798a | Docs | Datadog PR Page | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.96 s 13.85 s [+0.0%; +1.6%] (maybe worse)
startup:insecure-bank:tracing:Agent 12.95 s 13.00 s [-0.9%; +0.1%] (no difference)
startup:petclinic:appsec:Agent 17.03 s 16.77 s [+0.7%; +2.4%] (maybe worse)
startup:petclinic:iast:Agent 17.02 s 17.04 s [-1.0%; +0.7%] (no difference)
startup:petclinic:profiling:Agent 16.70 s 16.80 s [-1.5%; +0.2%] (no difference)
startup:petclinic:sca:Agent 16.94 s 16.98 s [-1.1%; +0.6%] (no difference)
startup:petclinic:tracing:Agent 16.16 s 16.10 s [-0.6%; +1.3%] (no difference)

Commit: 798798ab · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@PerfectSlayer
PerfectSlayer marked this pull request as draft July 28, 2026 09:59
@PerfectSlayer
PerfectSlayer force-pushed the bbujon/smoke-tests-decoded-span-assertions branch from 5134699 to d7f5363 Compare July 28, 2026 16:42
@PerfectSlayer

Copy link
Copy Markdown
Contributor Author

@autotest review

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: FAIL

Unordered trace matching can reject a valid trace when a reusable childOfPrevious matcher first probes a non-matching candidate: assertSpan mutates the matcher to that candidate's parent ID, so later candidates are checked against stale state. Keep the resolved parent matcher local instead of mutating the configured matcher.

Open Bits AI session

🤖 Datadog Autotest · Commit d7f5363 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

Comment thread dd-smoke-tests/src/main/java/datadog/smoketest/trace/SpanMatcher.java Outdated
@PerfectSlayer
PerfectSlayer force-pushed the bbujon/smoke-tests-decoded-span-assertions branch from d7f5363 to 798798a Compare July 28, 2026 17:31
@PerfectSlayer
PerfectSlayer marked this pull request as ready for review July 28, 2026 17:34

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

The decoder and assertion paths do not show a confirmed behavioral regression or diff-only production hazard. Adversarial cases were reviewed for malformed JSON, unsigned IDs, missing fields, duplicate/unordered trace matching, ancestry ordering, partial trace chunks, and nested meta_struct values; execution was blocked before tests ran because the checkout requires JDK 25 and the sandbox has only JDK 8/11/17/21.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Datadog Autotest · Commit 798798a · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: testing Testing type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant