Add APM test agent trace decoding - #12089
Conversation
There was a problem hiding this comment.
The new JSON decoder leaves v0.4 meta_struct entries as base64 strings, although the shared DecodedSpan contract and existing v0.4 decoder expose their decoded JSON values. Any smoke test that inspects AppSec or other structured span metadata will receive the wrong type or fail when casting it, so this path needs a custom decode before shipping.
🤖 Datadog Autotest · Commit 0bfd4d7 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
| public long getDuration() { | ||
| return this.duration; | ||
| } | ||
|
|
There was a problem hiding this comment.
Decode base64 meta_struct payloads
Smoke tests inspecting AppSec or other structured span metadata will see the wrong type and can fail or silently miss the metadata they are intended to validate.
Assertion details
- Input: A /test/traces JSON response containing v0.4 meta_struct, for example meta_struct:{"appsec":"<base64 of JSON {"triggers":3}>"}. The test-agent wire representation uses binary meta_struct values, and JSON encoding represents those byte arrays as base64 strings.
- Expected:
getMetaStruct() returns the decoded structured value, matching SpanV04 and the DecodedSpan contract, so callers can inspect nested maps and lists. - Actual:
SpanJson declares metaStruct as Map<String,Object> and returns it directly. Moshi therefore exposes the base64 text as a String instead of decoding the bytes and parsing the JSON payload.
Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
There was a problem hiding this comment.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0bfd4d7e74
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @Json(name = "meta_struct") | ||
| Map<String, Object> metaStruct; |
There was a problem hiding this comment.
Decode binary meta_struct entries before exposing them
When a trace contains meta_struct, each v0.4 entry is a MessagePack-encoded binary payload (see TraceMapperV0_4.MetaStructWriter), which /test/traces represents as a base64 JSON string. Deserializing directly into Map<String, Object> therefore exposes strings instead of the structured objects returned by the existing v0.4 decoder, so AppSec/LLM smoke tests that inspect or cast these values will fail or evaluate the wrong data. Decode each base64 value and unpack its MessagePack payload; the current synthetic test should also use the wire representation rather than an already-decoded object.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
It does not apply here. From Claude:
Thanks — this is the right instinct in general (in the raw msgpack v0.4/v1 wire formats, meta_struct values are binary MessagePack blobs, which is exactly why SpanV04/SpanV1 decode them). But it doesn't apply to the JSON this decoder consumes: the dd-apm-test-agent decodes meta_struct server-side before serving /test/traces and /test/session/traces, so the JSON already carries nested objects, not base64.
From ddapm_test_agent/trace.py at the pinned v1.44.0:
meta_struct: NotRequired[Dict[str, Dict[str, Any]]] # decoded dicts, not stringsdef _parse_meta_struct(value):
return {key: msgpack.unpackb(val_bytes) for key, val_bytes in value.items()}
_parse_meta_struct is applied via _flexible_decode_meta_struct(payload) inside decode_v04(...), and handle_session_traces / handle_test_traces simply web.json_response(traces) those already-decoded traces. Source: https://github.com/DataDog/dd-apm-test-agent/blob/v1.44.0/ddapm_test_agent/trace.py#L363-L367So typing meta_struct as base64 Map<String,String> and re-decoding would actually break against the real agent (the value is a nested JSON object, not a string). Keeping Map<String,Object> and letting the value deserialize directly is correct here. Not making this change; verified against the agent source (and the existing metaStructEmptyWhenAbsentAndAMapWhenPresent fixture uses the real nested shape).
This comment has been minimized.
This comment has been minimized.
0bfd4d7 to
d14912e
Compare
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
d14912e to
fa90299
Compare
What Does This Do
This PR adds APM test agent trace decoding.
Motivation
The goal is to retrieve the full traces captured by the APM test agent in order to evaluate them during smoke tests.
Unlike our mock backend, using the test agent will avoid testing against ourselves and our decoders.
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: APMLP-1247