Skip to content

Commit 65ae689

Browse files
committed
fix(appsec): Go runtime _dd.appsec.enabled missing on aws.lambda
Two related bugs caused the `_dd.appsec.enabled` metric and other AppSec tags to be absent from the `aws.lambda` invocation span for Go (and Java) runtimes when using extension-side App & API Protection enablement. **Bug 1 — race condition (context deleted by placeholder span)** Go's tracer emits a placeholder `aws.lambda` span with `resource = "dd-tracer-serverless-span"` in its `/v0.4/traces` flush. AppSec's `service_entry_span_mut` was matching this placeholder (same name, `request_id` in meta). Depending on tokio scheduling, the placeholder could arrive and be processed by AppSec _after_ `/runtime/invocation/ response` had already set `response_seen = true`. In that case, `Processor::process_span` would tag the placeholder (harmless, it gets dropped by `ChunkProcessor`) and then _delete the AppSec context_ (the "finalized" branch). When `process_on_platform_runtime_done` later sent the extension-built `aws.lambda` span via `send_ctx_spans`, the context was gone and no tags were applied. Fix: filter placeholder spans out of `service_entry_span_mut` by excluding spans whose `resource == INVOCATION_SPAN_RESOURCE`. These spans are always dropped before reaching the backend, so tagging them is both pointless and harmful. **Bug 2 — `_dd.appsec.enabled` never pre-set on the invocation span** `enrich_ctx_at_platform_done` calls `inferrer.complete_inferred_spans` which propagates `_dd.appsec.enabled` from the invocation span to the inferred trigger span (e.g. `aws.lambda.url`). However, AppSec has not yet run on the invocation span at that point, so the metric is not in `invocation_span.metrics`. `propagate_appsec` therefore falls back to the `serverless_appsec_enabled` config flag for the _inferred_ span (which always got the tag) but never sets it on the _invocation_ span itself. If the AppSec context could not be found at flush time for any reason, `aws.lambda` shipped without `_dd.appsec.enabled`. Fix: in `enrich_ctx_at_platform_done`, pre-set `_dd.appsec.enabled = 1.0` on the invocation span when AAP is enabled, before calling `complete_inferred_spans`. This resolves the pre-existing TODO comment (`// todo(duncanista): Add missing metric tags for ASM`), ensures the inferred span inherits the value from the actual metric rather than the config fallback, and makes the tag present even when the AppSec context is unavailable. Both issues are specific to Go (and Java) because only those runtimes use the placeholder span pattern. Python and Node emit the `aws.lambda` span directly in their tracer payload, which is not filtered and is not subject to the context-deletion race. JJ-Change-Id: xltnyl
1 parent 9f49e35 commit 65ae689

3 files changed

Lines changed: 24 additions & 3 deletions

File tree

bottlecap/src/appsec/processor/mod.rs

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -151,10 +151,19 @@ impl Processor {
151151
/// Returns the first `aws.lambda` span from the provided trace, if one
152152
/// exists.
153153
///
154+
/// Placeholder spans (resource == `INVOCATION_SPAN_RESOURCE`) emitted by
155+
/// Go and Java tracers are excluded: they are always dropped by the chunk
156+
/// processor before reaching the backend, so tagging them would waste the
157+
/// AppSec context and trigger a premature context deletion that would leave
158+
/// the real, extension-built `aws.lambda` span untagged.
159+
///
154160
/// # Returns
155161
/// The span on which security information will be attached.
156162
pub fn service_entry_span_mut(trace: &mut [Span]) -> Option<&mut Span> {
157-
trace.iter_mut().find(|span| span.name == "aws.lambda")
163+
trace.iter_mut().find(|span| {
164+
span.name == "aws.lambda"
165+
&& span.resource != crate::traces::INVOCATION_SPAN_RESOURCE
166+
})
158167
}
159168

160169
/// Processes an intercepted [`Span`].

bottlecap/src/lifecycle/invocation/processor.rs

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -615,7 +615,6 @@ impl Processor {
615615
);
616616
}
617617

618-
// todo(duncanista): Add missing metric tags for ASM
619618
// Add dynamic and trigger tags
620619
context
621620
.invocation_span
@@ -626,6 +625,19 @@ impl Processor {
626625
context.invocation_span.meta.extend(trigger_tags);
627626
}
628627

628+
// Ensure _dd.appsec.enabled is present on the invocation span when AAP is enabled.
629+
// complete_inferred_spans (called below) propagates this metric from the invocation
630+
// span to the inferred trigger span. AppSec's process_span will set it again from the
631+
// security context when it runs, but this baseline guarantees the tag is always present
632+
// even when the context cannot be found at flush time.
633+
if self.config.serverless_appsec_enabled {
634+
context
635+
.invocation_span
636+
.metrics
637+
.entry("_dd.appsec.enabled".to_string())
638+
.or_insert(1.0);
639+
}
640+
629641
self.inferrer
630642
.complete_inferred_spans(&context.invocation_span);
631643

bottlecap/src/traces/mod.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ const DNS_LOCAL_HOST_ADDRESS_URL_PREFIX: &str = "127.0.0.1";
3838
const AWS_XRAY_DAEMON_ADDRESS_URL_PREFIX: &str = "169.254.79.129";
3939

4040
// Name of the placeholder invocation span set by Java and Go tracers
41-
const INVOCATION_SPAN_RESOURCE: &str = "dd-tracer-serverless-span";
41+
pub(crate) const INVOCATION_SPAN_RESOURCE: &str = "dd-tracer-serverless-span";
4242

4343
#[allow(clippy::doc_markdown)]
4444
/// Header used for additional tags when sending APM data to the Datadog intake

0 commit comments

Comments
 (0)