fix(appsec): Go runtime _dd.appsec.enabled missing on aws.lambda

[RomainMuller]

Problem

Two related bugs caused _dd.appsec.enabled and other AppSec tags to be absent from the aws.lambda invocation span for Go (and Java) runtimes when using extension-side App & API Protection.

The symptom was flaky integration tests: sometimes all AppSec tags were missing, and even when that race was won, _dd.appsec.enabled was consistently absent from aws.lambda while the inferred trigger span (aws.lambda.url, aws.httpapi, etc.) always had it.

Root causes

Bug 1 — race condition: AppSec context deleted by placeholder span

Go's tracer emits a placeholder aws.lambda span with resource = "dd-tracer-serverless-span" alongside its child spans in /v0.4/traces. AppSecProcessor::service_entry_span_mut was matching this placeholder (it has name = "aws.lambda" and request_id in meta). Depending on tokio scheduling, the placeholder could reach Processor::process_span after /runtime/invocation/response had set response_seen = true. In that case, process_span would tag the placeholder — harmless, since ChunkProcessor drops it — but then delete the AppSec context. When process_on_platform_runtime_done later sent the extension-built aws.lambda span via send_ctx_spans, the context was gone and no tags were applied.

Fix: filter placeholder spans out of service_entry_span_mut by excluding spans whose resource == INVOCATION_SPAN_RESOURCE ("dd-tracer-serverless-span"). These spans are always dropped before reaching the backend; tagging them is both pointless and harmful.

Bug 2 — _dd.appsec.enabled never pre-set on the invocation span

enrich_ctx_at_platform_done calls inferrer.complete_inferred_spans, which propagates _dd.appsec.enabled from the invocation span to the inferred trigger span via propagate_appsec. However, AppSec has not yet run on the invocation span at that point, so _dd.appsec.enabled is not in invocation_span.metrics. propagate_appsec falls back to the serverless_appsec_enabled config flag for the inferred span (so it always got the tag) but never sets it on the invocation span itself. If AppSec's context was unavailable at flush time for any reason, aws.lambda shipped without the tag. This also explains an existing // todo(duncanista): Add missing metric tags for ASM comment at that exact location.

Fix: pre-set _dd.appsec.enabled = 1.0 on the invocation span in enrich_ctx_at_platform_done when AAP is enabled, before calling complete_inferred_spans. This ensures the inferred span inherits from the actual metric value rather than the config fallback, and guarantees the tag is present even when the AppSec security context cannot be found at flush time.

Why Go/Java only

Only Go and Java use the placeholder span pattern. Python and Node emit their aws.lambda span directly in their tracer payload with resource = function_name, so service_entry_span_mut correctly identifies it, and the context-deletion race cannot happen.

Changes

File	Change
bottlecap/src/traces/mod.rs	Make INVOCATION_SPAN_RESOURCE pub(crate)
bottlecap/src/appsec/processor/mod.rs	Filter placeholder spans in service_entry_span_mut
bottlecap/src/lifecycle/invocation/processor.rs	Pre-set _dd.appsec.enabled on invocation span before complete_inferred_spans

Testing

• Existing appsec_processor_test integration test passes.
• The race condition is no longer reproducible in Go + Lambda URL integration tests.
• _dd.appsec.enabled is now consistently present on the aws.lambda span.

[Copilot]

Pull request overview

Fixes missing/flaky AppSec tags (notably _dd.appsec.enabled) on the aws.lambda invocation span for Go/Java runtimes when using extension-side App & API Protection, by avoiding placeholder-span races and ensuring the metric is present early enough for propagation.

Changes:

• Expose INVOCATION_SPAN_RESOURCE within the crate so non-traces modules can reliably identify placeholder invocation spans.
• Update AppSec span selection to ignore Go/Java placeholder aws.lambda spans (resource == INVOCATION_SPAN_RESOURCE) to prevent premature AppSec context deletion.
• Pre-set _dd.appsec.enabled = 1.0 on the invocation span when AAP is enabled, before inferred-span completion/propagation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
bottlecap/src/traces/mod.rs	Makes the placeholder invocation span resource constant available crate-wide.
bottlecap/src/appsec/processor/mod.rs	Filters out placeholder aws.lambda spans when selecting the service-entry span for AppSec tagging.
bottlecap/src/lifecycle/invocation/processor.rs	Ensures _dd.appsec.enabled is present on the invocation span prior to inferred span propagation.

[purple4reina]

Running the e2e tests here: https://gitlab.ddbuild.io/DataDog/serverless-e2e-tests/-/pipelines/110011644

[purple4reina]

Running the e2e tests here: gitlab.ddbuild.io/DataDog/serverless-e2e-tests/-/pipelines/110011644

lambda-features/tests/test_spans.py::test_appsec_attack_detection[golang-appsecextension] XPASS [ 94%]

The test was flaky though, sometimes passing, sometimes not, but in this case it passed.