perf(profiling): bound task stack collection

[taegyunkim]

Description

Bounds asyncio stack collection and applies an explicit task-priority frame budget after logical stack stitching.

• materializes at most DD_PROFILING_MAX_FRAMES physical thread frames
• materializes at most DD_PROFILING_MAX_FRAMES coroutine frames per task and stops assembling a logical task chain once its frame budget is full
• gives logical task frames first claim on the final reporting budget
• fills remaining capacity with synchronous leaf frames, followed by lower-priority event-loop roots
• emits an inline synchronous-frame omission marker when physical frames are removed between retained logical segments
• counts injected native locations while selecting frames, preventing physical context from displacing task context in the exporter
• conservatively reports task context alone when an on-CPU asyncio boundary lies beyond the bounded physical stack
• scales the Echion frame cache to 4 * max_frames, bounded between 256 and 1,024 entries

This is stacked only on #19073, which propagates the configured limit into Echion and applies it to plain thread stacks. Merged #19076 supplies the 600-location backend and configuration limit on main.

JIRA: PROF-14213

Testing

• Added deterministic low-limit regressions covering:
  • task-only output when task frames fill the budget
  • preservation of task context when a deep synchronous stack hides the asyncio boundary
  • use of remaining capacity for synchronous context and an inline omission marker
• Verified both new task-priority regressions fail on the parent implementation and pass with this change.
• Recreated the Python 3.14 base environment to force a clean native extension build.
• Passed 11 targeted tests in normal and fast-copy configurations, 22 test executions total:
  • three low-limit task frame-budget regressions
  • native frame-limit and cache scaling
  • plain stack truncation
  • asyncio basic and task labels
  • deep gather stitching
  • recursive on-CPU coroutine and task stitching
  • on-CPU and off-CPU asyncio wall-time stacks
• After fixing clang-tidy findings, reran the three low-limit regressions in normal and fast-copy configurations, 6 test executions passed.
• Reworked the deep synchronous-stack regression to use an explicitly created task, avoiding uvloop's wrapper-only top-level task, and verified all three regressions with uvloop on Python 3.9 and 3.14.
• Confirmed the revised deep-stack regression fails on the parent implementation and passes with this change.
• Ran the complete profiling clang-tidy analysis with clang-tidy 18, including fuzz harnesses.
• Ran scripts/lint checks, C formatting, spelling, and git diff --check. Broad checks reported only existing repository warnings.
• scripts/run-benchmarks --list found no benchmark suite matching the changed Echion files.

Risks

Low DD_PROFILING_MAX_FRAMES values now intentionally prefer logical task ancestry over physical synchronous frames. This changes which portion of a mixed stack is reported, but preserves a bounded and logically meaningful task trace instead of allowing a deep physical stack to hide task context.

If the asyncio boundary is outside the bounded physical candidates, the sampler does not guess at overlap or concatenate potentially duplicated segments. It reports the bounded logical task stack alone.

The smaller frame cache can increase cache churn in workloads with many distinct code locations. It retains a 256-entry floor and scales with configured stack depth to balance reuse and memory.

Additional Notes

Task selection and render ordering are intentionally separate. Task frames receive budgeting priority, while retained segments are still rendered in logical leaf-to-root order.

The uppercase MAX_STACK_DISCOVERY_DEPTH constant remains a hard internal safety bound. The configured reporting and materialization budget is stored separately in stack_max_frames.

[cit-pr-commenter-54b7da]

Codeowners resolved as

ddtrace/internal/datadog/profiling/stack/fuzz/fuzz_echion_stacks.cpp    @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/fuzz/fuzz_echion_task_unwind.cpp  @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/src/echion/threads.cc          @DataDog/profiling-python
tests/profiling/collector/test_asyncio_task_frame_budget.py             @DataDog/profiling-python

[cit-pr-commenter-54b7da]

Circular import analysis

⚠️ Existing circular imports

There are 61 circular imports that already exist on the base branch and have not been changed by this PR.

Show existing cycles (showing 5 of 61 shortest)

ddtrace.internal.datastreams -> ddtrace.internal.datastreams.aiokafka -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.kafka -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.google_cloud_pubsub -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.botocore -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.kombu -> ddtrace.internal.datastreams

To see all cycles, download the cycles-base.json and cycles-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/cycles.py compare cycles-base.json cycles-pr.json