feat(profiling): link task samples to origin task

[gyuheon0h]

PROF-15417
local

Description

When an asyncio task offloads blocking work to a ThreadPoolExecutor (await loop.run_in_executor(None, fn)), that work runs on a pool worker thread that has no asyncio task of its own. Today those samples can't be tied back to the task that submitted and is awaiting them.

This adds two pprof labels on the executor worker thread's samples:

• origin task id -- id() of the submitting asyncio task
• origin task name -- that task's name

origin task id is deliberately the same value the profiler already emits as task id for the task on its event-loop thread, so the offloaded work correlates back to the awaiting task with no new join key.

The futures integration already wraps ThreadPoolExecutor.submit and runs the work through an intermediate _wrap_execution on the worker thread. We capture the current asyncio task at submit time, and at execution time register a thread_id -> {task id, name} entry in a native per-thread map for the duration of the job, clearing it in a finally. The stack renderer stamps the labels by looking that map up by sampled thread id. The worker thread isnt predicted ahead of time, rather, the link is made on whichever worker the pool assigns and torn down when the job completes, so pool reuse is handled correctly.

Testing

Risks

Additional Notes

[gyuheon0h]

• feat(profiling): link task samples to origin task #19082 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[cit-pr-commenter-54b7da]

Codeowners resolved as

ddtrace/contrib/internal/futures/threading.py                           @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/internal/datadog/profiling/dd_wrapper/include/libdatadog_helpers.hpp  @DataDog/profiling-python
ddtrace/internal/datadog/profiling/dd_wrapper/include/sample.hpp        @DataDog/profiling-python
ddtrace/internal/datadog/profiling/dd_wrapper/src/sample.cpp            @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/CMakeLists.txt                 @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/__init__.py                    @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/__init__.pyi                   @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/_stack.pyi                     @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/include/origin_task_links.hpp  @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/include/sampler.hpp            @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/src/origin_task_links.cpp      @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/src/sampler.cpp                @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/src/stack.cpp                  @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/src/stack_renderer.cpp         @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/test/CMakeLists.txt            @DataDog/profiling-python
ddtrace/internal/datadog/profiling/stack/test/test_origin_task_links.cpp  @DataDog/profiling-python
releasenotes/notes/profiling-asyncio-origin-task-c0ffee1234567890.yaml  @DataDog/apm-python
tests/contrib/futures/test_origin_task_linking.py                       @DataDog/apm-core-python @DataDog/apm-idm-python
tests/profiling/collector/test_asyncio_executor.py                      @DataDog/profiling-python

[cit-pr-commenter-54b7da]

Circular import analysis

⚠️ Existing circular imports

There are 61 circular imports that already exist on the base branch and have not been changed by this PR.

Show existing cycles (showing 5 of 61 shortest)

ddtrace.internal.datastreams -> ddtrace.internal.datastreams.kombu -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.aiokafka -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.botocore -> ddtrace.internal.datastreams
ddtrace.internal.core -> ddtrace._trace.span -> ddtrace.internal.core
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.kafka -> ddtrace.internal.datastreams

To see all cycles, download the cycles-base.json and cycles-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/cycles.py compare cycles-base.json cycles-pr.json

[datadog-datadog-prod-us1]

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 83a5e40 | Docs | Datadog PR Page | Give us feedback!}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6293ddaa8e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[taegyunkim]

With this PR, do we expect to see any difference on UI (timeline maybe?) for applications using ThreadPoolExecutor? or this is PR is a necessary step towards to have such changes?

[gyuheon0h]

With this PR, do we expect to see any difference on UI (timeline maybe?) for applications using ThreadPoolExecutor? or this is PR is a necessary step towards to have such changes?

@taegyunkim Nothing changed in UI after this. But its a necessary step

[taegyunkim]

Thanks! now it's just formatting ci

[wantsui]

LGTM from an integrations standpoint