fix #4251: wrap BackgroundTasks in their own span#4368
Conversation
|
Thanks for this! The fix to Please could you also commit fixes from:
|
Only the request/ASGI middleware was traced; background tasks were not. Background tasks ran after the response with no span, causing child spans to attach to a closed parent span. This change patches BackgroundTask.__call__ to wrap execution in a new span. Each background task now creates a child span of the request, ensuring a correct trace hierarchy. Add three regression tests for background task instrumentation.
d65500d to
ab9b962
Compare
|
Thanks @davidgss04 ! Forgot to also mention: please add an entry to |
tammy-baylis-swi
left a comment
There was a problem hiding this comment.
Thanks again! This lgtm. 👍 The maintainers will also have to review this.
|
This PR has been automatically marked as stale because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 days of this comment. |
|
Good evening @tammy-baylis-swi, Regarding the failing tests, I noticed that they are all related to Python 3.9 environments. To investigate, I synced my local repository with Based on this, it seems the issue is not related to the changes in this PR, but rather to the current test configuration or dependency versions for Python 3.9. I also observed that some recent passing PRs do not appear to run Python 3.9 tests. Do you have any guidance on how this should be handled? |
aabmass
left a comment
There was a problem hiding this comment.
I'm OK with this PR but I think I remember seeing something related to this in FastAPI upstream. If the background tasks are explicitly run by FastAPI with an isolated contextvars.Context instance, they will end up detached from the request by default. IIRC FastAPI made this the default behavior.
Description
Fixes #4251
FastAPI background tasks were not properly instrumented. While request spans were created correctly, background tasks executed after the response without their own span. As a result, any spans created inside a background task were incorrectly attached directly to the request span, which had already finished, leading to broken trace hierarchies and inaccurate timing information.
The fix patches
BackgroundTask.__call__to wrap its execution in a dedicated span. This ensures that each background task runs within its own span, correctly parented to the request span, and that any spans created inside the task are properly nested.The patch is applied only once using a guard (
hasattr) to avoid double instrumentation, and the original method is restored duringuninstrument_appto prevent side effects.Type of change
Testing
Added three regression tests:
test_background_task_span_parents_inner_spans— verifies that background tasks create a wrapper span and that spans created inside the task are correctly parentedtest_uninstrument_app_restores_background_task_call— ensures the originalBackgroundTask.__call__is restored after uninstrumentationtest_background_task_span_not_duplicated_on_double_instrument_app— verifies that repeated instrumentation does not create duplicate spans or apply the patch multiple timesDoes This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.