Skip to content

fix #4251: wrap BackgroundTasks in their own span#4368

Merged
aabmass merged 6 commits into
open-telemetry:mainfrom
davidgss04:fix-bug-4251
May 8, 2026
Merged

fix #4251: wrap BackgroundTasks in their own span#4368
aabmass merged 6 commits into
open-telemetry:mainfrom
davidgss04:fix-bug-4251

Conversation

@davidgss04
Copy link
Copy Markdown
Contributor

Description

Fixes #4251

FastAPI background tasks were not properly instrumented. While request spans were created correctly, background tasks executed after the response without their own span. As a result, any spans created inside a background task were incorrectly attached directly to the request span, which had already finished, leading to broken trace hierarchies and inaccurate timing information.

The fix patches BackgroundTask.__call__ to wrap its execution in a dedicated span. This ensures that each background task runs within its own span, correctly parented to the request span, and that any spans created inside the task are properly nested.

The patch is applied only once using a guard (hasattr) to avoid double instrumentation, and the original method is restored during uninstrument_app to prevent side effects.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Testing

Added three regression tests:

  • test_background_task_span_parents_inner_spans — verifies that background tasks create a wrapper span and that spans created inside the task are correctly parented

  • test_uninstrument_app_restores_background_task_call — ensures the original BackgroundTask.__call__ is restored after uninstrumentation

  • test_background_task_span_not_duplicated_on_double_instrument_app — verifies that repeated instrumentation does not create duplicate spans or apply the patch multiple times

Does This PR Require a Core Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added

@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla Bot commented Mar 27, 2026

CLA Signed

The committers listed above are authorized under a signed CLA.

@tammy-baylis-swi
Copy link
Copy Markdown
Contributor

Thanks for this! The fix to Background tracing makes sense to me and there is no semconv requirement to add HTTP attributes to that particular span. Metrics generated will be the same.

Please could you also commit fixes from:

  • tox -e precommit
  • tox -e lint-instrumentation-fastapi

@tammy-baylis-swi tammy-baylis-swi moved this to Reviewed PRs that need fixes in Python PR digest Mar 27, 2026
Only the request/ASGI middleware was traced; background tasks
were not.

Background tasks ran after the response with no span, causing
child spans to attach to a closed parent span.

This change patches BackgroundTask.__call__ to wrap execution
in a new span.

Each background task now creates a child span of the request,
ensuring a correct trace hierarchy.

Add three regression tests for background task instrumentation.
@tammy-baylis-swi tammy-baylis-swi requested a review from a team March 30, 2026 16:38
@tammy-baylis-swi
Copy link
Copy Markdown
Contributor

Thanks @davidgss04 ! Forgot to also mention: please add an entry to CHANGELOG.md linking this PR.

Copy link
Copy Markdown
Contributor

@tammy-baylis-swi tammy-baylis-swi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again! This lgtm. 👍 The maintainers will also have to review this.

@tammy-baylis-swi tammy-baylis-swi requested a review from a team March 30, 2026 23:07
@tammy-baylis-swi tammy-baylis-swi moved this from Reviewed PRs that need fixes to Approved PRs in Python PR digest Mar 30, 2026
@github-actions
Copy link
Copy Markdown

This PR has been automatically marked as stale because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 days of this comment.
If you're still working on this, please add a comment or push new commits.

@github-actions github-actions Bot added the Stale label Apr 28, 2026
@davidgss04
Copy link
Copy Markdown
Contributor Author

Good evening @tammy-baylis-swi,

Regarding the failing tests, I noticed that they are all related to Python 3.9 environments. To investigate, I synced my local repository with upstream/main and ran the same failing tox environments locally. The failures occur during dependency resolution, as the current opentelemetry-api (from main) now requires Python >=3.10, making the Python 3.9 environments unsatisfiable.

Based on this, it seems the issue is not related to the changes in this PR, but rather to the current test configuration or dependency versions for Python 3.9.

I also observed that some recent passing PRs do not appear to run Python 3.9 tests.

Do you have any guidance on how this should be handled?

@github-actions github-actions Bot removed the Stale label Apr 29, 2026
Copy link
Copy Markdown
Member

@aabmass aabmass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this PR but I think I remember seeing something related to this in FastAPI upstream. If the background tasks are explicitly run by FastAPI with an isolated contextvars.Context instance, they will end up detached from the request by default. IIRC FastAPI made this the default behavior.

@aabmass aabmass moved this from Approved PRs to Ready for merge in Python PR digest May 8, 2026
@aabmass aabmass enabled auto-merge (squash) May 8, 2026 20:59
@aabmass aabmass merged commit 915165b into open-telemetry:main May 8, 2026
753 checks passed
@github-project-automation github-project-automation Bot moved this from Ready for merge to Done in Python PR digest May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[fastapi] BackgroundTasks produce child spans that outlive their closed parent span

3 participants