Traces are silently dropped in long-running workers

[kiansdi]

[OpenAI Agents SDK] Traces are silently dropped in long-running workers (Celery, FastAPI background tasks, etc.)

Package: openai-agents (Python)
Date: November 2025
Impact: High – many production users run agents via background workers

Description

When using agents.trace() inside long-living processes (Celery workers, FastAPI background tasks, RQ, Dramatiq, etc.), traces are created correctly (valid trace IDs in logs, nesting works) but never appear on https://platform.openai.com/traces.

Reason: the internal BatchTraceProcessor only flushes automatically on process shutdown or when the buffer is full. Long-running workers never shut down → traces remain buffered forever.

Minimal Reproduction

# Celery task – trace never reaches the dashboard
@celery.task
def demo():
    with agents.trace("celery_test"):
        Agent(model="gpt-4o-mini").run("Say hello")
    # ← missing flush → trace lost

Running the exact same code as a standalone script works because shutdown triggers the flush.

Working Fix (3 lines)

from agents.tracing import GLOBAL_TRACE_PROVIDER

def flush_traces():
    GLOBAL_TRACE_PROVIDER._multi_processor.force_flush()

Call flush_traces() at the end of every task/pipeline:

with agents.trace("my_job"):
    # all agent calls
    ...
flush_traces()   # ← required in workers

Result: full nested traces appear instantly on the dashboard.

Suggested Documentation Addition

Note for long-running workers
In Celery, FastAPI background tasks, RQ, Dramatiq, or any long-living worker process, the trace processor does not auto-flush.
Call flush_traces() (or a future public API) at the end of each job, otherwise traces will remain buffered and never appear on the dashboard.

Why this is hard to discover

• No error/warning is emitted
• All top models (Gemini 3, Claude 4.5, Codex-5, etc.) suggest the usual suspects (env vars, async context, .trace_id) – none solve it
• The real answer did not exist publicly until now

Thank you for the excellent Agents SDK – adding this short note would save the community weeks of debugging pain.

[yuyuma]

We have the same exact issue as described above (no / inconsistent traces) for our agent that sits behind a FastAPI route, but the suggested fix to force flush does not work for us unfortunately. Unsure whether the root cause is as described (buffered traces) or something else.

Note: had to update the code slightly when attempting the suggested fix given we were unable to import GLOBAL_TRACE_PROVIDER directly.

from agents.tracing import get_trace_provider
def flush_traces():
    get_trace_provider()._multi_processor.force_flush()

[kiansdi]

@yuyuma
This would be even a better practice instead of importing a global constant which I done
Did it ultimately work for you?

[yuyuma]

Did it ultimately work for you?

Unfortunately did not help

[mvanhorn]

I've submitted a PR to fix this: #2735

This adds a public flush_traces() function that you can call at the end of each task in long-running workers to ensure traces are exported. Usage:

from agents import flush_traces

@celery_app.task
def my_task():
    with trace("my_task"):
        # ... do work ...
        pass
    flush_traces()

No more need to access private internals like _multi_processor.force_flush().

[kiansdi]

I've submitted a PR to fix this: #2735

Nice to see you found this inspiring
I noticed that another issue referencing this one was covered in a milestone and led to a PR that was already merged
By the way, looking forward to seeing what we get further here
Thanks for sharing with us