Skip to content

Commit 246da2c

Browse files
authored
Symfony messenger telemetry refactor (#2466)
* feat(flow-php/telemetry): time-based flush for batching processors - add optional max_batch_age (monotonic hrtime) to span/metric/log batching processors * feat(flow-php/telemetry): derive trace id from active span; consumed messages get their own trace - Context holds the active SpanContext instead of a standalone trace id; root spans generate a new TraceId (OTEL-aligned) - remove Context::withTraceId/create and the context(?TraceId) DSL seeding; withActiveSpan takes a SpanContext - symfony-telemetry-bundle: each consumed Messenger message is its own trace, linked to the producer (link mode) - symfony-telemetry-bundle: configurable worker-span link (link_to_worker, default true) + per-message flush subscriber * refactor(flow-php/telemetry): derive span parent solely from Context - remove per-tracer span stack; active span lives in shared Context - parent resolution falls through to Context::activeSpan() only - nesting popped via Scope detach instead of stack pop - add cross-tracer parenting + attached-context conformance tests
1 parent 4baa42a commit 246da2c

32 files changed

Lines changed: 1162 additions & 410 deletions

File tree

documentation/components/bridges/symfony-telemetry-bundle.md

Lines changed: 41 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -508,18 +508,34 @@ processor:
508508

509509
Batches items before export.
510510

511-
| Option | Type | Default | Description |
512-
|--------------|---------|---------|-----------------------------------------|
513-
| `batch_size` | integer | `512` | Number of items per batch |
514-
| `exporter` | string | - | Name of a top-level exporter (required) |
511+
| Option | Type | Default | Description |
512+
|-----------------|---------|---------|------------------------------------------------------------------------------------------------------|
513+
| `batch_size` | integer | `512` | Number of items per batch |
514+
| `max_batch_age` | float | `null` | Max seconds before a partial batch is force-exported, measured from the first buffered signal; `null` disables time-based flush |
515+
| `exporter` | string | - | Name of a top-level exporter (required) |
515516

516517
```yaml
517518
processor:
518519
type: batching
519520
batch_size: 512
521+
max_batch_age: 15.0 # export at least every 15s in long-running processes
520522
exporter: otlp
521523
```
522524

525+
By default a batch is exported only when it reaches `batch_size`, when `flush()`/`shutdown()` is called, or
526+
when the process exits. In a request-scoped app that is fine — the request ends and the buffer drains. In a
527+
**long-running process** (Symfony Messenger workers, daemons, or persistent runtimes such as ReactPHP,
528+
RoadRunner, Swoole, AMP, or FrankenPHP worker mode) a low-rate signal can sit in the buffer until 512
529+
accumulate or the process stops. Set `max_batch_age` (e.g. `15.0`) to bound how long any single signal waits
530+
before export; leave it unset for request-scoped apps where behavior is unchanged.
531+
532+
> **PHP limitation — idle processes.** PHP has no background timer thread, so the age deadline is evaluated
533+
> only when a signal is processed, not on a wall-clock schedule. A **fully idle** process — nothing ending
534+
> spans or processing metrics/logs — cannot self-flush a partial batch on the age trigger. That residual
535+
> case needs an external flush: the Messenger worker-event flush subscriber, a runtime loop calling
536+
> `Telemetry::flush()`, or `shutdown()` when the process exits. The deadline is measured from a monotonic
537+
> clock (`hrtime`), so it is immune to wall-clock adjustments.
538+
523539
#### composite
524540

525541
Combines multiple processors. Each child references its own exporter by name.
@@ -1083,21 +1099,38 @@ flow_telemetry:
10831099
enabled: true
10841100
context_propagation: true # Propagate context across message boundaries
10851101
propagation_style: link # How the consumer span relates to the producer span
1102+
link_to_worker: true # Link each message trace back to the messenger:consume worker span
10861103
```
10871104

10881105
When `context_propagation` is enabled, `propagation_style` controls how a consumed message's span
10891106
relates to the producing (publishing) span:
10901107

1091-
- `link` (default) — the consumer span stays in the worker's own trace (under the `messenger:consume`
1092-
console span) and carries a span link back to the producer span. Producer and consumer get separate,
1093-
clean traces connected by a link. Recommended for decoupled, batch, or long-delay queues, where
1094-
continuing the trace would otherwise absorb the entire queue wait into a single span's duration.
1108+
- `link` (default) — each consumed message is the **root of its own trace** and carries a span link back
1109+
to the producer span (per the OpenTelemetry messaging conventions). Producer and consumer get separate,
1110+
clean traces connected by a link, and a long-running worker no longer collapses every message into one
1111+
trace. Recommended for decoupled, batch, or long-delay queues, where continuing the trace would otherwise
1112+
absorb the entire queue wait into a single span's duration.
10951113
- `continue` — the consumer span adopts the producer's trace and becomes its child, so
10961114
publish → queue → consume is one continuous distributed trace. Fine for fast, 1:1 processing.
10971115

10981116
`propagation_style` has no effect when `context_propagation` is `false` (there is nothing to relate to).
10991117
The producer side is identical in both modes — the telemetry stamp is always written on dispatch.
11001118

1119+
`link_to_worker` (default `true`) adds a span link from each consumed message's trace back to the active
1120+
`messenger:consume` worker span, so you can pivot from a message trace to the worker that processed it.
1121+
Set it to `false` to omit the link (e.g. if the consume command is not traced).
1122+
1123+
Buffered telemetry is flushed **after each handled or failed message** while the worker keeps running, so
1124+
consumer-side traces/logs/metrics export promptly instead of only when the `messenger:consume` worker
1125+
stops. (Without it, signals emitted inside handlers sit in the batching processors — default batch size
1126+
512 — and stay invisible until the buffer fills or the worker exits.) Failures flush too, so error spans
1127+
and exception logs are visible even when the message is retried or sent to the failure transport. This is
1128+
independent of `context_propagation` / `propagation_style`.
1129+
1130+
Flushing per message means one exporter round-trip per message. For high-throughput workers, set
1131+
[`max_batch_age`](#batching) on the batching processor so the batch coalesces across messages and a single
1132+
idle worker still exports on a time bound rather than per message.
1133+
11011134
#### Twig
11021135

11031136
Traces Twig template rendering.

documentation/upgrading.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,28 @@ incoming and injecting outgoing W3C trace headers:
5656
composer require flow-php/symfony-http-foundation-telemetry-bridge
5757
```
5858

59+
### 3) `flow-php/telemetry` - trace id derived from the active span; root spans start a new trace
60+
61+
| Before | After |
62+
|-------------------------------------|-------------------------------------------------------------------------|
63+
| `Context::create()` | `Context::root()` |
64+
| `Context::withTraceId(TraceId)` | removed |
65+
| `$context->traceId` (property) | `$context->traceId(): ?TraceId` (derived from the active span) |
66+
| `Context::withActiveSpan(SpanId)` | `Context::withActiveSpan(SpanContext)` |
67+
| `context(?TraceId, ?Baggage)` (DSL) | `context(?Baggage)` |
68+
| a root span reused the context trace id | each root span generates a new `TraceId` |
69+
70+
`Context` no longer stores a standalone trace id; attach the active span as a `SpanContext` to keep
71+
subsequent spans in the same trace.
72+
73+
### 4) `flow-php/symfony-telemetry-bundle` - each consumed Messenger message is its own trace
74+
75+
| Before | After |
76+
|--------------------------------------------------------------|-----------------------------------------------------------------------------|
77+
| all messages in a `messenger:consume` run shared one trace | each handled message is a new trace root, linked to the producer (`link` mode) |
78+
79+
`continue` mode still joins the producer's trace.
80+
5981
---
6082

6183
## Upgrading from 0.39.x to 0.40.x

0 commit comments

Comments
 (0)