Messenger Instrumentation - Trace seems incomplete

[kochen]

I have an application with a small (single message) messenger consumer that has this flow (handler):

• get the message
• twig an HTML
• generate a PDF
• upload to S3
• publish a (completion) message - this goes into a messenger (RabbitMQ) queue

OpenTelemetry Bundle instrumentation:

open_telemetry:
    service:
        namespace: 'App'
        name: 'PDFGenerator'
        version: '1.0.0'
        environment: '%kernel.environment%'

    instrumentation:
        console:
            tracing:
                enabled: true
            metering:
                enabled: true
        doctrine:
            tracing:
                enabled: true
            metering:
                enabled: true
        http_client:
            tracing:
                enabled: true
            metering:
                enabled: true
        http_kernel:
            tracing:
                enabled: true
            metering:
                enabled: true
        messenger:
            tracing:
                enabled: true
            metering:
                enabled: true
        twig:
            tracing:
                enabled: true
            metering:
                enabled: true

...

Looking at the traces of the consumer, I see a single block of twig:

I would expect to see the entire flow of the service here.

[kochen]

I've tried excluding the messenger:consume command from the console instrumentation as suggested here, but it just "ungrouped" (unspanned?) the traces:

[kochen]

removing the console instrumentation, disabled tracing completely - seems that other instrumentations do not work (?).

[kochen]

I added the missing middleware also didn't help. it just added these 2 rows to the trace:

[gaelreyrol]

Hi @kochen, thanks for the detailed report!

This is indeed the same root cause as #172. The current console instrumentation (TraceableConsoleEventSubscriber) creates a single span for the entire messenger:consume command on ConsoleEvents::COMMAND and only ends it on ConsoleEvents::TERMINATE. Since messenger:consume is designed to run indefinitely, that root span never ends and never gets exported to your collector — which is why the trace appears incomplete.

As you discovered, excluding messenger:consume from console instrumentation doesn't solve the problem either: it removes the only active span/scope in the OpenTelemetry context, so all other instrumentations (Twig, HttpClient, Doctrine…) produce orphan spans with no parent to group them together.

What's missing is per-message span management for Messenger workers. Instead of one span for the whole command lifetime, we need a new root span started and ended for each consumed message. This is exactly what PR #173 is working on:

• A WorkerMessageEventSubscriber that starts a span on WorkerMessageReceivedEvent and ends it on WorkerMessageHandledEvent / WorkerMessageFailedEvent — giving each message its own trace root that other instrumentations can attach to as children
• Automatic exclusion of messenger:consume from console instrumentation
• Trace context propagation via a TraceStamp so that producer and consumer traces can be linked together

Unfortunately there's no clean workaround until this lands. I'll prioritize getting #173 across the finish line.