Configure trace sampling from Tracing attribute

[gaelreyrol]

We should be able to configure trace sampling from the #[Trace] attribute, whether it's from a command or a route.

Example:

#[Traceable(ratio: 0.1)]
#[Route('/healthcheck', methods: ['GET'])]
public function healthcheck(): Response
{
    return $this->json([
        'status' => 'ok',
    ]);
}

[jvocampings]

Hi

I'm trying to do the implementation here : https://github.com/jvocampings/opentelemetry-bundle/pull/1/files

I don't really see how to configure the sampler from that. I'll continue to read the code of the bundle. Maybe from an Extension to read route's attributes then set the sampler dynamically in the container ?

[gaelreyrol]

I think there are two ways of doing this:

1. Instead of using the sampler mecanism from OpenTelemetry, implement our own sampling to skip span creation in the TraceableHttpKernelEventSubscriber, the sampling would be based on a request whereas OpenTelemetry sampling is based on a span. The downside of this is that spans would never exist, registered samplers on the OpenTelemetry side would not be able to act on them.
2. Add the sampling context as span attributes when the span is created in the TraceableHttpKernelEventSubscriber, this way when receiving the span in the sampler, we could act on it. The downside of this is that if you decided to not use this sampler or not extend it, the sampling would not happen anymore.

I prever the first option, as it would provide a clear separation, sampling from the bundle attribute means the span is not created whereas sample sampling from OpenTelemetry means the span exists but is not sent.

It could be explained like this:

1. Incoming HTTP request
2. Instrumentation (aka span creation or not based on a set of rules) <- here
3. Request handling
4. Sampling (aka span sampling by OpenTelemetry)
5. Request termination
6. Spans flush/export

It is worth to be noted that some filtering is already done in TraceableHttpKernelEventSubscriber for example, if you choose to exclude routes when auto instrumentation mode is enabled.

[jvocampings]

I think it could leads to an issue with propagation if the span is not created. Even if the span is not sampled by OpenTelemetry (sent to the collector), we still want to add the trace id to any HTTP request sent or dispatched message no ?

What I mean is:

1. Incoming HTTP request
2. Instrumentation -> span is not created (sampling decision)
3. Request handling -> our app sends an HTTP request (without trace id because we don't create any span)
4. The receiver receives an HTTP request without any trace id -> no propagation.

[gaelreyrol]

Well the only issue is when the incoming HTTP request has a trace id from another service. Other than that, it seems logic and expected that no trace id is propagated to other components (http client, messenger etc) if no instrumentation is implemented, I mean, why would propagate a trace id that has been sampled nor created.

[gaelreyrol]

That being said, we can have both ways implemented, but the second option should be more generic I think.

For example:

#[Traceable(attributes: [
    'ratio' => 0.8,
    'origin' => 'request()->headers->get("Origin")',
])]
#[Route('/healthcheck', methods: ['GET'])]
public function healthcheck(): Response
{
    return $this->json([
        'status' => 'ok',
    ]);
}

In this example we could automatically add values to the span attributes based on raw values and by using the expression language component.

[jvocampings]

Well the only issue is when the incoming HTTP request has a trace id from another service. Other than that, it seems logic and expected that no trace id is propagated to other components (http client, messenger etc) if no instrumentation is implemented, I mean, why would propagate a trace id that has been sampled nor created.

I have one use case in mind: adding the trace id in log's context and being able to fetch all logs related to a trace (sampled or not). Without propagation, we'll miss some logs.

[gaelreyrol]

You're right but I am feeling that it has less something to do with tracing capability than the logging one. Let me explain, you can implement this behavior without this bundle but with a monolog processor: https://symfony.com/doc/current/logging/processors.html#adding-a-session-request-token. I would to prefer to ease filtering/sampling capabilities by not creating spans at all and not rely on OpenTelemetry sampling, than having limited options.

Still, I am not closed to the other way, so if you want to try it by passing data from #[Traceable] properties to the Span attributes so you can act on them from the OpenTelemetry sampling implementation, feel free, I will happily review it and work on it with you 🙂.