feat(appender-tracing): add builder support for custom instrumentation scope

[DmitryAstafyev]

Changes

This PR adds OpenTelemetryTracingBridgeBuilder::with_scope() to allow configuring the OpenTelemetry InstrumentationScope used by the appender logger.

The implementation also updates OpenTelemetryTracingBridgeBuilder to hold the logger provider until build(), so the builder can expose a set of configuration methods:

• with_scope()
• with_span_attribute_allowlist()
• and potentially additional builder options in the future

Design Notes

This change introduces a lifetime parameter on OpenTelemetryTracingBridgeBuilder<'a>. That was done to keep the builder aligned with the existing style, where the bridge is constructed from a provider reference rather than from a pre-created logger.

In return, this keeps the API shape cleaner and gives us a more natural builder surface with related configuration methods grouped together.

Compatibility

Technically, adding the lifetime to the public builder type is a breaking API change.

However, this builder API has not yet shipped in a released version, so this does not affect production users of published crates. In practice, the change is limited to the unreleased API surface.

Testing

Added a unit test (tracing_appender_with_custom_scope) covering with_scope() and verifying that the configured InstrumentationScope is propagated to emitted logs.

Related

• Fix: [Feature]: Allow passing InstrumentationScope when creating an instance of OpenTelemetryTracingBridge #3415

Merge requirement checklist

• CONTRIBUTING guidelines followed
• Unit tests added/updated (if applicable)
• Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
• Changes in public API reviewed (if applicable)

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: DmitryAstafyev / name: DmitryAstafyev (248e5ee, 3f3f304, 453f7a1, 56e5ecb, 866bae7)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.0%. Comparing base (fb2af9d) to head (53d106f).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #3428   +/-   ##
=====================================
  Coverage   83.0%   83.0%           
=====================================
  Files        130     130           
  Lines      27952   27992   +40     
=====================================
+ Hits       23210   23250   +40     
  Misses      4742    4742           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[DmitryAstafyev]

Hello @cijothomas, could you please take a look at this PR? It's my first time contributing here, and I'm ready to correct/change if it's required. Many thanks in advance for review.

[lalitb]

The new scope support looks fine, but this changes the public builder API in a breaking way. OpenTelemetryTracingBridgeBuilder was previously an owned public type, and this change makes it borrow the provider and adds a lifetime parameter. That can break downstream code that names the builder type explicitly, stores it, or returns it from helper functions. For example:

fn make_builder<P, L>(
    provider: &P,
) -> opentelemetry_appender_tracing::layer::OpenTelemetryTracingBridgeBuilder<P, L>
where
    P: opentelemetry::logs::LoggerProvider<Logger = L> + Send + Sync,
    L: opentelemetry::logs::Logger + Send + Sync,
{
    opentelemetry_appender_tracing::layer::OpenTelemetryTracingBridge::builder(provider)
}

Can we keep the builder owned while still adding with_scope()?

[DmitryAstafyev]

Thank you @lalitb for the review and for highlighting this.

You are correct: my original proposal introduced a breaking API change by adding a lifetime to the public OpenTelemetryTracingBridgeBuilder type. That is problematic for downstream users who explicitly name, store, or return the builder type.

My goal was to keep configuration cohesive in the builder (with_scope() as a builder method), while still borrowing the provider. In practice, that design required a lifetime on the public builder, which weakens API stability.

I have updated the implementation to remove the lifetime and keep the builder owned, as requested.

The current trade-off is configuration split across two entry points:

• OpenTelemetryTracingBridge::with_scope(...) for scope configuration
• builder methods (for example with_span_attribute_allowlist(...)) for additional options

This preserves compatibility, but is less uniform than a fully builder-centric API.

From an API stability perspective, I also see a risk in a public builder whose signature may need to grow over time. If you agree, we could consider a more stable construction surface (for example, keep builder(provider) minimal/stable and expose additional dedicated constructors for optional configuration like scope).

Would you prefer one of these directions?

1. Keep builder(provider) public and stable, and expose scope via a separate public constructor (for example builder_with_scope(...)) so we avoid growing the builder signature.

2. Make OpenTelemetryTracingBridge::builder(...) non-public and keep only stable public constructors (new, with_scope, etc.), so internal constructor changes do not affect public API compatibility.

[lalitb]

Thanks for the update and the well-thought-out options @DmitryAstafyev. I'd go with your Option 1 - keep builder(provider) unchanged and add builder_with_scope(provider, scope) as a new entry point.

The scope isn't a configuration property of the bridge itself - it's a parameter needed alongside the provider to create the underlying logger. Since it must be known at the same time as the provider, a separate builder_with_scope() entry point is the natural fit - it makes that requirement explicit in the API rather than hiding it behind an Option parameter.

also, note that you need to sign CLA before this can be moved further.

[DmitryAstafyev]

Thanks @lalitb, for your response. I've made the requested changes.

• builder(provider: &P) stay as it was
• builder_with_scope(...) has beed added
• with_scope(...) has been added as a constructor

Let me know, please, if I should do something else.

[lalitb]

Thanks @DmitryAstafyev LGTM. Will approve/merge this once the CLA is signed.

[cijothomas]

@DmitryAstafyev Can you share bit more about the intended use-case? Since OTLP Exporter override scope-name with target, this may not have the intended effect, so want to make sure we have a good grasp of the scenario.

[dmitryastafyev-dyna]

@DmitryAstafyev Can you share bit more about the intended use-case? Since OTLP Exporter override scope-name with target, this may not have the intended effect, so want to make sure we have a good grasp of the scenario.

Thanks for your question @cijothomas . I hope I've gotten your question in right way.

We are interested in not loosing attributes, overwriting scope-name with target is okey. The use case is simple: we need to attach per-invocation metadata to every log record's scope in OTLP output. This metadata is dynamic and cannot live on the Resource level, so it belongs on the InstrumentationScope. Without builder_with_scope, there was no API to pass a custom scope to the bridge, which forced us to work around it by wrapping the provider.

If the concern is that scope attributes were being discarded by the OTLP exporter when a target is present - that was indeed a real issue, but it appears to have been fixed by 3332 which was merged recently. With both fixes in place the full use case should work as intended for us.

[cijothomas]

@DmitryAstafyev Can you share bit more about the intended use-case? Since OTLP Exporter override scope-name with target, this may not have the intended effect, so want to make sure we have a good grasp of the scenario.

Thanks for your question @cijothomas . I hope I've gotten your question in right way.

We are interested in not loosing attributes, overwriting scope-name with target is okey. The use case is simple: we need to attach per-invocation metadata to every log record's scope in OTLP output. This metadata is dynamic and cannot live on the Resource level, so it belongs on the InstrumentationScope. Without builder_with_scope, there was no API to pass a custom scope to the bridge, which forced us to work around it by wrapping the provider.

If the concern is that scope attributes were being discarded by the OTLP exporter when a target is present - that was indeed a real issue, but it appears to have been fixed by 3332 which was merged recently. With both fixes in place the full use case should work as intended for us.

Thanks for the explanation @DmitryAstafyev! One clarification — the InstrumentationScope is set once when the bridge is created and stays fixed for its lifetime. So by "per-invocation metadata," do you mean per-process-invocation (e.g., a CLI/batch job that runs and exits)? In a long-running server, the scope attributes wouldn't change across requests — per-request data would need to go as log record attributes instead. Once you share more details, I can tell if there is better way to do it.!

If you need per-request metadata in a long-running server, tracing span attributes would be a better fit — we recently added support for propagating span attributes to log records via the experimental_span_attributes feature (it won't be stored as Scope level, but at LogRecord level)

[dmitryastafyev-dyna]

@DmitryAstafyev Can you share bit more about the intended use-case? Since OTLP Exporter override scope-name with target, this may not have the intended effect, so want to make sure we have a good grasp of the scenario.

Thanks for your question @cijothomas . I hope I've gotten your question in right way.
We are interested in not loosing attributes, overwriting scope-name with target is okey. The use case is simple: we need to attach per-invocation metadata to every log record's scope in OTLP output. This metadata is dynamic and cannot live on the Resource level, so it belongs on the InstrumentationScope. Without builder_with_scope, there was no API to pass a custom scope to the bridge, which forced us to work around it by wrapping the provider.
If the concern is that scope attributes were being discarded by the OTLP exporter when a target is present - that was indeed a real issue, but it appears to have been fixed by 3332 which was merged recently. With both fixes in place the full use case should work as intended for us.

Thanks for the explanation @DmitryAstafyev! One clarification — the InstrumentationScope is set once when the bridge is created and stays fixed for its lifetime. So by "per-invocation metadata," do you mean per-process-invocation (e.g., a CLI/batch job that runs and exits)? In a long-running server, the scope attributes wouldn't change across requests — per-request data would need to go as log record attributes instead. Once you share more details, I can tell if there is better way to do it.!

If you need per-request metadata in a long-running server, tracing span attributes would be a better fit — we recently added support for propagating span attributes to log records via the experimental_span_attributes feature (it won't be stored as Scope level, but at LogRecord level)

Thanks @cijothomas for your question. Yes, our use case is per-invocation, not per-request in a long-running server. Sorry for confusion in my prev-reply. For each function invocation we create a fresh OpenTelemetryTracingBridge with invocation-scoped metadata (e.g. invocation ID etc.). Span attributes on individual log records wouldn't be a fit here, since the metadata describes the bridge's operational context rather than individual log events. A native builder_with_scope would let us drop the wrapper-workaround we currently rely on.

[dmitryastafyev-dyna]

Hello @cijothomas, any news about the progress?

[dmitryastafyev-dyna]

Hello @cijothomas, @lalitb, I've rebased PR to master. Could you please take a look. Thanks in advance!

[cijothomas]

Hello @cijothomas, any news about the progress?

Apologies for the delay. Before going further on the API, can you share a minimal example of the per-invocation use case? Specifically:

Is the process short-lived (one invocation per process), or long-running with many invocations in-process?
The right solution could be - Resource, tracing span attributes onto each LogRecord, or something else.

The reason this matters: as written, builder_with_scope(provider, scope) looks like the supplied scope is used directly, but the bridge uses a single placeholder Logger and the scope name on every record comes from target.
Only the scope's attributes / version / schema_url are kept.

Renaming to with_scope_attributes is clearer about what's accepted but doesn't fix the per-target replication, and gets harder if we ever move to one Logger per target. Hence wanting to understand the scenario before deciding whether this should be a bridge-level API at all.

https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/design/logs.md#performance, section 5 explains why we are in this tricky situation. We certainly want to make sure the scenarios is well understood before expanding the API here else this would get more complex.

[DmitryAstafyev]

@cijothomas thanks for your question.

I read our previous discussion, and I think I may have confused you and a little messed up things. Sorry if that happened.

Let me clarify the lifecycle first:

• What is an invocation? - it's one run of user code.
• How many invocations live in one process? - one invocation per process.
• Is a new LoggerProvider created per invocation? - one per process/run.
• Is a new OpenTelemetryTracingBridge created per invocation? - one per
  process/run.
• Can two invocations log through the same subscriber/provider at the same
  time? - not in our case.
• Can logs from different invocations end up in the same OTLP batch?- not in our case.

I also wasn't correct when talking about log record attributes. We do not want to copy these attributes into every log record. What we need is scope attributes under ResourceLogs.ScopeLogs.scope.attributes.

The attributes contain some important metadata for us. With the current builder(P), the bridge creates the logger with provider.logger(""). That creates an empty scope, so scope.attributes are empty. After export, we do not see the attributes on the endpoint.

With provider.logger_with_scope(scope) inside builder_with_scope, the logger keeps the provided scope. This gives us scope attributes, including after OTLP export.

I agree that the method name could be better. with_scope_attributes, like you suggested, sounds reasonable for me . Please let me know if that would be enough.

I hope I've answered your question more clearly this time :)

addon:

The right solution could be - Resource

I will back with an answer tomorrow. Have to check.

[cijothomas]

@DmitryAstafyev

Given the lifecycle you described (one process per invocation, metadata constant for the whole run), I think Resource might actually be a cleaner fit than Scope here - it's process-scoped, set once at startup, and flows through as ResourceLogs.resource.attributes on every export, which sounds like exactly what you need.

(also - what backend are you using? Some might not support scope attributes, and might require Collector massaging to move attributes around to the right place)

[dmitryastafyev-dyna]

Thanks @cijothomas for your response. I took time to check the proposed change and the alternatives against our system. Sorry, I can't share the code, but I'll try to explain.

In our model one process = one invocation. On the Resource we keep only stable identity that does not change between invocations (service name, version, etc.). We do not put per-invocation data (like the invocation id and others) there. The Resource is shared by traces and metrics in the same process, so anything we add to it also lands on our metrics and traces. Metrics are aggregated across many invocations on the backend, and the backend groups them by Resource identity. Per-invocation data in the Resource would make every invocation a separate series and affect aggregation.

One way around this would be separate Resources per signal. But I would not do it. Resource is the identity of the entity that produces the telemetry, and backends correlate logs, metrics, and traces by matching that identity. In other words, using different Resources for what is actually the same entity does not look right. This is also reflected in the spec, where faas.instance / faas.name / faas.version are Resource attributes, but faas.invocation_id is a span attribute.

Another work-around is the per-record option (span attributes on each LogRecord). I mean copying the some data (attrs) into every record. We can do this with a custom logger, but it isn't efficient: the value is constant for the whole run, so the right way is one copy at ScopeLogs.scope.attributes. It's what builder_with_scope_attributes (I renamed the method) gives us.

what backend are you using?

This is a data-model question (scope vs Resource vs per-record), independent of the backend. Our backend reads scope-level attributes, so it's not a blocker for us.

@cijothomas, with the rename and the use-case clarified, is there any remaining blocker, or are we good to merge?

[cijothomas]

@dmitryastafyev-dyna Thanks. It's more clear now. Let me re-review the changes with this context.

[cijothomas]

@dmitryastafyev-dyna Thanks. It's more clear now. Let me re-review the changes with this context.

https://github.com/open-telemetry/opentelemetry-rust/pull/3428/changes#r3473102342 - Once this is addressed, we can merge.

[dmitryastafyev-dyna]

@dmitryastafyev-dyna Thanks. It's more clear now. Let me re-review the changes with this context.

https://github.com/open-telemetry/opentelemetry-rust/pull/3428/changes#r3473102342 - Once this is addressed, we can merge.

Thanks @cijothomas . Done.

[cijothomas]

Agree

[cijothomas]

@DmitryAstafyev Queue to merge. Thanks for patiently working on this!