Skip to content

feat(sdk): add otel.sdk.processor.log.processed self-diagnostics metric#3514

Merged
cijothomas merged 4 commits into
open-telemetry:mainfrom
cijothomas:cijothomas/sdk-self-diagnostics-log-processed
May 26, 2026
Merged

feat(sdk): add otel.sdk.processor.log.processed self-diagnostics metric#3514
cijothomas merged 4 commits into
open-telemetry:mainfrom
cijothomas:cijothomas/sdk-self-diagnostics-log-processed

Conversation

@cijothomas

@cijothomas cijothomas commented May 11, 2026

Copy link
Copy Markdown
Member

Adds the otel.sdk.processor.log.processed counter to BatchLogProcessor, following the OTel SDK metrics semantic conventions.

Gated behind the existing experimental_metrics_bound_instruments feature flag — bound instruments make this practical on the hot path (~1.8 ns per emit vs ~50 ns with unbound).

Changes:

  • BatchLogProcessor emits a counter on every emit() call with three attribute variants: success, queue_full, and already_shutdown
  • OTLP examples reordered to set up MeterProvider first (so the processor gets a real meter)
  • Shutdown order corrected to tracer → logger → meter
  • Design doc added at docs/design/observability.md
  • Ignored test added (runs in isolation in CI via test.sh)

Ordering requirement: global::set_meter_provider() must be called before creating the LoggerProvider. Rust's global::meter() returns a snapshot, not a delegate. See the design doc for details.

@codecov

codecov Bot commented May 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 49.33333% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.9%. Comparing base (284a37d) to head (b85cf83).

Files with missing lines Patch % Lines
opentelemetry-sdk/src/logs/batch_log_processor.rs 49.3% 38 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main   #3514     +/-   ##
=======================================
- Coverage   82.9%   82.9%   -0.1%     
=======================================
  Files        130     130             
  Lines      27311   27386     +75     
=======================================
+ Hits       22659   22711     +52     
- Misses      4652    4675     +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cijothomas cijothomas force-pushed the cijothomas/sdk-self-diagnostics-log-processed branch from 9ed6cfd to 36d0e77 Compare May 11, 2026 20:20
let instance_id = INSTANCE_COUNTER.fetch_add(1, Ordering::Relaxed);
let component_name = format!("batching_log_processor/{instance_id}");

let meter = opentelemetry::global::meter("otel.sdk");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up suggestion (not a blocker for this PR): detect & warn when bound counters resolve to noop

If the user constructs the LoggerProvider before global::set_meter_provider, the three BoundCounters permanently bind to NoopBoundSyncInstrument, add(1) becomes a no-op, and no metric is ever emitted. The user gets no signal — just total == 0. The OTLP examples work around it by reordering setup, but the failure mode is silent.

The current public API can't detect this: MeterProvider is type-erased, doesn't extend Any, and has no is_noop() method. Detecting "was set_meter_provider called?" via a flag isn't enough either — NoopMeterProvider is pub and a valid MeterProvider, so a user can legitimately install it explicitly. We need to ask the actual provider whether it's a no-op.

Proposed addition: defaulted is_noop on MeterProvider

// opentelemetry/src/metrics/meter.rs
pub trait MeterProvider {
    // existing methods...

    /// Returns `true` if this provider is a no-op implementation that
    /// silently discards all measurements. Defaults to `false`; the
    /// built-in `NoopMeterProvider` overrides to `true`. SDK components
    /// can use this to warn at construction when self-diagnostics metrics
    /// would be silently dropped.
    fn is_noop(&self) -> bool { false }
}

// opentelemetry/src/metrics/noop.rs
impl MeterProvider for NoopMeterProvider {
    // existing methods...
    fn is_noop(&self) -> bool { true }
}

Backwards-compatible — existing MeterProvider impls don't need to change.

Usage in BatchLogProcessor::new

let provider = opentelemetry::global::meter_provider();
if provider.is_noop() {
    otel_warn!(
        name: "BatchLogProcessor.SelfDiagnosticsUnavailable",
        message = "BatchLogProcessor obtained a no-op MeterProvider from the global \
                   registry; otel.sdk.processor.log.processed will not record. Call \
                   global::set_meter_provider with a real MeterProvider before \
                   constructing the LoggerProvider to enable self-diagnostics."
    );
}
let meter = provider.meter("otel.sdk");
// ... build counter, bind() three times as today

Benefits

  • Catches both the "forgot to set the provider" and "set it after the LoggerProvider" cases — the warning fires whenever the bound counters would silently drop measurements.
  • One virtual call at construction; zero impact on add() hot path.
  • Works on injected providers too, in case any SDK component later accepts a MeterProvider parameter instead of reaching for the global.
  • Reusable by BatchSpanProcessor, PeriodicReader, and any future component that emits self-diagnostics.

Does not fix the late-rebind case (user replaces the global provider with a different real one after construction) — that needs the late-binding global::meter() work already noted in docs/design/observability.md.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree. I think we need to solve the fundamental underlying issue - that is the proper solution!

@cijothomas

Copy link
Copy Markdown
Member Author

@bantonsson PTAL! thanks in advance.

@cijothomas cijothomas added this pull request to the merge queue May 26, 2026
@cijothomas cijothomas removed this pull request from the merge queue due to a manual request May 26, 2026
@cijothomas cijothomas added this pull request to the merge queue May 26, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks May 26, 2026
@cijothomas cijothomas closed this May 26, 2026
@cijothomas cijothomas reopened this May 26, 2026
@cijothomas cijothomas added this pull request to the merge queue May 26, 2026
Merged via the queue into open-telemetry:main with commit 6e1fbbe May 26, 2026
25 of 26 checks passed
@cijothomas cijothomas deleted the cijothomas/sdk-self-diagnostics-log-processed branch May 26, 2026 21:54
jsurany pushed a commit to jsurany/opentelemetry-rust that referenced this pull request Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

3 participants