Support on-demand exporting of metrics

[felixbarny]

I have a use case where I don't want to periodically export metrics but only do so on demand.

The use case is implementing a protocol adapter. Instead of monitoring the application itself, it's about reading a proprietary metrics protocol with the goal to convert it to OTLP. Maybe there are better ways to do it but I was thinking about creating OTel metrics from the proprietary metrics protocol using the metrics SDK and then exporting them via the OTLP exporter.

An alternative to a dedicated MetricReader implementation may be to use a PeriodicMetricReader with a very long delay and setting an executor that doesn't spawn a thread by default. Something like this:

PeriodicMetricReader.builder(exporter)
    .setExecutor(Executors.newScheduledThreadPool(0))
    .setInterval(Duration.ofNanos(Long.MAX_VALUE))
    .build()

While that works, it seems like it's bending the intended use case for this metric reader implementation and doesn't lead to very readable code. Something like this would be a lot nicer:

new OnDemandMetricReader(exporter)

I've implemented a POC for this approach: #7392

An alternative to using the Java metrics SDK could be to implement a custom OTel collector receiver for the proprietary protocol, similar to the prometheusremotewritereceiver. While that works, it would be nice if something like this was possible in the Java ecosystem, too.

Let me know what y'all think. Does the use case sound sensible? Do you think that supporting that would be in scope for opentelemetry-java?

[jkwatson]

My suspicion is that this is not something we'd want to include in the default SDK, as it's not something that is included in the specification. It easy enough to implement yourself, and provide your own, as you've shown in the PoC PR.

[felixbarny]

I realized I have been thinking of this the wrong way. For this "protocol adapter" use case, you wouldn't track metrics in a MeterProvider and use a MetricReader to report them. The thing that made it obvious to me is that it would always use the resource of the current process instead of a resource that's based on the incoming metrics data. So that use case is better served by manually creating metrics via ImmutableMetricData#create* and exporting directly via MetricExporter#export, without even creating a MeterProvider.