Skip to content

Commit 184216d

Browse files
trasktrask
committed
Split runtime telemetry JFR config from experimental metrics
1 parent dfbf501 commit 184216d

15 files changed

Lines changed: 212 additions & 88 deletions

File tree

‎instrumentation/runtime-telemetry/README.md‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,21 @@
11
# Settings for the Runtime Telemetry instrumentation
22

3-
| System property | Type | Default | Description |
4-
| ------------------------------------------------------------------------------------- | ------- | ------- | --------------------------------------------------------------------------------------- |
5-
| `otel.instrumentation.runtime-telemetry.emit-experimental-metrics` | Boolean | `false` | Enable the capture of experimental metrics. |
6-
| `otel.instrumentation.runtime-telemetry.experimental.prefer-jfr` | Boolean | `false` | Prefer JFR over JMX for metrics where both collection methods are available (Java 17+). |
7-
| `otel.instrumentation.runtime-telemetry.experimental.package-emitter.enabled` | Boolean | `false` | Enable creating events for JAR libraries used by the application. |
8-
| `otel.instrumentation.runtime-telemetry.experimental.package-emitter.jars-per-second` | Integer | 10 | The number of JAR files processed per second. |
3+
| System property | Type | Default | Description |
4+
| ------------------------------------------------------------------------------------- | ------- | ------- | ------------------------------------------------------------------------------------------ |
5+
| `otel.instrumentation.runtime-telemetry.emit-experimental-metrics` | Boolean | `false` | Enable the capture of experimental JMX-based metrics. |
6+
| `otel.instrumentation.runtime-telemetry.jfr.enabled` | Boolean | `false` | Enable the default JFR-based runtime metrics on Java 17+. |
7+
| `otel.instrumentation.runtime-telemetry.jfr.enable-all` | Boolean | `false` | Enable all JFR-based runtime metrics on Java 17+, including metrics that overlap with JMX. |
8+
| `otel.instrumentation.runtime-telemetry.experimental.package-emitter.enabled` | Boolean | `false` | Enable creating events for JAR libraries used by the application. |
9+
| `otel.instrumentation.runtime-telemetry.experimental.package-emitter.jars-per-second` | Integer | 10 | The number of JAR files processed per second. |
910

1011
## Deprecated Properties (to be removed in 3.0)
1112

1213
| System property | Type | Default | Description |
1314
| ------------------------------------------------------------------------ | ------- | ------- | --------------------------------------------------------------------------------- |
1415
| `otel.instrumentation.runtime-telemetry.capture-gc-cause` | Boolean | `false` | Enable the capture of the jvm.gc.cause attribute. Will always be captured in 3.0. |
1516
| `otel.instrumentation.runtime-telemetry.emit-experimental-telemetry` | Boolean | `false` | Use `emit-experimental-metrics` instead. |
17+
| `otel.instrumentation.runtime-telemetry.experimental.prefer-jfr` | Boolean | `false` | Use `jfr.enable-all` instead. |
1618
| `otel.instrumentation.runtime-telemetry.package-emitter.enabled` | Boolean | `false` | Use `experimental.package-emitter.enabled` instead. |
1719
| `otel.instrumentation.runtime-telemetry.package-emitter.jars-per-second` | Integer | 10 | Use `experimental.package-emitter.jars-per-second` instead. |
18-
| `otel.instrumentation.runtime-telemetry-java17.enabled` | Boolean | `false` | Deprecated. Use `emit-experimental-metrics` for experimental JFR features. |
19-
| `otel.instrumentation.runtime-telemetry-java17.enable-all` | Boolean | `false` | Deprecated. Use `emit-experimental-metrics` and `experimental.prefer-jfr`. |
20+
| `otel.instrumentation.runtime-telemetry-java17.enabled` | Boolean | `false` | Deprecated. Use `jfr.enabled` instead. |
21+
| `otel.instrumentation.runtime-telemetry-java17.enable-all` | Boolean | `false` | Deprecated. Use `jfr.enable-all` instead. |

‎instrumentation/runtime-telemetry/library/README.md‎

Lines changed: 49 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,29 @@ meter_provider:
8787

8888
## Metrics
8989

90+
### Configuration
91+
92+
The set of emitted metrics depends on three independent knobs. In autoconfigured
93+
environments, these map to system properties; programmatically, use the corresponding
94+
builder methods.
95+
96+
- `otel.instrumentation.runtime-telemetry.emit-experimental-metrics=true`
97+
(`emitExperimentalMetrics()`): enables additional JMX-based metrics that are not yet
98+
stable in the semantic conventions.
99+
- `otel.instrumentation.runtime-telemetry.jfr.enabled=true` (Java 17+): enables the
100+
default set of JFR-based metrics. These metrics have no JMX equivalent, so they
101+
complement the JMX metrics without duplication.
102+
- `otel.instrumentation.runtime-telemetry.jfr.enable-all=true` (Java 17+): additionally
103+
sources metrics from JFR instead of JMX wherever a JFR equivalent exists (see
104+
[JFR-based (Overlap with JMX)](#jfr-based-overlap-with-jmx) below). The corresponding
105+
JMX metrics are suppressed.
106+
107+
> **Warning**: JFR events might not be available for all JVMs or with a GraalVM native
108+
> image, therefore limiting the produced metrics. The original implementation was done
109+
> for Hotspot. OpenJ9 currently (Nov. 2025) only has the VM-level JFR implementation. So
110+
> events emitted at the Java level (ie. in jdk.jfr) will not be present. Meaning,
111+
> jdk.SocketRead, jdk.SocketWrite won't work.
112+
90113
### Stable Metrics (enabled by default)
91114

92115
These metrics are collected via JMX on all Java versions:
@@ -107,9 +130,7 @@ These metrics are collected via JMX on all Java versions:
107130

108131
### Experimental Metrics
109132

110-
These metrics are enabled with `emitExperimentalMetrics()`:
111-
112-
**JMX-based (all Java versions):**
133+
#### JMX-based (all Java versions)
113134

114135
| Metric | Description |
115136
| -------- | ----------- |
@@ -119,12 +140,7 @@ These metrics are enabled with `emitExperimentalMetrics()`:
119140
| `jvm.memory.init` | Measure of initial memory requested |
120141
| `jvm.system.cpu.utilization` | System-wide CPU utilization |
121142

122-
**JFR-based (Java 17+ only):**
123-
124-
> **Warning**: JFR events might not be available for all JVMs or with a GraalVM native image,
125-
> therefore limiting the produced metrics. The original implementation was done for Hotspot. OpenJ9
126-
> currently (Nov. 2025) only has the VM-level JFR implementation. So events emitted at the Java
127-
> level (ie. in jdk.jfr) will not be present. Meaning, jdk.SocketRead, jdk.SocketWrite won't work.
143+
#### JFR-based (Java 17+ only)
128144

129145
| Metric | Description |
130146
| -------- | ----------- |
@@ -134,6 +150,30 @@ These metrics are enabled with `emitExperimentalMetrics()`:
134150
| `jvm.network.io` | Network I/O bytes |
135151
| `jvm.network.time` | Network I/O time |
136152

153+
#### JFR-based (Overlap with JMX)
154+
155+
When `jfr.enable-all=true`, the following metrics are sourced from JFR instead of JMX
156+
(the JMX-based registration is suppressed to avoid duplicates):
157+
158+
| Metric |
159+
| -------- |
160+
| `jvm.buffer.count` |
161+
| `jvm.buffer.memory.limit` |
162+
| `jvm.buffer.memory.used` |
163+
| `jvm.class.count` |
164+
| `jvm.class.loaded` |
165+
| `jvm.class.unloaded` |
166+
| `jvm.cpu.count` |
167+
| `jvm.cpu.recent_utilization` |
168+
| `jvm.gc.duration` |
169+
| `jvm.memory.committed` |
170+
| `jvm.memory.init` |
171+
| `jvm.memory.limit` |
172+
| `jvm.memory.used` |
173+
| `jvm.memory.used_after_last_gc` |
174+
| `jvm.system.cpu.utilization` |
175+
| `jvm.thread.count` |
176+
137177
## Garbage Collector Dependent Metrics
138178

139179
The attributes reported on the memory metrics (`jvm.memory.*`) and gc metrics (`jvm.gc.*`) are dependent on the garbage collector used by the application, since each garbage collector organizes memory pools differently and has different strategies for reclaiming memory during garbage collection.

‎instrumentation/runtime-telemetry/library/src/main/java/io/opentelemetry/instrumentation/runtimetelemetry/RuntimeTelemetryBuilder.java‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,6 @@ public final class RuntimeTelemetryBuilder {
3939
Experimental.internalSetEmitExperimentalMetrics(
4040
(builder, emit) -> {
4141
builder.emitExperimentalMetrics = emit;
42-
if (emit) {
43-
builder.jfrConfig.enableExperimentalFeatures();
44-
}
4542
});
4643
Experimental.internalSetPreferJfrMetrics(
4744
(builder, prefer) -> builder.preferJfrMetrics = prefer);

‎instrumentation/runtime-telemetry/library/src/main/java/io/opentelemetry/instrumentation/runtimetelemetry/internal/Cpu.java‎

Lines changed: 1 addition & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@
1010
import io.opentelemetry.api.metrics.Meter;
1111
import java.util.ArrayList;
1212
import java.util.List;
13-
import java.util.function.IntSupplier;
1413
import java.util.function.Supplier;
1514
import javax.annotation.Nullable;
1615

@@ -33,16 +32,12 @@ public class Cpu {
3332
/** Register observers for java runtime CPU metrics. */
3433
public static List<AutoCloseable> registerObservers(Meter meter) {
3534
return INSTANCE.registerObservers(
36-
meter,
37-
Runtime.getRuntime()::availableProcessors,
38-
CpuMethods.processCpuTime(),
39-
CpuMethods.processCpuUtilization());
35+
meter, CpuMethods.processCpuTime(), CpuMethods.processCpuUtilization());
4036
}
4137

4238
// Visible for testing
4339
List<AutoCloseable> registerObservers(
4440
Meter meter,
45-
IntSupplier availableProcessors,
4641
@Nullable Supplier<Long> processCpuTime,
4742
@Nullable Supplier<Double> processCpuUtilization) {
4843
List<AutoCloseable> observables = new ArrayList<>();
@@ -62,14 +57,6 @@ List<AutoCloseable> registerObservers(
6257
}
6358
}));
6459
}
65-
observables.add(
66-
meter
67-
.upDownCounterBuilder("jvm.cpu.count")
68-
.setDescription("Number of processors available to the Java virtual machine.")
69-
.setUnit("{cpu}")
70-
.buildWithCallback(
71-
observableMeasurement ->
72-
observableMeasurement.record(availableProcessors.getAsInt())));
7360
if (processCpuUtilization != null) {
7461
observables.add(
7562
meter

‎instrumentation/runtime-telemetry/library/src/main/java/io/opentelemetry/instrumentation/runtimetelemetry/internal/CpuCount.java‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
/*
2+
* Copyright The OpenTelemetry Authors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package io.opentelemetry.instrumentation.runtimetelemetry.internal;
7+
8+
import static java.util.Collections.singletonList;
9+
10+
import io.opentelemetry.api.metrics.Meter;
11+
import java.util.List;
12+
import java.util.function.IntSupplier;
13+
14+
/**
15+
* Registers measurements that generate the JVM CPU count metric.
16+
*
17+
* <p>This class is internal and is hence not for public use. Its APIs are unstable and can change
18+
* at any time.
19+
*/
20+
public class CpuCount {
21+
22+
// Visible for testing
23+
static final CpuCount INSTANCE = new CpuCount();
24+
25+
/** Register observers for the JVM CPU count metric. */
26+
public static List<AutoCloseable> registerObservers(Meter meter) {
27+
return INSTANCE.registerObservers(meter, Runtime.getRuntime()::availableProcessors);
28+
}
29+
30+
// Visible for testing
31+
List<AutoCloseable> registerObservers(Meter meter, IntSupplier availableProcessors) {
32+
AutoCloseable observable =
33+
meter
34+
.upDownCounterBuilder("jvm.cpu.count")
35+
.setDescription("Number of processors available to the Java virtual machine.")
36+
.setUnit("{cpu}")
37+
.buildWithCallback(
38+
observableMeasurement ->
39+
observableMeasurement.record(availableProcessors.getAsInt()));
40+
return singletonList(observable);
41+
}
42+
43+
protected CpuCount() {}
44+
}

‎instrumentation/runtime-telemetry/library/src/main/java/io/opentelemetry/instrumentation/runtimetelemetry/internal/Experimental.java‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,13 +23,13 @@ public final class Experimental {
2323
private static volatile BiConsumer<RuntimeTelemetryBuilder, Boolean> setPreferJfrMetrics;
2424

2525
/**
26-
* Sets whether experimental metrics should be emitted. Experimental metrics are those not marked
27-
* as stable in the <a
26+
* Sets whether experimental JMX-based metrics should be emitted. Experimental metrics are those
27+
* not marked as stable in the <a
2828
* href="https://github.com/open-telemetry/semantic-conventions/blob/main/docs/runtime/jvm-metrics.md">semantic
2929
* conventions</a>.
3030
*
3131
* @param builder the runtime telemetry builder
32-
* @param emitExperimentalMetrics {@code true} to emit experimental metrics
32+
* @param emitExperimentalMetrics {@code true} to emit experimental JMX metrics
3333
*/
3434
public static void setEmitExperimentalMetrics(
3535
RuntimeTelemetryBuilder builder, boolean emitExperimentalMetrics) {

‎instrumentation/runtime-telemetry/library/src/main/java/io/opentelemetry/instrumentation/runtimetelemetry/internal/Internal.java‎

Lines changed: 30 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -284,8 +284,8 @@ private static void configureJava17EnableAll(
284284
RuntimeTelemetryBuilder builder, DeclarativeConfigProperties config) {
285285
logger.warning(
286286
"otel.instrumentation.runtime-telemetry-java17.enable-all is deprecated and will be"
287-
+ " removed in 3.0. Use otel.instrumentation.runtime-telemetry.emit-experimental-metrics"
288-
+ " and otel.instrumentation.runtime-telemetry.experimental.prefer-jfr instead.");
287+
+ " removed in 3.0. Use otel.instrumentation.runtime-telemetry.jfr.enable-all"
288+
+ " instead.");
289289
// For backward compatibility: route JMX metrics to java8 scope, JFR metrics to java17 scope
290290
Internal.setJmxInstrumentationName(builder, "io.opentelemetry.runtime-telemetry-java8");
291291
Internal.setJfrInstrumentationName(builder, "io.opentelemetry.runtime-telemetry-java17");
@@ -311,27 +311,43 @@ private static void configureJava17EnableAll(
311311
private static void configureJava17Enabled(RuntimeTelemetryBuilder builder) {
312312
logger.warning(
313313
"otel.instrumentation.runtime-telemetry-java17.enabled is deprecated and will be"
314-
+ " removed in 3.0. Use otel.instrumentation.runtime-telemetry.emit-experimental-metrics"
315-
+ " for experimental JFR features.");
316-
// Enable default JFR features: context switches, CPU count, locks, allocations, network I/O
317-
Internal.setEnableJfrFeature(builder, "CONTEXT_SWITCH_METRICS");
314+
+ " removed in 3.0. Use otel.instrumentation.runtime-telemetry.jfr.enabled instead.");
315+
enableDefaultJfrFeatures(builder);
316+
// Preserve legacy behavior of the deprecated runtime-telemetry-java17 module, which enabled
317+
// CPU_COUNT_METRICS by default (emitted as jvm.cpu.limit via useLegacyJfrCpuCountMetric).
318318
Internal.setEnableJfrFeature(builder, "CPU_COUNT_METRICS");
319-
Internal.setEnableJfrFeature(builder, "LOCK_METRICS");
320-
Internal.setEnableJfrFeature(builder, "MEMORY_ALLOCATION_METRICS");
321-
Internal.setEnableJfrFeature(builder, "NETWORK_IO_METRICS");
322319
Internal.setUseLegacyJfrCpuCountMetric(builder, true);
323320
// For backward compatibility: OLD java17 module used java8's JMX factory, so JMX -> java8 scope
324321
Internal.setJmxInstrumentationName(builder, "io.opentelemetry.runtime-telemetry-java8");
325322
Internal.setJfrInstrumentationName(builder, "io.opentelemetry.runtime-telemetry-java17");
326323
}
327324

325+
private static void enableDefaultJfrFeatures(RuntimeTelemetryBuilder builder) {
326+
Internal.setEnableJfrFeature(builder, "CONTEXT_SWITCH_METRICS");
327+
Internal.setEnableJfrFeature(builder, "LOCK_METRICS");
328+
Internal.setEnableJfrFeature(builder, "MEMORY_ALLOCATION_METRICS");
329+
Internal.setEnableJfrFeature(builder, "NETWORK_IO_METRICS");
330+
}
331+
328332
private static void configureUnified(
329333
RuntimeTelemetryBuilder builder, DeclarativeConfigProperties config) {
330334
// Check if user is using new unified config options
331335
boolean emitExperimentalMetrics =
332336
config.getBoolean("emit_experimental_metrics/development", false);
333-
boolean preferJfr = config.getBoolean("prefer_jfr/development", false);
334-
boolean newConfig = emitExperimentalMetrics || preferJfr;
337+
Boolean jfrEnabled = config.get("jfr").getBoolean("enabled");
338+
Boolean jfrEnableAll = config.get("jfr").getBoolean("enable_all");
339+
Boolean deprecatedPreferJfr = config.getBoolean("prefer_jfr/development");
340+
if (deprecatedPreferJfr != null) {
341+
logger.warning(
342+
"otel.instrumentation.runtime-telemetry.experimental.prefer-jfr is deprecated and"
343+
+ " will be removed in 3.0. Use"
344+
+ " otel.instrumentation.runtime-telemetry.jfr.enable-all instead.");
345+
}
346+
347+
boolean enableAllJfr =
348+
jfrEnableAll != null ? jfrEnableAll : Boolean.TRUE.equals(deprecatedPreferJfr);
349+
boolean enableDefaultJfr = Boolean.TRUE.equals(jfrEnabled);
350+
boolean newConfig = emitExperimentalMetrics || enableDefaultJfr || enableAllJfr;
335351

336352
if (newConfig) {
337353
// New unified config: Use new instrumentation name for both JMX and JFR
@@ -357,9 +373,10 @@ private static void configureUnified(
357373
Experimental.setEmitExperimentalMetrics(builder, true);
358374
}
359375

360-
// Apply prefer_jfr
361-
if (preferJfr) {
376+
if (enableAllJfr) {
362377
Experimental.setPreferJfrMetrics(builder, true);
378+
} else if (enableDefaultJfr) {
379+
enableDefaultJfrFeatures(builder);
363380
}
364381

365382
// Apply capture_gc_cause

‎instrumentation/runtime-telemetry/library/src/main/java/io/opentelemetry/instrumentation/runtimetelemetry/internal/JmxRuntimeMetricsFactory.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ public static List<AutoCloseable> buildObservables(
2525
if (!preferJfrMetrics) {
2626
observables.addAll(Classes.registerObservers(meter));
2727
observables.addAll(Cpu.registerObservers(meter));
28+
observables.addAll(CpuCount.registerObservers(meter));
2829
observables.addAll(GarbageCollector.registerObservers(meter, captureGcCause));
2930
observables.addAll(MemoryPools.registerObservers(meter));
3031
observables.addAll(Threads.registerObservers(meter));

‎instrumentation/runtime-telemetry/library/src/main/java17/io/opentelemetry/instrumentation/runtimetelemetry/internal/JfrFeature.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
*
1111
* <p>Features that overlap with stable JMX-based instrumentation are disabled by default to avoid
1212
* duplicate metrics. Experimental features (those not marked stable in the semantic conventions)
13-
* are also disabled by default and require {@code emit_experimental_metrics=true} to enable.
13+
* are also disabled by default and require explicit JFR opt-in to enable.
1414
*
1515
* <p>This class is internal and is hence not for public use. Its APIs are unstable and can change
1616
* at any time.

‎instrumentation/runtime-telemetry/library/src/test/java/io/opentelemetry/instrumentation/runtimetelemetry/internal/CpuCountTest.java‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
/*
2+
* Copyright The OpenTelemetry Authors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package io.opentelemetry.instrumentation.runtimetelemetry.internal;
7+
8+
import static io.opentelemetry.sdk.testing.assertj.OpenTelemetryAssertions.assertThat;
9+
10+
import io.opentelemetry.instrumentation.testing.junit.InstrumentationExtension;
11+
import io.opentelemetry.instrumentation.testing.junit.LibraryInstrumentationExtension;
12+
import java.util.function.IntSupplier;
13+
import org.junit.jupiter.api.Test;
14+
import org.junit.jupiter.api.extension.RegisterExtension;
15+
16+
class CpuCountTest {
17+
18+
@RegisterExtension
19+
static final InstrumentationExtension testing = LibraryInstrumentationExtension.create();
20+
21+
@Test
22+
void registerObservers() {
23+
IntSupplier availableProcessors = () -> 8;
24+
25+
CpuCount.INSTANCE.registerObservers(
26+
testing.getOpenTelemetry().getMeter("test"), availableProcessors);
27+
28+
testing.waitAndAssertMetrics(
29+
"test",
30+
"jvm.cpu.count",
31+
metrics ->
32+
metrics.anySatisfy(
33+
metricData ->
34+
assertThat(metricData)
35+
.hasDescription(
36+
"Number of processors available to the Java virtual machine.")
37+
.hasUnit("{cpu}")
38+
.hasLongSumSatisfying(
39+
count ->
40+
count
41+
.isNotMonotonic()
42+
.hasPointsSatisfying(point -> point.hasValue(8)))));
43+
}
44+
}

0 commit comments

Comments
 (0)