diff --git a/content/en/opentelemetry/integrations/runtime_metrics/_index.md b/content/en/opentelemetry/integrations/runtime_metrics/_index.md
index 287427d4400..47ae6d2d3d4 100644
--- a/content/en/opentelemetry/integrations/runtime_metrics/_index.md
+++ b/content/en/opentelemetry/integrations/runtime_metrics/_index.md
@@ -24,6 +24,8 @@ Datadog supports OpenTelemetry runtime metrics for the following languages:
- Java
- .NET
- Go
+- NodeJS
+- Python
For details about host and container metrics mapping, see [OpenTelemetry Metrics Mapping][1].
@@ -51,10 +53,23 @@ If you use [OpenTelemetry manual instrumentation][4], follow the guides for your
- [Java 8][5]
- [Java 17][6]
+#### Collector configuration
+
+The `jvm.gc.collections.count` and `jvm.gc.collections.elapsed` metrics require the [Delta-to-Rate Processor][8] in the OpenTelemetry Collector. Additionally, ensure that `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta` is set as well.
+
+```yaml
+processors:
+ deltatorate:
+ metrics:
+ - jvm.gc.collections.count
+ - jvm.gc.collections.elapsed
+```
+
[3]: https://opentelemetry.io/docs/instrumentation/java/automatic/
[4]: https://opentelemetry.io/docs/instrumentation/java/manual/
[5]: https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/runtime-telemetry/runtime-telemetry-java8/library
[6]: https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/runtime-telemetry/runtime-telemetry-java17/library
+[8]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/deltatorateprocessor
{{% /tab %}}
@@ -62,8 +77,27 @@ If you use [OpenTelemetry manual instrumentation][4], follow the guides for your
OpenTelemetry Go applications are [instrumented manually][3]. To enable runtime metrics, see the documentation for the [runtime package][4].
+#### Collector configuration
+
+The `process.runtime.go.gc.pause_total_ns` and `process.runtime.go.gc.count` metrics require the [Delta-to-Cumulative Processor][9] and the [Transform Processor][8] in the OpenTelemetry Collector. The Delta-to-Cumulative Processor converts delta sums to cumulative sums, and the Transform Processor converts cumulative sums to gauges. Additionally, ensure that `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta` is set to handle other metrics as well.
+
+```yaml
+processors:
+ deltatocumulative:
+ metrics:
+ - process.runtime.go.gc.pause_total_ns
+ - process.runtime.go.gc.count
+ transform:
+ metric_statements:
+ - context: metric
+ statements:
+ - convert_sum_to_gauge() where name == "process.runtime.go.gc.pause_total_ns" or name == "process.runtime.go.gc.count"
+```
+
[3]: https://opentelemetry.io/docs/instrumentation/go/manual/
[4]: https://pkg.go.dev/go.opentelemetry.io/contrib/instrumentation/runtime
+[8]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor
+[9]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/deltatocumulativeprocessor
{{% /tab %}}
@@ -87,10 +121,62 @@ The default metric export interval for the .NET OTel SDK differs from the defaul
OTEL_METRIC_EXPORT_INTERVAL=10000
```
+#### Collector configuration
+
+The `dotnet.process.cpu.time` metric requires the [Delta-to-Rate Processor][8] in the OpenTelemetry Collector. Additionally, ensure that `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta` is set as well.
+
+```yaml
+processors:
+ deltatorate:
+ metrics:
+ - dotnet.process.cpu.time
+```
+
[3]: https://opentelemetry.io/docs/instrumentation/net/automatic/
[4]: https://opentelemetry.io/docs/instrumentation/net/manual/
[5]: https://github.com/open-telemetry/opentelemetry-dotnet-contrib/tree/main/src/OpenTelemetry.Instrumentation.Runtime
[7]: https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#periodic-exporting-metricreader
+[8]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/deltatorateprocessor
+
+{{% /tab %}}
+
+{{% tab "NodeJS" %}}
+
+#### Automatic instrumentation
+
+If you use [OpenTelemetry automatic instrumentation][3] for Node.js applications, runtime metrics are enabled by default through the [`@opentelemetry/instrumentation-runtime-node`][5] package.
+
+**Note**: Runtime metrics are only exported if a `MeterProvider` and metric exporter are configured. Set the `OTEL_METRICS_EXPORTER` environment variable or programmatically configure a `metricReader` in your SDK initialization.
+
+#### Manual instrumentation
+
+If you use [OpenTelemetry manual instrumentation][4], see the documentation for the [`@opentelemetry/instrumentation-runtime-node`][5] library.
+
+[3]: https://opentelemetry.io/docs/zero-code/js/
+[4]: https://opentelemetry.io/docs/languages/js/instrumentation/
+[5]: https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/packages/instrumentation-runtime-node
+
+{{% /tab %}}
+
+{{% tab "Python" %}}
+
+Runtime metrics are not enabled by default for Python applications. Install the [`opentelemetry-instrumentation-system-metrics`][5] package:
+
+```shell
+pip install opentelemetry-instrumentation-system-metrics
+```
+
+If you use [automatic instrumentation][3], `opentelemetry-instrument` discovers and enables the package after installation. If you use [manual instrumentation][4], enable it in your application:
+
+```python
+from opentelemetry.instrumentation.system_metrics import SystemMetricsInstrumentor
+
+SystemMetricsInstrumentor().instrument()
+```
+
+[3]: https://opentelemetry.io/docs/languages/python/automatic/
+[4]: https://opentelemetry.io/docs/languages/python/instrumentation/
+[5]: https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-system-metrics
{{% /tab %}}
@@ -107,132 +193,77 @@ After setup is complete, you can view runtime metrics in:
## Data collected
-When using OpenTelemetry runtime metrics with Datadog, you receive both:
-- Original OpenTelemetry runtime metrics
-- Mapped Datadog runtime metrics for equivalent metrics
+The following tables list the OpenTelemetry runtime metrics used in Datadog's out-of-the-box in-app experiences.
-The OpenTelemetry runtime metrics have the following prefixes based on their source:
+
OpenTelemetry runtime metrics are mapped to Datadog by metric name. Do not rename host metrics for OpenTelemetry runtime metrics as this breaks the mapping.
-| Source | Prefix |
-| --- | --- |
-| [OTel Collector Datadog Exporter][100] | `otel.process.runtime.*` |
-| [Datadog Agent OTLP Ingest][101] | `process.runtime.*` |
+{{< tabs >}}
+{{< tab "Java" >}}
-The following tables list the Datadog runtime metrics that are supported through OpenTelemetry mapping. "N/A" indicates that there is no OpenTelemetry equivalent metric available.
+JVM Instrumentation
+These metrics are collected when using the latest version of the OpenTelemetry Java SDK.
+{{< mapping-table resource="jvm-instrumentation.csv">}}
- OpenTelemetry runtime metrics are mapped to Datadog by metric name. Do not rename host metrics for OpenTelemetry runtime metrics as this breaks the mapping.
+JVM Contrib
+These metrics are collected when using the OpenTelemetry JMX Metrics Gatherer.
+{{< mapping-table resource="jvm-contrib.csv">}}
-[100]: /opentelemetry/setup/collector_exporter/
-[101]: /opentelemetry/setup/otlp_ingest_in_the_agent
+JVM Deprecated
+These metrics are collected when using OpenTelemetry Java SDK versions 1.32.0 and earlier.
+{{< mapping-table resource="jvm-deprecated.csv">}}
-{{< tabs >}}
-{{% tab "Java" %}}
+{{< /tab >}}
-| Datadog metric | Description | OpenTelemetry metric |
-| --- | --- | --- |
-| `jvm.heap_memory` | The total Java heap memory used. | `process.runtime.jvm.memory.usage`
`jvm.memory.used` |
-| `jvm.heap_memory_committed` | The total Java heap memory committed to be used. | `process.runtime.jvm.memory.committed`
`jvm.memory.committed` |
-| `jvm.heap_memory_init` | The initial Java heap memory allocated. | `process.runtime.jvm.memory.init`
`jvm.memory.init` |
-| `jvm.heap_memory_max` | The maximum Java heap memory available. | `process.runtime.jvm.memory.limit`
`jvm.memory.limit` |
-| `jvm.non_heap_memory` | The total Java non-heap memory used. Non-heap memory is: `Metaspace + CompressedClassSpace + CodeCache`. | `process.runtime.jvm.memory.usage`
`jvm.memory.used` |
-| `jvm.non_heap_memory_committed` | The total Java non-heap memory committed to be used. | `process.runtime.jvm.memory.committed`
`jvm.memory.committed` |
-| `jvm.non_heap_memory_init` | The initial Java non-heap memory allocated. | `process.runtime.jvm.memory.init`
`jvm.memory.init` |
-| `jvm.non_heap_memory_max` | The maximum Java non-heap memory available. | `process.runtime.jvm.memory.limit`
`jvm.memory.limit` |
-| `jvm.gc.old_gen_size` | The current Java heap memory usage of the Old Generation memory pool. | `process.runtime.jvm.memory.usage`
`jvm.memory.used` |
-| `jvm.gc.eden_size` | The current Java heap memory usage of the Eden memory pool. | `process.runtime.jvm.memory.usage`
`jvm.memory.used` |
-| `jvm.gc.survivor_size` | The current Java heap memory usage of the Survivor memory pool. | `process.runtime.jvm.memory.usage`
`jvm.memory.used` |
-| `jvm.gc.metaspace_size` | The current Java non-heap memory usage of the Metaspace memory pool. | `process.runtime.jvm.memory.usage`
`jvm.memory.used` |
-| `jvm.thread_count` | The number of live threads. | `process.runtime.jvm.threads.count`
`jvm.thread.count` |
-| `jvm.loaded_classes` | Number of classes currently loaded. | `process.runtime.jvm.classes.current_loaded`
`jvm.class.count` |
-| `jvm.cpu_load.system` | Recent CPU utilization for the whole system. | `process.runtime.jvm.system.cpu.utilization`
`jvm.system.cpu.utilization` |
-| `jvm.cpu_load.process` | Recent CPU utilization for the process. | `process.runtime.jvm.cpu.utilization`
`jvm.cpu.recent_utilization` |
-| `jvm.buffer_pool.direct.used` | Measure of memory used by direct buffers. | `process.runtime.jvm.buffer.usage`
`jvm.buffer.memory.usage` |
-| `jvm.buffer_pool.direct.count` | Number of direct buffers in the pool. | `process.runtime.jvm.buffer.count`
`jvm.buffer.count` |
-| `jvm.buffer_pool.direct.limit` | Measure of total memory capacity of direct buffers. | `process.runtime.jvm.buffer.limit`
`jvm.buffer.memory.limit` |
-| `jvm.buffer_pool.mapped.used` | Measure of memory used by mapped buffers. | `process.runtime.jvm.buffer.usage`
`jvm.buffer.memory.usage` |
-| `jvm.buffer_pool.mapped.count` | Number of mapped buffers in the pool. | `process.runtime.jvm.buffer.count`
`jvm.buffer.count` |
-| `jvm.buffer_pool.mapped.limit` | Measure of total memory capacity of mapped buffers. | `process.runtime.jvm.buffer.limit`
`jvm.buffer.memory.limit` |
-| `jvm.gc.parnew.time` | The approximate accumulated garbage collection time elapsed. | N/A |
-| `jvm.gc.cms.count` | The total number of garbage collections that have occurred. | N/A |
-| `jvm.gc.major_collection_count` | The rate of major garbage collections. Set `new_gc_metrics: true` to receive this metric. | N/A |
-| `jvm.gc.minor_collection_count` | The rate of minor garbage collections. Set `new_gc_metrics: true` to receive this metric. | N/A |
-| `jvm.gc.major_collection_time` | The fraction of time spent in major garbage collection. Set `new_gc_metrics: true` to receive this metric. | N/A |
-| `jvm.gc.minor_collection_time` | The fraction of time spent in minor garbage collection. Set `new_gc_metrics: true` to receive this metric. | N/A |
-| `jvm.os.open_file_descriptors` | The number of open file descriptors. | N/A |
+{{< tab "Go" >}}
-{{% /tab %}}
+{{< mapping-table resource="go-contrib-runtime.csv">}}
-{{% tab "Go" %}}
+{{< /tab >}}
-| Datadog metric | Description | OpenTelemetry metric |
-| --- | --- | --- |
-| `runtime.go.num_goroutine` | Number of goroutines spawned. | `process.runtime.go.goroutines` |
-| `runtime.go.num_cgo_call` | Number of CGO calls made. |`process.runtime.go.cgo.calls` |
-| `runtime.go.mem_stats.lookups` | Number of pointer lookups performed by the runtime. | `process.runtime.go.mem.lookups` |
-| `runtime.go.mem_stats.heap_alloc` | Bytes of allocated heap objects. | `process.runtime.go.mem.heap_alloc` |
-| `runtime.go.mem_stats.heap_sys` | Bytes of heap memory obtained from the operating system. | `process.runtime.go.mem.heap_sys` |
-| `runtime.go.mem_stats.heap_idle` | Bytes in idle (unused) spans. | `process.runtime.go.mem.heap_idle` |
-| `runtime.go.mem_stats.heap_inuse` | Bytes in in-use spans. | `process.runtime.go.mem.heap_inuse` |
-| `runtime.go.mem_stats.heap_released` | Bytes of physical memory returned to the operating system. | `process.runtime.go.mem.heap_released` |
-| `runtime.go.mem_stats.heap_objects` | Number of allocated heap objects. | `process.runtime.go.mem.heap_objects` |
-| `runtime.go.mem_stats.pause_total_ns` | Cumulative nanoseconds in garbage collection (GC). | `process.runtime.go.gc.pause_total_ns` |
-| `runtime.go.mem_stats.num_gc` | Number of completed GC cycles. | `process.runtime.go.gc.count` |
-| `runtime.go.num_cpu` | Number of CPUs detected by the runtime. | N/A |
-| `runtime.go.mem_stats.alloc` | Bytes of allocated heap objects. | N/A |
-| `runtime.go.mem_stats.total_alloc` | Cumulative bytes allocated for heap objects. | N/A |
-| `runtime.go.mem_stats.sys` | Total bytes of memory obtained from the operating system. | N/A |
-| `runtime.go.mem_stats.mallocs` | Cumulative count of heap objects allocated. | N/A |
-| `runtime.go.mem_stats.frees` | Cumulative count of heap objects freed. | N/A |
-| `runtime.go.mem_stats.stack_inuse` | Bytes in stack spans. | N/A |
-| `runtime.go.mem_stats.stack_sys` | Bytes of stack memory obtained from the operating system. | N/A |
-| `runtime.go.mem_stats.m_span_inuse` | Bytes of allocated mspan structures. | N/A |
-| `runtime.go.mem_stats.m_span_sys` | Bytes of memory obtained from the operating system for mspan structures. | N/A |
-| `runtime.go.mem_stats.m_cache_inuse` | Bytes of allocated mcache structures. | N/A |
-| `runtime.go.mem_stats.m_cache_sys` | Bytes of memory obtained from the operating system for mcache structures. | N/A |
-| `runtime.go.mem_stats.buck_hash_sys` | Bytes of memory in profiling bucket hash tables. | N/A |
-| `runtime.go.mem_stats.gc_sys` | Bytes of memory in garbage collection metadata. | N/A |
-| `runtime.go.mem_stats.other_sys` | Bytes of memory in miscellaneous off-heap. | N/A |
-| `runtime.go.mem_stats.next_gc` | Target heap size of the next GC cycle. | N/A |
-| `runtime.go.mem_stats.last_gc` | Last garbage collection finished, as nanoseconds since the UNIX epoch. | N/A |
-| `runtime.go.mem_stats.num_forced_gc` | Number of GC cycles that were forced by the application calling the GC function. | N/A |
-| `runtime.go.mem_stats.gc_cpu_fraction` | Fraction of this program's available CPU time used by the GC since the program started. | N/A |
-| `runtime.go.gc_stats.pause_quantiles.min` | Distribution of GC pause times: minimum values. | N/A |
-| `runtime.go.gc_stats.pause_quantiles.25p` | Distribution of GC pause times: 25th percentile. | N/A |
-| `runtime.go.gc_stats.pause_quantiles.50p` | Distribution of GC pause times: 50th percentile. | N/A |
-| `runtime.go.gc_stats.pause_quantiles.75p` | Distribution of GC pause times: 75th percentile. | N/A |
-| `runtime.go.gc_stats.pause_quantiles.max` | Distribution of GC pause times: maximum values. | N/A |
+{{< tab ".NET" >}}
-{{% /tab %}}
+.NET System.Runtime
+These metrics are emitted by the .NET runtime's built-in System.Runtime meter on .NET 9.0 and later. The OpenTelemetry SDK collects and exports them automatically.
+{{< mapping-table resource="dotnet.csv">}}
-{{% tab ".NET" %}}
+.NET Contrib Runtime
+These metrics are collected by the OpenTelemetry.Instrumentation.Runtime package. On .NET 9.0 and later, these overlap with the System.Runtime metrics above.
+{{< mapping-table resource="dotnet-contrib-runtime.csv">}}
+
+.NET Contrib Process
+These metrics are collected by the OpenTelemetry.Instrumentation.Process package.
+{{< mapping-table resource="dotnet-contrib-process.csv">}}
+
+{{< /tab >}}
+
+{{< tab "NodeJS" >}}
+
+Node Contrib Runtime
+These metrics are emitted from auto-instrumentation with the latest version of the OpenTelemetry Node.js SDK.
+{{< mapping-table resource="node-contrib-runtime.csv">}}
+
+Node Contrib Host
+These metrics are collected by the @opentelemetry/host-metrics package. This package is not included in OpenTelemetry automatic instrumentation and must be installed and configured manually.
+{{< mapping-table resource="node-contrib-host.csv">}}
+
+{{< /tab >}}
+
+{{% tab "Python" %}}
+
+The following table lists the conceptual equivalences between OpenTelemetry and Datadog Python runtime metrics. There are no direct mappings due to mismatching metric types between the two systems.
-| Datadog metric | Description | OpenTelemetry metric |
+| Datadog metric | Description | OpenTelemetry metric |
| --- | --- | --- |
-| `runtime.dotnet.threads.contention_count` | The number of times a thread stopped to wait on a lock. | `process.runtime.dotnet.`
`monitor.lock_contention.count` |
-| `runtime.dotnet.exceptions.count` | The number of first-chance exceptions. | `process.runtime.dotnet.`
`exceptions.count` |
-| `runtime.dotnet.gc.size.gen0` | The size of the gen 0 heap. | `process.runtime.dotnet.`
`gc.heap.size` |
-| `runtime.dotnet.gc.size.gen1` | The size of the gen 1 heap. | `process.runtime.dotnet.`
`gc.heap.size` |
-| `runtime.dotnet.gc.size.gen2` | The size of the gen 2 heap. | `process.runtime.dotnet.`
`gc.heap.size` |
-| `runtime.dotnet.gc.size.loh` | The size of the large object heap. | `process.runtime.dotnet.`
`gc.heap.size` |
-| `runtime.dotnet.gc.count.gen0` | The number of gen 0 garbage collections. | `process.runtime.dotnet.`
`gc.collections.count` |
-| `runtime.dotnet.gc.count.gen1` | The number of gen 1 garbage collections. | `process.runtime.dotnet.`
`gc.collections.count` |
-| `runtime.dotnet.gc.count.gen2` | The number of gen 2 garbage collections. | `process.runtime.dotnet.`
`gc.collections.count` |
-| `runtime.dotnet.cpu.system` | The number of milliseconds executing in the kernel. | N/A |
-| `runtime.dotnet.cpu.user` | The number of milliseconds executing outside the kernel. | N/A |
-| `runtime.dotnet.cpu.percent` | The percentage of total CPU used by the application. | N/A |
-| `runtime.dotnet.mem.committed` | Memory usage. | N/A |
-| `runtime.dotnet.threads.count` | The number of threads. | N/A |
-| `runtime.dotnet.threads.workers_count` | The number of workers in the threadpool. (.NET Core only) | N/A |
-| `runtime.dotnet.threads.contention_time` | The cumulated time spent by threads waiting on a lock. (.NET Core only) | N/A |
-| `runtime.dotnet.gc.memory_load` | The percentage of the total memory used by the process. The garbage collection (GC) changes its behavior when this value gets above 85. (.NET Core only) | N/A |
-| `runtime.dotnet.gc.pause_time` | The amount of time the GC paused the application threads. (.NET Core only) | N/A |
-| `runtime.dotnet.aspnetcore.`
`requests.total` | The total number of HTTP requests received by the server. (.NET Core only) | N/A |
-| `runtime.dotnet.aspnetcore.`
`requests.failed` | The number of failed HTTP requests received by the server. (.NET Core only) | N/A |
-| `runtime.dotnet.aspnetcore.`
`requests.current` | The total number of HTTP requests that have started but not yet stopped. (.NET Core only) | N/A |
-| `runtime.dotnet.aspnetcore.`
`requests.queue_length` | The current length of the server HTTP request queue. (.NET Core only) | N/A |
-| `runtime.dotnet.aspnetcore.`
`connections.total` | The total number of HTTP connections established to the server. (.NET Core only) | N/A |
-| `runtime.dotnet.aspnetcore.`
`connections.current` | The current number of active HTTP connections to the server. (.NET Core only) | N/A |
-| `runtime.dotnet.aspnetcore.`
`connections.queue_length` | The current length of the HTTP server connection queue. (.NET Core only) | N/A |
+| `runtime.python.cpu.time.sys` | Number of seconds executing in the kernel. | `process.cpu.time` (`type: system`) |
+| `runtime.python.cpu.time.user` | Number of seconds executing outside the kernel. | `process.cpu.time` (`type: user`) |
+| `runtime.python.cpu.percent` | CPU utilization percentage. OTel divides by 100 × number of cores. | `process.cpu.utilization` |
+| `runtime.python.cpu.ctx_switch.voluntary` | Number of voluntary context switches. | `process.context_switches` (`type: voluntary`) |
+| `runtime.python.cpu.ctx_switch.involuntary` | Number of involuntary context switches. | `process.context_switches` (`type: involuntary`) |
+| `runtime.python.gc.count.gen0` | Number of generation 0 objects. | `process.runtime.{python_implementation}.gc_count` (`count: 0`) |
+| `runtime.python.gc.count.gen1` | Number of generation 1 objects. | `process.runtime.{python_implementation}.gc_count` (`count: 1`) |
+| `runtime.python.gc.count.gen2` | Number of generation 2 objects. | `process.runtime.{python_implementation}.gc_count` (`count: 2`) |
+| `runtime.python.mem.rss` | Resident set memory. | `process.memory.usage` |
+| `runtime.python.thread_count` | Number of threads. | `process.thread.count` |
{{% /tab %}}