|
1 | 1 | # Metrics |
2 | 2 |
|
3 | | -This packages contains utilities for collecting metrics in `@message-queue-toolkit` |
| 3 | +This package contains utilities for collecting metrics in `@message-queue-toolkit`. |
| 4 | + |
| 5 | +## Installation |
| 6 | + |
| 7 | +```sh |
| 8 | +npm install @message-queue-toolkit/metrics |
| 9 | +``` |
| 10 | + |
| 11 | +## Overview |
| 12 | + |
| 13 | +All metrics implement the `MessageMetricsManager` interface from `@message-queue-toolkit/core`, which means they can be passed directly to any `AbstractQueueService` via the `messageMetricsManager` option. |
| 14 | + |
| 15 | +```ts |
| 16 | +import { PrometheusMessageProcessingTimeMetric } from '@message-queue-toolkit/metrics' |
| 17 | + |
| 18 | +const metric = new PrometheusMessageProcessingTimeMetric({ |
| 19 | + name: 'message_processing_duration_ms', |
| 20 | + helpDescription: 'Time spent processing a message', |
| 21 | + buckets: [10, 50, 100, 500, 1000], |
| 22 | +}) |
| 23 | + |
| 24 | +// Pass to your queue service |
| 25 | +const service = new MyQueueService({ messageMetricsManager: metric }) |
| 26 | +``` |
| 27 | + |
| 28 | +--- |
4 | 29 |
|
5 | 30 | ## Prometheus metrics |
6 | 31 |
|
7 | | -Metrics that use [Prometheus](https://prometheus.io/) toolkit and [prom-client](https://github.com/siimon/prom-client) library |
| 32 | +All Prometheus metrics use [prom-client](https://github.com/siimon/prom-client) under the hood. |
| 33 | + |
| 34 | +### Base parameters |
| 35 | + |
| 36 | +All metrics accept `PrometheusMetricParams`: |
| 37 | + |
| 38 | +| Field | Type | Required | Description | |
| 39 | +|---|---|---|---| |
| 40 | +| `name` | `string` | yes | Prometheus metric name | |
| 41 | +| `helpDescription` | `string` | yes | Prometheus metric description | |
| 42 | +| `buckets` | `number[]` | histograms only | Histogram bucket boundaries | |
| 43 | +| `messageVersion` | `string \| (metadata) => string \| undefined` | no | Static version string or function to extract version from message metadata | |
| 44 | +| `labelNames` | `Labels[]` | when `Labels` is specified | Names of the custom labels to register. Must not overlap with `DefaultLabels` (`queue`, `messageType`, `version`, `result`) — TypeScript enforces this at compile time | |
| 45 | + |
| 46 | +An optional second argument accepts a custom `prom-client` instance (useful for testing or multi-registry setups). |
| 47 | + |
| 48 | +--- |
| 49 | + |
| 50 | +### Histogram metrics (time-based) |
| 51 | + |
| 52 | +Use `Histogram` to measure message timing. Base labels registered on every observation: |
| 53 | + |
| 54 | +| Label | Value | |
| 55 | +|---|---| |
| 56 | +| `messageType` | Message type identifier | |
| 57 | +| `version` | Resolved message version | |
| 58 | +| `queue` | Queue or topic name | |
| 59 | +| `result` | Processing result status (`consumed`, `published`, `retryLater`, `error`) | |
| 60 | + |
| 61 | +#### Built-in implementations |
| 62 | + |
| 63 | +**`PrometheusMessageProcessingTimeMetric`** |
| 64 | +Measures elapsed time from when processing started to when it ended. |
| 65 | +``` |
| 66 | +value = messageProcessingEndTimestamp - messageProcessingStartTimestamp |
| 67 | +``` |
| 68 | + |
| 69 | +**`PrometheusMessageLifetimeMetric`** |
| 70 | +Measures elapsed time from when the message was originally sent to when it was fully processed. Includes any time the message spent waiting in the queue. |
| 71 | +``` |
| 72 | +value = messageProcessingEndTimestamp - messageTimestamp |
| 73 | +``` |
| 74 | +Skips observation if `messageTimestamp` is not available. |
| 75 | + |
| 76 | +**`PrometheusMessageQueueTimeMetric`** |
| 77 | +Measures elapsed time from when the message was originally sent to when processing started (i.e., queue wait time only). |
| 78 | +``` |
| 79 | +value = messageProcessingStartTimestamp - messageTimestamp |
| 80 | +``` |
| 81 | +Skips observation if `messageTimestamp` is not available. |
| 82 | + |
| 83 | +#### Custom histogram with extra labels |
| 84 | + |
| 85 | +Extend `PrometheusMessageTimeMetric` to add custom labels. Pass `labelNames` in the params and override `getLabelValuesForProcessedMessage`. Custom label names must not conflict with `DefaultLabels` — using a reserved name (e.g. `'result'`) will produce a TypeScript compile error: |
| 86 | + |
| 87 | +```ts |
| 88 | +import { PrometheusMessageTimeMetric } from '@message-queue-toolkit/metrics' |
| 89 | +import type { ProcessedMessageMetadata } from '@message-queue-toolkit/core' |
| 90 | +import type { LabelValues } from 'prom-client' |
| 91 | + |
| 92 | +class MyProcessingTimeMetric extends PrometheusMessageTimeMetric<MyMessage, 'env'> { |
| 93 | + protected calculateObservedValue(metadata: ProcessedMessageMetadata<MyMessage>): number | null { |
| 94 | + return metadata.messageProcessingEndTimestamp - metadata.messageProcessingStartTimestamp |
| 95 | + } |
| 96 | + |
| 97 | + protected getLabelValuesForProcessedMessage(): LabelValues<'env'> { |
| 98 | + return { env: process.env.NODE_ENV ?? 'unknown' } |
| 99 | + } |
| 100 | +} |
| 101 | + |
| 102 | +const metric = new MyProcessingTimeMetric({ |
| 103 | + name: 'message_processing_duration_ms', |
| 104 | + helpDescription: 'Processing time by environment', |
| 105 | + buckets: [10, 50, 100, 500], |
| 106 | + labelNames: ['env'], |
| 107 | +}) |
| 108 | +``` |
| 109 | + |
| 110 | +--- |
| 111 | + |
| 112 | +### Counter metrics (event-based) |
| 113 | + |
| 114 | +Use `Counter` to count message events. Base labels registered on every increment: |
| 115 | + |
| 116 | +| Label | Value | |
| 117 | +|---|---| |
| 118 | +| `messageType` | Message type identifier | |
| 119 | +| `version` | Resolved message version | |
| 120 | +| `queue` | Queue or topic name | |
| 121 | +| `result` | Processing result status (`consumed`, `published`, `retryLater`, `error`) | |
| 122 | + |
| 123 | +#### Built-in implementations |
| 124 | + |
| 125 | +**`PrometheusMessageResultCounter`** |
| 126 | +Counts all processed messages using only the built-in base labels. No extra configuration needed. |
| 127 | + |
| 128 | +```ts |
| 129 | +import { PrometheusMessageResultCounter } from '@message-queue-toolkit/metrics' |
| 130 | + |
| 131 | +const metric = new PrometheusMessageResultCounter({ |
| 132 | + name: 'messages_total', |
| 133 | + helpDescription: 'Number of messages processed', |
| 134 | +}) |
| 135 | +``` |
| 136 | + |
| 137 | +**`PrometheusMessageErrorCounter`** |
| 138 | +Counts only messages that result in an error. Adds an `errorReason` label. Skips all non-error messages. |
| 139 | + |
| 140 | +```ts |
| 141 | +import { PrometheusMessageErrorCounter } from '@message-queue-toolkit/metrics' |
| 142 | + |
| 143 | +const metric = new PrometheusMessageErrorCounter({ |
| 144 | + name: 'message_errors_total', |
| 145 | + helpDescription: 'Number of messages that failed processing', |
| 146 | + labelNames: ['errorReason'], |
| 147 | +}) |
| 148 | +``` |
| 149 | + |
| 150 | +#### Custom counter with extra labels |
| 151 | + |
| 152 | +Extend `PrometheusMessageCounter` and implement `calculateCount`. Override `getLabelValuesForProcessedMessage` when adding custom labels. Same as histograms, custom label names must not conflict with `DefaultLabels`: |
| 153 | + |
| 154 | +```ts |
| 155 | +import { PrometheusMessageCounter } from '@message-queue-toolkit/metrics' |
| 156 | +import type { ProcessedMessageMetadata } from '@message-queue-toolkit/core' |
| 157 | +import type { LabelValues } from 'prom-client' |
| 158 | + |
| 159 | +class MyRegionCounter extends PrometheusMessageCounter<MyMessage, 'region'> { |
| 160 | + protected calculateCount(): number | null { |
| 161 | + return 1 |
| 162 | + } |
| 163 | + |
| 164 | + protected override getLabelValuesForProcessedMessage( |
| 165 | + metadata: ProcessedMessageMetadata<MyMessage>, |
| 166 | + ): LabelValues<'region'> { |
| 167 | + return { region: metadata.message.region } |
| 168 | + } |
| 169 | +} |
| 170 | + |
| 171 | +const metric = new MyRegionCounter({ |
| 172 | + name: 'messages_by_region_total', |
| 173 | + helpDescription: 'Number of messages processed, by region', |
| 174 | + labelNames: ['region'], |
| 175 | +}) |
| 176 | +``` |
| 177 | + |
| 178 | +When no custom labels are needed, omit `labelNames` and skip overriding `getLabelValuesForProcessedMessage`: |
| 179 | + |
| 180 | +```ts |
| 181 | +class MyConsumedCounter extends PrometheusMessageCounter<MyMessage> { |
| 182 | + protected calculateCount(metadata: ProcessedMessageMetadata<MyMessage>): number | null { |
| 183 | + return metadata.processingResult.status === 'consumed' ? 1 : null |
| 184 | + } |
| 185 | +} |
| 186 | + |
| 187 | +const metric = new MyConsumedCounter({ |
| 188 | + name: 'messages_consumed_total', |
| 189 | + helpDescription: 'Number of successfully consumed messages', |
| 190 | +}) |
| 191 | +``` |
8 | 192 |
|
9 | | -### MessageProcessingPrometheusMetric |
10 | | -Abstract class implementing `MessageMetricsManager` interface, that can be injected into `AbstractQueueService` from `@message-queue-toolkit/core`. |
| 193 | +--- |
11 | 194 |
|
12 | | -It uses [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram) metric to collect message processing times with labels: |
13 | | -- `messageType` - message type |
14 | | -- `version` - message version |
15 | | -- `queue` - name of the queue or topic |
16 | | -- `result` - processing result |
| 195 | +### Using multiple metrics together |
17 | 196 |
|
18 | | -See [MessageProcessingPrometheusMetric.ts](lib/prometheus/MessageProcessingPrometheusMetric.ts) for available parameters. |
| 197 | +`MessageMultiMetricManager` aggregates multiple `MessageMetricsManager` instances and fans out each `registerProcessedMessage` call to all of them. |
19 | 198 |
|
20 | | -There are following non-abstract implementations available: |
21 | | -- `MessageProcessingTimeMetric` - registers elapsed time from start to the end of message processing |
22 | | -- `MessageLifetimeMetric` - registers elapsed time from the point where message was initially sent, to the point when it was processed. |
23 | | -Note: if message is waiting in the queue due to high load or barrier, the waiting time is included in the measurement |
| 199 | +```ts |
| 200 | +import { |
| 201 | + MessageMultiMetricManager, |
| 202 | + PrometheusMessageProcessingTimeMetric, |
| 203 | + PrometheusMessageResultCounter, |
| 204 | + PrometheusMessageErrorCounter, |
| 205 | +} from '@message-queue-toolkit/metrics' |
24 | 206 |
|
25 | | -### MessageProcessingMultiMetrics |
26 | | -Implementation of `MessageMetricsManager` that allows to use multiple `MessageProcessingPrometheusMetric` instances. |
| 207 | +const metricsManager = new MessageMultiMetricManager([ |
| 208 | + new PrometheusMessageProcessingTimeMetric({ |
| 209 | + name: 'message_processing_duration_ms', |
| 210 | + helpDescription: 'Message processing time', |
| 211 | + buckets: [10, 50, 100, 500, 1000], |
| 212 | + }), |
| 213 | + new PrometheusMessageResultCounter({ |
| 214 | + name: 'messages_total', |
| 215 | + helpDescription: 'Messages processed', |
| 216 | + }), |
| 217 | + new PrometheusMessageErrorCounter({ |
| 218 | + name: 'message_errors_total', |
| 219 | + helpDescription: 'Messages that failed processing', |
| 220 | + labelNames: ['errorReason'], |
| 221 | + }), |
| 222 | +]) |
27 | 223 |
|
| 224 | +const service = new MyQueueService({ messageMetricsManager: metricsManager }) |
| 225 | +``` |
0 commit comments