Skip to content

Commit 0044ca8

Browse files
authored
feat: Metrics extended labels (#414)
* metricParams is protected * Adding PrometheusMessageCounter * Error counter migrated and status fixes * Adding tests * Adding PrometheusMessageByStatusCounter * Update to have backward compatible change * Allowing adding labels on PrometheusMessageTimeMetric * Readme improvement * Lint fixes * Typo fix * AI comment for consistency * AI suggestion to simplify code * Symplifying type * readme updated * lint fix * Test fix
1 parent 7b8b4ef commit 0044ca8

12 files changed

Lines changed: 761 additions & 58 deletions

packages/metrics/README.md

Lines changed: 214 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,225 @@
11
# Metrics
22

3-
This packages contains utilities for collecting metrics in `@message-queue-toolkit`
3+
This package contains utilities for collecting metrics in `@message-queue-toolkit`.
4+
5+
## Installation
6+
7+
```sh
8+
npm install @message-queue-toolkit/metrics
9+
```
10+
11+
## Overview
12+
13+
All metrics implement the `MessageMetricsManager` interface from `@message-queue-toolkit/core`, which means they can be passed directly to any `AbstractQueueService` via the `messageMetricsManager` option.
14+
15+
```ts
16+
import { PrometheusMessageProcessingTimeMetric } from '@message-queue-toolkit/metrics'
17+
18+
const metric = new PrometheusMessageProcessingTimeMetric({
19+
name: 'message_processing_duration_ms',
20+
helpDescription: 'Time spent processing a message',
21+
buckets: [10, 50, 100, 500, 1000],
22+
})
23+
24+
// Pass to your queue service
25+
const service = new MyQueueService({ messageMetricsManager: metric })
26+
```
27+
28+
---
429

530
## Prometheus metrics
631

7-
Metrics that use [Prometheus](https://prometheus.io/) toolkit and [prom-client](https://github.com/siimon/prom-client) library
32+
All Prometheus metrics use [prom-client](https://github.com/siimon/prom-client) under the hood.
33+
34+
### Base parameters
35+
36+
All metrics accept `PrometheusMetricParams`:
37+
38+
| Field | Type | Required | Description |
39+
|---|---|---|---|
40+
| `name` | `string` | yes | Prometheus metric name |
41+
| `helpDescription` | `string` | yes | Prometheus metric description |
42+
| `buckets` | `number[]` | histograms only | Histogram bucket boundaries |
43+
| `messageVersion` | `string \| (metadata) => string \| undefined` | no | Static version string or function to extract version from message metadata |
44+
| `labelNames` | `Labels[]` | when `Labels` is specified | Names of the custom labels to register. Must not overlap with `DefaultLabels` (`queue`, `messageType`, `version`, `result`) — TypeScript enforces this at compile time |
45+
46+
An optional second argument accepts a custom `prom-client` instance (useful for testing or multi-registry setups).
47+
48+
---
49+
50+
### Histogram metrics (time-based)
51+
52+
Use `Histogram` to measure message timing. Base labels registered on every observation:
53+
54+
| Label | Value |
55+
|---|---|
56+
| `messageType` | Message type identifier |
57+
| `version` | Resolved message version |
58+
| `queue` | Queue or topic name |
59+
| `result` | Processing result status (`consumed`, `published`, `retryLater`, `error`) |
60+
61+
#### Built-in implementations
62+
63+
**`PrometheusMessageProcessingTimeMetric`**
64+
Measures elapsed time from when processing started to when it ended.
65+
```
66+
value = messageProcessingEndTimestamp - messageProcessingStartTimestamp
67+
```
68+
69+
**`PrometheusMessageLifetimeMetric`**
70+
Measures elapsed time from when the message was originally sent to when it was fully processed. Includes any time the message spent waiting in the queue.
71+
```
72+
value = messageProcessingEndTimestamp - messageTimestamp
73+
```
74+
Skips observation if `messageTimestamp` is not available.
75+
76+
**`PrometheusMessageQueueTimeMetric`**
77+
Measures elapsed time from when the message was originally sent to when processing started (i.e., queue wait time only).
78+
```
79+
value = messageProcessingStartTimestamp - messageTimestamp
80+
```
81+
Skips observation if `messageTimestamp` is not available.
82+
83+
#### Custom histogram with extra labels
84+
85+
Extend `PrometheusMessageTimeMetric` to add custom labels. Pass `labelNames` in the params and override `getLabelValuesForProcessedMessage`. Custom label names must not conflict with `DefaultLabels` — using a reserved name (e.g. `'result'`) will produce a TypeScript compile error:
86+
87+
```ts
88+
import { PrometheusMessageTimeMetric } from '@message-queue-toolkit/metrics'
89+
import type { ProcessedMessageMetadata } from '@message-queue-toolkit/core'
90+
import type { LabelValues } from 'prom-client'
91+
92+
class MyProcessingTimeMetric extends PrometheusMessageTimeMetric<MyMessage, 'env'> {
93+
protected calculateObservedValue(metadata: ProcessedMessageMetadata<MyMessage>): number | null {
94+
return metadata.messageProcessingEndTimestamp - metadata.messageProcessingStartTimestamp
95+
}
96+
97+
protected getLabelValuesForProcessedMessage(): LabelValues<'env'> {
98+
return { env: process.env.NODE_ENV ?? 'unknown' }
99+
}
100+
}
101+
102+
const metric = new MyProcessingTimeMetric({
103+
name: 'message_processing_duration_ms',
104+
helpDescription: 'Processing time by environment',
105+
buckets: [10, 50, 100, 500],
106+
labelNames: ['env'],
107+
})
108+
```
109+
110+
---
111+
112+
### Counter metrics (event-based)
113+
114+
Use `Counter` to count message events. Base labels registered on every increment:
115+
116+
| Label | Value |
117+
|---|---|
118+
| `messageType` | Message type identifier |
119+
| `version` | Resolved message version |
120+
| `queue` | Queue or topic name |
121+
| `result` | Processing result status (`consumed`, `published`, `retryLater`, `error`) |
122+
123+
#### Built-in implementations
124+
125+
**`PrometheusMessageResultCounter`**
126+
Counts all processed messages using only the built-in base labels. No extra configuration needed.
127+
128+
```ts
129+
import { PrometheusMessageResultCounter } from '@message-queue-toolkit/metrics'
130+
131+
const metric = new PrometheusMessageResultCounter({
132+
name: 'messages_total',
133+
helpDescription: 'Number of messages processed',
134+
})
135+
```
136+
137+
**`PrometheusMessageErrorCounter`**
138+
Counts only messages that result in an error. Adds an `errorReason` label. Skips all non-error messages.
139+
140+
```ts
141+
import { PrometheusMessageErrorCounter } from '@message-queue-toolkit/metrics'
142+
143+
const metric = new PrometheusMessageErrorCounter({
144+
name: 'message_errors_total',
145+
helpDescription: 'Number of messages that failed processing',
146+
labelNames: ['errorReason'],
147+
})
148+
```
149+
150+
#### Custom counter with extra labels
151+
152+
Extend `PrometheusMessageCounter` and implement `calculateCount`. Override `getLabelValuesForProcessedMessage` when adding custom labels. Same as histograms, custom label names must not conflict with `DefaultLabels`:
153+
154+
```ts
155+
import { PrometheusMessageCounter } from '@message-queue-toolkit/metrics'
156+
import type { ProcessedMessageMetadata } from '@message-queue-toolkit/core'
157+
import type { LabelValues } from 'prom-client'
158+
159+
class MyRegionCounter extends PrometheusMessageCounter<MyMessage, 'region'> {
160+
protected calculateCount(): number | null {
161+
return 1
162+
}
163+
164+
protected override getLabelValuesForProcessedMessage(
165+
metadata: ProcessedMessageMetadata<MyMessage>,
166+
): LabelValues<'region'> {
167+
return { region: metadata.message.region }
168+
}
169+
}
170+
171+
const metric = new MyRegionCounter({
172+
name: 'messages_by_region_total',
173+
helpDescription: 'Number of messages processed, by region',
174+
labelNames: ['region'],
175+
})
176+
```
177+
178+
When no custom labels are needed, omit `labelNames` and skip overriding `getLabelValuesForProcessedMessage`:
179+
180+
```ts
181+
class MyConsumedCounter extends PrometheusMessageCounter<MyMessage> {
182+
protected calculateCount(metadata: ProcessedMessageMetadata<MyMessage>): number | null {
183+
return metadata.processingResult.status === 'consumed' ? 1 : null
184+
}
185+
}
186+
187+
const metric = new MyConsumedCounter({
188+
name: 'messages_consumed_total',
189+
helpDescription: 'Number of successfully consumed messages',
190+
})
191+
```
8192

9-
### MessageProcessingPrometheusMetric
10-
Abstract class implementing `MessageMetricsManager` interface, that can be injected into `AbstractQueueService` from `@message-queue-toolkit/core`.
193+
---
11194

12-
It uses [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram) metric to collect message processing times with labels:
13-
- `messageType` - message type
14-
- `version` - message version
15-
- `queue` - name of the queue or topic
16-
- `result` - processing result
195+
### Using multiple metrics together
17196

18-
See [MessageProcessingPrometheusMetric.ts](lib/prometheus/MessageProcessingPrometheusMetric.ts) for available parameters.
197+
`MessageMultiMetricManager` aggregates multiple `MessageMetricsManager` instances and fans out each `registerProcessedMessage` call to all of them.
19198

20-
There are following non-abstract implementations available:
21-
- `MessageProcessingTimeMetric` - registers elapsed time from start to the end of message processing
22-
- `MessageLifetimeMetric` - registers elapsed time from the point where message was initially sent, to the point when it was processed.
23-
Note: if message is waiting in the queue due to high load or barrier, the waiting time is included in the measurement
199+
```ts
200+
import {
201+
MessageMultiMetricManager,
202+
PrometheusMessageProcessingTimeMetric,
203+
PrometheusMessageResultCounter,
204+
PrometheusMessageErrorCounter,
205+
} from '@message-queue-toolkit/metrics'
24206

25-
### MessageProcessingMultiMetrics
26-
Implementation of `MessageMetricsManager` that allows to use multiple `MessageProcessingPrometheusMetric` instances.
207+
const metricsManager = new MessageMultiMetricManager([
208+
new PrometheusMessageProcessingTimeMetric({
209+
name: 'message_processing_duration_ms',
210+
helpDescription: 'Message processing time',
211+
buckets: [10, 50, 100, 500, 1000],
212+
}),
213+
new PrometheusMessageResultCounter({
214+
name: 'messages_total',
215+
helpDescription: 'Messages processed',
216+
}),
217+
new PrometheusMessageErrorCounter({
218+
name: 'message_errors_total',
219+
helpDescription: 'Messages that failed processing',
220+
labelNames: ['errorReason'],
221+
}),
222+
])
27223

224+
const service = new MyQueueService({ messageMetricsManager: metricsManager })
225+
```

packages/metrics/lib/index.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
export * from './MessageMultiMetricManager.ts'
2+
export * from './prometheus/metrics/message-error/PrometheusMessageCounter.ts'
23
export * from './prometheus/metrics/message-error/PrometheusMessageErrorCounter.ts'
4+
export * from './prometheus/metrics/message-error/PrometheusMessageResultCounter.ts'
35
export * from './prometheus/metrics/message-time/PrometheusMessageLifetimeMetric.ts'
46
export * from './prometheus/metrics/message-time/PrometheusMessageProcessingTimeMetric.ts'
57
export * from './prometheus/metrics/message-time/PrometheusMessageQueueTimeMetric.ts'

packages/metrics/lib/prometheus/PrometheusMessageMetric.ts

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,16 +9,19 @@ import type { MessageVersionGeneratingFunction, PrometheusMetricParams } from '.
99
export abstract class PrometheusMessageMetric<
1010
MessagePayload extends object,
1111
MetricType extends Metric,
12-
MetricParams extends
13-
PrometheusMetricParams<MessagePayload> = PrometheusMetricParams<MessagePayload>,
12+
Labels extends string = never,
13+
MetricParams extends PrometheusMetricParams<MessagePayload, Labels> = PrometheusMetricParams<
14+
MessagePayload,
15+
Labels
16+
>,
1417
> implements MessageMetricsManager<MessagePayload>
1518
{
1619
/** Fallbacks to null if metrics are disabled on app level */
1720
protected readonly metric: MetricType
1821

1922
protected readonly messageVersionGeneratingFunction: MessageVersionGeneratingFunction<MessagePayload>
2023

21-
private readonly metricParams: MetricParams
24+
protected readonly metricParams: MetricParams
2225

2326
/**
2427
* @param metricParams - metrics parameters (see PrometheusMetricParams)

0 commit comments

Comments
 (0)