Skip to content

Commit e3c504a

Browse files
committed
README.md update
1 parent cf14378 commit e3c504a

1 file changed

Lines changed: 58 additions & 3 deletions

File tree

README.md

Lines changed: 58 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,8 @@ They implement the following public methods:
4040
* `policyConfig` - SQS only - configuration for queue access policies (see [SQS Policy Configuration](#sqs-policy-configuration) for more information);
4141
* `deletionConfig` - automatic cleanup of resources;
4242
* `handlerSpy` - allow awaiting certain messages to be published (see [Handler Spies](#handler-spies) for more information);
43-
* `logMessages` - add logs for processed messages.
43+
* `logMessages` - add debug logs for processed messages. When enabled, logs structured metadata including message id, type, timestamps, and queue name. For privacy reasons, the full message payload is not logged by default. See [Message Logging](#message-logging) for more details.
44+
* `messageMetadataField` - which field in the message contains metadata for logging purposes (by default it is `metadata`).
4445
* `payloadStoreConfig` - configuration for payload offloading. This option enables the external storage of large message payloads to comply with message size limitations of the queue system. For more details on setting this up, see [Payload Offloading](#payload-offloading).
4546
* `messageDeduplicationConfig` - configuration for store-based message deduplication on publisher level. For more details on setting this up, see [Publisher-level store-based-message deduplication](#publisher-level-store-based-message-deduplication).
4647
* `enablePublisherDeduplication` - enable store-based publisher-level deduplication. For more details on setting this up, see [Publisher-level store-based-message deduplication](#publisher-level-store-based-message-deduplication).
@@ -110,7 +111,8 @@ Multi-schema consumers support multiple message types via handler configs. They
110111
* `consumerOverrides` – available only for SQS consumers;
111112
* `deadLetterQueue` - available only for SQS and SNS consumers (see [Dead Letter Queue](#dead-letter-queue) for more information);
112113
* `handlerSpy` - allow awaiting certain messages to be published (see [Handler Spies](#handler-spies) for more information);
113-
* `logMessages` - add logs for processed messages.
114+
* `logMessages` - add debug logs for processed messages. When enabled, logs structured metadata including message id, type, timestamps, and queue name. For privacy reasons, the full message payload is not logged by default. To include custom message data in logs, configure `messageLogFormatter` on your handlers. See [Message Logging](#message-logging) for more details.
115+
* `messageMetadataField` - which field in the message contains metadata for logging purposes (by default it is `metadata`).
114116
* `payloadStoreConfig` - configuration for payload offloading. This option enables the external storage of large message payloads to comply with message size limitations of the queue system. For more details on setting this up, see [Payload Offloading](#payload-offloading).
115117
* `concurrentConsumersAmount` - configuration for specifying the number of concurrent consumers to create. Available only for SQS and SNS consumers
116118
* `messageDeduplicationConfig` - configuration for store-based message deduplication on consumer level. For more details on setting this up, see [Consumer-level store-based-message deduplication](#consumer-level-store-based-message-deduplication).
@@ -173,7 +175,9 @@ export class SqsPermissionConsumer extends AbstractSqsConsumer<
173175
preHandlerBarrier: async (message) => {
174176
// do barrier check here
175177
return true
176-
}
178+
},
179+
// Optional: customize what message data is logged (see Message Logging section)
180+
messageLogFormatter: (message) => ({ id: message.id, type: message.type }),
177181
},
178182
)
179183
.addConfig(PERMISSIONS_REMOVE_MESSAGE_SCHEMA,
@@ -642,6 +646,56 @@ const result = await myConsumer.handlerSpy.waitForMessageWithId('1')
642646
expect(result.processingResult).toEqual({ status: 'consumed' })
643647
```
644648

649+
## Message Logging
650+
651+
When `logMessages` is enabled, processed messages are logged at the `debug` level with structured metadata. For privacy reasons, the full message payload is **not logged by default** to avoid exposing sensitive data.
652+
653+
### What is logged by default
654+
655+
Each log entry includes processed message metadata:
656+
- `messageId` - unique identifier of the message
657+
- `messageType` - type of the message
658+
- `queueName` - name of the queue or topic
659+
- `messageTimestamp` - when the message was originally sent
660+
- `messageProcessingStartTimestamp` - when processing started
661+
- `messageProcessingEndTimestamp` - when processing completed
662+
- `messageDeduplicationId` - deduplication id (if deduplication is enabled)
663+
- `messageMetadata` - contents of the metadata field (configurable via `messageMetadataField`)
664+
- `processingResult` - outcome of processing (e.g., `{ status: 'consumed' }` or `{ status: 'published' }`)
665+
666+
### Custom message logging with messageLogFormatter
667+
668+
If you need to include additional message data in logs, you can configure a `messageLogFormatter` on your handler. This formatter receives the message and returns the data to be logged:
669+
670+
```typescript
671+
new MessageHandlerConfigBuilder<SupportedMessages, ExecutionContext>()
672+
.addConfig(
673+
MY_MESSAGE_SCHEMA,
674+
async (message, context) => {
675+
// handler logic
676+
return { result: 'success' }
677+
},
678+
{
679+
// Only log specific fields, excluding sensitive data
680+
messageLogFormatter: (message) => ({
681+
id: message.id,
682+
type: message.type,
683+
// Exclude sensitive fields like email, password, etc.
684+
}),
685+
},
686+
)
687+
.build()
688+
```
689+
690+
When a `messageLogFormatter` is provided, its output is included in the log under the `message` key alongside the processed message metadata.
691+
692+
### Configuration options
693+
694+
| Option | Default | Description |
695+
|--------|---------|-------------|
696+
| `logMessages` | `false` | Enable debug logging for processed messages |
697+
| `messageMetadataField` | `'metadata'` | Field in the message containing metadata to include in logs |
698+
645699
## Payload Offloading
646700
Payload offloading allows you to manage large message payloads by storing them in external storage, bypassing any message size restrictions imposed by queue systems.
647701

@@ -773,6 +827,7 @@ It needs to implement the following methods:
773827
- `messageProcessingStartTimestamp` - the timestamp when the processing of the message started
774828
- `messageProcessingEndTimestamp` - the timestamp when the processing of the message finished
775829
- `messageDeduplicationId` - the deduplication id of the message, in case deduplication is enabled
830+
- `messageMetadata` - contents of the message metadata field (configurable via `messageMetadataField`)
776831

777832
See [@message-queue-toolkit/metrics](packages/metrics/README.md) for concrete implementations
778833

0 commit comments

Comments
 (0)