Skip to content

Adjust SQS limits #403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kibertoad merged 1 commit into from
Feb 20, 2026
Merged

Adjust SQS limits #403

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ services:
restart: on-failure

localstack:
image: localstack/localstack:4.10.0
image: localstack/localstack:4.13.1
network_mode: bridge
hostname: localstack
ports:
Expand Down
2 changes: 1 addition & 1 deletion examples/sns-sqs/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
services:

localstack:
image: localstack/localstack:4.0.2
image: localstack/localstack:4.13.1
network_mode: bridge
hostname: localstack
ports:
Expand Down
12 changes: 6 additions & 6 deletions packages/sqs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ Publishers send messages to SQS queues. They handle:
- Message validation against Zod schemas
- Automatic serialization
- Optional deduplication (preventing duplicate sends)
- Optional payload offloading (for messages > 256KB)
- Optional payload offloading (for messages > 1 MiB)
- FIFO-specific concerns (MessageGroupId, MessageDeduplicationId)

### Consumers
Expand Down Expand Up @@ -380,7 +380,7 @@ When using `creationConfig`, the queue will be created automatically if it doesn

// Other attributes
DelaySeconds: '0', // Default delay for all messages
MaximumMessageSize: '262144', // 256 KB (default maximum)
MaximumMessageSize: '1048576', // 1 MiB (default maximum)
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

// 1 MiB (default maximum) is ambiguous — clarify whether this is the new per-queue default or just the new maximum allowed value.

The MaximumMessageSize queue attribute previously defaulted to 262,144 bytes (256 KiB). If you've explicitly set the MaximumMessageSize attribute, you might need to adjust it to a new desired value. This wording implies the default was not automatically raised to 1 MiB — only the maximum ceiling was. If that's the case, setting MaximumMessageSize: '1048576' here doesn't represent the default but rather explicitly opts into the new maximum.

📝 Suggested comment clarification 
-        MaximumMessageSize: '1048576',      // 1 MiB (default maximum)
+        MaximumMessageSize: '1048576',      // 1 MiB (new maximum; set explicitly to opt in)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
MaximumMessageSize: '1048576', // 1 MiB (default maximum)
MaximumMessageSize: '1048576', // 1 MiB (new maximum; set explicitly to opt in)
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@packages/sqs/README.md` at line 383, The inline comment on MaximumMessageSize
is ambiguous; update the comment next to the MaximumMessageSize attribute to
state that '1048576' is the new maximum allowed per-message size (1 MiB) rather
than a changed default, and clarify that leaving the attribute unset preserves
the previous default (262144 bytes) while explicitly setting MaximumMessageSize:
'1048576' opts the queue into the larger cap; reference the MaximumMessageSize
attribute in the README to make this distinction explicit.

},
tags: {
Environment: 'production',
Expand Down Expand Up @@ -457,7 +457,7 @@ When using `locatorConfig`, you connect to an existing queue without creating it
// Optional - Payload Offloading
payloadStoreConfig: {
payloadStore: s3Store, // S3-based payload store
maxPayloadSize: 256 * 1024, // 256 KB
maxPayloadSize: 1024 * 1024, // 1 MiB
},

// Optional - Deletion
Expand Down Expand Up @@ -755,7 +755,7 @@ Prevents processing the same message multiple times:

### Payload Offloading

For messages larger than 256 KB, store the payload externally (e.g., S3):
For messages larger than 1 MiB, store the payload externally (e.g., S3):

```typescript
import { S3PayloadStore } from '@message-queue-toolkit/s3-payload-store'
Expand All @@ -769,15 +769,15 @@ const payloadStore = new S3PayloadStore({
{
payloadStoreConfig: {
payloadStore,
maxPayloadSize: 256 * 1024, // 256 KB threshold
maxPayloadSize: 1024 * 1024, // 1 MiB threshold
},
}

// Large message is automatically offloaded
await publisher.publish({
id: '123',
messageType: 'document.processed',
largeData: hugeArrayOfData, // If total size > 256 KB, stored in S3
largeData: hugeArrayOfData, // If total size > 1 MiB, stored in S3
})
```

Expand Down
2 changes: 1 addition & 1 deletion packages/sqs/lib/sqs/AbstractSqsService.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import type { SQSMessage } from '../types/MessageTypes.ts'
import { deleteSqs, initSqs } from '../utils/sqsInitter.ts'

// https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html
export const SQS_MESSAGE_MAX_SIZE = 256 * 1024 // 256KB
export const SQS_MESSAGE_MAX_SIZE = 1024 * 1024 // 1 MiB
Comment on lines 11 to +12
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

SQS_MESSAGE_MAX_SIZE is a publicly exported constant — this is a behavioral breaking change that needs a changelog entry.

SQS_MESSAGE_MAX_SIZE is re-exported from packages/sqs/lib/index.ts. Any consumer who derived their maxPayloadSize from this constant (e.g., payloadStoreConfig: { maxPayloadSize: SQS_MESSAGE_MAX_SIZE }) will now silently skip S3 offloading for messages between 256 KiB and 1 MiB. Infrastructure relying on those messages flowing through S3 (lifecycle policies, monitoring, auditing) could be affected. A semver-major bump or explicit migration note in the changelog is warranted.

Amazon SQS increased the maximum message payload size from 256 KiB to 1 MiB for standard and FIFO queues, so the constant value itself is correct.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@packages/sqs/lib/sqs/AbstractSqsService.ts` around lines 11 - 12, You
exported SQS_MESSAGE_MAX_SIZE (now 1 MiB) which is a behavioral breaking change
for consumers who derived maxPayloadSize from it—add a clear changelog entry and
migration note and either (A) publish this change as a semver-major release or
(B) re-introduce a backwards-compatible alias like a deprecated
SQS_LEGACY_MAX_SIZE = 256 * 1024 and document preferred usage; update the
package release notes (and packages/sqs/lib/index.ts re-export) to call out the
change, recommended migration steps for consumers using
payloadStoreConfig.maxPayloadSize, and which symbol (SQS_MESSAGE_MAX_SIZE vs
SQS_LEGACY_MAX_SIZE) they should use going forward.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Verify the `@aws-sdk/client-sqs` version in use and check for large-message tests.

# 1. Show declared version across all package.json files (excluding node_modules)
echo "=== Declared `@aws-sdk/client-sqs` version ==="
fd 'package.json' --type f | xargs grep '"@aws-sdk/client-sqs"' 2>/dev/null

# 2. Check if any test or source files exercise message sizes near/above 262144 bytes
echo -e "\n=== Large-message test/usage references ==="
rg -n --type ts -C 3 'SQS_MESSAGE_MAX_SIZE|262144|1048576|maxPayloadSize'

Repository: kibertoad/message-queue-toolkit

Length of output: 6192

🏁 Script executed:

#!/bin/bash
# Check how payload offloading handles large messages and look for validator constraints

# 1. Look at payload offloading implementation
echo "=== Payload offloading implementation ==="
fd 'payloadOffloa' --type ts | head -5 | xargs -I {} bash -c 'echo "File: {}"; head -50 {}'

# 2. Search for test message creation to see if large messages are actually sent
echo -e "\n=== Test message creation in payload offloading tests ==="
rg -A 10 'describe.*payload offloading' packages/sqs/test/consumers/SqsPermissionConsumer.payloadOffloading.spec.ts | head -30

Repository: kibertoad/message-queue-toolkit

Length of output: 1332

🌐 Web query:

@aws-sdk/client-sqs 3.812.0 validator maxLength 262144 SQS 1 MiB limit when updated

💡 Result:

Amazon SQS raised the maximum message payload size to 1,048,576 bytes (1 MiB) on August 4, 2025 (from 256 KiB). [1] The SQS quotas/docs now state “Message size … maximum is 1,048,576 bytes (1 MiB)”, and the FAQ likewise says MaximumMessageSize can be set up to 1 MiB. [2][3]

So if @aws-sdk/client-sqs@3.812.0 is still enforcing a maxLength: 262144 validator, that limit reflects the old 256 KiB (262,144 bytes) model and the SDK’s generated validation metadata hasn’t caught up to the service change yet.

When will the SDK update? I couldn’t find any AWS-published date/commit guaranteeing when @aws-sdk/client-sqs will change that validator; as of today (Feb 20, 2026), the authoritative service limit is already 1 MiB in AWS docs, but the SDK-side validator may still be outdated. [2][3]

Practical implication: SQS can accept up to 1 MiB (subject to the queue’s MaximumMessageSize), but the JS v3 client may block >262,144-byte MessageBody locally until the model/validator is updated (or you bypass/patch that validation). [2][3]

Sources
[1] Amazon SQS “What’s New” announcement (Aug 4, 2025)
[2] SQS Developer Guide – message quotas (Message size max 1,048,576 bytes)
[3] SQS FAQ – configuring MaximumMessageSize up to 1 MiB

🏁 Script executed:

#!/bin/bash
# Look at the actual test to see message sizes being tested

echo "=== Test file content (first 150 lines) ==="
cat -n packages/sqs/test/consumers/SqsPermissionConsumer.payloadOffloading.spec.ts | head -150

Repository: kibertoad/message-queue-toolkit

Length of output: 6846

Upgrade @aws-sdk/client-sqs to 3.990.0 or verify the version includes the 1 MiB validator.

The project pins @aws-sdk/client-sqs@3.812.0, which may still enforce a client-side validation limit of 262,144 bytes for MessageBody (the pre-August 2025 limit). AWS announced the 1 MiB limit increase on August 4, 2025, but the SDK's Smithy-generated validators lag behind the service model update. The current tests work because they use S3 payload offloading—only sending a small pointer message to SQS—rather than testing direct 1 MiB message sends.

If any code attempts to send a MessageBody between 256 KiB and 1 MiB directly to SQS without offloading, the client-side validator will reject it before the request reaches AWS. Either upgrade the SDK to a version released after the service model was updated, or document that SQS_MESSAGE_MAX_SIZE represents the SQS service limit, not the SDK's validator limit for direct messages.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@packages/sqs/lib/sqs/AbstractSqsService.ts` around lines 11 - 12, The
SQS_MESSAGE_MAX_SIZE constant declares the SQS service limit (1 MiB) but the
pinned `@aws-sdk/client-sqs` may still enforce the older 256 KiB client-side
validator; either upgrade the SDK to a version that includes the 1 MiB validator
(>= 3.990.0) or explicitly document the mismatch where SQS_MESSAGE_MAX_SIZE is
defined in AbstractSqsService.ts so callers know the SDK may reject MessageBody
>256 KiB; update package.json dependency for `@aws-sdk/client-sqs` to >=3.990.0
and/or add a comment next to SQS_MESSAGE_MAX_SIZE clarifying it reflects the
service limit not the SDK validator limit.

export const SQS_RESOURCE_ANY = Symbol('any')
export const SQS_RESOURCE_CURRENT_QUEUE = Symbol('current_queue')

Expand Down
2 changes: 1 addition & 1 deletion packages/sqs/lib/utils/sqsUtils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ export async function deleteQueue(
/**
* Calculates the size of an outgoing SQS message.
*
* SQS imposes a 256 KB limit on the total size of a message, which includes both the message body and any metadata (attributes).
* SQS imposes a 1 MiB limit on the total size of a message, which includes both the message body and any metadata (attributes).
* This function currently computes the size based solely on the message body, as no attributes are included at this time.
* For future updates, if message attributes are added, their sizes should also be considered.
*
Expand Down
Loading