Skip to content

Commit 8970771

Browse files
committed
Address review comments
1 parent dbc10e1 commit 8970771

13 files changed

Lines changed: 278 additions & 243 deletions

README.md

Lines changed: 44 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -244,34 +244,36 @@ Both publishers and consumers accept a queue name and configuration as parameter
244244

245245
If you do not want to create a new queue/topic, you can set `queueLocator` field for `queueConfiguration`. In that case `message-queue-toolkit` will not attempt to create a new queue or topic, and instead throw an error if they don't already exist.
246246

247-
## Resource Availability Polling (Eventual Consistency Mode)
247+
## Startup Resource Polling (Eventual Consistency Mode)
248248

249249
When using `locatorConfig` to reference existing queues or topics, the default behavior is to fail immediately if the resource doesn't exist. However, in some deployment scenarios (especially with cross-service dependencies), resources may not be available at startup time.
250250

251-
The `resourceAvailabilityConfig` option enables "eventual consistency mode" where consumers poll for resources to become available instead of failing immediately.
251+
The `startupResourcePolling` option within `locatorConfig` enables "eventual consistency mode" where consumers poll for resources to become available instead of failing immediately.
252252

253253
### Use Case: Cross-Service Dependencies
254254

255255
Consider two services that need to subscribe to each other's topics:
256256
- **Service A** needs to subscribe to **Service B's** topic
257257
- **Service B** needs to subscribe to **Service A's** topic
258258

259-
Without eventual consistency mode, neither service can deploy first because the other's topic doesn't exist yet. With `resourceAvailabilityConfig`, both services can start simultaneously and wait for the other's resources to appear.
259+
Without eventual consistency mode, neither service can deploy first because the other's topic doesn't exist yet. With `startupResourcePolling`, both services can start simultaneously and wait for the other's resources to appear.
260260

261261
### Configuration
262262

263263
```typescript
264+
import { NO_TIMEOUT } from '@message-queue-toolkit/core'
265+
264266
const consumer = new MySnsSqsConsumer(dependencies, {
265267
locatorConfig: {
266268
topicArn: 'arn:aws:sns:...',
267269
queueUrl: 'https://sqs...',
268-
subscriptionArn: '...'
269-
},
270-
// Enable eventual consistency mode
271-
resourceAvailabilityConfig: {
272-
enabled: true, // Enable polling for resource availability
273-
pollingIntervalMs: 5000, // Check every 5 seconds (default: 5000)
274-
timeoutMs: 300000, // Fail after 5 minutes (optional, default: no timeout)
270+
subscriptionArn: '...',
271+
// Enable eventual consistency mode
272+
startupResourcePolling: {
273+
enabled: true, // Enable polling for resource availability
274+
pollingIntervalMs: 5000, // Check every 5 seconds (default: 5000)
275+
timeoutMs: 300000, // Fail after 5 minutes (required)
276+
}
275277
}
276278
})
277279
```
@@ -280,42 +282,61 @@ const consumer = new MySnsSqsConsumer(dependencies, {
280282

281283
| Option | Type | Default | Description |
282284
|--------|------|---------|-------------|
283-
| `enabled` | `boolean` | `false` | Enable eventual consistency mode |
285+
| `enabled` | `boolean` | - | Must be set to `true` to enable polling |
284286
| `pollingIntervalMs` | `number` | `5000` | Interval between availability checks (ms) |
285-
| `timeoutMs` | `number` | `undefined` | Maximum wait time before throwing `ResourceAvailabilityTimeoutError`. If not set, polls indefinitely |
287+
| `timeoutMs` | `number \| NO_TIMEOUT` | - (required) | Maximum wait time before throwing `StartupResourcePollingTimeoutError`. Use `NO_TIMEOUT` to poll indefinitely |
286288

287289
### Environment-Specific Configuration
288290

289291
```typescript
292+
import { NO_TIMEOUT } from '@message-queue-toolkit/core'
293+
294+
// Production - with timeout
295+
{
296+
locatorConfig: {
297+
queueUrl: '...',
298+
startupResourcePolling: {
299+
enabled: true,
300+
timeoutMs: 5 * 60 * 1000, // 5 minutes
301+
}
302+
}
303+
}
304+
290305
// Development/Staging - poll indefinitely
291306
{
292-
resourceAvailabilityConfig: {
293-
enabled: true,
294-
pollingIntervalMs: 5000,
307+
locatorConfig: {
308+
queueUrl: '...',
309+
startupResourcePolling: {
310+
enabled: true,
311+
timeoutMs: NO_TIMEOUT,
312+
}
295313
}
296314
}
297315

298-
// Production - poll with timeout to catch misconfigurations
316+
// Custom timeout and interval
299317
{
300-
resourceAvailabilityConfig: {
301-
enabled: true,
302-
pollingIntervalMs: 10000,
303-
timeoutMs: 5 * 60 * 1000, // 5 minutes
318+
locatorConfig: {
319+
queueUrl: '...',
320+
startupResourcePolling: {
321+
enabled: true,
322+
pollingIntervalMs: 10000,
323+
timeoutMs: 10 * 60 * 1000, // 10 minutes
324+
}
304325
}
305326
}
306327
```
307328

308329
### Error Handling
309330

310-
When `timeoutMs` is configured and the resource doesn't become available within the specified time, a `ResourceAvailabilityTimeoutError` is thrown:
331+
When `timeoutMs` is configured and the resource doesn't become available within the specified time, a `StartupResourcePollingTimeoutError` is thrown:
311332

312333
```typescript
313-
import { ResourceAvailabilityTimeoutError } from '@message-queue-toolkit/core'
334+
import { StartupResourcePollingTimeoutError } from '@message-queue-toolkit/core'
314335

315336
try {
316337
await consumer.init()
317338
} catch (error) {
318-
if (error instanceof ResourceAvailabilityTimeoutError) {
339+
if (error instanceof StartupResourcePollingTimeoutError) {
319340
console.error(`Resource ${error.resourceName} not available after ${error.timeoutMs}ms`)
320341
}
321342
}

packages/core/lib/index.ts

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -106,11 +106,11 @@ export { isShallowSubset, objectMatches } from './utils/matchUtils.ts'
106106
export { type ParseMessageResult, parseMessage } from './utils/parseUtils.ts'
107107
export { objectToBuffer } from './utils/queueUtils.ts'
108108
export {
109-
isResourceAvailabilityWaitingEnabled,
110-
type ResourceAvailabilityCheckResult,
111-
ResourceAvailabilityTimeoutError,
109+
isStartupResourcePollingEnabled,
110+
type StartupResourcePollingCheckResult,
111+
StartupResourcePollingTimeoutError,
112112
type WaitForResourceOptions,
113113
waitForResource,
114-
} from './utils/resourceAvailabilityUtils.ts'
114+
} from './utils/startupResourcePollingUtils.ts'
115115
export { toDatePreprocessor } from './utils/toDateProcessor.ts'
116116
export { waitAndRetry } from './utils/waitUtils.ts'

packages/core/lib/queues/AbstractQueueService.ts

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,6 @@ import type {
4646
ProcessedMessageMetadata,
4747
QueueDependencies,
4848
QueueOptions,
49-
ResourceAvailabilityConfig,
5049
} from '../types/queueOptionsTypes.ts'
5150
import { isRetryDateExceeded } from '../utils/dateUtils.ts'
5251
import { streamWithKnownSizeToString } from '../utils/streamUtils.ts'
@@ -126,7 +125,6 @@ export abstract class AbstractQueueService<
126125
protected readonly creationConfig?: QueueConfiguration
127126
protected readonly locatorConfig?: QueueLocatorType
128127
protected readonly deletionConfig?: DeletionConfig
129-
protected readonly resourceAvailabilityConfig?: ResourceAvailabilityConfig
130128
protected readonly payloadStoreConfig?:
131129
| MakeRequired<SinglePayloadStoreConfig, 'serializer'>
132130
| MakeRequired<MultiPayloadStoreConfig, 'serializer'>
@@ -162,7 +160,6 @@ export abstract class AbstractQueueService<
162160
this.creationConfig = options.creationConfig
163161
this.locatorConfig = options.locatorConfig
164162
this.deletionConfig = options.deletionConfig
165-
this.resourceAvailabilityConfig = options.resourceAvailabilityConfig
166163
this.payloadStoreConfig = options.payloadStoreConfig
167164
? {
168165
serializer: jsonStreamStringifySerializer,

packages/core/lib/types/queueOptionsTypes.ts

Lines changed: 59 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -118,13 +118,6 @@ export type CommonQueueOptions = {
118118
deletionConfig?: DeletionConfig
119119
payloadStoreConfig?: PayloadStoreConfig
120120
messageDeduplicationConfig?: MessageDeduplicationConfig
121-
/**
122-
* Configuration for eventual consistency mode.
123-
* When enabled, the consumer will poll for the topic/queue to become available
124-
* instead of failing immediately when using locatorConfig.
125-
* This is useful for handling cross-service dependencies during deployment.
126-
*/
127-
resourceAvailabilityConfig?: ResourceAvailabilityConfig
128121
}
129122

130123
export type CommonCreationConfigType = {
@@ -138,57 +131,82 @@ export type DeletionConfig = {
138131
}
139132

140133
/**
141-
* Configuration for eventual consistency mode when resources may not exist at startup.
134+
* Symbol to indicate no timeout - polling will continue indefinitely.
135+
* Use this for dev/staging environments where you want to wait indefinitely for resources.
136+
*
137+
* @example
138+
* {
139+
* locatorConfig: {
140+
* queueUrl: '...',
141+
* startupResourcePolling: {
142+
* enabled: true,
143+
* timeoutMs: NO_TIMEOUT,
144+
* }
145+
* }
146+
* }
147+
*/
148+
export const NO_TIMEOUT = Symbol('NO_TIMEOUT')
149+
150+
/**
151+
* Configuration for startup resource polling mode when resources may not exist at startup.
142152
*
143153
* This is useful in scenarios where services have cross-dependencies:
144154
* - Service A needs to subscribe to Service B's topic
145155
* - Service B needs to subscribe to Service A's topic
146156
* - Neither can deploy first without the other's topic existing
147157
*
148-
* When `resourceAvailabilityConfig` is provided, the consumer will poll for the
149-
* topic/queue to become available instead of failing immediately.
158+
* When `startupResourcePolling` is provided with `enabled: true` inside `locatorConfig`,
159+
* the consumer will poll for the topic/queue to become available instead of failing immediately.
150160
*
151161
* @example
152-
* // Development/staging - poll indefinitely
162+
* // Enable with 5 minute timeout
153163
* {
154-
* resourceAvailabilityConfig: {
155-
* pollingIntervalMs: 5000,
164+
* locatorConfig: {
165+
* queueUrl: '...',
166+
* startupResourcePolling: {
167+
* enabled: true,
168+
* timeoutMs: 5 * 60 * 1000,
169+
* }
156170
* }
157171
* }
158172
*
159173
* @example
160-
* // Production - poll with timeout to catch misconfigurations
174+
* // Poll indefinitely (useful for dev/staging environments)
161175
* {
162-
* resourceAvailabilityConfig: {
163-
* timeoutMs: 5 * 60 * 1000, // 5 minutes
164-
* pollingIntervalMs: 10000,
176+
* locatorConfig: {
177+
* queueUrl: '...',
178+
* startupResourcePolling: {
179+
* enabled: true,
180+
* timeoutMs: NO_TIMEOUT,
181+
* }
165182
* }
166183
* }
167184
*
168185
* @example
169-
* // Temporarily disable without removing config
186+
* // Custom timeout and interval
170187
* {
171-
* resourceAvailabilityConfig: {
172-
* enabled: false,
173-
* timeoutMs: 5 * 60 * 1000,
188+
* locatorConfig: {
189+
* queueUrl: '...',
190+
* startupResourcePolling: {
191+
* enabled: true,
192+
* timeoutMs: 10 * 60 * 1000, // 10 minutes
193+
* pollingIntervalMs: 10000,
194+
* }
174195
* }
175196
* }
176197
*/
177-
export type ResourceAvailabilityConfig = {
198+
export type StartupResourcePollingConfig = {
178199
/**
179200
* Controls whether polling is enabled.
180-
* Default: true (when resourceAvailabilityConfig is provided)
181-
* Set to false to temporarily disable polling without removing the config.
201+
* Must be set to true to enable polling.
182202
*/
183203
enabled?: boolean
184204

185205
/**
186206
* Maximum time in milliseconds to wait for the resource to become available.
187-
* If not set or set to 0, will poll indefinitely (useful for dev/staging environments).
188-
* For production, it's recommended to set a reasonable timeout (e.g., 5 minutes).
189-
* Default: undefined (no timeout - poll indefinitely)
207+
* Use `NO_TIMEOUT` to disable timeout and poll indefinitely.
190208
*/
191-
timeoutMs?: number
209+
timeoutMs: number | typeof NO_TIMEOUT
192210

193211
/**
194212
* Interval in milliseconds between polling attempts.
@@ -197,6 +215,19 @@ export type ResourceAvailabilityConfig = {
197215
pollingIntervalMs?: number
198216
}
199217

218+
/**
219+
* Base type for queue locator configurations that includes startup resource polling.
220+
* Protocol-specific locator types should extend this.
221+
*/
222+
export type BaseQueueLocatorType = {
223+
/**
224+
* Configuration for startup resource polling mode.
225+
* When enabled, the consumer will poll for the resource to become available
226+
* instead of failing immediately.
227+
*/
228+
startupResourcePolling?: StartupResourcePollingConfig
229+
}
230+
200231
type NewQueueOptions<CreationConfigType extends CommonCreationConfigType> = {
201232
creationConfig?: CreationConfigType
202233
}

0 commit comments

Comments
 (0)