Skip to content

Commit a9ecc03

Browse files
committed
DRIVERS-3446 - Add prose tests to verify correct retry behavior when a mix of overload and non-overload errors are encountered
DRIVERS-3427: Remove token bucket disclaimer from backpressure speci… (mongodb#1923) DRIVERS-3436 - Refine `withTransaction` timeout error wrapping semantics and label propagation in spec and prose tests (mongodb#1920) Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com> DRIVERS-3325 allow keyAltName in encryptedFields spec test (mongodb#1853)
1 parent 5afa5e4 commit a9ecc03

3 files changed

Lines changed: 104 additions & 5 deletions

File tree

source/client-backpressure/client-backpressure.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -159,9 +159,9 @@ The retry policy in this specification is separate from the other retry policies
159159
specifications. Drivers MUST ensure:
160160

161161
- When a failed attempt is retried, backoff MUST be applied if and only if the error is an overload error.
162-
- If an overload error is encountered:
162+
- If an overload error is encountered at any point during an operation's retry loop:
163163
- Regardless of whether CSOT is enabled or not, the maximum number of retries for any retry policy becomes
164-
`MAX_RETRIES`.
164+
`MAX_RETRIES`. This includes retryable non-overload errors following the encountered overload error.
165165
- If CSOT is enabled and a command has been retried at least `MAX_RETRIES` times, it MUST NOT be retried further.
166166

167167
#### Pseudocode
@@ -307,7 +307,7 @@ The Node and Python drivers will provide the reference implementations. See
307307
The client backpressure retry loop is primarily concerned with spreading out retries to avoid retry storms. The exact
308308
sleep duration is not critical to the intended behavior, so long as we sleep at least as long as we say we will.
309309

310-
### Why override existing maximum number of retry attempt defaults for retryable reads and writes if an overload error is received?
310+
### Why override existing maximum number of retry attempt defaults for retryable reads and writes if an overload error is received at any point during an operation's retry loop?
311311

312312
Load-shedded errors indicate that the request was rejected by the server to minimize load, not that the command failed
313313
for logical reasons. So, when determining the number of retries an operation should attempt:
@@ -325,8 +325,8 @@ specifications with load-shedding behavior:
325325
[retryable writes](../retryable-writes/retryable-writes.md) and
326326
[CSOT](../client-side-operations-timeout/client-side-operations-timeout.md) specifications when no overload errors are
327327
encountered.
328-
- Adjusting the maximum number of retry attempts to 5 if an overload error is returned from the server gives requests
329-
more opportunities to succeed and helps reduce application errors.
328+
- Adjusting the maximum number of retry attempts to MAX_RETRIES if an overload error is returned from the server gives
329+
requests more opportunities to succeed and helps reduce application errors.
330330
- An alternative approach would be to retry once if we don't receive an overload error, in which case we'd retry 5
331331
times. The approach chosen allows for additional retries in scenarios where a non-overload error fails on a retry
332332
with an overload error.

source/retryable-reads/tests/README.md

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,57 @@ This test MUST be executed against a MongoDB 4.4+ replica set that has at least
217217

218218
6. Assert that both events occurred on the same server.
219219

220+
### 1: Test that drivers set the maximum number of retries for all retryable read errors when an overload error is encountered
221+
222+
This test MUST be executed against a MongoDB 4.4+ server that supports `retryReads=true` and has enabled the
223+
`configureFailPoint` command with the `errorLabels` option.
224+
225+
1. Create a client.
226+
227+
2. Configure a fail point with error code `91` (ShutdownInProgress) with the `RetryableError` and
228+
`SystemOverloadedError` error labels:
229+
230+
```javascript
231+
{
232+
configureFailPoint: "failCommand",
233+
mode: {times: 1},
234+
data: {
235+
failCommands: ["find"],
236+
errorLabels: ["RetryableError", "SystemOverloadedError"],
237+
errorCode: 91
238+
}
239+
}
240+
```
241+
242+
3. Via the command monitoring CommandFailedEvent, configure a fail point with error code `91` (ShutdownInProgress) and
243+
the `RetryableError` label:
244+
245+
```javascript
246+
{
247+
configureFailPoint: "failCommand",
248+
mode: "alwaysOn",
249+
data: {
250+
failCommands: ["find"],
251+
errorLabels: ["RetryableError"],
252+
errorCode: 91
253+
}
254+
}
255+
```
256+
257+
Configure the second fail point command only if the failed event is for the first error configured in step 2.
258+
259+
4. Attempt a `findOne` operation on any record for any database and collection. Expect the `findOne` to fail with a
260+
server error. Assert that `MAX_ADAPTIVE_RETRIES + 1` attempts were made.
261+
262+
5. Disable the fail point:
263+
264+
```javascript
265+
{
266+
configureFailPoint: "failCommand",
267+
mode: "off"
268+
}
269+
```
270+
220271
## Changelog
221272

222273
- 2026-03-31: Add additional prose test for overload retargeting.

source/retryable-writes/tests/README.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -419,6 +419,54 @@ to cover the same sequence of events.
419419
}
420420
```
421421

422+
#### Case 4: Test that drivers set the maximum number of retries for all retryable write errors when an overload error is encountered
423+
424+
1. Create a client with `retryWrites=true`.
425+
426+
2. Configure a fail point with error code `91` (ShutdownInProgress) with the `RetryableError` and
427+
`SystemOverloadedError` error labels:
428+
429+
```javascript
430+
{
431+
configureFailPoint: "failCommand",
432+
mode: {times: 1},
433+
data: {
434+
failCommands: ["insert"],
435+
errorLabels: ["RetryableError", "SystemOverloadedError"],
436+
errorCode: 91
437+
}
438+
}
439+
```
440+
441+
3. Via the command monitoring CommandFailedEvent, configure a fail point with error code `91` (ShutdownInProgress) and
442+
the `RetryableWriteError` and `RetryableError` labels:
443+
444+
```javascript
445+
{
446+
configureFailPoint: "failCommand",
447+
mode: "alwaysOn",
448+
data: {
449+
failCommands: ["insert"],
450+
errorLabels: ["RetryableError", "RetryableWriteError"],
451+
errorCode: 91
452+
}
453+
}
454+
```
455+
456+
Configure the second fail point command only if the failed event is for the first error configured in step 2.
457+
458+
4. Attempt an `insertOne` operation on any record for any database and collection. Expect the `insertOne` to fail with a
459+
server error. Assert that `MAX_ADAPTIVE_RETRIES + 1` attempts were made.
460+
461+
5. Disable the fail point:
462+
463+
```javascript
464+
{
465+
configureFailPoint: "failCommand",
466+
mode: "off"
467+
}
468+
```
469+
422470
## Changelog
423471

424472
- 2026-04-02: Fix test for error propagation behavior when multiple errors are encountered.

0 commit comments

Comments
 (0)