DRIVERS-3535 - Client Backpressure with baseBackoffMS

source/client-backpressure/client-backpressure.md

@@ -129,12 +129,16 @@ rules:
         - To retry `runCommand`, both [retryWrites](../retryable-writes/retryable-writes.md#retrywrites) and
             [retryReads](../retryable-reads/retryable-reads.md#retryreads) MUST be enabled. See
             [Why must both `retryWrites` and `retryReads` be enabled to retry runCommand?](client-backpressure.md#why-must-both-retrywrites-and-retryreads-be-enabled-to-retry-runcommand)
-3. If the request is eligible for retry (as outlined in step 2 above), the client MUST apply exponential backoff
-    according to the following formula: `backoff = jitter * min(MAX_BACKOFF, BASE_BACKOFF * 2^(attempt - 1))`
-    - `jitter` is a random jitter value between 0 and 1.
-    - `BASE_BACKOFF` is constant 100ms.
-    - `MAX_BACKOFF` is 10000ms.
-    - This results in delays of 100ms and 200ms before accounting for jitter.
+3. If the request is eligible for retry (as outlined in step 2 above), the client MUST apply backoff according to the
+    following formula: `backoff = jitter * min(MAX_BACKOFF, BASE_BACKOFF * 2^attempt)`
+    1. `jitter` is a random jitter value between 0 and 1. This averages out to `0.5`.
+    2. `MAX_BACKOFF` is 10000ms.
+    3. `BASE_BACKOFF` is constant 100ms.
+    4. `attempt` is the retry number. The first retry is `attempt = 1`, the second is `attempt = 2`, and so on.
+    5. This results in delays of 100ms and 200ms after accounting for the average jitter.
+    6. If `baseBackoffMS` is present on the error and has a positive value, the client MUST use that value instead of
+        `BASE_BACKOFF`. `baseBackoffMS` represents a server-supplied base backoff to use in place of the driver's
+        default.
 4. If the request is eligible for retry (as outlined in step 2 above) and `enableOverloadRetargeting` is enabled, the
     client MUST add the previously used server's address to the list of deprioritized server addresses for
     [server selection](../server-selection/server-selection.md). Drivers MUST expose `enableOverloadRetargeting` as a
@@ -214,8 +218,11 @@ def execute_command_retryable(command, ...):
 
             if is_overload:
                 jitter = random.random() # Random float between [0.0, 1.0).
-                backoff = jitter * min(MAX_BACKOFF, BASE_BACKOFF * 2 ** (attempt - 1))
-
+                base_backoff = BASE_BACKOFF
+                # If present on the error, baseBackoffMS overrides the base backoff
+                if exc.base_backoff_ms:
+                    base_backoff = exc.base_backoff_ms / 1000 # Convert from milliseconds to seconds
+                backoff = jitter * min(MAX_BACKOFF, base_backoff * 2 ** attempt)
                 # If the delay exceeds the deadline, bail early.
                 if _csot.get_timeout():
                     if time.monotonic() + backoff > _csot.get_deadline():
@@ -433,6 +440,11 @@ to understand and configure.
 
 ## Changelog
 
+- 2026-07-08: Update exponential backoff formula to use the attempt number as the exponent instead of one less than the
+    attempt number.
+
+- 2026-06-16: Add support for baseBackoffMS backoff calculation.
+
 - 2026-04-14: Clarify correct retry behavior when a mix of overload and non-overload errors are encountered.
 
 - 2026-03-30: Introduce phase 1 support without token buckets.

source/client-backpressure/tests/README.md

@@ -53,10 +53,10 @@ executed against a MongoDB 4.4+ server that has enabled the `configureFailPoint`
     6. Compare the time between the two runs.
 
         ```python
-        assertTrue(absolute_value(with_backoff_time - (no_backoff_time + 0.3 seconds)) < 0.3 seconds)
+        assertTrue(absolute_value(with_backoff_time - (no_backoff_time + 0.6 seconds)) < 0.6 seconds)
         ```
 
-        The sum of 2 backoffs is 0.3 seconds. There is a 0.3-second window to account for potential variance between the
+        The sum of 2 backoffs is 0.6 seconds. There is a 0.6-second window to account for potential variance between the
         two runs.
 
 #### Test 2: REMOVED
@@ -119,3 +119,66 @@ option.
 5. Assert that the raised error contains both the `RetryableError` and `SystemOverloadedError` error labels.
 
 6. Assert that the total number of started commands is `maxAdaptiveRetries` + 1 (2).
+
+#### Test 5: Overload Errors with baseBackoffMS override base backoff
+
+Drivers SHOULD test that overload errors with `baseBackoffMS` override the default backoff duration. This test MUST be
+executed against a MongoDB 9.0+ server that has enabled the `configureFailPoint` command with the `errorLabels` option.
+
+1. Let `client` be a `MongoClient`.
+
+2. Let `coll` be a collection.
+
+3. Configure the random number generator used for exponential backoff jitter to always return a number as close as
+    possible to `1`.
+
+4. Configure the following failPoint:
+
+    ```javascript
+        {
+            configureFailPoint: 'failCommand',
+            mode: 'alwaysOn',
+            data: {
+                failCommands: ['insert'],
+                errorCode: 462, // IngressRequestRateLimitExceeded
+                errorLabels: ['SystemOverloadedError', 'RetryableError']
+            }
+        }
+    ```
+
+5. Insert the document `{ a: 1 }`. Expect that the command errors. Measure the duration of the command execution.
+
+    ```javascript
+       const start = performance.now();
+       expect(
+        await coll.insertOne({ a: 1 }).catch(e => e)
+       ).to.be.an.instanceof(MongoServerError);
+       const end = performance.now();
+    ```
+
+6. Run the following command to set up `baseBackoffMS` on overload errors.
+
+    ```python
+    client.admin.command("setParameter", 1, externalClientBaseBackoffMS=50)
+    ```
+
+7. Execute step 5 again.
+
+8. Assert that the server attached `baseBackoffMS` to the error and that the driver parsed it.
+
+9. Run the following command to disable `baseBackoffMS` on overload errors.
+
+    ```python
+    client.admin.command("setParameter", 1, externalClientBaseBackoffMS=0)
+    ```
+
+10. Assert absolute bounds on each run's duration.
+
+    ```python
+    assertGreaterEqual(exponential_backoff_time, 0.6)
+    assertGreaterEqual(with_base_backoff_ms_time, 0.3)
+    assertLess(with_base_backoff_ms_time, 0.6)
+    ```
+
+    A run can never be faster than the sum of its backoffs. With jitter pinned to 1, the default backoffs are
+    `0.2 + 0.4 = 0.6s` and the `baseBackoffMS=50` backoffs are `0.1 + 0.2 = 0.3s`.

source/mongodb-handshake/handshake.md

@@ -55,6 +55,10 @@ Drivers MUST use the `OP_MSG` protocol for all handshakes if their minWireVersio
 MUST use legacy hello for the first message of the initial handshake, and include `helloOk:true` in the handshake
 request.
 
+Drivers MUST include `backpressure: "2"` in their handshake request in order to explicitly version their supported
+version of the client backpressure specification. The value of `backpressure` MUST be the string `"2"` and not a literal
+number `2`.
+
 If the legacy handshake response includes `helloOk: true`, then subsequent topology monitoring commands MUST use the
 `hello` command. If the legacy handshake response does not include `helloOk: true`, then subsequent topology monitoring
 commands MUST use the legacy hello command. Additionally, note that if the server does not understand `OP_MSG`, the
@@ -85,7 +89,7 @@ if stable_api_configured or client_options.load_balanced:
     cmd = {"hello": 1}
 else:
     cmd = {"legacy hello": 1, "helloOk": 1}
-cmd["backpressure"] = True
+cmd["backpressure"] = "2"
 cmd["client"] = client_metadata
 if client_options.compressors:
     cmd["compression"] = client_options.compressors
@@ -561,6 +565,7 @@ support the `hello` command, the `helloOk: true` argument is ignored and the leg
 
 ## Changelog
 
+- 2026-06-25: Clarify the client backpressure component of the handshake.
 - 2026-06-11: Clarify that there is no new behavior as a result of only using OP_MSG for all handshakes.
 - 2026-06-05: Use OP_MSG for all handshakes.
 - 2025-09-04: Clarify that drivers do not append the same metadata multiple times.

source/mongodb-handshake/tests/README.md

@@ -499,7 +499,7 @@ Before each test case, perform the setup.
 
 8. Assert that `initialClientMetadata` is identical to `updatedClientMetadata`.
 
-### Test 9: Handshake documents include `backpressure: true`
+### Test 9: Handshake documents include `backpressure: "2"`
 
 These tests require a mechanism for observing handshake documents sent to the server.
 
@@ -511,4 +511,4 @@ These tests require a mechanism for observing handshake documents sent to the se
 
 3. Assert that for every handshake document intercepted:
 
-    1. The document has a field `backpressure` whose value is `true`.
+    1. The document has a field `backpressure` whose value is `"2"`.

source/transactions-convenient-api/tests/README.md

@@ -105,13 +105,14 @@ Drivers should test that retries within `withTransaction` do not occur immediate
         ```
 5. Compare the durations of the two runs.
     ```python
-    assertTrue(absolute_value(with_backoff_time - (no_backoff_time + 1.8 seconds)) < 0.5 seconds)
+    assertTrue(absolute_value(with_backoff_time - (no_backoff_time + 3.6 seconds)) < 0.5 seconds)
     ```
-    The sum of 13 backoffs is roughly 1.8 seconds. There is a half-second window to account for potential variance
+    The sum of 13 backoffs is roughly 3.6 seconds. There is a half-second window to account for potential variance
     between the two runs.
 
 ## Changelog
 
+- 2206-07-08: Update Backoff test to use updated exponential formula.
 - 2026-04-02: [DRIVERS-3436](https://github.com/mongodb/specifications/pull/1920) Refine withTransaction timeout error
     wrapping semantics and label propagation in spec and prose tests
 - 2026-03-03: Clarify exponential backoff jitter upper bound.

source/transactions-convenient-api/transactions-convenient-api.md

@@ -123,7 +123,7 @@ This method should perform the following sequence of actions:
 
 2. If `transactionAttempt` > 0:
 
-    1. Calculate `backoffMS` to be `jitter * min(BACKOFF_INITIAL * 1.5 ** (transactionAttempt - 1), BACKOFF_MAX)`. If
+    1. Calculate `backoffMS` to be `jitter * min(BACKOFF_INITIAL * 1.5 ** (transactionAttempt), BACKOFF_MAX)`. If
         elapsed time + `backoffMS` > `TIMEOUT_MS`, then propagate the previously encountered error to the caller of
         `withTransaction` as per [timeout error propagation](#timeout-error-propagation) and return immediately.
         Otherwise, sleep for `backoffMS`.
@@ -218,7 +218,7 @@ withTransaction(callback, options) {
 
     retryTransaction: while (true) {
         if (transactionAttempt > 0) {
-            var backoff = Math.random() * min(BACKOFF_INITIAL * 1.5 ** (transactionAttempt - 1), 
+            var backoff = Math.random() * min(BACKOFF_INITIAL * 1.5 ** (transactionAttempt), 
                                               BACKOFF_MAX);
 
             if (Date.now() + backoff - startTime >= timeout) {
@@ -446,6 +446,9 @@ provides an implementation of a technique already described in the MongoDB 4.0 d
 
 ## Changelog
 
+- 2026-07-08: Update exponential backoff formula to use the attempt number as the exponent instead of one less than the
+    attempt number.
+
 - 2026-04-02: [DRIVERS-3436](https://github.com/mongodb/specifications/pull/1920) Refine withTransaction timeout error
     wrapping semantics and label propagation in spec and prose tests.