test(kernel-node-runtime): Fix flaky remote-comms E2E tests (#927)

## Description

Two E2E tests in `remote-comms.test.ts` were consistently flaky on
`main`, failing more often than passing:

1. **"rejects new messages when queue reaches MAX_QUEUE limit"** — timed
out at 120s
2. **"resolves promise after reconnection when retries have not been
exhausted"** — failed with `URL redemption timed out after 8000ms`

**Root cause:** Both tests used `testBackoffOptions` with `ackTimeoutMs:
2_000`. The `RemoteHandle` gives up after `ackTimeoutMs × (MAX_RETRIES +
1) = 8s`, which is too short for kernel2 to restart (new DB connection +
kernel init + libp2p + relay connection + vat launch).

## Changes

- Increase `ackTimeoutMs` to `5_000` for both tests, giving a 20s
give-up window instead of 8s
- Extract named options objects (`queueTestOptions`, `reconnectOptions`)
for clarity and consistency between `setupAliceAndBob` and
`restartKernelAndReloadVat` calls
- Restructure the reconnection test to establish the connection first
(completing URL redemption), then stop kernel2 before sending the test
message — eliminating both the URL redemption race and the
non-deterministic message delivery race
- Remove unnecessary `delay()` calls that were masking the underlying
timing issue

## Testing

Both tests pass reliably after the fix. Verified by running the full
`remote-comms.test.ts` E2E suite twice — all 67 E2E tests pass across
all 11 test files (including both previously-flaky tests).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Low risk: changes are limited to E2E test timing/configuration and
message sequencing; main risk is masking real regressions by increasing
timeouts or altering test ordering.
> 
> **Overview**
> Stabilizes `remote-comms.test.ts` E2E coverage by introducing per-test
option objects that override `ackTimeoutMs` (to `5_000`) and related
rate-limit settings, and then using those same options consistently
across `setupAliceAndBob` and `restartKernelAndReloadVat`.
> 
> Reworks the "reconnect without exhausting retries" test to first
establish a connection (complete URL redemption), then stop the remote
kernel and queue the message while it’s definitely offline, removing
race-prone delays and making the queued-message reconnection behavior
deterministic.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
11d93ec. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: sirtimid

packages/kernel-node-runtime/test/e2e/remote-comms.test.ts

@@ -569,20 +569,25 @@ describe.sequential('Remote Communications E2E', () => {
     it(
       'rejects new messages when queue reaches MAX_QUEUE limit',
       async () => {
-        // Use high rate limits to avoid rate limiting interference with queue limit test
+        // Use high rate limits to avoid rate limiting interference with queue limit test.
         // maxConnectionAttemptsPerMinute is needed because async kernel service invocations
-        // can cause multiple concurrent connection attempts when processing many messages
+        // can cause multiple concurrent connection attempts when processing many messages.
+        // ackTimeoutMs must be long enough for kernel2 to restart + reconnect before
+        // the RemoteHandle gives up (give-up fires after ackTimeoutMs × (MAX_RETRIES + 1)).
+        const queueTestOptions = {
+          ...testBackoffOptions,
+          maxMessagesPerSecond: 500,
+          maxConnectionAttemptsPerMinute: 500,
+          ackTimeoutMs: 5_000,
+        };
+
         const { aliceRef, bobURL } = await setupAliceAndBob(
           kernel1,
           kernel2,
           kernelStore1,
           kernelStore2,
           testRelays,
-          {
-            maxMessagesPerSecond: 500,
-            maxConnectionAttemptsPerMinute: 500,
-            ...testBackoffOptions,
-          },
+          queueTestOptions,
         );
 
         await sendRemoteMessage(kernel1, aliceRef, bobURL, 'hello', ['Alice']);
@@ -607,7 +612,7 @@ describe.sequential('Remote Communications E2E', () => {
           false,
           testRelays,
           bobConfig,
-          testBackoffOptions,
+          queueTestOptions,
         );
         // eslint-disable-next-line require-atomic-updates
         kernel2 = restartResult.kernel;
@@ -1069,44 +1074,49 @@ describe.sequential('Remote Communications E2E', () => {
     it(
       'resolves promise after reconnection when retries have not been exhausted',
       async () => {
+        // Use a longer ACK timeout so the RemoteHandle doesn't give up before
+        // kernel2 can restart and reconnect (give-up = ackTimeoutMs × 4).
+        const reconnectOptions = {
+          ...testBackoffOptions,
+          ackTimeoutMs: 5_000,
+        };
+
         const { aliceRef, bobURL } = await setupAliceAndBob(
           kernel1,
           kernel2,
           kernelStore1,
           kernelStore2,
           testRelays,
-          testBackoffOptions,
+          reconnectOptions,
         );
 
-        // Send a message that creates a promise with remote as decider
+        // Establish the connection and complete URL redemption first so that
+        // the subsequent message doesn't race URL redemption against kernel2.stop().
+        await sendRemoteMessage(kernel1, aliceRef, bobURL, 'hello', ['Alice']);
+
+        // Stop kernel2 so the next message is guaranteed to be pending
+        await kernel2.stop();
+
+        // Send a message while kernel2 is down - it will be queued
         const messagePromise = kernel1.queueMessage(
           aliceRef,
           'sendRemoteMessage',
           [bobURL, 'hello', ['Alice']],
         );
 
-        // Stop kernel2 before it can respond
-        await kernel2.stop();
-
-        // Wait a bit for connection loss to be detected
-        await delay(100);
-
-        // Restart kernel2 quickly (before max retries, since default is infinite)
+        // Restart kernel2 (before max retries, since default is infinite)
         // The promise should remain unresolved and resolve normally after reconnection
         const bobConfig = makeRemoteVatConfig('Bob');
         const restartResult = await restartKernelAndReloadVat(
           dbFilename2,
           false,
           testRelays,
           bobConfig,
-          testBackoffOptions,
+          reconnectOptions,
         );
         // eslint-disable-next-line require-atomic-updates
         kernel2 = restartResult.kernel;
 
-        // Wait for reconnection
-        await delay(200);
-
         // The message should eventually be delivered and resolved
         // The promise was never rejected because retries weren't exhausted
         const result = await messagePromise;