Skip to content

Commit 6ec66c4

Browse files
committed
Raise the e2e execution rate cap above other scenarios' usage
The e2e worker's per-org hourly cap of 3 was low enough that ordinary scenarios (toolkits-mcp, no-auth-connection, connect-handoff) tripped the backstop mid-test once the suite ran them against the shared worker. Lift it to 20 (still exhaustible by the backstop scenario) and move the value into one shared constant consumed by both the boot env and the test.
1 parent 78f7b83 commit 6ec66c4

3 files changed

Lines changed: 23 additions & 5 deletions

File tree

e2e/cloud/mcp-execution-limits.test.ts

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ import {
2222
} from "../../apps/cloud/src/engine/execution-limit-messages";
2323
import { scenario } from "../src/scenario";
2424
import { Autumn, Billing, Mcp, Target } from "../src/services";
25+
import { E2E_EXECUTION_RATE_LIMIT } from "../setup/execution-limits";
2526
import type { Identity } from "../src/target";
2627

2728
const emailOf = (identity: Identity): string => identity.credentials?.email ?? identity.label;
@@ -38,8 +39,9 @@ const orgIdOf = (bearer: string): string => {
3839

3940
// The e2e worker is booted with a small EXECUTION_RATE_LIMIT_PER_HOUR so the
4041
// backstop is reachable with real executions (cloud.boot.ts). The prod cap of
41-
// 1000/hour can't be. Kept in sync with that env value.
42-
const RATE_LIMIT = 3;
42+
// 1000/hour can't be. One shared constant (setup/execution-limits.ts) keeps
43+
// this and the boot env from drifting apart.
44+
const RATE_LIMIT = E2E_EXECUTION_RATE_LIMIT;
4345

4446
scenario(
4547
"Billing · an org out of executions is blocked before the code ever runs",

e2e/setup/cloud.boot.ts

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import { createEmulator } from "@executor-js/emulate";
1212

1313
import { bootProcesses, waitForHttp } from "./boot";
1414
import { AUTUMN_PLAN_SEED } from "./autumn-plans";
15+
import { E2E_EXECUTION_RATE_LIMIT } from "./execution-limits";
1516

1617
export const cloudDir = fileURLToPath(new URL("../../apps/cloud/", import.meta.url));
1718

@@ -92,9 +93,10 @@ export const bootCloud = async (options: CloudBootOptions): Promise<CloudBooted>
9293
ALLOW_LOCAL_NETWORK: "true",
9394
// Shrink the per-org hourly execution cap (prod default 1000) to a number
9495
// the rate-limit-backstop scenario can actually exhaust with real
95-
// executions. Reaches the worker via CLOUDFLARE_INCLUDE_PROCESS_ENV, same
96-
// as ALLOW_LOCAL_NETWORK.
97-
EXECUTION_RATE_LIMIT_PER_HOUR: "3",
96+
// executions — but see execution-limits.ts: it must stay above every other
97+
// scenario's per-org execute count. Reaches the worker via
98+
// CLOUDFLARE_INCLUDE_PROCESS_ENV, same as ALLOW_LOCAL_NETWORK.
99+
EXECUTION_RATE_LIMIT_PER_HOUR: String(E2E_EXECUTION_RATE_LIMIT),
98100
// Throwaway PGlite on its own port + dir so it never fights `bun dev`.
99101
DEV_DB_PORT: String(options.dbPort),
100102
DEV_DB_PATH: dbPath,

e2e/setup/execution-limits.ts

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
// The e2e worker's per-org hourly execution cap (EXECUTION_RATE_LIMIT_PER_HOUR).
2+
// One constant with two consumers, the boot recipe env (cloud.boot.ts) and the
3+
// rate-limit backstop scenario (cloud/mcp-execution-limits.test.ts), so they
4+
// cannot drift apart.
5+
//
6+
// Picking the value is a squeeze from both sides. It must be LOW enough that
7+
// the backstop scenario can exhaust it with real sequential executions (prod's
8+
// 1000/hour cannot be reached in a test), and HIGH enough that no other
9+
// scenario trips it: the counter is per organization and every `execute` a
10+
// scenario runs counts against its org, so this must exceed the busiest
11+
// single-org scenario's execute count (currently toolkits-mcp at ~8) with
12+
// comfortable headroom. If a scenario ever fails with the rate-limit backstop
13+
// message, it outgrew this cap: raise it here, never in the boot env alone.
14+
export const E2E_EXECUTION_RATE_LIMIT = 20;

0 commit comments

Comments
 (0)