fix(telemetry): classify 5xx + rate-limit errors; harden DoS test thresholds

Two follow-ups from the v1.4.0 pre-merge ritual, both surfaced as
[KNOWN GAP] tests in earlier rounds. Neither blocks the merge; both
strengthen the test suite and improve telemetry diagnostics.

## (1) classifyError ERROR_PATTERNS coverage

Round 2 fuzz testing pinned that real provider errors like
"APIError: Service unavailable (503)" and "Rate limit exceeded.
Retry after 60s" classified as `"unknown"` because ERROR_PATTERNS
only matched the prefixed forms ("status code: 5", "http 429").
That defeated the diagnostic value of `agent_outcome.error_class`
for two of the most common provider failure modes.

Adds these phrases to the existing `http_error` class so they
classify into the right bucket:

  "service unavailable", "rate limit", "rate_limit",
  "retry after", "too many requests", "503", "502", "504"

Why `http_error` and not a new `rate_limit` class: the upstream
event schema uses `http_error` for any non-2xx HTTP failure;
rate-limit responses ARE HTTP 429s, just stated in different
prose. Keeping the same bucket avoids fragmenting telemetry
dashboards.

Tests:
- v140-merge-fuzz.test.ts: replace the 2 `[KNOWN GAP]` tests
  with positive assertions:
  - "APIError 503 classifies as 'http_error' (Service unavailable)"
  - "rate-limit error classifies as 'http_error' (Retry after)"

## (2) DoS test threshold hardening

Round 3 chaos tests asserted regex DoS resistance with fixed
`<100ms` wall-clock thresholds. CI runners under load can flake
on absolute thresholds (slow scheduling, contended I/O, etc.),
even though the actual runtime is fine relative to a baseline.

Replaces fixed thresholds with a relative-budget approach:
- Take the min of 5 baseline runs on a trivial input (filters
  scheduler jitter, 1ms floor).
- Run a single warmup invocation before the timed adversarial
  call (so JIT-cold doesn't penalize first-run).
- Adversarial input must complete within `50 × baseline`.

Why 50× and not the original implicit 100ms-on-a-fast-runner
(maybe ~200×): catastrophic backtracking on these regexes
would be 1000×+ baseline, so 50× is a generous yet still-strict
budget. Real measurements stayed within ~3× baseline locally.

Tests still pass 5/5 runs locally. The new code prints a
descriptive error message ("X took Yms (baseline Zms × 50 budget
= Wms)") so a future regression has a clear diagnostic.

Combined verification:
- All 4 telemetry/test/upstream suites pass: 87 tests, 3593
  assertions, 0 fail.
- Chaos suite alone: 24/24 pass × 5 consecutive runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: anandgupta42

packages/opencode/src/altimate/telemetry/index.ts

@@ -925,6 +925,17 @@ export namespace Telemetry {
         "http 429",
         "http 451",
         "http 403",
+        // R3 audit: real provider 5xx + rate-limit messages don't carry "status code:" prefix.
+        // Add bare phrases so 503 / 502 / 504 + Retry-After / "rate limit exceeded" classify
+        // out of "unknown" into http_error (preserving diagnostic specificity in agent_outcome.error_class).
+        "service unavailable",
+        "rate limit",
+        "rate_limit",
+        "retry after",
+        "too many requests",
+        "503",
+        "502",
+        "504",
       ],
     },
     // altimate_change end

packages/opencode/test/upstream/v140-merge-chaos.test.ts

@@ -26,33 +26,65 @@ import path from "path"
 const repoRoot = path.resolve(import.meta.dir, "..", "..", "..", "..")
 
 // ---------- Regex DoS resistance ----------
+//
+// We measure adversarial-input runtime relative to a baseline of a benign
+// short input. This is more robust against CI runner variability than a
+// fixed wall-clock threshold: a slow runner that takes 50ms on the baseline
+// gets a 50× budget (2500ms) for adversarial input; a fast runner that takes
+// 0.5ms gets a 25ms budget. Either way, catastrophic-backtracking regressions
+// (which would be 100×–1000× slower than baseline) get caught.
+//
+// We also include a single warmup invocation before each timed run so JIT
+// optimization doesn't artificially penalize the first call.
 describe("v1.4.0 chaos — maskString regex DoS resistance", () => {
-  test("10000 backslashes processed in <100ms", () => {
-    const input = '"' + "\\\\".repeat(10000) + '"'
+  // Baseline: trivial input. Computed once per describe block.
+  const BASELINE_INPUT = "no secrets here, just plain text"
+  const BUDGET_MULTIPLIER = 50 // adversarial input must be at most 50× baseline
+
+  function baselineMs(): number {
+    // Take min of 5 runs to filter out scheduler jitter
+    const samples: number[] = []
+    for (let i = 0; i < 5; i++) {
+      const t0 = performance.now()
+      Telemetry.maskString(BASELINE_INPUT)
+      samples.push(performance.now() - t0)
+    }
+    // 1ms floor so we don't divide-by-zero on very-fast runners
+    return Math.max(1, Math.min(...samples))
+  }
+
+  function probe(input: string, label: string): void {
+    Telemetry.maskString(input) // warmup
     const t0 = performance.now()
     Telemetry.maskString(input)
-    expect(performance.now() - t0).toBeLessThan(100)
+    const dt = performance.now() - t0
+    const baseline = baselineMs()
+    const budget = baseline * BUDGET_MULTIPLIER
+    if (dt > budget) {
+      throw new Error(
+        `DoS regression: "${label}" took ${dt.toFixed(2)}ms (baseline ${baseline.toFixed(2)}ms × ${BUDGET_MULTIPLIER} budget = ${budget.toFixed(2)}ms)`,
+      )
+    }
+  }
+
+  test("10000 backslashes — runtime stays within 50× baseline", () => {
+    const input = '"' + "\\\\".repeat(10000) + '"'
+    expect(() => probe(input, "10000 backslashes")).not.toThrow()
   })
 
-  test("alternating quotes (10k) processed in <100ms", () => {
+  test("alternating quotes (10k) — runtime stays within 50× baseline", () => {
     const input = '""""'.repeat(2500)
-    const t0 = performance.now()
-    Telemetry.maskString(input)
-    expect(performance.now() - t0).toBeLessThan(100)
+    expect(() => probe(input, "alternating quotes")).not.toThrow()
   })
 
-  test("evil escape pattern (1k×3) processed in <100ms", () => {
+  test("evil escape pattern (1k×3) — runtime stays within 50× baseline", () => {
     const input = '"' + "\\\\.".repeat(1000) + '"'
-    const t0 = performance.now()
-    Telemetry.maskString(input)
-    expect(performance.now() - t0).toBeLessThan(100)
+    expect(() => probe(input, "evil escape pattern")).not.toThrow()
   })
 
-  test("1000 concurrent sk-ant prefixes processed in <100ms", () => {
+  test("1000 concurrent sk-ant prefixes — runtime stays within 50× baseline", () => {
     const input = ("sk-ant-" + "a".repeat(20) + " ").repeat(1000)
-    const t0 = performance.now()
-    Telemetry.maskString(input)
-    expect(performance.now() - t0).toBeLessThan(100)
+    expect(() => probe(input, "sk-ant prefix flood")).not.toThrow()
   })
 })
 

packages/opencode/test/upstream/v140-merge-fuzz.test.ts

@@ -232,31 +232,32 @@ describe("v1.4.0 failure injection — synthetic provider errors flow through di
     expect(out.error_class).not.toBe("unknown")
   })
 
-  // KNOWN GAP: classifyError patterns lack "503" / "Service unavailable" /
-  // "Rate limit" / "Retry after" keywords. Real provider errors get
-  // classified as "unknown" — diagnostic value lost. Fix: extend
-  // ERROR_PATTERNS in altimate/telemetry/index.ts. Tests below pin the
-  // current behavior so a future fix flips them automatically.
-  test("[KNOWN GAP] APIError 503 classifies as 'unknown' (pattern coverage hole)", () => {
+  // Once gapped: classifyError patterns lacked "503" / "Service unavailable" /
+  // "Rate limit" / "Retry after" / "Too many requests" keywords, so real
+  // provider 5xx and rate-limit errors classified as "unknown" — defeating
+  // the diagnostic value of agent_outcome.error_class. ERROR_PATTERNS now
+  // includes those phrases under the http_error class. Tests below pin
+  // the new classifications.
+  test("APIError 503 classifies as 'http_error' (Service unavailable)", () => {
     const out = Telemetry.deriveAgentOutcomeReason({
       outcome: "error",
       lastToolName: "edit",
       lastMessageError: "APIError: Service unavailable (503)",
       abortReason: null,
       lastErrorClass: "",
     })
-    expect(out.error_class).toBe("unknown") // flip when pattern added
+    expect(out.error_class).toBe("http_error")
     expect(out.reason).toContain("Service unavailable")
   })
 
-  test("[KNOWN GAP] rate-limit error classifies as 'unknown' (pattern coverage hole)", () => {
+  test("rate-limit error classifies as 'http_error' (Retry after)", () => {
     const out = Telemetry.deriveAgentOutcomeReason({
       outcome: "error",
       lastToolName: null,
       lastMessageError: "Rate limit exceeded. Retry after 60s",
       abortReason: null,
       lastErrorClass: "",
     })
-    expect(out.error_class).toBe("unknown") // flip when pattern added
+    expect(out.error_class).toBe("http_error")
   })
 })