Skip to content

fix(smoketests): raise devbox snapshot test timeouts to 120s - #816

Closed
jrvb-rl wants to merge 1 commit into
mainfrom
fix/snapshot-smoketest-timeout
Closed

fix(smoketests): raise devbox snapshot test timeouts to 120s#816
jrvb-rl wants to merge 1 commit into
mainfrom
fix/snapshot-smoketest-timeout

Conversation

@jrvb-rl

@jrvb-rl jrvb-rl commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

User description

Description

Raises the per-test Jest timeout on both tests in the devbox snapshots suite (tests/smoketests/snapshots.test.ts) from 30s to 120s.

Motivation

These two tests hard-coded a 30_000 per-test timeout that overrode the smoketest default of 300s (jest.config.tstestTimeout: runSmoketests ? 300000 : 120000). They were the only snapshot-flow tests capped at 30s.

Under concurrent CI load (smoke-tests.yml runs the full suite with --maxWorkers=800%), createAndAwaitRunning + snapshotDisk tail latency climbs well past 30s. Repro'd against dev:

Concurrency Max observed test duration
1 (sequential) 3.7s
10 9.7s
30 21.8s

The backend serves these operations in batches of ~4, adding ~3s of queued latency per tier — so the 30s cap breaches under enough simultaneous devbox creation (the failing run also had axon.test.ts at 83s). This matches the observed CI failure: snapshot devbox (30078 ms)Exceeded timeout of 30000 ms.

120s gives comfortable headroom while staying well under the 300s smoketest default, so a genuinely hung create/snapshot still fails fast.

Changes

  • tests/smoketests/snapshots.test.ts: snapshot devbox timeout 30s → 120s
  • tests/smoketests/snapshots.test.ts: launch devbox from snapshot (deprecated polling path) timeout 30s → 120s (same createAndAwaitRunning-under-load path)

Testing

  • Unit tests added
  • Integration tests added
  • Smoke Tests added/updated
  • Tested locally

Ran the edited test against dev: passes sequentially (10/10) and under 10- and 30-way parallelism (30/30, 60/60) — no failures, tail latency ~22s stays under the new 120s cap.

Breaking Changes

None.

Note

The timeout bump de-flakes CI but does not address the underlying ~4-wide batching / throughput ceiling on devbox create+snapshot, which may warrant a separate backend investigation.

Checklist

  • PR title follows Conventional Commits format
  • Documentation updated (if needed)
  • Breaking changes documented (if applicable)

🤖 Generated with Claude Code


CodeAnt-AI Description

Raise flaky devbox snapshot smoketest timeouts

What Changed

  • The two devbox snapshot smoke tests now allow up to 120 seconds instead of 30 seconds
  • Snapshot creation and launching from a snapshot no longer fail as often under CI load when these steps take longer than expected

Impact

✅ Fewer flaky snapshot smoke test failures
✅ More reliable CI runs for devbox snapshots
✅ Fewer timeout errors during busy test runs

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

The two tests in the `devbox snapshots` suite hard-coded a 30s per-test
timeout, overriding the 300s smoketest default in jest.config.ts. Under
concurrent CI load, devbox create+snapshot tail latency exceeds 30s
(measured up to ~22s at 30-way parallelism), causing flaky timeouts.
Raise both to 120s for headroom while still failing fast on a genuine hang.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codeant-ai

codeant-ai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

CodeAnt AI is reviewing your PR.

@codeant-ai

codeant-ai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@reflex-loop

reflex-loop Bot commented Jul 9, 2026

Copy link
Copy Markdown

🔄 Reflex agent status: Working

The agent is working on this pull request.

This PR was created by Reflex.

View the agent run →

This comment updates in place as the agent works.

@codeant-ai codeant-ai Bot added the size:XS This PR changes 0-9 lines, ignoring generated files label Jul 9, 2026
@codeant-ai

codeant-ai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

CodeAnt AI finished reviewing your PR.

@jrvb-rl
jrvb-rl enabled auto-merge (squash) July 9, 2026 02:36
@jrvb-rl
jrvb-rl requested review from gautam-rl and wall-rl July 9, 2026 02:36
@jrvb-rl

jrvb-rl commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

As discussed, closing this in favor of pushing out the underlying slowness.

@jrvb-rl jrvb-rl closed this Jul 9, 2026
auto-merge was automatically disabled July 9, 2026 18:30

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS This PR changes 0-9 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant