Skip to content

[FIX][JS] capturePerf shouldn't break when we have an external runner (batch size = 1)#1509

Merged
Saga4 merged 2 commits into
mainfrom
fix/js-capture-perf-for-external-runner
Feb 17, 2026
Merged

[FIX][JS] capturePerf shouldn't break when we have an external runner (batch size = 1)#1509
Saga4 merged 2 commits into
mainfrom
fix/js-capture-perf-for-external-runner

Conversation

@mohammedahmed18
Copy link
Copy Markdown
Contributor

@mohammedahmed18 mohammedahmed18 commented Feb 17, 2026

this will fix the false positive issue we are having caused by the jest runner not actually executing the function inside the test case because capturePerf exited at the start of the first loop before calling fn(..), which makes it look like it's much much faster than the original code

@Saga4 Saga4 merged commit cc678ba into main Feb 17, 2026
28 of 30 checks passed
@Saga4 Saga4 deleted the fix/js-capture-perf-for-external-runner branch February 17, 2026 21:17
@claude
Copy link
Copy Markdown
Contributor

claude Bot commented Feb 17, 2026

PR Review Summary

Prek Checks

✅ All checks pass — ruff check and ruff format reported no issues.

Mypy

✅ No type errors found in changed Python files (codeflash/version.py).

Code Review

✅ No critical issues found.

Changes reviewed:

  • packages/codeflash/runtime/capture.js: Adds !hasExternalLoopRunner guards to 5 early-exit conditions in the capturePerf loop. When an external loop runner (Jest) manages batching, batchSize is already set to 1, so the loop body executes exactly once. The guards prevent the early-exit checks (shared time limit, stability, max loops, error) from incorrectly breaking out before the single execution completes. Logic is sound — the async path (_capturePerfAsync) doesn't need the same guards since it's only entered when batchSize > 1, which can't happen with an external runner.
  • codeflash/version.py: Auto-generated version bump (no logic change).

Test Coverage

File Stmts Miss Coverage
codeflash/version.py 1 0 100%
packages/codeflash/runtime/capture.js N/A (JS)

Overall Python coverage: 79% (2374 passed, 8 failed, 57 skipped)

Notes:

  • The primary change is in capture.js, a JavaScript runtime file not covered by Python pytest/coverage tooling.
  • The 8 test failures (test_tracer.py) are pre-existing on main and unrelated to this PR.
  • capturePerf has no dedicated JS unit tests (packages/codeflash/runtime/__tests__/ only contains dynamic-env-vars.test.js). Consider adding JS tests for this critical benchmarking function in a follow-up.

Last updated: 2026-02-17

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants