Run benchmarks in test mode by default instead of excluding them (#56418)#56418
Closed
rubennorte wants to merge 2 commits intofacebook:mainfrom
Closed
Run benchmarks in test mode by default instead of excluding them (#56418)#56418rubennorte wants to merge 2 commits intofacebook:mainfrom
rubennorte wants to merge 2 commits intofacebook:mainfrom
Conversation
meta-cla
bot
added
the
CLA Signed
|
@rubennorte has exported this pull request. If you are a Meta employee, you can view the originating Diff in D100464314. |
meta-codesync
bot
changed the title
rubennorte
force-pushed
the
export-D100464314
branch
from
9d1d352 to
d498b3b
Compare
rubennorte
added a commit
to rubennorte/react-native
that referenced
this pull request
Apr 11, 2026
…ebook#56418) Summary: Changelog: [internal] D92150163 excluded benchmark tests by default when running `yarn fantom` without `--benchmarks`. This was incorrect because it means benchmarks could silently break without being caught. This changes the behavior so: 1. By default (without `--benchmarks`), benchmarks run in test mode (single iteration for correctness only), ensuring they do not break. 2. With `--benchmarks`, benchmarks run in full benchmark mode (multiple iterations for performance measurement). Also renames `FANTOM_FORCE_TEST_MODE` to `FANTOM_RUN_BENCHMARKS` and `forceTestModeForBenchmarks` to `runBenchmarks` to better reflect the intent (opt-in to full benchmarks rather than opt-in to test mode). Differential Revision: D100464314
rubennorte
added a commit
to rubennorte/react-native
that referenced
this pull request
Apr 11, 2026
…ebook#56418) Summary: Pull Request resolved: facebook#56418 Changelog: [internal] D92150163 excluded benchmark tests by default when running `yarn fantom` without `--benchmarks`. This was incorrect because it means benchmarks could silently break without being caught. This changes the behavior so: 1. By default (without `--benchmarks`), benchmarks run in test mode (single iteration for correctness only), ensuring they do not break. 2. With `--benchmarks`, benchmarks run in full benchmark mode (multiple iterations for performance measurement). Also renames `FANTOM_FORCE_TEST_MODE` to `FANTOM_RUN_BENCHMARKS` and `forceTestModeForBenchmarks` to `runBenchmarks` to better reflect the intent (opt-in to full benchmarks rather than opt-in to test mode). Differential Revision: D100464314
rubennorte
force-pushed
the
export-D100464314
branch
from
d498b3b to
8ebc33e
Compare
Summary: Changelog: [internal] The `FANTOM_BENCHMARK_FILENAME_RE` and `FANTOM_BENCHMARK_SUITE_RE` regexes used the `g` (global) flag, which makes `.test()` stateful — it preserves `lastIndex` between calls. When `getFantomTestConfigs` is called for multiple test files in the same process, the stale `lastIndex` from a previous successful match causes `.test()` to miss the match on the next file, returning `false`. This results in benchmark files intermittently getting the default dev config (`isJsOptimized: false`, `dev: true`), causing them to fail at runtime with "Benchmarks should not be run in development mode". Removes the `g` flag from both regexes since they are used with `.test()` across different strings and should be stateless. Differential Revision: D100603612
…ebook#56418) Summary: Changelog: [internal] D92150163 excluded benchmark tests by default when running `yarn fantom` without `--benchmarks`. This was incorrect because it means benchmarks could silently break without being caught. This changes the behavior so: 1. By default (without `--benchmarks`), benchmarks run in test mode (single iteration for correctness only), ensuring they do not break. 2. With `--benchmarks`, benchmarks run in full benchmark mode (multiple iterations for performance measurement). Also renames `FANTOM_FORCE_TEST_MODE` to `FANTOM_RUN_BENCHMARKS` and `forceTestModeForBenchmarks` to `runBenchmarks` to better reflect the intent (opt-in to full benchmarks rather than opt-in to test mode). Reviewed By: sammy-SC Differential Revision: D100464314
rubennorte
force-pushed
the
export-D100464314
branch
from
8ebc33e to
e1deed8
Compare
|
This pull request has been merged in 7efb4b5. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Changelog: [internal]
D92150163 excluded benchmark tests by default when running
yarn fantomwithout--benchmarks. This was incorrect because it means benchmarks could silently break without being caught.This changes the behavior so:
--benchmarks), benchmarks run in test mode (single iteration for correctness only), ensuring they do not break.--benchmarks, benchmarks run in full benchmark mode (multiple iterations for performance measurement).Also renames
FANTOM_FORCE_TEST_MODEtoFANTOM_RUN_BENCHMARKSandforceTestModeForBenchmarkstorunBenchmarksto better reflect the intent (opt-in to full benchmarks rather than opt-in to test mode).Reviewed By: sammy-SC
Differential Revision: D100464314