Commit 9075ad2
fix: continue benchmark looping when some tests fail but timing markers exist
Previously, the benchmark loop stopped immediately when Maven returned
non-zero (any test failure). This was too aggressive because:
- Generated tests may have some failures
- Passing tests still produce valid timing markers
- We need multiple loops for accurate measurements
Now the loop continues if timing markers are present, only stopping when:
- No timing markers are found (all tests failed)
- Target duration is reached
- Max loops is reached
This allows proper multi-loop benchmarking even when some generated
tests fail, improving measurement accuracy.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>1 parent 79fbd2b commit 9075ad2
1 file changed
Lines changed: 25 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
640 | 640 | | |
641 | 641 | | |
642 | 642 | | |
| 643 | + | |
| 644 | + | |
643 | 645 | | |
644 | | - | |
645 | | - | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
646 | 657 | | |
647 | 658 | | |
648 | 659 | | |
| |||
840 | 851 | | |
841 | 852 | | |
842 | 853 | | |
843 | | - | |
| 854 | + | |
844 | 855 | | |
845 | | - | |
846 | | - | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
847 | 867 | | |
848 | 868 | | |
849 | 869 | | |
| |||
0 commit comments