⚡️ Speed up function discover_parameters_unittest by 252% in PR #753 (test_cache_revival)

[codeflash-ai]

⚡️ This pull request contains optimizations for PR #753

If you approve this dependent PR, these changes will be merged into the original PR branch test_cache_revival.

This PR will be automatically closed if the original PR is merged.

---

📄 252% (2.52x) speedup for discover_parameters_unittest in codeflash/discovery/discover_unit_tests.py

⏱️ Runtime : 350 microseconds → 99.4 microseconds (best of 264 runs)

⚡️ This change will improve the performance of the following benchmarks:

{benchmark_info_improved}

🔻 This change will degrade the performance of the following benchmarks:

Benchmark File :: Function	Original Runtime	Expected New Runtime	Slowdown
tests.benchmarks.test_benchmark_merge_test_results::test_benchmark_merge_test_results	44.2 milliseconds	44.3 milliseconds	0.12%

📝 Explanation and details

The optimized code achieves a 252% speedup through two key optimizations:

1. Early Exit for No Underscores: Added a check if '_' not in function_name: that immediately returns False for strings without underscores. This avoids unnecessary splitting operations for simple function names, providing significant speedups (51-69% faster) for cases like single words or empty strings.

2. Right-Split Optimization: Replaced split("_") with rsplit("_", 1) which only splits from the right once, creating exactly 2 parts instead of potentially hundreds. This dramatically reduces memory allocation and processing time, especially for long function names with many underscores.

Performance Impact by Test Type:

• Simple cases (no underscores): 51-69% faster due to early exit
• Valid numeric suffixes: 21-50% faster from efficient rsplit
• Large-scale tests: 660-3584% faster - the rsplit optimization shines here, avoiding expensive operations on strings with hundreds of parts
• Edge cases with non-numeric suffixes: Slight 2-13% slowdown due to the additional underscore check, but this is minimal compared to the gains

The optimizations preserve exact functionality while being most effective for complex function names with many parts, which are common in parameterized test scenarios.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	🔘 None Found
🌀 Generated Regression Tests	✅ 61 Passed
⏪ Replay Tests	✅ 100 Passed
🔎 Concolic Coverage Tests	🔘 None Found
📊 Tests Coverage	100.0%

🌀 Generated Regression Tests and Runtime

from __future__ import annotations

# imports
import pytest  # used for our unit tests
from codeflash.discovery.discover_unit_tests import \
    discover_parameters_unittest

# unit tests

# --------------------------
# Basic Test Cases
# --------------------------

def test_basic_valid_with_numeric_suffix():
    # Should detect numeric suffix and split correctly
    codeflash_output = discover_parameters_unittest("foo_123"); result = codeflash_output # 1.29μs -> 992ns (30.2% faster)

def test_basic_valid_with_long_name_and_numeric_suffix():
    # Should handle longer names
    codeflash_output = discover_parameters_unittest("bar_baz_qux_456"); result = codeflash_output # 1.44μs -> 962ns (50.0% faster)

def test_basic_no_numeric_suffix():
    # Should return False if no numeric suffix
    codeflash_output = discover_parameters_unittest("foo_bar"); result = codeflash_output # 852ns -> 972ns (12.3% slower)

def test_basic_single_word():
    # Should return False for single word with no underscore
    codeflash_output = discover_parameters_unittest("foobar"); result = codeflash_output # 621ns -> 411ns (51.1% faster)

def test_basic_numeric_suffix_not_last():
    # Should not match if numeric is not last part
    codeflash_output = discover_parameters_unittest("foo_123_bar"); result = codeflash_output # 902ns -> 1.03μs (12.6% slower)

# --------------------------
# Edge Test Cases
# --------------------------

def test_edge_empty_string():
    # Should handle empty string
    codeflash_output = discover_parameters_unittest(""); result = codeflash_output # 611ns -> 371ns (64.7% faster)

def test_edge_underscore_only():
    # Should handle string of underscores
    codeflash_output = discover_parameters_unittest("_"); result = codeflash_output # 832ns -> 892ns (6.73% slower)

def test_edge_numeric_only():
    # Should handle string that is only numeric
    codeflash_output = discover_parameters_unittest("123"); result = codeflash_output # 602ns -> 391ns (54.0% faster)

def test_edge_underscore_and_numeric():
    # Should handle underscore and numeric only
    codeflash_output = discover_parameters_unittest("_123"); result = codeflash_output # 1.31μs -> 1.02μs (28.6% faster)

def test_edge_multiple_underscores_and_numeric():
    # Should handle multiple underscores before numeric
    codeflash_output = discover_parameters_unittest("___123"); result = codeflash_output # 1.44μs -> 952ns (51.6% faster)

def test_edge_trailing_underscore_no_numeric():
    # Should not match if trailing underscore with no numeric
    codeflash_output = discover_parameters_unittest("foo_"); result = codeflash_output # 841ns -> 972ns (13.5% slower)

def test_edge_leading_underscore_and_numeric_suffix():
    # Should handle leading underscore and valid numeric suffix
    codeflash_output = discover_parameters_unittest("_foo_123"); result = codeflash_output # 1.47μs -> 981ns (50.1% faster)

def test_edge_numeric_suffix_with_leading_zeros():
    # Should handle numeric suffix with leading zeros
    codeflash_output = discover_parameters_unittest("foo_00123"); result = codeflash_output # 1.26μs -> 1.01μs (24.8% faster)

def test_edge_non_numeric_suffix():
    # Should not match if last part is not purely digits
    codeflash_output = discover_parameters_unittest("foo_12a"); result = codeflash_output # 912ns -> 1.03μs (11.6% slower)

def test_edge_suffix_is_negative_number():
    # Should not match if last part is negative number (not purely digits)
    codeflash_output = discover_parameters_unittest("foo_-123"); result = codeflash_output # 911ns -> 1.03μs (11.7% slower)

def test_edge_suffix_is_float():
    # Should not match if last part is a float
    codeflash_output = discover_parameters_unittest("foo_123.45"); result = codeflash_output # 971ns -> 1.03μs (5.82% slower)

def test_edge_suffix_is_alphanumeric():
    # Should not match if last part is alphanumeric
    codeflash_output = discover_parameters_unittest("foo_123abc"); result = codeflash_output # 941ns -> 1.00μs (6.09% slower)

def test_edge_suffix_is_empty_after_underscore():
    # Should not match if there's an empty last part
    codeflash_output = discover_parameters_unittest("foo_"); result = codeflash_output # 831ns -> 921ns (9.77% slower)

def test_edge_multiple_numeric_suffixes():
    # Should only consider the last part as the numeric suffix
    codeflash_output = discover_parameters_unittest("foo_123_456"); result = codeflash_output # 1.47μs -> 1.01μs (45.6% faster)

def test_edge_all_underscores():
    # Should handle string with only underscores
    codeflash_output = discover_parameters_unittest("____"); result = codeflash_output # 882ns -> 901ns (2.11% slower)

def test_edge_underscore_and_non_numeric():
    # Should not match if last part is non-numeric after underscore
    codeflash_output = discover_parameters_unittest("_foo_bar"); result = codeflash_output # 972ns -> 992ns (2.02% slower)

# --------------------------
# Large Scale Test Cases
# --------------------------

def test_large_scale_many_parts_with_numeric_suffix():
    # Should handle long function names with many underscores and numeric suffix
    name = "_".join(["part"] * 999) + "_42"
    codeflash_output = discover_parameters_unittest(name); result = codeflash_output # 49.8μs -> 1.35μs (3584% faster)

def test_large_scale_many_parts_without_numeric_suffix():
    # Should handle long function names with many underscores but no numeric suffix
    name = "_".join(["part"] * 1000)
    codeflash_output = discover_parameters_unittest(name); result = codeflash_output # 27.8μs -> 1.51μs (1735% faster)

def test_large_scale_numeric_suffix_is_large_number():
    # Should handle very large numeric suffix
    name = "foo_" + "9" * 100
    codeflash_output = discover_parameters_unittest(name); result = codeflash_output # 1.51μs -> 1.24μs (21.7% faster)

def test_large_scale_all_numeric_parts():
    # Should only match last part as numeric suffix
    name = "_".join(["123"] * 999) + "_456"
    codeflash_output = discover_parameters_unittest(name); result = codeflash_output # 38.1μs -> 1.29μs (2844% faster)

def test_large_scale_all_numeric_parts_no_suffix():
    # Should not match if last part is not numeric
    name = "_".join(["123"] * 999) + "_foo"
    codeflash_output = discover_parameters_unittest(name); result = codeflash_output # 27.4μs -> 1.37μs (1899% faster)

def test_large_scale_empty_parts():
    # Should handle many empty parts before numeric suffix
    name = ("_" * 999) + "_789"
    codeflash_output = discover_parameters_unittest(name); result = codeflash_output # 20.3μs -> 1.32μs (1438% faster)
    # The split will result in a lot of empty strings, last part is '789'
    expected_prefix = "_" * 998  # 999 underscores split gives 1000 parts, last is '789'

def test_large_scale_suffix_is_zero():
    # Should handle suffix '0'
    name = "foo_" + "0"
    codeflash_output = discover_parameters_unittest(name); result = codeflash_output # 1.25μs -> 1.05μs (19.0% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
#------------------------------------------------
from __future__ import annotations

# imports
import pytest  # used for our unit tests
from codeflash.discovery.discover_unit_tests import \
    discover_parameters_unittest

# unit tests

# --- Basic Test Cases ---

def test_basic_with_numeric_suffix():
    # Function name with numeric suffix
    codeflash_output = discover_parameters_unittest("process_data_42"); result = codeflash_output # 1.65μs -> 1.12μs (47.2% faster)

def test_basic_without_numeric_suffix():
    # Function name without numeric suffix
    codeflash_output = discover_parameters_unittest("process_data"); result = codeflash_output # 982ns -> 1.01μs (2.96% slower)

def test_basic_single_word():
    # Single word function name, no underscores
    codeflash_output = discover_parameters_unittest("data"); result = codeflash_output # 631ns -> 380ns (66.1% faster)

def test_basic_multiple_underscores_and_numeric_suffix():
    # Multiple underscores with numeric suffix
    codeflash_output = discover_parameters_unittest("load_test_case_100"); result = codeflash_output # 1.56μs -> 1.14μs (36.9% faster)

def test_basic_numeric_suffix_is_zero():
    # Numeric suffix is zero
    codeflash_output = discover_parameters_unittest("run_simulation_0"); result = codeflash_output # 1.54μs -> 1.08μs (42.5% faster)

# --- Edge Test Cases ---

def test_edge_empty_string():
    # Empty string as function name
    codeflash_output = discover_parameters_unittest(""); result = codeflash_output # 630ns -> 371ns (69.8% faster)

def test_edge_underscore_only():
    # Function name is just underscores
    codeflash_output = discover_parameters_unittest("_"); result = codeflash_output # 882ns -> 922ns (4.34% slower)

def test_edge_trailing_underscore():
    # Function name ends with underscore
    codeflash_output = discover_parameters_unittest("process_data_"); result = codeflash_output # 862ns -> 992ns (13.1% slower)

def test_edge_leading_underscore():
    # Function name starts with underscore
    codeflash_output = discover_parameters_unittest("_42"); result = codeflash_output # 1.26μs -> 1.00μs (25.9% faster)

def test_edge_multiple_numeric_parts():
    # Function name with multiple numeric parts, only last is considered
    codeflash_output = discover_parameters_unittest("test_1_2_3"); result = codeflash_output # 1.55μs -> 1.01μs (53.5% faster)

def test_edge_suffix_is_not_integer():
    # Suffix is not an integer
    codeflash_output = discover_parameters_unittest("parse_data_final"); result = codeflash_output # 972ns -> 1.08μs (10.2% slower)

def test_edge_suffix_is_float():
    # Suffix is a float
    codeflash_output = discover_parameters_unittest("parse_data_3.14"); result = codeflash_output # 952ns -> 1.07μs (11.2% slower)

def test_edge_suffix_is_negative_integer():
    # Suffix is negative integer (not handled by isdigit)
    codeflash_output = discover_parameters_unittest("parse_data_-42"); result = codeflash_output # 902ns -> 962ns (6.24% slower)

def test_edge_suffix_is_alphanumeric():
    # Suffix is alphanumeric
    codeflash_output = discover_parameters_unittest("parse_data_42a"); result = codeflash_output # 911ns -> 1.01μs (9.98% slower)

def test_edge_suffix_is_empty_after_split():
    # Function name ends with underscore, so last part is empty string
    codeflash_output = discover_parameters_unittest("function_"); result = codeflash_output # 841ns -> 922ns (8.79% slower)

def test_edge_suffix_is_large_number():
    # Suffix is a very large number
    codeflash_output = discover_parameters_unittest("big_data_999999999"); result = codeflash_output # 1.64μs -> 1.09μs (50.5% faster)

def test_edge_suffix_is_zero_padded():
    # Suffix is zero-padded
    codeflash_output = discover_parameters_unittest("test_case_0001"); result = codeflash_output # 1.46μs -> 982ns (49.0% faster)

# --- Large Scale Test Cases ---

def test_large_scale_long_function_name_with_numeric_suffix():
    # Very long function name with numeric suffix at the end
    long_name = "a_" * 499 + "b_123"
    expected_prefix = "a_" * 499 + "b"
    codeflash_output = discover_parameters_unittest(long_name); result = codeflash_output # 12.2μs -> 1.29μs (841% faster)

def test_large_scale_long_function_name_without_numeric_suffix():
    # Very long function name without numeric suffix
    long_name = "x_" * 999 + "y"
    codeflash_output = discover_parameters_unittest(long_name); result = codeflash_output # 10.5μs -> 1.38μs (660% faster)

def test_large_scale_many_underscores_and_numeric_suffix():
    # Function name with many underscores and numeric suffix
    parts = ["part{}".format(i) for i in range(999)]
    function_name = "_".join(parts) + "_77"
    expected_prefix = "_".join(parts)
    codeflash_output = discover_parameters_unittest(function_name); result = codeflash_output # 60.7μs -> 1.66μs (3548% faster)

def test_large_scale_suffix_is_max_length_integer():
    # Suffix is a very large integer string (close to 1000 digits)
    large_num = "9" * 999
    function_name = "data_" + large_num
    codeflash_output = discover_parameters_unittest(function_name); result = codeflash_output # 2.93μs -> 2.60μs (12.8% faster)

def test_large_scale_numeric_suffix_in_middle():
    # Numeric part in the middle, not at end
    function_name = "part_123_part"
    codeflash_output = discover_parameters_unittest(function_name); result = codeflash_output # 982ns -> 1.00μs (2.00% slower)

def test_large_scale_all_numeric_parts():
    # All parts are numeric except the first
    function_name = "a_1_2_3_4_5_6_7_8_9_10"
    expected_prefix = "a_1_2_3_4_5_6_7_8_9"
    codeflash_output = discover_parameters_unittest(function_name); result = codeflash_output # 1.60μs -> 992ns (61.6% faster)

def test_large_scale_suffix_is_empty_string():
    # Suffix is an empty string after split
    function_name = "x_" * 999 + ""
    codeflash_output = discover_parameters_unittest(function_name); result = codeflash_output # 10.5μs -> 1.37μs (662% faster)

# --- Additional Edge Cases ---

def test_edge_suffix_is_space():
    # Suffix is a space
    codeflash_output = discover_parameters_unittest("test_case_ "); result = codeflash_output # 891ns -> 972ns (8.33% slower)

def test_edge_suffix_is_tab():
    # Suffix is a tab character
    codeflash_output = discover_parameters_unittest("test_case_\t"); result = codeflash_output # 811ns -> 922ns (12.0% slower)

def test_edge_suffix_is_special_characters():
    # Suffix is special characters
    codeflash_output = discover_parameters_unittest("test_case_!@#"); result = codeflash_output # 962ns -> 991ns (2.93% slower)

def test_edge_suffix_is_unicode_digits():
    # Suffix is unicode digits (e.g., Arabic-Indic digits)
    codeflash_output = discover_parameters_unittest("test_case_١٢٣"); result = codeflash_output # 2.00μs -> 1.56μs (28.2% faster)

def test_edge_suffix_is_mixed_unicode_and_ascii_digits():
    # Suffix is mixed unicode and ascii digits
    codeflash_output = discover_parameters_unittest("test_case_1٢3"); result = codeflash_output # 1.71μs -> 1.41μs (21.3% faster)

def test_edge_suffix_is_newline():
    # Suffix is a newline character
    codeflash_output = discover_parameters_unittest("test_case_\n"); result = codeflash_output # 851ns -> 902ns (5.65% slower)

def test_edge_suffix_is_multiple_underscores():
    # Suffix is multiple underscores
    codeflash_output = discover_parameters_unittest("test_case___"); result = codeflash_output # 912ns -> 952ns (4.20% slower)

# --- Determinism Test ---

def test_determinism():
    # Multiple calls with same input should yield same output
    input_name = "repeatable_123"
    codeflash_output = discover_parameters_unittest(input_name); result1 = codeflash_output # 1.29μs -> 1.00μs (28.9% faster)
    codeflash_output = discover_parameters_unittest(input_name); result2 = codeflash_output # 591ns -> 621ns (4.83% slower)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

⏪ Replay Tests and Runtime

Test File::Test Function	Original ⏱️	Optimized ⏱️	Speedup
benchmarks/codeflash_replay_tests_cv_23o2q/test_tests_benchmarks_test_benchmark_merge_test_results__replay_test_0.py::test_codeflash_discovery_discover_unit_tests_discover_parameters_unittest_test_benchmark_merge_test_results	33.9μs	35.6μs	-4.67%⚠️

To edit these changes git checkout codeflash/optimize-pr753-2025-09-23T14.02.28 and push.