⚡️ Speed up function existing_tests_source_for by 43% in PR #363 (part-1-windows-fixes)

[codeflash-ai]

⚡️ This pull request contains optimizations for PR #363

If you approve this dependent PR, these changes will be merged into the original PR branch part-1-windows-fixes.

This PR will be automatically closed if the original PR is merged.

---

📄 43% (0.43x) speedup for existing_tests_source_for in codeflash/result/create_pr.py

⏱️ Runtime : 6.13 milliseconds → 4.28 milliseconds (best of 364 runs)

📝 Explanation and details

Here is an optimized version of your program, rewritten to minimize unnecessary work, allocation, and redundant computation, addressing the main bottlenecks surfaced by your profiling data.

• Tabulate: Main performance issue is repeated function calls and list comprehensions inside loops. The column/row transforms, especially for header formatting and alignments, are the heaviest. We reduce allocation, avoid repeated calls when not needed, and specialize “headers” and “no headers” branches.
• existing_tests_source_for: Avoids unnecessary dict lookups and string formatting by grouping updates, and directly iterates/precomputes keys, minimizing set/dict operations.
• General: Inline tiny helpers, use local variables to reduce global lookups, and use tuple/list comprehension where possible.

Note: All logic, side-effects, return values, and signatures are preserved exactly per your requirements.

Summary of main optimizations.

• No repeated list comprehensions in tight loops, especially for column and header formatting.
• Locals for small globals (MIN_PADDING, width_fn, etc.), and cache path computation in existing_tests_source_for.
• No repeated dict/set membership tests; minimized lookups to once per unique key.
• Fast header/row formatting with minimal allocations and in-place width calculations.

You should observe a faster runtime and lower memory usage, especially on large tables or when invoked many times. All function behaviors and signatures are precisely preserved.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	✅ 11 Passed
🌀 Generated Regression Tests	✅ 16 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	🔘 None Found
📊 Tests Coverage	100.0%

⚙️ Existing Unit Tests and Runtime

Test File::Test Function	Original ⏱️	Optimized ⏱️	Speedup
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_complex_module_path_conversion	254μs	259μs	⚠️-1.81%
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_filters_out_generated_tests	280μs	284μs	⚠️-1.43%
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_missing_optimized_runtime	123μs	130μs	⚠️-4.97%
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_missing_original_runtime	122μs	127μs	⚠️-4.10%
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_multiple_runtimes_uses_minimum	241μs	249μs	⚠️-3.22%
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_multiple_tests_sorted_output	360μs	363μs	⚠️-0.747%
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_no_test_files_returns_empty_string	572ns	621ns	⚠️-7.89%
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_single_test_with_improvement	245μs	250μs	⚠️-1.97%
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_single_test_with_regression	253μs	259μs	⚠️-2.42%
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_test_without_class_name	242μs	247μs	⚠️-1.91%
test_existing_tests_source_for.py::TestExistingTestsSourceFor.test_zero_runtime_values	122μs	127μs	⚠️-4.28%

🌀 Generated Regression Tests and Runtime

from dataclasses import dataclass, field
from pathlib import Path
from typing import Optional

# imports
import pytest
from codeflash.result.create_pr import existing_tests_source_for

# --- Stubs for dependencies and data structures used by existing_tests_source_for ---

@dataclass(frozen=True)
class InvocationId:
    test_module_path: str
    test_class_name: Optional[str]
    test_function_name: str

@dataclass(frozen=True)
class FunctionCalledInTest:
    tests_in_file: 'TestsInFile'

@dataclass(frozen=True)
class TestsInFile:
    test_file: Path

@dataclass
class TestConfig:
    tests_root: Path
from codeflash.result.create_pr import existing_tests_source_for

# --- Unit Tests ---

# Utility for producing InvocationId and FunctionCalledInTest objects
def make_invocation(test_module_path, test_class_name, test_function_name):
    return InvocationId(
        test_module_path=test_module_path,
        test_class_name=test_class_name,
        test_function_name=test_function_name,
    )

def make_func_called(test_file_path):
    return FunctionCalledInTest(
        tests_in_file=TestsInFile(test_file=Path(test_file_path).resolve())
    )

# 1. Basic Test Cases

def test_no_tests_for_function_returns_empty():
    # No tests for this function
    function_to_tests = {}
    test_cfg = TestConfig(tests_root=Path("/tests"))
    codeflash_output = existing_tests_source_for(
        "some.module.func",
        function_to_tests,
        test_cfg,
        {},
        {},
    ); result = codeflash_output # 521ns -> 632ns (17.6% slower)

def test_one_test_case_speedup():
    # One test, optimized is faster
    test_file = "/tests/test_file.py"
    func_name = "my.module.func"
    invocation = make_invocation("tests.test_file", None, "test_func")
    function_to_tests = {
        func_name: {make_func_called(test_file)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    original_runtimes = {invocation: [2000]}
    optimized_runtimes = {invocation: [1000]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 111μs -> 118μs (5.19% slower)

def test_one_test_case_slowdown():
    # One test, optimized is slower
    test_file = "/tests/test_file.py"
    func_name = "my.module.func"
    invocation = make_invocation("tests.test_file", "TestClass", "test_func")
    function_to_tests = {
        func_name: {make_func_called(test_file)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    original_runtimes = {invocation: [1000]}
    optimized_runtimes = {invocation: [2000]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 107μs -> 112μs (4.58% slower)

def test_multiple_tests_and_files():
    # Multiple tests in different files, both speedup and slowdown
    test_file1 = "/tests/test_file1.py"
    test_file2 = "/tests/test_file2.py"
    func_name = "my.module.func"
    inv1 = make_invocation("tests.test_file1", None, "test_func1")
    inv2 = make_invocation("tests.test_file2", None, "test_func2")
    function_to_tests = {
        func_name: {make_func_called(test_file1), make_func_called(test_file2)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    original_runtimes = {inv1: [5000], inv2: [10000]}
    optimized_runtimes = {inv1: [4000], inv2: [12000]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 133μs -> 139μs (4.67% slower)

def test_only_include_tests_in_function_to_tests():
    # Should only include tests that are in function_to_tests
    test_file = "/tests/test_file.py"
    func_name = "my.module.func"
    inv1 = make_invocation("tests.test_file", None, "test_func1")
    inv2 = make_invocation("tests.test_file", None, "test_func2")
    # Only test_func1 is in function_to_tests
    function_to_tests = {
        func_name: {make_func_called(test_file)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    original_runtimes = {inv1: [1000], inv2: [2000]}
    optimized_runtimes = {inv1: [500], inv2: [1000]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 132μs -> 111μs (18.6% faster)

# 2. Edge Test Cases

def test_zero_runtimes_are_ignored():
    # If either runtime is zero, that row should not appear
    test_file = "/tests/test_file.py"
    func_name = "my.module.func"
    inv = make_invocation("tests.test_file", None, "test_func")
    function_to_tests = {
        func_name: {make_func_called(test_file)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    # original is 0, optimized is nonzero
    original_runtimes = {inv: [0]}
    optimized_runtimes = {inv: [1000]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 104μs -> 109μs (4.27% slower)
    # optimized is 0, original is nonzero
    original_runtimes = {inv: [1000]}
    optimized_runtimes = {inv: [0]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 97.9μs -> 103μs (5.24% slower)

def test_missing_optimized_or_original_runtime():
    # If an invocation is only present in one of the runtime dicts, should not appear
    test_file = "/tests/test_file.py"
    func_name = "my.module.func"
    inv = make_invocation("tests.test_file", None, "test_func")
    function_to_tests = {
        func_name: {make_func_called(test_file)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    # Only original
    original_runtimes = {inv: [1000]}
    optimized_runtimes = {}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 103μs -> 109μs (4.95% slower)
    # Only optimized
    original_runtimes = {}
    optimized_runtimes = {inv: [1000]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 97.8μs -> 101μs (3.52% slower)

def test_minimum_runtime_selected():
    # If multiple runtimes, should use the minimum
    test_file = "/tests/test_file.py"
    func_name = "my.module.func"
    inv = make_invocation("tests.test_file", None, "test_func")
    function_to_tests = {
        func_name: {make_func_called(test_file)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    original_runtimes = {inv: [1000, 2000, 1500]}
    optimized_runtimes = {inv: [800, 900, 1000]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 104μs -> 108μs (3.86% slower)

def test_handles_class_and_function_level_tests():
    # Should correctly format both class and function level tests
    test_file = "/tests/test_file.py"
    func_name = "my.module.func"
    inv1 = make_invocation("tests.test_file", "TestClass", "test_func")
    inv2 = make_invocation("tests.test_file", None, "test_func2")
    function_to_tests = {
        func_name: {make_func_called(test_file)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    original_runtimes = {inv1: [1000], inv2: [2000]}
    optimized_runtimes = {inv1: [500], inv2: [1000]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 131μs -> 109μs (19.8% faster)

def test_handles_nontrivial_relative_paths():
    # If tests_root is not /, should show relative path
    test_file = "/repo/tests/unit/test_file.py"
    func_name = "my.module.func"
    inv = make_invocation("tests.unit.test_file", None, "test_func")
    function_to_tests = {
        func_name: {make_func_called(test_file)}
    }
    test_cfg = TestConfig(tests_root=Path("/repo/tests"))
    original_runtimes = {inv: [1000]}
    optimized_runtimes = {inv: [500]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 107μs -> 112μs (4.68% slower)

# 3. Large Scale Test Cases

def test_many_tests_and_files():
    # 100 files, 10 tests each
    num_files = 10
    num_tests_per_file = 10
    func_name = "my.module.func"
    function_to_tests = {}
    original_runtimes = {}
    optimized_runtimes = {}
    test_cfg = TestConfig(tests_root=Path("/tests"))
    func_tests = set()
    for i in range(num_files):
        test_file = f"/tests/test_file_{i}.py"
        func_tests.add(make_func_called(test_file))
        for j in range(num_tests_per_file):
            test_func = f"test_func_{j}"
            inv = make_invocation(f"tests.test_file_{i}", None, test_func)
            # original is always 1000+j ns, optimized is always 500+j ns
            original_runtimes[inv] = [1000 + j]
            optimized_runtimes[inv] = [500 + j]
    function_to_tests[func_name] = func_tests
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output
    # Should contain all test files and test functions
    for i in range(num_files):
        for j in range(num_tests_per_file):
            expected = f"`test_file_{i}.py::test_func_{j}`"

def test_large_runtime_values():
    # Handles very large nanosecond values (seconds)
    test_file = "/tests/test_file.py"
    func_name = "my.module.func"
    inv = make_invocation("tests.test_file", None, "test_func")
    function_to_tests = {
        func_name: {make_func_called(test_file)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    original_runtimes = {inv: [2_000_000_000]}
    optimized_runtimes = {inv: [1_000_000_000]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 107μs -> 111μs (3.44% slower)

def test_handles_non_ascii_test_names_and_paths():
    # Handles unicode in file names and test names
    test_file = "/tests/тестовый_файл.py"
    func_name = "my.module.func"
    inv = make_invocation("tests.тестовый_файл", None, "тест_функция")
    function_to_tests = {
        func_name: {make_func_called(test_file)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    original_runtimes = {inv: [1000]}
    optimized_runtimes = {inv: [500]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 109μs -> 114μs (4.21% slower)

def test_handles_duplicate_test_names_in_different_files():
    # Two files with same test function name, both should appear
    test_file1 = "/tests/test1.py"
    test_file2 = "/tests/test2.py"
    func_name = "my.module.func"
    inv1 = make_invocation("tests.test1", None, "test_func")
    inv2 = make_invocation("tests.test2", None, "test_func")
    function_to_tests = {
        func_name: {make_func_called(test_file1), make_func_called(test_file2)}
    }
    test_cfg = TestConfig(tests_root=Path("/tests"))
    original_runtimes = {inv1: [1000], inv2: [2000]}
    optimized_runtimes = {inv1: [500], inv2: [1000]}
    codeflash_output = existing_tests_source_for(
        func_name,
        function_to_tests,
        test_cfg,
        original_runtimes,
        optimized_runtimes,
    ); result = codeflash_output # 131μs -> 139μs (5.38% slower)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

To edit these changes git checkout codeflash/optimize-pr363-2025-07-04T00.27.49 and push.