Faster test results analysis (CF-625)

[aseembits93]

User description

The current code was not optimized as it was using multiple list and dictionary comprehensions which could be collated into a single for loop.

---

PR Type

Enhancement

---

Description

• Refactor runtime grouping into single loop

• Decouple capture module dependency removal

• Duplicate VerificationType with extra members

• Add TODO for duplicated class notice

---

Changes walkthrough 📝

	Relevant files
Enhancement	models.py Refactor runtime data grouping logic                                          codeflash/models/models.py • Added TODO about duplicated VerificationType class • Replaced comprehensions with single loop + defaultdict • Improved logging for missing runtimes +13/-16
Dependencies	codeflash_capture.py Decouple capture from core module                                                codeflash/verification/codeflash_capture.py • Removed external VerificationType import • Added local VerificationType class with extra variants • Added no-dependency comment and imported Enum locally +8/-2

---

Need help?

Type /help how to ... in the comments thread for any questions about PR-Agent usage.

Check out the documentation for more information.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Logic Bug The new usable_runtime_data_by_test_case logic appends runtimes for failed tests instead of passed ones. Passed tests with valid runtimes are not collected, leading to incorrect data aggregation. usable_id_to_runtime = defaultdict(list) for result in self.test_results: if result.did_pass: if not result.runtime: msg = ( f"Ignoring test case that passed but had no runtime -> {result.id}, " f"Loop # {result.loop_index}, Test Type: {result.test_type}, " f"Verification Type: {result.verification_type}" ) logger.debug(msg) else: usable_id_to_runtime[result.id].append(result.runtime) return usable_id_to_runtime Missing Import defaultdict is used on line 540 but not imported. Add from collections import defaultdict to avoid NameError. usable_id_to_runtime = defaultdict(list) Duplicate Class VerificationType is duplicated in this module and in codeflash/models/models.py. Centralize or share the definition to prevent maintenance drift. class VerificationType(str, Enum): FUNCTION_CALL = ( "function_call" # Correctness verification for a test function, checks input values and output values) ) INIT_STATE_FTO = "init_state_fto" # Correctness verification for fto class instance attributes after init INIT_STATE_HELPER = "init_state_helper" # Correctness verification for helper class instance attributes after init

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Append only passed runtimes Update the conditional logic to only append runtimes for passing tests with valid runtime values and log missing runtimes otherwise. codeflash/models/models.py [542-551] if result.did_pass: - if not result.runtime: + if result.runtime: + usable_id_to_runtime[result.id].append(result.runtime) + else: msg = ( f"Ignoring test case that passed but had no runtime -> {result.id}, " f"Loop # {result.loop_index}, Test Type: {result.test_type}, " f"Verification Type: {result.verification_type}" ) logger.debug(msg) -else: - usable_id_to_runtime[result.id].append(result.runtime) Suggestion importance[1-10]: 10 __ Why: The current logic mistakenly appends runtimes for failed tests; this change fixes the bug by only collecting runtimes for passed tests and logs missing runtime cases.	High
General	Simplify enum assignment Simplify the enum member assignment by removing unnecessary parentheses and consolidating it into a single line for consistency. codeflash/verification/codeflash_capture.py [15-17] -FUNCTION_CALL = ( - "function_call" # Correctness verification for a test function, checks input values and output values) -) +FUNCTION_CALL = "function_call" # Correctness verification for a test function, checks input values and output values Suggestion importance[1-10]: 3 __ Why: This reduction of parentheses is a minor stylistic cleanup that has no functional impact but improves consistency.	Low

[aseembits93]

#197 a similar optimization was found by codeflash, I closed it as the diff was larger.