Commit 5d585d9
authored
Optimize _should_include_method
This optimization achieves a **21% runtime improvement** (from 2.30ms to 1.89ms) by eliminating repeated pattern matching overhead in the method filtering logic.
## Key Optimizations
**1. Pre-compiled Pattern Matching (~87% time reduction in pattern checks)**
The original code's major bottleneck was spending 87% of total time in fnmatch operations:
- 48.1% in include_patterns check (25.6ms)
- 39.1% in exclude_patterns check (20.7ms)
The optimization pre-compiles glob patterns into regex objects in `FunctionFilterCriteria.__post_init__()`:
```python
self._include_regexes = [re.compile(fnmatch.translate(p)) for p in self.include_patterns]
self._exclude_regexes = [re.compile(fnmatch.translate(p)) for p in self.exclude_patterns]
```
This eliminates the need to:
- Import fnmatch 1,155 times per run (once per pattern check)
- Convert glob patterns to regex on every method evaluation
- Rebuild pattern matching state repeatedly
**2. Dedicated Pattern Matching Methods**
The new `matches_include_patterns()` and `matches_exclude_patterns()` methods provide cleaner interfaces and enable the pre-compiled regex optimization. Pattern matching time drops from 45.9ms to just 3.1ms in the profiler results.
**3. Added Missing Implementation**
The optimized code includes the `_node_has_return()` method implementation that was referenced but missing from the original code, ensuring the analyzer works correctly without relying on external dependencies.
## Test Results Analysis
The optimization shows dramatic improvements for pattern-heavy workloads:
- **Pattern matching tests**: 44-66% faster (e.g., `test_include_patterns_allows_when_matching_and_blocks_when_not` improved 57-66%)
- **Simple checks** (abstract, constructor): 22-25% faster due to reduced overhead
- **Return type checks**: Slight regressions (7-26% slower) are acceptable trade-offs, as these aren't pattern-matching bottlenecks
The bulk test (`test_bulk_processing_of_many_methods_runs_and_counts_expected_inclusions`) processes 1,000 methods with pattern matching—exactly the workload that benefits most from pre-compiled patterns.
## Impact
This optimization is particularly valuable when:
- Processing large codebases with many methods to filter
- Using complex glob patterns (wildcards, multiple patterns)
- Running discovery operations repeatedly during development cycles
The 21% overall speedup comes primarily from eliminating redundant work in the most frequently executed code path (pattern matching), making method discovery operations substantially faster without changing behavior.1 parent 8c1a3a4 commit 5d585d9
2 files changed
Lines changed: 26 additions & 10 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
| 11 | + | |
10 | 12 | | |
11 | 13 | | |
12 | 14 | | |
| |||
28 | 30 | | |
29 | 31 | | |
30 | 32 | | |
31 | | - | |
| 33 | + | |
| 34 | + | |
32 | 35 | | |
33 | 36 | | |
34 | 37 | | |
| |||
171 | 174 | | |
172 | 175 | | |
173 | 176 | | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
174 | 194 | | |
175 | 195 | | |
176 | 196 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
136 | 136 | | |
137 | 137 | | |
138 | 138 | | |
139 | | - | |
140 | | - | |
141 | | - | |
142 | | - | |
143 | | - | |
| 139 | + | |
| 140 | + | |
144 | 141 | | |
145 | 142 | | |
146 | | - | |
147 | | - | |
| 143 | + | |
| 144 | + | |
148 | 145 | | |
149 | | - | |
150 | | - | |
| 146 | + | |
151 | 147 | | |
152 | 148 | | |
153 | 149 | | |
| |||
0 commit comments