Performance optimizations for large repositories

[stephenway]

Description

Add performance optimizations to handle large repositories more efficiently and improve analysis speed.

Type of Change

• ⚡ Performance improvement

Problem Statement

Large repositories with many files and complex commit histories can be slow to analyze, leading to poor user experience in CI/CD pipelines and CLI usage.

Proposed Solution

Implement several performance optimizations:

1. File-level caching: Cache analysis results for unchanged files
2. Parallel processing: Analyze multiple files concurrently
3. Incremental analysis: Only analyze changed files when possible
4. Memory optimization: Stream large diffs instead of loading entirely into memory
5. Smart filtering: Skip analysis for certain file types or patterns

Performance Impact

• Faster analysis for large repositories (1000+ files)
• Reduced memory usage during analysis
• Better CI/CD pipeline performance
• Improved CLI responsiveness

Implementation Ideas

1. Add caching layer with file hash-based invalidation
2. Implement worker threads for parallel file analysis
3. Add configuration options for performance tuning
4. Create performance benchmarks and tests
5. Add progress indicators for long-running analyses

Acceptance Criteria

• Analysis time reduced by 50% for large repositories
• Memory usage stays under 500MB for typical repos
• Progress indicators show during long analyses
• Caching can be disabled via configuration
• Performance benchmarks pass

Additional Context

This is important for enterprise adoption where repositories can be very large. The optimizations should be backward compatible and configurable.