Skip to content

performance-tuning.md

Latest commit

 

History

History
359 lines (251 loc) · 8.4 KB

performance-tuning.md

File metadata and controls

359 lines (251 loc) · 8.4 KB
Raw

Performance Tuning Guide

This guide provides recommendations for optimizing Git Spark's performance when analyzing large repositories and generating reports.

Performance Benchmarks

Based on testing with the git-spark repository itself and various open-source projects:

Repository Size Commits Files Analysis Time Memory Usage
Small <1,000 <500 <5 seconds <100MB
Medium 1,000-10,000 500-2,000 10-30 seconds 100-300MB
Large 10,000-100,000 2,000-10,000 1-5 minutes 300-500MB
Very Large >100,000 >10,000 5-15 minutes 500MB-1GB

Note: Actual performance depends on hardware (CPU, disk I/O), repository structure, and analysis options.

Optimization Strategies

1. Limit Analysis Time Range

The most effective way to improve performance is to limit the analysis window:

# Analyze only the last 30 days (fastest)
git-spark --days=30

# Analyze a specific quarter
git-spark --since=2026-01-01 --until=2026-03-31

# Analyze last 7 days for quick health checks
git-spark --days=7 --format=console

Impact: Reducing from 1 year to 30 days can improve speed by 10-20x for large repositories.

2. Use Branch Filtering

Analyze specific branches instead of the entire repository:

# Analyze only main branch
git-spark --branch=main

# Analyze feature branch
git-spark --branch=feature/new-analytics

Impact: 30-50% faster for repositories with many branches.

3. Exclude Non-Code Files

Skip documentation, configuration, and generated files:

# Exclude markdown and text files
git-spark --exclude-extensions=.md,.txt

# Exclude multiple file types
git-spark --exclude-extensions=.md,.txt,.log,.json,.xml,.yaml,.yml

# Focus only on source code
git-spark --exclude-extensions=.md,.txt,.json,.xml,.yaml,.yml,.lock,.sum

Impact: 20-40% faster analysis and smaller reports for documentation-heavy repositories.

4. Use Console Format for Quick Checks

For rapid analysis during development:

# Quick console output (no HTML generation)
git-spark --days=7 --format=console

# JSON for automation (faster than HTML)
git-spark --days=30 --format=json --output=./reports

Impact: Console format is 5-10x faster than HTML for large datasets.

5. Avoid Heavy Analysis Mode for Initial Runs

The --heavy flag enables expensive analyses. Skip it for initial exploration:

# Standard analysis (default, faster)
git-spark --days=30

# Heavy analysis (more detailed, slower)
git-spark --days=30 --heavy

Impact: Heavy mode adds 20-30% to analysis time but provides deeper insights.

Memory Management

Git Spark uses streaming processing to handle large repositories efficiently.

Default Configuration

The default buffer size (200MB) works well for most repositories:

{
  "performance": {
    "maxBuffer": 200,
    "chunkSize": 1000
  }
}

For Very Large Repositories (>100k commits)

Increase buffer size in .git-spark.json:

{
  "performance": {
    "maxBuffer": 500,
    "chunkSize": 2000
  }
}

Warning: Higher buffer sizes increase memory usage but improve performance for large datasets.

Memory-Constrained Environments

Reduce buffer size for low-memory systems (e.g., CI/CD runners):

{
  "performance": {
    "maxBuffer": 50,
    "chunkSize": 500
  }
}

HTML Report Optimization

Progressive Table Pagination

Large HTML reports use progressive loading for performance:

  • Initial render: First 50 rows
  • Click "Show more": Load additional rows incrementally
  • Full dataset embedded as JSON for export

This keeps initial load time fast even for repos with 1000+ authors or files.

Dark Mode Performance

Dark mode uses CSS variables for instant theme switching with minimal overhead. No re-rendering required.

SVG Charts vs. External Libraries

Git Spark uses native SVG charts instead of Chart.js:

Benefits:

  • No external JavaScript loading
  • Smaller bundle size
  • Faster initial page load
  • Offline-capable reports

CI/CD Integration Performance

GitHub Actions

Optimize for CI/CD environments:

- name: Run git-spark analysis
  run: |
    git-spark --days=7 --format=json --output=./reports
    # Skip heavy analysis in CI

Caching in CI/CD

Cache the .git-spark-cache directory between runs:

- uses: actions/cache@v3
  with:
    path: .git-spark-cache
    key: git-spark-cache-${{ github.sha }}
    restore-keys: |
      git-spark-cache-

Impact: 50-70% faster subsequent CI runs when Git history and generated artifacts are reused.

Disk I/O Optimization

Use SSD Storage

Git Spark performs many small file reads. SSD storage provides:

  • 5-10x faster analysis on large repositories
  • Better performance for Git Spark cache and generated reports

File Cache Location

Place cache on fastest available disk:

{
  "performance": {
    "cacheDir": "/path/to/fast/disk/.git-spark-cache"
  }
}

Troubleshooting Performance Issues

Slow Analysis (>10 minutes for <10k commits)

  1. Check disk I/O: Use iostat or Task Manager to monitor disk activity
  2. Verify Git performance: Run git log --oneline | wc -l to test git speed
  3. Disable heavy analysis: Test without --heavy to isolate expensive calculations
  4. Reduce date range: Test with --days=7 to verify baseline performance

High Memory Usage (>1GB)

  1. Reduce buffer size in .git-spark.json
  2. Use smaller chunks: Set chunkSize: 500 or lower
  3. Limit analysis range: Use --days to reduce dataset
  4. Check for memory leaks: Update to latest version

Slow HTML Report Loading

  1. Check report size: HTML files >10MB may load slowly
  2. Use JSON instead: For programmatic access, JSON is faster
  3. Reduce analysis range: Fewer commits = smaller HTML
  4. Limit hotspot tables: Configure maxHotspots in .git-spark.json

Configuration Reference

Creating Your Configuration

The easiest way to create a .git-spark.json configuration file is with the interactive wizard:

# Interactive setup with prompts
git-spark init

# Quick setup with defaults
git-spark init --yes

The wizard guides you through setting days to analyze, output format, and file exclusions.

Sample High-Performance Configuration

{
  "analysis": {
    "excludeExtensions": [".md", ".txt", ".log"],
    "excludeAuthors": ["dependabot[bot]", "github-actions[bot]"]
  },
  "performance": {
    "maxBuffer": 300,
    "enableCaching": true,
    "cacheDir": ".git-spark-cache",
    "chunkSize": 1500
  },
  "output": {
    "defaultFormat": "json",
    "outputDir": "./reports"
  }
}

Sample Memory-Efficient Configuration

{
  "performance": {
    "maxBuffer": 50,
    "enableCaching": true,
    "chunkSize": 500
  },
  "output": {
    "defaultFormat": "console",
    "outputDir": "./reports"
  }
}

Best Practices Summary

  1. ✅ Start small: Use --days=7 for initial testing
  2. ✅ Use caching: Enable all caching options for repeated analysis
  3. ✅ Exclude non-code files: Use --exclude-extensions for faster analysis
  4. ✅ Choose right format: Console for quick checks, JSON for automation, HTML for presentations
  5. ✅ Monitor resources: Watch memory and disk I/O during analysis
  6. ✅ Update regularly: New versions often include performance improvements
  7. ✅ Use SSD storage: Dramatically improves I/O-bound operations
  8. ✅ Cache in CI/CD: Reuse .git-spark-cache between runs

Performance Metrics

Real-World Examples

Git Spark Repository (small):

  • Commits: ~138 (at v1.3.0)
  • Analysis time: 2-3 seconds
  • HTML size: ~500KB
  • Memory: ~50MB

Medium Open Source Project:

  • Commits: ~5,000
  • Analysis time: 15-20 seconds
  • HTML size: ~2MB
  • Memory: ~200MB

Large Enterprise Repository:

  • Commits: ~50,000
  • Analysis time: 2-3 minutes
  • HTML size: ~8MB
  • Memory: ~400MB

Getting Help

If you experience persistent performance issues:

  1. Check GitHub Issues for known issues
  2. Run with --log-level=debug to diagnose problems
  3. Share performance metrics when reporting issues:
    • Repository size (commits, files, authors)
    • Analysis time
    • Memory usage
    • System specifications

Last Updated: July 2026 Git Spark Version: 1.4.0+