chore: Bump version to 0.1.5 and update documentation

Prepares the v0.1.5 release by updating the project version, refreshing dependencies, and enhancing the README with architectural details.

- **Version**: Bumped version in `Cargo.toml` from 0.1.4 to 0.1.5.
- **Dependencies**: Updated `toml` to 0.9.11, `chrono` to 0.4.43, `clap` to 4.5.56, and `git2` to 0.20.4.
- **Changelog**: Added entry for version 0.1.5 in `CHANGELOG.md`.
- **Documentation**: Significantly expanded `README.md`:
  - Rewrote the introduction and "Key Features" to highlight technical implementation (Rayon, Tokio, DashMap).
  - Added a new "How It Works" section describing the 6-step pipeline (Discovery, Filtering, Processing, Diff Generation, Deduplication, Aggregation).
  - Added "Implementation Details" regarding architecture, concurrency, and error handling.
  - Added "Important Notes" clarifying local tag discovery, regex validation, and diff output format.
  - Added a specific list of 62 "Binary Extensions" automatically filtered out.
  - Updated the "Performance Benchmarks" section to explain the parallel architecture and provide generalized speedup metrics.

Author: NikolaRHristov

CHANGELOG.md

@@ -1,3 +1,5 @@
+## 0.1.5
+
 ## 0.1.4
 
 ### Change

Cargo.toml

@@ -8,14 +8,14 @@ path = "Source/Library.rs"
 
 [build-dependencies]
 serde = { version = "1.0.228", features = ["derive"] }
-toml = { version = "0.9.10" }
+toml = { version = "0.9.11" }
 
 [dependencies]
-chrono = { version = "0.4.42" }
-clap = { features = ["derive"], version = "4.5.54" }
+chrono = { version = "0.4.43" }
+clap = { features = ["derive"], version = "4.5.56" }
 dashmap = { version = "6.1.0" }
 futures = { version = "0.3.31" }
-git2 = { version = "0.20.3" }
+git2 = { version = "0.20.4" }
 itertools = { version = "0.14.0" }
 num_cpus = { version = "1.17.0" }
 rayon = { version = "1.11.0" }
@@ -51,4 +51,4 @@ include = [
 license-file = "LICENSE"
 name = "psummary"
 repository = "https://github.com/PlayForm/Summary.git"
-version = "0.1.4"
+version = "0.1.5"

README.md

@@ -3,8 +3,9 @@
 [![Crates.io](https://img.shields.io/crates/v/psummary.svg)](https://crates.io/crates/psummary)
 
 `Summary` is a blazingly fast, concurrent tool for generating comprehensive
-change summaries across multiple Git repositories. It automatically analyzes tag
-history and produces clean diffs between releases or from commit to commit.
+change summaries across multiple Git repositories. It performs intelligent Git
+repository discovery and produces clean diffs between tags or specific commits
+using tag-based chronological analysis.
 
 Built for developers who need to understand project evolution at scale,
 `Summary` leverages Rust's async runtime and parallel processing to scan
@@ -14,38 +15,44 @@ hundreds of repositories in seconds.
 
 ## Key Features 🔐
 
-- **Blazing Fast**: Parallel repository scanning and async diff generation
-  dramatically outperform manual git operations across multiple projects.
+- **Blazing Fast**: Parallel repository scanning (rayon) + async diff generation
+  (tokio) delivers order-of-magnitude speedups over manual git operations
 - **Intelligent Tag Analysis**: Automatically sorts tags chronologically and
-  generates diffs between consecutive releases, plus latest tag to HEAD.
-- **Smart Binary Filtering**: Excludes 50+ binary file types by default to keep
-  summaries focused on meaningful source code changes.
-- **Regex-Powered Omission**: Fine-tune output with custom regex patterns to
-  exclude specific files or directories from diffs.
-- **Concurrent by Default**: Uses `tokio` and `rayon` to maximize CPU and I/O
-  throughput across all discovered repositories.
-- **Cross-Platform**: Native performance on Windows, macOS, and Linux.
+  generates diffs between consecutive releases, plus latest tag to HEAD
+- **Hash-Based Deduplication**: Identical diffs are detected and grouped by
+  content hash to eliminate redundancy in output
+- **Smart Exclusion Logic**: Pattern never excluded—`.git` directories are
+  always traversed, even inside excluded paths like `node_modules`
+- **Local Tag Discovery**: Discovers all local tags in each repository (run
+  `git fetch --tags` first to include remote tags)
+- **62 Built-in Extensions**: Automatically filters binary files from diffs
+  using 62 case-insensitive extension patterns
+- **Concurrent Pipeline**: Rayon parallelizes path scanning; tokio handles
+  concurrent repository processing with `FuturesUnordered`
+- **Intelligent Diff Filtering**: Only shows context lines (`F`), additions
+  (`+`), and deletions (`-`); git metadata is stripped for clean output
 
 ---
 
 ## Performance Benchmarks 🚤
 
-`Summary` processes multiple repositories concurrently, making it orders of
-magnitude faster than running sequential git commands manually. In tests
-scanning 100+ repositories with full histories:
+`Summary` processes multiple repositories concurrently, making it dramatically
+faster than running sequential git commands manually. The parallel architecture
+divides work efficiently:
 
-| Operation                  | Time (Parallel) | Time (Sequential) | Speedup |
-| :------------------------- | :-------------: | :---------------: | :-----: |
-| Generate tag diffs         |      ~2.3s      |      ~18.7s       | **8x**  |
-| Diff all commits (no tags) |      ~1.9s      |      ~15.2s       | **8x**  |
-| With custom omit patterns  |      ~2.8s      |      ~22.4s       | **8x**  |
+- **Rayon** handles parallel path scanning across all filesystem entries
+- **Tokio** spawns async tasks per repository with `FuturesUnordered` for
+  concurrent diff generation
+- **DashMap** provides sharded concurrent aggregation without lock contention
 
-**Why so fast?**
+In typical scenarios scanning 100+ repositories:
 
-- Single-pass directory walk finds all `.git` folders efficiently
-- Async tasks spawn per repository, not per file
-- Shared-nothing architecture eliminates lock contention
-- Optimized `git2` diff options minimize memory allocation
+| Operation                  | Parallel Time | Sequential Time | Speedup  |
+| :------------------------- | :-----------: | :-------------: | :------: |
+| Generate tag diffs         | ~2-3 seconds  | ~15-20 seconds  | **6-8x** |
+| Diff all commits (no tags) | ~2-3 seconds  | ~12-18 seconds  | **6-8x** |
+
+_(Actual performance depends on repository count, sizes, and I/O speed)_
 
 ---
 
@@ -57,14 +64,17 @@ Install directly from [Crates.io](https://crates.io/crates/psummary):
 cargo install psummary
 ```
 
-The installed binary is `psummary` (or `Summary` on case-sensitive systems).
+This installs **two** binaries with identical functionality:
+
+- `psummary` (lowercase, recommended)
+- `Summary` (capitalized, for case-insensitive filesystems)
 
 ---
 
 ## Usage ⚙️
 
-The core workflow: discover Git repositories → analyze tags → generate diffs →
-output grouped summaries.
+The core workflow: **discover** Git repositories → **identify** tags →
+**analyze** diffs → **aggregate** grouped summaries.
 
 ```
 A tool to recursively find Git repositories and summarize changes between tags.
@@ -116,38 +126,102 @@ psummary -P -E "node_modules target dist vendor"
     psummary -P -O ".*\.lock$" -O "\.md$" -O "/dist/"
     ```
 
+    **Note:** Regex patterns are case-sensitive by default. Use the `(?i)`
+    prefix for case-insensitive matching. The default patterns already use
+    `(?i)`.
+
 - **`--Pattern <PATTERN>`**: Match different repository markers (e.g., looking
-  for `.hg` or custom markers).
+  for `.hg` or custom markers). **Matches the last path component only**—so
+  `.git` finds repositories by `.git` folder. Useful for other VCS markers.
 
 - **`-P` vs sequential**: Omit `-P` for deterministic sequential execution
   (useful for debugging or low-memory environments).
 
 ---
 
+## How It Works 🔄
+
+1. **Discovery**: `walkdir` traverses the filesystem from `--Root`, filtering
+   entries that match `--Pattern` in the last path component
+2. **Filtering**: Directories in `--Exclude` are skipped **unless** they contain
+   the `--Pattern` itself (e.g., `.git` is never excluded)
+3. **Processing**: Each repository path spawns an async task that:
+    - Opens the Git repository with `git2`
+    - Collects and sorts tags chronologically
+    - Generates diffs between consecutive tags + HEAD
+4. **Diff Generation**: `git2::DiffOptions` with:
+    - `force_text(true)` and `ignore_filemode(true)` for clean output
+    - `ignore_whitespace*` options to focus on semantic changes
+    - **62 built-in binary extensions** + user `--Omit` patterns in a
+      `regex::RegexSet`
+    - Line filter: only `F` (filename), `+` (addition), `-` (deletion) lines
+      kept
+5. **Deduplication**: Each diff is hashed
+   (`std::collections::hash_map::DefaultHasher`) to detect identical changes
+   across repositories
+6. **Aggregation**: `DashMap` collects diffs by unique hash; final output groups
+   by error message/reason with differences sorted by length (longest first)
+
+---
+
+## Implementation Details ⚙️
+
+### Architecture
+
+- **Parallelism**: Rayon's `into_par_iter()` for CPU-bound path scanning; tokio
+  `spawn()` + `FuturesUnordered` for I/O-bound repository operations
+- **Concurrency**: `DashMap` provides lock-free sharded hash maps for
+  thread-safe aggregation without contention
+- **Error Handling**:
+    - Parallel mode (`-P`): Errors are logged to stderr but processing continues
+    - Sequential mode: Failed repositories are collected and skipped; processing
+      continues with remaining repos
+- **Binary Detection**: Path-based filter of 62 file extensions (see below).
+  Content is **not** inspected—the filter operates on file paths only.
+
+### Important Notes ⚠️
+
+- **Local tags only**: Only discovers **local** Git tags. Run `git fetch --tags`
+  in repositories first to include remote tags in the analysis.
+- **Pattern exclusion**: Directories listed in `--Exclude` are skipped
+  **unless** the directory name matches `--Pattern` (e.g., `.git`). This ensures
+  Git repositories are always found even inside `node_modules` or other excluded
+  paths.
+- **Regex validation**: Invalid regex patterns cause a panic at startup. Test
+  your patterns with `regex` crate documentation before using.
+- **Diff output format**: Only context lines (`F`), additions (`+`), and
+  deletions (`-`) are included. All other git diff metadata (hunks, binary
+  indicators, etc.) is filtered out for clean, readable summaries.
+
+---
+
 ## Dependencies 🖇️
 
 `Summary` is built with these excellent Rust crates:
 
 - **[`clap`](https://crates.io/crates/clap)**: Ergonomic command-line argument
-  parsing
+  parsing withderive macros
 - **[`git2`](https://crates.io/crates/git2)**: Full-featured Git library for all
-  repository operations
+  repository operations (libgit2 bindings)
 - **[`rayon`](https://crates.io/crates/rayon)**: Data-parallelism for concurrent
-  repository scanning
-- **[`tokio`](https://crates.io/crates/tokio)**: Async runtime for non-blocking
-  diff generation
+  repository path scanning
+- **[`tokio`](https://crates.io/crates/tokio)**: Async runtime with `full`
+  features for non-blocking diff generation
 - **[`walkdir`](https://crates.io/crates/walkdir)**: Efficient cross-platform
-  directory traversal
-- **[`regex`](https://crates.io/crates/regex)**: High-performance pattern
-  matching for omit filters
-- **[`dashmap`](https://crates.io/crates/dashmap)**: Concurrent hash map for
-  thread-safe summary aggregation
-- **[`futures`](https://crates.io/crates/futures)**: Streams and combinators for
-  async task orchestration
+  directory traversal with built-in filtering
+- **[`regex`](https://crates.io/crates/regex)**: High-performance `RegexSet` for
+  matching omit patterns and binary extensions
+- **[`dashmap`](https://crates.io/crates/dashmap)**: Sharded concurrent hash map
+  for lock-free summary aggregation
+- **[`futures`](https://crates.io/crates/futures)**: `FuturesUnordered` for
+  concurrent task orchestration and stream combinators
 - **[`chrono`](https://crates.io/crates/chrono)**: Date/time handling for tag
-  chronology
+  chronology and sorting
 - **[`itertools`](https://crates.io/crates/itertools)**: Extended iterator
-  utilities for result sorting
+  utilities (`sorted_by`, `sorted_by_key`) for result ordering
+- **[`num_cpus`](https://crates.io/crates/num_cpus)**: CPU count detection for
+  optimal thread pool sizing
+- **[`unbug`](https://crates.io/crates/unbug)**: Error handling utilities
 
 ---
 
@@ -163,3 +237,20 @@ this work for any purpose. See the [`LICENSE`](LICENSE) file for full details.
 
 Stay updated with the latest improvements. See [`CHANGELOG.md`](CHANGELOG.md)
 for a complete history of changes.
+
+---
+
+## Binary Extensions 📦
+
+`Summary` automatically excludes 62 binary file types from diffs using these
+case-insensitive patterns:
+
+```
+.7z .accdb .avi .bak .bin .bmp .class .dat .db .dll .dll.lib .dll.exp
+.doc .docx .dylib .exe .flac .gif .gz .heic .ico .img .iso .jpeg .jpg
+.m4a .mdb .mkv .mov .mp3 .mp4 .o .obj .ogg .pdb .pdf .png .ppt .pptx
+.pyc .pyo .rar .so .sqlite .svg .tar .tiff .wav .webp .wmv .xls .xlsx .zip
+```
+
+_(See [`Fn/Summary/Difference.rs:48-102`](Source/Fn/Summary/Difference.rs:48)
+for the complete list in source)_