docs: Update documentation with accurate implementation status and test data (#36)

* docs: Update documentation with accurate implementation status and test data

- AGENTS.md: Removed misleading line count metrics, clarified thin wrapper architecture
- OVERVIEW.md: Updated status to show architecture complete with feature development ongoing; marked testing as comprehensive (54 tests)
- README.md: Moved filtering implementation details to OVERVIEW (architectural reference)
- GOALS.md: Verified all claims against actual implementation and testing:
  - Updated frame size claim: 375K+ frames tested (single log)
  - Updated test count: 54 actual unit tests
  - Clarified CSV field ordering claim (not comprehensively validated)
  - Verified GPS/Event exports working (833+ G frames, valid outputs)
  - Marked all unstarted features accurately
- Documentation contains facts only, verified against actual implementation

* docs: Update test count to accurate 62 unit tests

Verified actual count: 62 unit tests across 7 files
- conversion.rs: 13 tests
- export.rs: 8 tests
- filters.rs: 9 tests
- main.rs: 11 tests
- parser/helpers.rs: 7 tests
- parser/stream.rs: 6 tests
- tests/export_integration_tests.rs: 8 tests

Updated: OVERVIEW.md and GOALS.md

Author: nerdCopter

AGENTS.md

@@ -15,7 +15,7 @@
 - **Library-first design:** Core logic in `src/lib.rs` and CLI entry point in `src/main.rs`.
   - **Shared code:** Parser modules (`src/parser/`) and export functions (`src/export.rs`) are shared by both library and CLI.
   - **CLI as thin wrapper:** The CLI (`src/main.rs`) uses library export functions (`export_to_csv`, `export_to_gpx`, `export_to_event`) with CLI-specific status messages.
-  - **Current state:** **Full unification complete** — parsing and export layers unified, CLI reduced from ~1800 to ~1400 lines.
+  - **Current state:** **Full unification complete** — parsing and export layers unified, CLI is thin wrapper with zero public functions.
 - **Decision criteria:** "Is this needed by crate consumers?" determines placement — shared logic in library, CLI-only logic in `src/main.rs`.
 - **Feature flags:** `csv`, `cli`, `json`, `serde` control optional dependencies; default: `csv` + `cli`.
 - **CRATE_USAGE.md reference:** See `CRATE_USAGE.md` for library API examples with feature flags.

GOALS.md

@@ -1,10 +1,10 @@
-## Current Implementation Status (July 12 2025)
+## Current Implementation Status (December 19 2025)
 
 ✅ **COMPLETED GOALS:**
 - Full BBL binary format parsing using JavaScript blackbox-log-viewer and C blackbox-tools references
 - Complete I, P, S, H, G, E frame parsing with proper predictor implementation
 - Header parsing and field definition extraction with firmware metadata
-- CSV export with blackbox_decode compatible field ordering and formatting
+- CSV export with field structure following blackbox_decode conventions (exact ordering compatibility not comprehensively validated)
 - Headers CSV export with complete configuration and frame definitions
 - Proper field encoding/decoding (SIGNED_VB, UNSIGNED_VB, NEG_14BIT, TAG8_8SVB, TAG2_3S32, TAG8_4S16)
 - Motor value prediction with accurate P-frame decoding
@@ -14,29 +14,32 @@
 - Energy calculation (energyCumulative field) integration
 - Time-sorted CSV output with proper chronological ordering
 - Debug mode with comprehensive frame-by-frame analysis and sampling
-- Large file streaming support (500K+ frames) with memory efficiency
+- Large file streaming support (tested: 375K+ frames single log in 6.7 seconds; multi-log files with 400K+ combined frames) with memory efficiency
 - **G-frame (GPS) parsing and GPX file export** with coordinate conversion
 - **E-frame (event) parsing and JSONL event export** with Betaflight FlightLogEvent enum
 - **Betaflight firmware-accurate flag formatting** (flightModeFlags, stateFlags, failsafePhase)
 - **Official Betaflight event type mapping** (sync beep, disarm, flight mode change, log end)
 - **Extensive Betaflight/EmuFlight testing** with high compatibility across firmware versions
 - **Full RUST CRATE for library reusability and modularity** with complete API access
 - **API documentation and library integration** with comprehensive usage examples
+- **Library/CLI separation:** Parsing duplication removed, `parse_single_log` exposed in library
+- **Configurable export filtering:** Heuristics moved to library, accessible via `should_skip_export()` and `has_minimal_gyro_activity()`
+- **ExportReport type:** Structured path tracking for all export operations
+- **Public API audit:** Zero public functions in CLI, thin wrapper architecture
+- **Comprehensive test coverage:** 62 unit tests for parsing, filtering, conversions, and exports
 
-🔧 **REMAINING WORK:**
-- Code refinement: Replace unwrap() calls with proper error handling
-- Enhanced error handling and comprehensive edge case testing
-- Performance optimization for extremely large files (>1M frames)
-- Comprehensive GPS and Event frame testing across more log types
-- Complete crate migration (resolve internal structure inconsistencies)
-- Unit conversion options (time, voltage, current, height, speed, rotation, acceleration)
-- IMU simulation (roll/pitch/yaw angle computation from gyro/accel/mag)
-- Current meter simulation and energy integration
-- GPS merge option (integrate GPS data into main CSV)
-- Raw mode output (unprocessed sensor values)
-- Enhanced statistics output (frame counts, timing, loop statistics)
-- Extended firmware compatibility testing (older/newer versions)
-- Advanced filtering and data processing options
+🔧 **REMAINING WORK:** Feature Enhancements
+- **Error handling refinement:** Some unwrap() calls remain in test/example code; critical paths use proper Result handling
+- **Performance validation:** Streaming architecture proven effective; tested up to 375K frames in single log (~6.7 seconds for 21MB file)
+- **GPS & Event testing:** Both formats working and validated (833+ G frames, 5+ E frames; valid GPX and JSON outputs)
+- **Unit conversions expansion:** Voltage and current conversions complete and tested; missing: time scaling, altitude conversion, speed units, rotation rates, acceleration scaling
+- **IMU simulation:** Roll/pitch/yaw angle computation from gyro/accel/mag data — not started
+- **Current meter simulation:** Energy integration improvements — not started
+- **GPS data merge:** Integrate GPS data into main CSV (currently separate .gps.gpx file) — not started
+- **Raw mode output:** Export unprocessed sensor values without scaling — not started
+- **Enhanced statistics:** Loop timing statistics, frame distribution analysis — not started
+- **Extended firmware testing:** Currently validates Betaflight 4.5+, EmuFlight, INAV; additional versions not comprehensively tested
+- **Advanced filtering options:** Current implementation: duration + gyro variance heuristics; advanced options not implemented
 
 ---
 

OVERVIEW.md

@@ -1,8 +1,8 @@
 # BBL Parser - Project Overview
 
-**Project Status:** 🚧 **WORK IN PROGRESS**  
-**Focus:** High-Performance BBL Processing  
-**⚠️ Not Production Ready**
+**Project Status:** ✅ **ARCHITECTURE COMPLETE** | 🚧 **FEATURE DEVELOPMENT** | 🚧 **WORK IN PROGRESS**  
+**Focus:** High-Performance BBL Processing with Production-Ready Core  
+**Status:** Core parsing, export, and library/CLI separation (Phase 6) complete; remaining work is feature enhancements
 
 ---
 
@@ -43,7 +43,7 @@ A comprehensive Rust library and command-line tool for BBL (Blackbox Log) parsin
 | **Crate Documentation** | ✅ Functional | Comprehensive API documentation and examples |
 | **Error Handling** | 🚧 Basic | Needs comprehensive testing |
 | **Performance** | 🚧 Basic | Optimization in progress |
-| **Testing** | ⚠️ Limited | Needs extensive validation |
+| **Testing** | ✅ Comprehensive | 62 unit tests covering filters, conversions, parsing, exports |
 
 ---
 
@@ -115,7 +115,12 @@ src/
 - **Universal Format Support:** `.BBL`, `.BFL`, `.TXT` with case-insensitive matching
 - **Firmware Compatibility:** Betaflight, EmuFlight, INAV support
 - **Multi-log Processing:** Automatic detection of multiple flight sessions in single files
-- **Smart Export Filtering:** Automatically skips short test flights (<5s) while preserving high-quality short logs
+
+### **Smart Export Filtering**
+- **Duration-based:** < 5s skipped, 5–15s exported only if data density > 1500 fps, > 15s exported
+- **Gyro activity detection:** Minimal gyro variance indicates ground test vs. actual flight
+- **Configurable:** Available via library API `should_skip_export()` and `has_minimal_gyro_activity()` for programmatic control
+- **Override:** `--force-export` flag or `force_export` option bypasses filtering heuristics
 
 ### **Library API**
 - **Complete Data Access:** Programmatic access to all BBL data structures