Skip to content

CHANGELOG.md

Latest commit

 

History

History
427 lines (343 loc) · 19.6 KB

CHANGELOG.md

File metadata and controls

427 lines (343 loc) · 19.6 KB
Raw

Changelog

All notable changes to EEG-Dash will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

[0.8.4] - 2026-07-01

Fixed

  • Replaced s3fs with boto3 for S3 downloads, unpinning botocore so eegdash no longer conflicts with other packages' botocore requirements (#398)

Changed

  • Documentation site: performance overhaul (virtual-scrolled dataset table, lazy-loaded Plotly, dropped a ~1.5 MB icon bundle, shrank two ~7 MB API pages), prefers-reduced-motion accessibility support, and deployment moved to the GitHub Pages Actions flow (#394, #395, #396)
  • Repository slimmed: dead history (scratch notebooks, caches, generated dumps) purged and uv.lock untracked, cutting a fresh clone from >200 MB to ~17 MB

[0.8.3] - 2026-06-23

Added

  • EEGDash.find now warns when an explicitly requested filter value (e.g. a misspelled task inside a list of tasks) matches no records, instead of silently dropping it (#141)

  • EEGDashDataset validates target_name: the field is auto-added to description_fields, and a ValueError (listing the available fields) is raised when the target is missing for every recording — typically a misspelled name such as "p-factor" for "p_factor" (#21)

  • EEGDashDataset gained a remove_nan_targets parameter (default False): when target_name is set and remove_nan_targets=True, recordings whose target is missing (None/NaN) are dropped with a warning (#22)

  • build_query_from_kwargs (and therefore EEGDashDataset/EEGChallengeDataset keyword filters) accept a compiled re.Pattern, translated into a MongoDB $regex query with IGNORECASE/MULTILINE/DOTALL flags mapped onto $options (#135)

Fixed

  • Co-installing eegdash alongside packages that declare an unbounded numpy>=2.1 no longer backtracks numba to 0.53.1 / llvmlite 0.36 (which fail to build on Python ≥3.10). numba is now floored (>=0.61, and >=0.60 on the Intel-Mac <0.61 path), so the resolver settles on a recent numba/llvmlite and caps numpy to numba's supported range instead (#170)

Documentation

  • EEGDashDataset and EEGChallengeDataset docstrings now enumerate the allowed keyword filters (ALLOWED_QUERY_FIELDS), document scalar/list ($in) usage, clarify that non-filter keywords such as target_name are forwarded to braindecode, and note that a misspelled filter name is silently forwarded rather than raising (#211)

[0.8.2] - 2026-05-30

Changed

  • NEMAR records no longer embed sidecar file contents in storage.sidecar_inline. Sidecars/companions are fetched on demand from data.nemar.org via nemar-py (version-pinned manifest, checksum-verified), with the GitHub-raw pointer kept as a legacy fallback. Shrinks records ~250 KB → ~1.5 KB (a 1000-record API page dropped from ~258 MB to ~1.7 MB) and avoids serving stale digest-time snapshots (#369)

Fixed

  • EEGDashDataset.download_all() now fetches NEMAR recordings, which were previously skipped because their _raw_uri is resolved lazily (#369)

[0.8.1] - 2026-05-29

Fixed

  • Restored feature-extraction and dataset-IO work unintentionally removed by #359: feature decorators/extractors, the complexity/connectivity/signal feature banks, and eegdash.dataset.io (#365)

Added

  • Ingestion pipeline: golden-test safety net (InjectionPlan / ValidationReport / montage snapshots) plus a DDD-lite glossary and ADRs documenting the typed-pipeline design (#364)

[0.8.0] - 2026-05-28

Added

  • New public APIs: eegdash.splits (subject-aware split utilities) and EEGDashDataset.summary / preview / filter / search_datasets for in-memory query and inspection
  • 22-tutorial sphinx-gallery rewrite plus HPC how-tos and concept pages
  • NEMAR onNNNNNN IDs (OpenNeuro mirrors on data.nemar.org) are recognised as the NEMAR backend (#361)
  • nemar-py>=0.2.0 runtime dependency: data.nemar.org access (manifest, BIDS-path → annex-key, streaming download) is delegated to the official client (#361)
  • Features module additions: spectral_peak_frequency, more connectivity features, preprocessor_as_feature wrapper, channel-picking on extractors
  • Documentation: per-dataset pages with embedded trace viewer + electrode explorer; SEO and performance overhaul

Changed

  • Bumped mne-bids minimum to 0.19.0
  • Ingestion pipeline refactor: pydantic-settings configs, shared seam modules, and new test infrastructure under eegdash-testing-data
  • eegdash.features.fit_feature_extractor kept as a backward-compat alias for the new feature-extraction entrypoint

Fixed

  • Loader robustness sweep across the 522-dataset catalog: companion-file auto-discovery (.fdt/.eeg/.vmrk), git-annex pointer resolution, split-FIF handling, BIDS metadata recovery (encoding, decimal separator, whitespace, acq_time timezone), and clean DataIntegrityError paths
  • NEMAR anonymous S3 + GitHub access works end-to-end; ingestion now reconciles source/storage.base against the dataset ID pattern, and sidecars are inlined at digest time to avoid runtime GitHub round-trips
  • _generate_coordsystem_json recognises the emg BIDS datatype and writes EMGCoordinateSystem / EMGCoordinateUnits per BEP 030 (#361)
  • Intel-Mac install (#294)

Removed

  • Legacy tutorial entrypoints superseded by the new gallery (examples/tutorial_minimal.py, examples/tutorial_transfer_learning.py, several examples/noplot_tutorial_*.py)

[0.7.2] - 2026-04-29

Added

  • NEMAR digest-time sidecar inlining: storage.sidecar_inline carries small text BIDS sidecars (TSV/JSON/README) so the runtime never round-trips to GitHub for an enriched record (#334)

Changed

  • Quieted canonical-alias collision logs in the dataset registry; refreshed uv.lock (#333)

[0.7.1] - 2026-04-29

Fixed

  • NEMAR anonymous S3 + GitHub access end-to-end: stop probing with ListBucket, use s3fs.get_file() for direct anonymous GetObject, and route via GitHub-pointer-first with the largefiles annex rule (#332)

[0.7.0] - 2026-04-28

Added

  • Pick-channels feature for feature extractors (#302)
  • Canonical-name aliases for dataset class registry (#306)
  • Feature-as-preprocessor wrapper, including preprocessor_as_feature (#308)
  • Montage registry pipeline with S3 FIF header fallback (#325)
  • Embedded electrode-explorer iframe on each dataset documentation page (#326)
  • Feature dataset and extractor improvements (#323)
  • Feature extractor addons (#331)

Fixed

  • Reconcile source / storage.base against dataset_id pattern in ingest (#329)
  • Feature column-naming consistency in extractor output (#330)
  • signal_decorrelation_time logic correction (#328)
  • NEMAR source detection and removal of three redirecting OpenNeuro IDs (#327)
  • Dataset hero badge row alignment by pinning each :width: to SVG natural size (#324)
  • requestIdleCallback signature plus shield-badge aspect ratio (#322)
  • Restore _rewrite_sitemap_index dropped during #318 rebase (#321)
  • Rewrite homepage entry to canonical bare-host URL in sitemap (#319)
  • SEO batch-2: short/missing metas, duplicate tags, plus local validator and audit log (#318)
  • Cap long meta descriptions broken by an apostrophe regex (#317)
  • P0+P1 figure fixes — Sankey SVG export, clinical split, growth log, bubble labels

Changed

  • Refactor bivariate iterator to enforce directed/undirected logic on metrics (#309, #310)
  • Convert feature class decorators to functions (#305)
  • Minimize runtime dependencies; replace parquet with safetensors for feature serialization

Performance

  • Drop 2.2 MiB per-page payload, recover 1.1K+ 404s, add GSC/Bing/IndexNow hooks (#315)
  • Vendor Fuse.js plus post-build asset fingerprinting (#320)

Documentation

  • Polish catalog charts, table, intro and extend author-year alias resolution (#307)
  • Structured data, Open Graph cards, meta descriptions, JS gating (#311)
  • Agent-readiness, library-first hero, paper-accurate citation block (#312)
  • llms.txt coverage, dataset_summary.md shrink, directive hints (#314)
  • Remove AI-isms across user-facing documentation (#316)
  • Redesigned social card with brand identity and code snippet

[0.6.0] - 2026-04-06

Added

  • Publication-quality chart redesign with PDF export (#299)
  • EEGDash API tutorial (#283)
  • Intel Mac system support with conditional numba and torch pinning (#294)
  • Preprocessor output type system with documentation and inspection (#278, #289, #290, #291, #292)
  • Feature extractor configuration files (#286)
  • Optional recording-info parameter for feature extractors (#284, #285)
  • on_error parameter to skip bad records during data loading (#276)
  • CTF direct reader for .ds directories (#279)
  • BrainVision handling with auto-repair and metadata generation (#236)
  • Enhanced documentation search with autocomplete and live search (#238)
  • fNIRS metadata extraction support (#238)
  • EDF/BDF metadata extraction via MNE
  • Auto-sync dataset CSV with git-annex size correction (#298)
  • Duration computation for recordings
  • datasets_dict dictionary for programmatic dataset access (#209)
  • HTTP API client (#214)
  • acq- BIDS entity support in record pipeline (#247)
  • Pathology and modality labels for dataset summary (#212)
  • Number of sessions column in dataset summary table
  • Unified color palette across plots and CSS tags

Fixed

  • Reliable data loading across 522 EEG datasets (#282)
  • Git-annex key path resolution to BIDS names for S3 downloads (#280)
  • BIDS TSV whitespace-padded n/a values (#278)
  • BIDS coordinate system validation and extended non-numeric run fallback (#277)
  • EEGLAB format: truncated .fdt handling (#266), epoch/trial mismatch (#275), latin-1 encoding (#269), error handling (#272)
  • CTF "Illegal date" handling for numeric dash dates (#262)
  • Corrupt MAT/EEGLAB file detection (#265)
  • Participants.tsv subject/session ID repair (#264)
  • Non-numeric run entities via check=False in BIDSPath (#263)
  • Empty or malformed channels.tsv handling (#270)
  • Auto-discovery and download of companion files .fdt, .eeg, .vmrk (#268)
  • Split FIF file handling and continuation file downloads (#273)
  • TSV encoding fallback order (#240, #241)
  • Case-insensitive BIDS sidecar matching for task entity mismatches (#242, #243)
  • Duplicate participant_id in participants.tsv (#244)
  • European comma decimal separators in BIDS TSV files (#257)
  • iEEG coordsystem.json key handling (#236)
  • Git-annex key pointer rewriting in VHDR files (#259, #261)
  • Corrupt data, bad timestamps, and invalid BIDS entity handling (#260)
  • Directory-based format recursive download support (#253)
  • Dynamic cumulative_sizes and accurate nchans computation (#243)
  • BIDSPath entity extraction for git-annex datasets (#245)
  • Subject/participant TSV fallback handling (#248)
  • Signal filter preprocessor bug (#288)
  • Anonymous hyperlinks in docstrings to avoid duplicate target warnings
  • MNE/Dataset compatibility with retry handlers for multiple failing datasets (#271)

Changed

  • Switched to released PyPI versions for MNE and MNE-BIDS dependencies
  • Required pandas>=2.0 (#210)
  • Consolidated _load_raw error handling into single retry loop
  • Expanded default BIDS modalities to include SNIRF
  • Bumped braindecode dependency to >=1.4.0

Documentation

  • Full docstrings for Features module (#281, #287)
  • Dataset documentation with tags, feedback section, and visualization updates (#235)
  • P3 oddball tutorial updates (#192)
  • Improved dataset page layout and README parser

[0.5.0] - 2026-01-07

Added

  • New EEGDash logo and branding assets (#200)
  • CNAME configuration for custom domain support (#187)
  • Favicon configuration for documentation
  • Preprocessors as standalone functions (#194)
  • Data digestion pipeline v2 (#193, #215)
  • Mass ingestion fixes and optimization (#227)
  • HPC, Docker, and Singularity usage examples
  • Age regression tutorial
  • iEEG documentation

Fixed

  • Warning spam during EEGChallengeDataset download (#226)
  • Sphinx documentation build warnings
  • Import path resolution in documentation generation
  • Completed missing entries in the dataset summary table

Changed

  • Updated repository organization to eegdash/EEGDash
  • Updated license from GPL to BSD-3-Clause (#199)
  • Updated color theme to match new logo (primary blue: #003D82, accent orange: #FF8000)
  • Updated GitHub references to new organization
  • Cleaned up legacy code and removed dead code (#182, #184)
  • Use API dataset summary for CI docs (#224)

Documentation

  • Parallelized documentation build for faster generation (#232)
  • Improved quickstart guide aesthetics (#232)
  • Regenerated dataset summary and documentation (#220)
  • Enhanced footer with logo branding
  • Updated tutorial documentation (#202)
  • Improved documentation styling and visual hierarchy

[0.4.1] - 2025-10-21

Added

  • Treemap visualization for dataset statistics
  • Sankey, bubble, and ridgeline plots for data exploration
  • Time estimation feature for tutorials
  • User guide documentation for EEGDash
  • Warning system for nonexistent query conditions
  • Full API documentation

Fixed

  • Python 3.10 compatibility issues with type annotations
  • S3 download timeout handling for large files
  • Cache directory inconsistencies in CI/CD
  • MongoDB connection warnings and error messages
  • Import errors in feature modules
  • Documentation generation warnings
  • Orphaned documentation files

Changed

  • Optimized offline mode performance (2x speedup)
  • Replaced isort with ruff for import sorting
  • Improved BIDS metadata caching efficiency
  • Enhanced dataset loading speed
  • Cleaned up legacy code and removed dead code

Performance

  • 2x speedup in offline dataset loading
  • Vectorized feature extraction
  • S3 download retry logic with exponential backoff

Documentation

  • Added user guide
  • Improved API documentation structure
  • Added tutorial time estimates
  • Updated logo and visual assets
  • Enhanced developer notes

[0.4.0] - 2025-10-11

Added

  • Dataset registry system for dynamic OpenNeuro dataset registration
  • Support for multiple data releases
  • Mini-release functionality for testing and development
  • Enhanced BIDS dataset integration
  • Feature extraction preprocessing as standalone functions
  • PyArrow support for saving/loading feature dataframes
  • Visualization tools for dataset distribution (bubble plots, Sankey diagrams)

Fixed

  • GitHub worker configuration to reduce costs (Linux-only by default)
  • Cache directory path resolution across different platforms
  • Download functionality for BIDS dependencies
  • Pre-commit hook configuration issues

Changed

  • Refactored load_eeg_attrs_from_bids_file for better modularity
  • Moved feature preprocessing logic to separate functions
  • Improved CI/CD caching strategy for datasets
  • Updated GitHub Actions workflow for better efficiency

Security

  • Improved MongoDB connection string handling
  • Enhanced environment variable management

[0.3.x] - 2025-09-xx

Added

  • Sphinx documentation system with GitHub Pages deployment
  • Custom API documentation pages
  • Sex classification tutorial
  • P3 oddball and audio task tutorials
  • Field consistency validation tests
  • Support for EEGLAB (.set) file format
  • Rich console output for better user experience

Fixed

  • Downloader module isolation and error handling
  • Cache directory default behavior
  • Import paths for various modules
  • Logo and branding assets

Changed

  • Improved documentation structure and organization
  • Enhanced tutorial examples
  • Better error messages and logging

Documentation

  • Initial documentation deployment to GitHub Pages
  • Added end-to-end examples
  • Created tutorial notebooks for common tasks

[0.2.0] - 2025-xx-xx

Added

  • NeurIPS 2025 challenge support
  • Custom S3 bucket specification capability
  • Support for braindecode 1.0+
  • Type hints for top-level functionality
  • Field consistency testing

Changed

  • Upgraded to latest braindecode version
  • Improved API consistency
  • Enhanced database query capabilities

Fixed

  • Pre-commit hook configurations
  • API compatibility with newer dependencies
  • Various import and dependency issues

[0.1.x] - Initial Releases

Added

  • Initial EEGDash API for MongoDB queries
  • EEGDashDataset for PyTorch-compatible data loading
  • EEGDashBaseDataset for single recording access
  • BIDS format support
  • S3 data downloading functionality
  • MongoDB connection management
  • Feature extraction framework with 60+ features:
    • Complexity features (entropy, Lempel-Ziv complexity)
    • Connectivity features (coherence, imaginary coherence)
    • CSP (Common Spatial Pattern) features
    • Dimensionality features (fractal dimensions, Hurst exponent)
    • Signal features (statistical measures)
    • Spectral features (power, entropy, bands)
  • OpenNeuro dataset integration
  • HBN (Healthy Brain Network) specific preprocessing
  • Braindecode integration for preprocessing
  • pytest-based testing infrastructure

Infrastructure

  • GitHub Actions CI/CD pipeline
  • Pre-commit hooks with ruff linting
  • Sphinx documentation setup
  • PyPI package publishing automation

Release Schedule

  • Patch releases (0.x.Y): Bug fixes, documentation updates (as needed)
  • Minor releases (0.X.0): New features, non-breaking changes (monthly to quarterly)
  • Major releases (X.0.0): Breaking changes, major refactors (when necessary)

Contributing

See CONTRIBUTING.md for guidelines on how to contribute to this project.

Links

Authors and Contributors

Core Team

Contributors

  • Pierre Guetschel
  • Vivian Chen
  • Christian Kothe
  • And many others who have contributed through pull requests and issue reports

Acknowledgments

EEG-DaSh is a collaborative initiative between the United States and Israel, supported by the National Science Foundation (NSF). The partnership brings together experts from:

  • Swartz Center for Computational Neuroscience (SCCN) at UC San Diego
  • Ben-Gurion University (BGU) in Israel

License

This project is licensed under the BSD 3-Clause License - see the LICENSE file for details.

Version History Summary

Version Release Date Key Features
0.7.2 2026-04-29 NEMAR digest-time sidecar inlining; runtime never round-trips to GitHub for enriched records
0.7.1 2026-04-29 NEMAR anonymous S3 access fix (GitHub-pointer-first, anonymous GetObject)
0.7.0 2026-04-28 Montage registry, electrode-explorer integration, feature-as-preprocessor, parquet→safetensors, SEO/perf overhaul
0.6.0 2026-04-06 Reliable loading across 522 datasets, publication charts, API tutorial, fNIRS support
0.5.0 2026-01-07 New branding, digestion pipeline v2, preprocessors as functions, BSD-3 license
0.4.1 2025-10-21 Performance optimizations, visualization tools, comprehensive documentation
0.4.0 2025-10-11 Dataset registry, feature preprocessing refactor, multi-release support
0.3.x 2025-09-xx Documentation system, tutorials, GitHub Pages deployment
0.2.0 2025-xx-xx NeurIPS challenge support, braindecode 1.0+ compatibility
0.1.x 2024-2025 Initial release with core functionality