Skip to content

adapters_implementation_summary.md

Latest commit

 

History

History
249 lines (200 loc) · 8.02 KB

adapters_implementation_summary.md

File metadata and controls

249 lines (200 loc) · 8.02 KB

BCI Adapters Implementation Summary

Overview

This document summarizes the implementation of device adapters for the BCI compression toolkit, including new adapters for Blackrock, Intan, and HDF5 data formats.

Completed Features

Device Adapters (4 total)

1. OpenBCI Adapter ✅

  • File: src/bci_compression/adapters/openbci.py
  • Devices: Cyton (8ch @ 250Hz), Daisy (16ch @ 250Hz)
  • Features: 10-20 electrode mapping, frontal/central/parietal/occipital grouping
  • Tests: 26 tests passing
  • Demo: examples/openbci_adapter_demo.py (6 scenarios)

2. Blackrock Adapter ✅ NEW

  • File: src/bci_compression/adapters/blackrock.py
  • Devices: Neuroport (96ch @ 30kHz), Cerebus (128ch @ 30kHz)
  • Features: Utah array grid layout, motor/sensory cortex regions, NEV file support
  • Tests: 19 tests passing
  • Key Capabilities:
    • 10x10 Utah array electrode mapping
    • High-frequency neural recording (30 kHz)
    • Grid-based channel grouping (rows + cortical regions)
    • NEV file loading (placeholder for future integration)

3. Intan Adapter ✅ NEW

  • File: src/bci_compression/adapters/intan.py
  • Devices: RHD2132 (32ch @ 20kHz), RHD2164 (64ch @ 20kHz), RHS128 (128ch @ 30kHz)
  • Features: Headstage type tracking, stimulation capability detection, RHD file support
  • Tests: Not yet created (pending)
  • Key Capabilities:
    • Support for RHD and RHS headstages
    • Stimulation capability tracking
    • High-frequency LFP recording
    • RHD file loading (placeholder for future integration)

4. HDF5 Adapter ✅ NEW

  • File: src/bci_compression/adapters/hdf5.py
  • Features: Generic HDF5 file loading, auto-detection from metadata, dataset exploration
  • Tests: Not yet created (pending)
  • Key Capabilities:
    • Flexible data path specification
    • Partial data loading (slicing support)
    • Metadata extraction
    • Dataset listing and exploration
    • Auto-detection of sampling rate and channel count

Real-World Examples (3 total) ✅ NEW

1. Streaming Compression Example

  • File: examples/streaming_compression_example.py
  • Features:
    • Real-time sliding window processing
    • Circular buffer management
    • Latency monitoring (<1ms achievable)
    • Throughput statistics
  • Demos:
    • Standard streaming (4s windows, 1s overlap)
    • Low-latency mode (400ms windows, 40ms overlap)

2. Multi-Device Pipeline Example

  • File: examples/multi_device_pipeline_example.py
  • Features:
    • Unified pipeline for multiple devices
    • Hierarchical compression (lossless for high-priority, lossy for others)
    • Channel alignment across devices
    • Multi-device streaming simulation
  • Demos:
    • Basic pipeline (OpenBCI + Blackrock + Intan)
    • Hierarchical compression strategy
    • Channel alignment and resampling
    • 5-second multi-device recording

3. Performance Profiling Script

  • File: scripts/profile_adapters.py
  • Features:
    • Adapter overhead measurement
    • Resampling performance benchmarks
    • Channel grouping efficiency
    • Memory usage analysis
    • Hot path profiling (cProfile)
  • Metrics:
    • Time per operation (ms)
    • Throughput (k samples/sec)
    • Memory usage (MB)
    • Full pipeline performance

Performance Results

Adapter Benchmarks (from profiling script)

OpenBCI Adapter:
  mapping              0.028ms    356,810k samples/s     0.61 MB
  resample_1000Hz      6.566ms      1,523k samples/s     0.61 MB
  channel_groups       0.027ms    365,103k samples/s     0.61 MB
  full_pipeline        0.059ms    169,948k samples/s     0.61 MB

Blackrock Adapter:
  mapping              2.488ms     12,059k samples/s    21.97 MB
  resample_1000Hz     49.278ms        609k samples/s    21.97 MB
  channel_groups       4.848ms      6,188k samples/s    21.97 MB
  full_pipeline        4.216ms      7,116k samples/s    21.97 MB

Intan Adapter:
  mapping              0.420ms     47,630k samples/s     9.77 MB
  resample_1000Hz     21.775ms        919k samples/s     9.77 MB
  channel_groups       1.724ms     11,600k samples/s     9.77 MB
  full_pipeline        1.803ms     11,090k samples/s     9.77 MB

Streaming Performance

  • Standard mode: 0.06ms average latency, 2.00x compression ratio
  • Low-latency mode: 0.091ms average latency, <1ms max
  • Throughput: 2,000 samples/sec (sufficient for real-time BCI)

Test Coverage

Component Tests Status
Core adapters 26 ✅ Passing
OpenBCI adapter 26 ✅ Passing
Blackrock adapter 19 ✅ Passing
Intan adapter 0 ⭕ Pending
HDF5 adapter 0 ⭕ Pending
Total 45 45 passing

API Examples

Quick Start

# OpenBCI
from bci_compression.adapters.openbci import OpenBCIAdapter
adapter = OpenBCIAdapter(device='cyton_8ch')
data = adapter.convert(raw_data)

# Blackrock
from bci_compression.adapters.blackrock import BlackrockAdapter
adapter = BlackrockAdapter(device='neuroport_96ch')
data = adapter.resample_to(raw_data, target_rate=1000)

# Intan
from bci_compression.adapters.intan import IntanAdapter
adapter = IntanAdapter(device='rhd2164_64ch')
groups = adapter.get_channel_groups()

# HDF5 (generic)
from bci_compression.adapters.hdf5 import HDF5Adapter
adapter = HDF5Adapter.from_hdf5('data.h5', data_path='/neural/raw')
data = adapter.load_data(start_sample=0, end_sample=10000)

Multi-Device Pipeline

from bci_compression.adapters import MultiDevicePipeline

pipeline = MultiDevicePipeline()
pipeline.add_device('openbci', openbci_adapter, priority='normal')
pipeline.add_device('blackrock', blackrock_adapter, priority='high')
pipeline.add_device('intan', intan_adapter, priority='normal')

compressed = pipeline.process_batch({
    'openbci': openbci_data,
    'blackrock': blackrock_data,
    'intan': intan_data
})

summary = pipeline.get_summary()

File Structure

src/bci_compression/adapters/
├── __init__.py              # Core adapter functions
├── openbci.py               # OpenBCI Cyton/Daisy
├── blackrock.py             # Blackrock Neuroport/Cerebus (NEW)
├── intan.py                 # Intan RHD/RHS headstages (NEW)
└── hdf5.py                  # Generic HDF5 loader (NEW)

examples/
├── openbci_adapter_demo.py              # OpenBCI demos
├── streaming_compression_example.py      # Streaming processing (NEW)
└── multi_device_pipeline_example.py     # Multi-device integration (NEW)

tests/
├── test_adapters.py         # Core adapter tests (26)
└── test_blackrock_adapter.py # Blackrock tests (19) (NEW)

scripts/
└── profile_adapters.py      # Performance profiling (NEW)

docs/
├── adapters_guide.md        # User guide
└── adapters_implementation_summary.md  # This document (NEW)

Next Steps

Immediate (High Priority)

  1. ✅ Create Blackrock adapter tests — COMPLETE (19 tests)
  2. ⭕ Create Intan adapter tests — PENDING
  3. ⭕ Create HDF5 adapter tests — PENDING

Short-term

  • Add NEV/NSx file parser (requires neo library)
  • Add RHD/RHS file parser (requires intanutil library)
  • Optimize resampling performance (GPU acceleration)
  • Add more compression algorithms to pipeline

Long-term

  • Add more device adapters (Ripple, Plexon, etc.)
  • Cloud storage integration for HDF5 files
  • Real-time streaming from hardware devices
  • Web dashboard for multi-device monitoring

Dependencies

Core

  • numpy
  • scipy
  • h5py

Optional (for file format support)

  • neo (for Blackrock NEV/NSx files)
  • intanutil (for Intan RHD/RHS files)

Changelog

2025-01-XX - Adapters Phase 2

  • ✅ Added Blackrock adapter (Neuroport 96ch, Cerebus 128ch)
  • ✅ Added Intan adapter (RHD2132, RHD2164, RHS128)
  • ✅ Added HDF5 generic adapter
  • ✅ Created streaming compression example
  • ✅ Created multi-device pipeline example
  • ✅ Created performance profiling script
  • ✅ Added 19 Blackrock adapter tests (all passing)
  • ✅ Verified streaming latency <1ms
  • ✅ Benchmarked adapter overhead and throughput

Contributors

Generated by GitHub Copilot with guidance from project requirements.