Skip to content

feat: Add GRU (Gated Recurrent Unit) implementation with multi-layer support and dropout#6

Merged
kholdrex merged 2 commits into
mainfrom
gru-implementation
Jul 2, 2025
Merged

feat: Add GRU (Gated Recurrent Unit) implementation with multi-layer support and dropout#6
kholdrex merged 2 commits into
mainfrom
gru-implementation

Conversation

@kholdrex

@kholdrex kholdrex commented Jul 2, 2025

Copy link
Copy Markdown
Member

This PR adds a complete GRU (Gated Recurrent Unit) implementation to the Rust LSTM library, providing an alternative RNN architecture that's simpler yet often as effective as LSTM for many sequence modeling tasks.

✨ Features Added

Core Components

  • GRUCell: Complete GRU cell with mathematically correct forward/backward passes
  • GRUNetwork: Multi-layer GRU networks with flexible architecture
  • Dropout Support: All dropout types (input, recurrent, output, variational, layer-specific)
  • Training Integration: Full compatibility with existing optimizers (SGD, Adam, RMSprop)

Key Implementation Details

  • 3-gate architecture: Reset gate, Update gate, New gate computations
  • Efficient caching: Forward pass caching for training efficiency
  • Gradient computation: Accurate backpropagation through time (BPTT)
  • Mode switching: Proper training/evaluation mode handling
  • Memory efficient: Optimized array operations with ndarray

🚀 Benefits

  1. Simpler Architecture: 2 gates vs LSTM's 3 gates = fewer parameters & faster training
  2. Comparable Performance: Often matches LSTM performance with less complexity
  3. Better Gradient Flow: Effective handling of vanishing gradient problem
  4. Computational Efficiency: Lower memory and computational overhead
  5. Versatility: Excellent for many sequence modeling tasks

@kholdrex kholdrex self-assigned this Jul 2, 2025
@kholdrex kholdrex added the enhancement New feature or request label Jul 2, 2025
@kholdrex
kholdrex merged commit a803990 into main Jul 2, 2025
1 check passed
@kholdrex
kholdrex deleted the gru-implementation branch July 2, 2025 01:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant