Skip to content

overview.md

Latest commit

 

History

History
157 lines (110 loc) · 5.47 KB

overview.md

File metadata and controls

157 lines (110 loc) · 5.47 KB
Raw

Hyperdimensional Computing (HDC) Overview

Introduction

Hyperdimensional Computing (HDC), also known as Vector Symbolic Architectures (VSA) or Computing with High-Dimensional Vectors, is a brain-inspired computing paradigm that performs computations using high-dimensional vectors (typically 1,000-10,000 dimensions). HDC provides a unified framework for representing and manipulating various types of data—from scalars to complex data structures—using vectors in high-dimensional spaces.

Key Concepts

High-Dimensional Vectors (Hypervectors)

The fundamental unit in HDC is a hypervector—a vector with thousands of dimensions. These vectors exhibit several unique properties:

  • Quasi-orthogonality: Random hypervectors are nearly orthogonal with high probability
  • Robustness: High tolerance to noise and component failures
  • Holographic: Information is distributed across all dimensions
  • Fixed-width: All data types use the same vector dimensionality

Core Operations

HDC relies on three fundamental operations:

  1. Binding (⊗): Combines two hypervectors to create a new hypervector that is dissimilar to both inputs

    • Binary vectors: XOR operation
    • Bipolar vectors: Element-wise multiplication
    • Creates associations between concepts

  2. Bundling (+): Superimposes multiple hypervectors to create a new hypervector similar to all inputs

    • Binary vectors: Majority voting
    • Bipolar vectors: Element-wise addition with sign
    • Creates sets or collections

  3. Permutation (ρ): Rearranges vector components to create a dissimilar but related vector

    • Used for encoding sequences and positions
    • Preserves distance relationships

Similarity Measures

HDC uses various similarity metrics to compare hypervectors:

  • Hamming distance: For binary vectors
  • Cosine similarity: For real-valued vectors
  • Dot product: Fast similarity approximation

Architecture Components

1. Hypervector Types

The package supports multiple hypervector representations:

from cognitive_computing.hdc import HDC, HDCConfig

# Binary hypervectors {0, 1}
binary_hdc = HDC(HDCConfig(dimension=10000, hypervector_type="binary"))

# Bipolar hypervectors {-1, +1}
bipolar_hdc = HDC(HDCConfig(dimension=10000, hypervector_type="bipolar"))

# Ternary hypervectors {-1, 0, +1}
ternary_hdc = HDC(HDCConfig(dimension=10000, hypervector_type="ternary"))

# Multi-level hypervectors
level_hdc = HDC(HDCConfig(dimension=10000, hypervector_type="level", levels=5))

2. Data Encoders

HDC provides encoders to convert various data types into hypervectors:

  • ScalarEncoder: Continuous values using thermometer or level encoding
  • CategoricalEncoder: Discrete categories with orthogonal hypervectors
  • SequenceEncoder: Ordered sequences using n-grams or positional encoding
  • SpatialEncoder: 2D/3D coordinates preserving spatial relationships
  • RecordEncoder: Structured data with field-value bindings
  • NGramEncoder: Text data using character or word n-grams

3. Associative Memory

The ItemMemory class provides content-addressable storage:

from cognitive_computing.hdc import ItemMemory

memory = ItemMemory(dimension=10000)
memory.add("apple", apple_vector)
memory.add("banana", banana_vector)

# Query with noisy vector
results = memory.query(noisy_apple, top_k=3)
cleaned, label = memory.cleanup(noisy_apple)

4. HDC Classifiers

Several classification algorithms leverage HDC principles:

  • OneShotClassifier: Learn from single examples
  • AdaptiveClassifier: Online learning with momentum
  • EnsembleClassifier: Combine multiple classifiers
  • HierarchicalClassifier: Multi-level classification

Applications

HDC is particularly well-suited for:

  1. Cognitive Computing: Modeling human-like reasoning and memory
  2. Sensor Fusion: Combining data from multiple sensors
  3. Natural Language Processing: Semantic representation of text
  4. Pattern Recognition: Robust classification with few examples
  5. Edge Computing: Efficient computation on resource-constrained devices
  6. Fault-Tolerant Systems: Graceful degradation with component failures

Advantages

  • Efficiency: Simple operations (XOR, addition) suitable for hardware
  • Robustness: Tolerates noise and hardware faults
  • Scalability: Fixed-width representations for all data types
  • Interpretability: Similarity measures provide semantic meaning
  • One-shot Learning: Learn from single examples
  • Compositionality: Complex structures from simple operations

Getting Started

from cognitive_computing.hdc import HDC, HDCConfig, ItemMemory

# Create HDC system
config = HDCConfig(dimension=10000, hypervector_type="bipolar")
hdc = HDC(config)

# Generate hypervectors
fruit = hdc.generate_hypervector()
red = hdc.generate_hypervector()
sweet = hdc.generate_hypervector()

# Create composite concept
apple = hdc.bind(fruit, hdc.bind(red, sweet))

# Bundle similar items
banana = hdc.bind(fruit, hdc.bind(yellow, sweet))
fruits = hdc.bundle([apple, banana])

# Store in memory
memory = ItemMemory(dimension=10000)
memory.add("apple", apple)
memory.add("fruits", fruits)

# Query with properties
red_fruit = hdc.bind(red, fruit)
results = memory.query(red_fruit, top_k=2)

Next Steps