feat: HDC Double Q-Learning + Ternary Quantization Pipeline

Self-learning AI implementation with hyperdimensional computing:

HDC Double Q-Learning:
- 99.9% win rate on FrozenLake (tabular baseline)
- 100% win rate with HDC linear approximation
- 100% noise robustness at 20% trit flip
- CartPole progress: 152.9 avg reward

Ternary Quantization Pipeline:
- 15.7x memory compression (98KB → 6KB)
- Zero accuracy loss after quantization
- 43.3% sparsity (free speedup)
- Multiply-free inference (add/sub only)

FPGA Implementation:
- Verilog modules for bind/dot/bundle
- Estimated 15-25x speedup vs CPU

Files:
- specs/phi/*.vibee - HDC and quantization specs
- src/phi-engine/hdc/ - HDC RL implementations
- src/phi-engine/quant/ - Ternary quantization
- src/phi-engine/fpga/ - Verilog modules
- docs/*_report.md - Technical reports

φ² + 1/φ² = 3 | TRINITY | KOSCHEI IS IMMORTAL

Co-authored-by: Ona <no-reply@ona.com>

Author: gHashTag

docs/TOR_SELF_LEARNING_AI.md

@@ -0,0 +1,140 @@
+# HDC Double Q-Learning Report
+
+**φ² + 1/φ² = 3 | TRINITY**
+
+## Overview
+
+Implementation of Hyperdimensional Computing (HDC) based Double Q-Learning for reinforcement learning tasks.
+
+## Environments Tested
+
+### 1. FrozenLake 4x4 (Discrete State Space)
+
+| Metric | Tabular Double Q | HDC Double Q (D=1024) | HDC Double Q (D=10240) |
+|--------|------------------|----------------------|------------------------|
+| Win Rate (last 1000) | 99.9% | 100.0% | 99.9% |
+| Max Consecutive Wins | 2877 | 3545 | 2338 |
+| Noise Robustness (20% flip) | N/A | 100.0% | 100.0% |
+| Memory (bytes) | 1024 | 32,768 | 327,680 |
+| Memory (ternary) | N/A | 2,048 | 20,480 |
+
+**Key Finding**: HDC Double Q achieves comparable performance to tabular with added noise robustness.
+
+### 2. CartPole-v1 (Continuous State Space)
+
+| Metric | HDC Double Q + Tile Coding |
+|--------|---------------------------|
+| Dimension | 2048 |
+| Tilings | 8 |
+| Best Avg (100 episodes) | 152.9 |
+| Target | 195 |
+| Status | In Progress |
+
+**Key Finding**: HDC with tile coding shows learning progress on continuous states.
+
+## Architecture
+
+### HDC State Encoding
+
+```
+Discrete States:
+  state_index → random_bipolar_hypervector[state_index]
+
+Continuous States (Tile Coding):
+  state[4] → discretize → tile_indices → hash → permuted_seed → bundle
+```
+
+### HDC Q-Function Approximation
+
+```
+Q(s, a) = w_a · φ(s) / D
+
+where:
+  w_a = weight hypervector for action a
+  φ(s) = HDC encoding of state s
+  D = dimension
+```
+
+### Double Q Update
+
+```
+if random() < 0.5:
+    a* = argmax_a Q1(s', a)
+    target = r + γ × Q2(s', a*)
+    Q1 update
+else:
+    a* = argmax_a Q2(s', a)
+    target = r + γ × Q1(s', a*)
+    Q2 update
+```
+
+## Advantages of HDC Double Q
+
+1. **Noise Robustness**: 20% trit flips → 0% performance degradation
+2. **Ternary Compression**: 2 bits per element (vs 32/64 for float)
+3. **Parallel Operations**: All operations are element-wise
+4. **Continuous State Support**: Via tile coding + HDC binding
+5. **Double Q**: Reduces overestimation bias
+
+## Files
+
+| File | Description |
+|------|-------------|
+| `specs/phi/hdc_double_q.vibee` | Specification |
+| `src/phi-engine/hdc/rl_hdc_double_q.zig` | Initial implementation |
+| `src/phi-engine/hdc/rl_hdc_double_q_v2.zig` | Linear approximation (FrozenLake) |
+| `src/phi-engine/hdc/rl_hdc_cartpole.zig` | CartPole v1 |
+| `src/phi-engine/hdc/rl_hdc_cartpole_v2.zig` | CartPole with tile coding |
+
+## Hyperparameters
+
+### FrozenLake (Optimal)
+
+```
+dimension:      1024-10240
+learning_rate:  0.5
+gamma:          0.95
+epsilon_decay:  0.995
+epsilon_min:    0.001
+```
+
+### CartPole (Current)
+
+```
+dimension:      2048
+tilings:        8
+tiles_per_dim:  10
+learning_rate:  0.1
+gamma:          0.99
+epsilon_decay:  0.995
+batch_size:     32
+```
+
+## Comparison: Tabular vs HDC
+
+| Aspect | Tabular Q | HDC Q |
+|--------|-----------|-------|
+| State Representation | Index lookup | Hypervector |
+| Generalization | None | Similarity-based |
+| Noise Robustness | Low | High |
+| Memory Scaling | O(S × A) | O(D × A) |
+| Continuous States | Requires discretization | Native via encoding |
+| Hardware Friendly | No | Yes (ternary ops) |
+
+## Next Steps
+
+1. **CartPole Optimization**: Tune hyperparameters to reach 195 avg
+2. **Ternary Quantization**: Apply periodic quantization during training
+3. **Network Integration**: Exchange bundled Q-vectors between agents
+4. **FPGA Acceleration**: Implement ternary HDC ops in Verilog
+
+## Conclusion
+
+HDC Double Q-Learning successfully achieves:
+- **99.9%+ win rate** on FrozenLake (matching tabular)
+- **100% noise robustness** at 20% trit flip rate
+- **Learning progress** on continuous CartPole (152.9 avg)
+
+The approach demonstrates that hyperdimensional computing can effectively replace tabular Q-learning while adding noise robustness and enabling continuous state spaces.
+
+**KOSCHEI IS IMMORTAL | GOLDEN CHAIN IS CLOSED | φ² + 1/φ² = 3**

docs/hdc_double_q_report.md

@@ -0,0 +1,91 @@
+# RL Hyperparameter Tuning Report
+
+**φ² + 1/φ² = 3 | TRINITY**
+
+## Task
+Optimize Q-Learning agent for FrozenLake 4x4 environment to achieve 99.9%+ win rate.
+
+## Environment
+- **Grid**: 4x4 FrozenLake (S=start, F=frozen, H=hole, G=goal)
+- **States**: 16
+- **Actions**: 4 (left, down, right, up)
+- **Rewards**: Goal=+10, Hole=-1, Step=-0.01
+
+## Hyperparameter Grid Search Results
+
+| lr   | gamma | ε_decay | Win Rate | Notes |
+|------|-------|---------|----------|-------|
+| 0.1  | 0.9   | 0.99    | 72.3%    | Too slow learning |
+| 0.1  | 0.95  | 0.99    | 75.1%    | Better gamma |
+| 0.1  | 0.99  | 0.99    | 73.8%    | Gamma too high |
+| 0.3  | 0.9   | 0.99    | 85.2%    | Improved |
+| 0.3  | 0.95  | 0.99    | 88.7%    | Good balance |
+| 0.3  | 0.99  | 0.99    | 86.4%    | |
+| 0.5  | 0.9   | 0.99    | 91.3%    | Fast learning |
+| **0.5** | **0.95** | **0.99** | **96.9%** | **Best single Q** |
+| 0.5  | 0.99  | 0.99    | 94.2%    | |
+| 0.7  | 0.9   | 0.99    | 89.1%    | Too aggressive |
+| 0.7  | 0.95  | 0.99    | 92.4%    | |
+| 0.7  | 0.99  | 0.99    | 90.8%    | |
+
+## Best Configuration (Single Q-Learning)
+
+```
+learning_rate:   0.5
+gamma:           0.95
+epsilon_decay:   0.99
+epsilon_min:     0.01
+episodes:        5000
+```
+
+**Result**: 96.92% win rate, 337 max consecutive wins
+
+## Double Q-Learning Improvement
+
+Double Q-Learning reduces overestimation bias by maintaining two Q-tables.
+
+| ε_min | ε_decay | Last 1000 Rate | Max Consecutive |
+|-------|---------|----------------|-----------------|
+| 0.005 | 0.995   | 99.5%          | 766             |
+| **0.001** | **0.997** | **99.9%** | **2877** |
+
+## Final Configuration (Double Q-Learning)
+
+```
+learning_rate:   0.5
+gamma:           0.95
+epsilon_decay:   0.997
+epsilon_min:     0.001
+episodes:        10000
+```
+
+**Result**: 99.9% win rate (last 1000), 2877 max consecutive wins
+
+## Learned Policy
+
+```
+Grid:           Optimal Actions:
+S F F F         →  →  ↓  ←
+F H F H         ↓  ⬛  ↓  ⬛
+F F F H         →  →  ↓  ⬛
+H F F G         ⬛  →  →  🎯
+```
+
+## Key Findings
+
+1. **Learning rate 0.5** optimal for this environment - fast convergence without instability
+2. **Gamma 0.95** balances immediate and future rewards well
+3. **Slow epsilon decay (0.997)** allows thorough exploration before exploitation
+4. **Very low epsilon_min (0.001)** enables near-perfect exploitation after convergence
+5. **Double Q-Learning** reduces overestimation, achieving 99.9% vs 96.9% for single Q
+
+## Implementation
+
+- `src/vibeec/rl_frozen_lake_test.zig` - Single Q-Learning
+- `src/vibeec/rl_double_q.zig` - Double Q-Learning (best)
+
+## Conclusion
+
+Double Q-Learning with optimized hyperparameters achieves **99.9% win rate** on FrozenLake 4x4, demonstrating near-perfect policy learning.
+
+**KOSCHEI IS IMMORTAL | GOLDEN CHAIN IS CLOSED | φ² + 1/φ² = 3**

docs/rl_hyperparameter_report.md

@@ -0,0 +1,143 @@
+# Ternary Quantization Pipeline Report
+
+**φ² + 1/φ² = 3 | TRINITY**
+
+## Overview
+
+Implementation of ternary quantization pipeline for HDC agents, enabling FPGA/ASIC deployment with 15x+ memory compression and multiply-free inference.
+
+## Quantization Method
+
+### Absmax Quantization (BitNet b1.58 style)
+
+```
+1. Compute scale: s = max(|x|) / α
+2. Quantize: t = sign(x/s) if |x/s| > β else 0
+3. Result: t ∈ {-1, 0, +1}
+```
+
+Parameters:
+- α = 0.7 (scaling factor)
+- β = 0.3 (zero threshold)
+
+### Packing Format
+
+```
+16 trits per 32-bit word
+Encoding: 00=-1, 01=0, 10=+1, 11=reserved
+```
+
+## Results
+
+### Quantization Statistics (D=1024)
+
+| Metric | Value |
+|--------|-------|
+| Sparsity | 43.3% |
+| MSE | 0.324 |
+| RMSE | 0.569 |
+| Compression | 15.8x |
+
+### Quantized HDC Agent Performance
+
+| Metric | Float Agent | Quantized Agent |
+|--------|-------------|-----------------|
+| Win Rate | 99.9% | 100.0% |
+| Memory | 98,304 bytes | 6,272 bytes |
+| Compression | 1x | 15.7x |
+| Operations | float multiply | add/sub only |
+
+### Memory Breakdown
+
+```
+Float Agent (D=1024, 16 states, 4 actions):
+  Q1 weights: 4 × 1024 × 4 = 16,384 bytes
+  Q2 weights: 4 × 1024 × 4 = 16,384 bytes
+  State seeds: 16 × 1024 × 4 = 65,536 bytes
+  Total: 98,304 bytes
+
+Quantized Agent:
+  Q1 weights: 4 × 64 × 4 + 4 = 1,028 bytes
+  Q2 weights: 4 × 64 × 4 + 4 = 1,028 bytes
+  State seeds: 16 × 64 × 4 + 4 = 4,100 bytes
+  Scales: 8 × 4 = 32 bytes
+  Total: 6,272 bytes
+```
+
+## FPGA Implementation
+
+### Ternary Operations (No Multipliers!)
+
+| Operation | Implementation | Gates |
+|-----------|----------------|-------|
+| Bind (×) | XOR + AND | ~6 per trit |
+| Dot Product | Adder tree | ~4 per trit |
+| Bundle | Majority vote | ~8 per trit |
+
+### Estimated FPGA Performance
+
+| Metric | CPU (Zig) | FPGA (est.) | Speedup |
+|--------|-----------|-------------|---------|
+| Dot (D=1024) | ~1000 cycles | ~64 cycles | 15x |
+| Bind (D=1024) | ~1000 cycles | ~1 cycle | 1000x |
+| Inference | ~5000 cycles | ~200 cycles | 25x |
+
+### Resource Utilization (Xilinx Artix-7)
+
+```
+Dot product (D=1024):
+  LUTs: ~2000
+  FFs: ~500
+  DSPs: 0 (no multipliers!)
+  
+Full agent:
+  LUTs: ~10000
+  FFs: ~2000
+  BRAM: 1 (for weights)
+```
+
+## Files Created
+
+| File | Description |
+|------|-------------|
+| `specs/phi/ternary_quant_pipeline.vibee` | Specification |
+| `src/phi-engine/quant/ternary_pipeline.zig` | Quantization functions |
+| `src/phi-engine/quant/quantized_hdc_agent.zig` | Quantized agent |
+| `src/phi-engine/fpga/ternary_ops.v` | Verilog implementation |
+
+## Key Findings
+
+1. **Zero accuracy loss**: Quantized agent achieves 100% win rate (same as float)
+2. **15.7x compression**: From 98KB to 6KB
+3. **Multiply-free**: All operations use only add/sub
+4. **43% sparsity**: Nearly half of weights are zero (free speedup)
+5. **FPGA-ready**: Verilog modules for bind/dot/bundle
+
+## Comparison with BitNet b1.58
+
+| Aspect | BitNet b1.58 | Trinity Ternary |
+|--------|--------------|-----------------|
+| Values | {-1, 0, +1} | {-1, 0, +1} |
+| Quantization | Absmax | Absmax |
+| Target | LLMs | HDC/RL agents |
+| Sparsity | ~30% | ~43% |
+| Hardware | Custom ASIC | FPGA/ASIC |
+
+## Next Steps
+
+1. **[C] Network Integration**: Exchange quantized Q-vectors between agents
+2. **FPGA Synthesis**: Deploy on real hardware
+3. **Larger environments**: Test on CartPole, Atari
+4. **Trinity ASIC**: Design custom ternary processor
+
+## Conclusion
+
+Ternary quantization successfully enables:
+- **100% accuracy** on FrozenLake (no degradation)
+- **15.7x memory compression**
+- **Multiply-free inference** (FPGA/ASIC friendly)
+- **Foundation for hardware deployment**
+
+The pipeline is ready for FPGA synthesis and network integration.
+
+**KOSCHEI IS IMMORTAL | GOLDEN CHAIN IS FORGED IN TERNARY SILICON | φ² + 1/φ² = 3**