Hardest-case benchmark profile is now far tougher

Author: OptimumAF

CHANGELOG.md

@@ -61,6 +61,7 @@ for versioning even while in research-stage development.
 - Enhanced benchmark visuals with a compact combined overview figure (static + interactive) and linked it in README/dashboard for faster comparison.
 - Added hardest-case dynamics GIF (training progression + inference decision-map evolution) and surfaced it near the top of README and docs dashboard.
 - Added an interactive Plotly hardest-case dynamics page with playback controls and circadian internals visualization (node/edge weights, chemical/plasticity state) on the docs dashboard.
+- Increased hardest-case difficulty substantially (higher drift/noise, lower phase-B train fraction, longer training horizon) and raised hidden-layer width in hardest-case runs for all three models.
 - Refreshed README benchmark section with a latest master verification run on 2026-02-28 and added raw output artifact under `docs/benchmarks/`.
 - Repositioned repository messaging to Circadian Predictive Coding as the primary focus.
 - Updated `README.md` with:

README.md

@@ -123,16 +123,18 @@ Raw benchmark output: [`docs/benchmarks/benchmark_master_cifar100_subset_2026-02
 
 Strengths:
 
-- Strong retention/adaptation balance under hard continual shift.
-- Hardest-case balanced score: circadian `0.922` vs predictive coding `0.916` vs backprop `0.889`.
-- Hardest-case retention ratio: circadian `0.994`.
-- Source: [`docs/benchmarks/benchmark_continual_shift_hardest_case_2026-02-28.txt`](docs/benchmarks/benchmark_continual_shift_hardest_case_2026-02-28.txt).
-- Dynamic capacity adaptation is observable and measurable (for hardest-case: mean splits `6.29`, hidden size `8 -> 14.29`).
+- Competitive retention/adaptation behavior under hard continual shift.
+- Strong balance in the moderate strength-case stress test (circadian balanced score `0.949` vs predictive coding `0.947` vs backprop `0.946`).
+- Sources:
+  - [`docs/benchmarks/benchmark_continual_shift_strength_case_2026-02-28.txt`](docs/benchmarks/benchmark_continual_shift_strength_case_2026-02-28.txt)
+  - [`docs/benchmarks/benchmark_continual_shift_hardest_case_2026-02-28.txt`](docs/benchmarks/benchmark_continual_shift_hardest_case_2026-02-28.txt)
+- Dynamic capacity adaptation is observable and measurable (updated hardest-case: mean splits `27.57`, hidden size `24 -> 51.57`).
 - Competitive behavior in moderate continual-shift stress tests with stable multi-seed performance.
 
 Weaknesses:
 
 - Not best on every benchmark; on the latest CIFAR-100 subset master check, predictive coding accuracy (`0.692`) was higher than circadian (`0.685`).
+- In the updated ultra-hard hardest-case setting, predictive coding currently leads circadian on balanced score (`0.844` vs `0.831`), even though circadian still outperforms backprop (`0.785`).
 - Extra algorithmic machinery (sleep scheduling, replay, split/prune controls) adds tuning burden and implementation complexity compared with fixed-width baselines.
 - Speed overhead can appear depending on configuration; in the latest CIFAR-100 subset master check, circadian train speed (`874.2` SPS) was lower than predictive coding (`965.2` SPS).
 - Results are regime-dependent; claims should be tied to specific benchmark settings and seeds instead of treated as universal.
@@ -195,7 +197,7 @@ Continual shift stress test (retention vs adaptation):
 python scripts/run_continual_shift_benchmark.py --profile strength-case --seeds 3,7,11,19,23,31,37
 ```
 
-Hardest continual-shift stress test (small starting capacity + heavy drift):
+Hardest continual-shift stress test (expanded hidden capacity + very heavy drift):
 
 ```powershell
 python scripts/run_continual_shift_benchmark.py --profile hardest-case --seeds 3,7,11,19,23,31,37

docs/benchmarks/benchmark_continual_shift_hardest_case_2026-02-28.txt

@@ -2,10 +2,10 @@ Continual Shift Benchmark
 -------------------------
 Phase A trains on base distribution; phase B trains on shifted/rotated distribution.
 Seeds: [3, 7, 11, 19, 23, 31, 37]
-Setup: hidden_dim=8, phaseA_epochs=90, phaseB_epochs=120, phaseA_noise=0.80, phaseB_noise=1.20
-Phase B transform: rotation=44.0 deg, translation=(0.90, -0.70)
-Phase B train fraction: 0.08
+Setup: hidden_dim=24, phaseA_epochs=120, phaseB_epochs=180, phaseA_noise=0.80, phaseB_noise=1.45
+Phase B transform: rotation=68.0 deg, translation=(1.60, -1.30)
+Phase B train fraction: 0.05
 
-Backprop: A_pre=0.975+/-0.014, A_post=0.888+/-0.081, B_post=0.889+/-0.031, retention=0.911+/-0.083, balanced=0.889+/-0.052
-Predictive coding: A_pre=0.979+/-0.013, A_post=0.958+/-0.018, B_post=0.874+/-0.028, retention=0.978+/-0.021, balanced=0.916+/-0.015
-Circadian predictive coding: A_pre=0.973+/-0.015, A_post=0.967+/-0.017, B_post=0.878+/-0.031, retention=0.994+/-0.021, balanced=0.922+/-0.014, sleep_events=6.29, splits=6.29, prunes=0.00, hidden_end=14.29
+Backprop: A_pre=0.975+/-0.009, A_post=0.714+/-0.145, B_post=0.856+/-0.025, retention=0.732+/-0.145, balanced=0.785+/-0.064
+Predictive coding: A_pre=0.971+/-0.010, A_post=0.852+/-0.130, B_post=0.836+/-0.037, retention=0.878+/-0.131, balanced=0.844+/-0.058
+Circadian predictive coding: A_pre=0.972+/-0.009, A_post=0.829+/-0.143, B_post=0.833+/-0.029, retention=0.852+/-0.148, balanced=0.831+/-0.064, sleep_events=13.86, splits=27.57, prunes=0.00, hidden_end=51.57

docs/figures/hardest_mode_dynamics.gif

@@ -40,21 +40,21 @@
 @dataclass(frozen=True)
 class HardestModeConfig:
     seed: int = 7
-    sample_count_phase_a: int = 500
-    sample_count_phase_b: int = 500
+    sample_count_phase_a: int = 700
+    sample_count_phase_b: int = 700
     test_ratio: float = 0.25
-    phase_b_train_fraction: float = 0.08
-    hidden_dim: int = 8
-    phase_a_epochs: int = 90
-    phase_b_epochs: int = 120
+    phase_b_train_fraction: float = 0.05
+    hidden_dim: int = 24
+    phase_a_epochs: int = 120
+    phase_b_epochs: int = 180
     phase_a_noise: float = 0.8
-    phase_b_noise: float = 1.2
-    phase_b_rotation_degrees: float = 44.0
-    phase_b_translation_x: float = 0.9
-    phase_b_translation_y: float = -0.7
+    phase_b_noise: float = 1.45
+    phase_b_rotation_degrees: float = 68.0
+    phase_b_translation_x: float = 1.6
+    phase_b_translation_y: float = -1.3
     sleep_interval_phase_a: int = 40
-    sleep_interval_phase_b: int = 8
-    snapshot_interval: int = 4
+    sleep_interval_phase_b: int = 6
+    snapshot_interval: int = 6
     decision_grid_size: int = 110
     latency_repeats: int = 10
     gif_duration_ms: int = 120
@@ -110,14 +110,14 @@ def parse_args() -> argparse.Namespace:
 def build_hardest_circadian_config() -> CircadianConfig:
     return CircadianConfig(
         use_reward_modulated_learning=False,
-        split_threshold=0.25,
+        split_threshold=0.22,
         prune_threshold=0.04,
-        max_split_per_sleep=1,
+        max_split_per_sleep=2,
         max_prune_per_sleep=0,
-        replay_steps=2,
-        replay_memory_size=10,
+        replay_steps=3,
+        replay_memory_size=14,
         replay_learning_rate=0.04,
-        replay_inference_steps=12,
+        replay_inference_steps=14,
         replay_inference_learning_rate=0.14,
     )
 

docs/figures/interactive_hardest_mode_dynamics.html

@@ -154,14 +154,14 @@ def _build_hardest_case_circadian_config() -> CircadianConfig:
     """Build circadian profile tuned for the hardest continual-shift setup."""
     return CircadianConfig(
         use_reward_modulated_learning=False,
-        split_threshold=0.25,
+        split_threshold=0.22,
         prune_threshold=0.04,
-        max_split_per_sleep=1,
+        max_split_per_sleep=2,
         max_prune_per_sleep=0,
-        replay_steps=2,
-        replay_memory_size=10,
+        replay_steps=3,
+        replay_memory_size=14,
         replay_learning_rate=0.04,
-        replay_inference_steps=12,
+        replay_inference_steps=14,
         replay_inference_learning_rate=0.14,
     )
 
@@ -173,19 +173,19 @@ def _build_baseline_circadian_config() -> CircadianConfig:
 def _build_profile_defaults(profile: str) -> ProfileDefaults:
     if profile == "hardest-case":
         return ProfileDefaults(
-            sample_count_phase_a=500,
-            sample_count_phase_b=500,
-            phase_b_train_fraction=0.08,
-            phase_a_epochs=90,
-            phase_b_epochs=120,
-            hidden_dim=8,
+            sample_count_phase_a=700,
+            sample_count_phase_b=700,
+            phase_b_train_fraction=0.05,
+            phase_a_epochs=120,
+            phase_b_epochs=180,
+            hidden_dim=24,
             phase_a_noise_scale=0.8,
-            phase_b_noise_scale=1.2,
-            phase_b_rotation_degrees=44.0,
-            phase_b_translation_x=0.9,
-            phase_b_translation_y=-0.7,
+            phase_b_noise_scale=1.45,
+            phase_b_rotation_degrees=68.0,
+            phase_b_translation_x=1.6,
+            phase_b_translation_y=-1.3,
             sleep_interval_phase_a=40,
-            sleep_interval_phase_b=8,
+            sleep_interval_phase_b=6,
         )
     return ProfileDefaults(
         sample_count_phase_a=500,