You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -459,7 +459,7 @@ print(f"Mean final score: {perf['mean_score']:.2f} ± {perf['std_score']:.2f}")
459
459
460
460
### Agent End-of-Episode Hook
461
461
462
-
The training loop calls `agent.on_episode_end()` after each episode if the method exists. The MLP agent uses this for target network updates. Linear and tile coding agents don't need it.
462
+
The training loop calls `agent.on_episode_end()` after each episode if the method exists. The Double DQN agent uses this for target network updates. Linear and tile coding agents don't need it.
463
463
464
464
---
465
465
@@ -490,13 +490,13 @@ Full experimental matrix: 20,000 episodes × 5 random seeds, 20×20 grid. Mean s

498
498
499
-
*Heatmap of mean final scores. MLP (Double DQN) dominates every cell; tile coding is competitive only on compact features.*
499
+
*Heatmap of mean final scores. Double DQN dominates every cell; tile coding is competitive only on compact features.*
500
500
501
501

502
502
@@ -506,9 +506,9 @@ Full experimental matrix: 20,000 episodes × 5 random seeds, 20×20 grid. Mean s
506
506
507
507
**Tile coding's performance degrades sharply with dimensionality.** Tile × compact (22.69) outperforms tile × local (3.21) by ~7× and tile × extended (1.32) by ~17×. Hash-based tile coding was designed for low-dimensional continuous spaces; at 109+ dimensions, hash collisions in the fixed-size table (262,144 entries) dominate and generalization breaks down. Extended actually performs *worse* than local despite having more information — more dimensions means more collisions.
508
508
509
-
**MLP is more robust to representation choice than tile coding.**MLP scores (24.26 / 22.51 / 29.05) stay competitive across all three representations. Tile coding's scores collapse monotonically with dimensionality (22.69 → 3.21 → 1.32). The gap is starkest on extended: MLP (29.05) vs tile coding (1.32) — a 22× difference on the same representation.
509
+
**Double DQN is more robust to representation choice than tile coding.**Double DQN scores (24.26 / 22.51 / 29.05) stay competitive across all three representations. Tile coding's scores collapse monotonically with dimensionality (22.69 → 3.21 → 1.32). The gap is starkest on extended: Double DQN (29.05) vs tile coding (1.32) — a 22× difference on the same representation.
510
510
511
-
**MLP variance tightens with richer representations.** Seed-to-seed std narrows from ±0.39 (compact) to ±0.32 (extended) — the richer feature space provides a more stable learning signal for the network.
511
+
**Double DQN variance tightens with richer representations.** Seed-to-seed std narrows from ±0.39 (compact) to ±0.32 (extended) — the richer feature space provides a more stable learning signal for the network.
512
512
513
513
**Linear FA benefits from richer representations.** Extended (3.90) is ~5× better than compact (0.80) for linear FA, because the extended representation explicitly encodes interaction terms that a linear model cannot learn on its own.
514
514
@@ -567,7 +567,7 @@ Full 3×3 matrix: 20,000 episodes × 5 random seeds × 9 configurations, 20×20
Tile coding uses fewer tilings for local/extended representations (4 vs 8) because the higher dimensionality already provides sufficient coverage with a 262,144-entry hash table.
573
573
@@ -581,7 +581,7 @@ Tile coding uses fewer tilings for local/extended representations (4 vs 8) becau
0 commit comments