Skip to content

Commit 7940a2c

Browse files
committed
transformerless_lm: v3 results — phi-power ties fib at the saturation floor
Three tier bases now tested (per_row scale, n=16): fibonacci/nor dense_crt +0.74 tied_substrate +0.59 fibonacci/rec dense_crt +0.77 tied_substrate +0.59 phi_power dense_crt +0.77 tied_substrate +0.59 phi_power (continuous Binet limit, ratio exactly phi) TIES the best Fibonacci configuration but does not beat it. All three bases hit the same floor: dense_crt: ~+0.74 nats (~0.30 per-row scale gain over per_tensor) tied_substrate: ~+0.59 nats (consistent 0.15 nat improvement over dense_crt) The substrate basis is not the bottleneck. The bottleneck is that weights trained with unconstrained float gradients have arbitrary fine structure that no log-spaced tier set can recover post-hoc. Standard 4-bit linear quantization on language models loses <0.1 nats; we are 6-7x worse. What this tells us: - Substrate as a POST-HOC compression scheme is bounded by how Fibonacci-shaped the float weights happen to be at convergence. - tied_substrate weights are MEASURABLY more substrate-shaped than dense_crt weights (+0.15 nat advantage at quantization), which confirms Principle A x Principle B compose, just not enough. - To clear the 0.1 nat bar we need quantization-aware training: weights snapped to the tier set inside the gradient loop, with straight-through-estimator surrogates so gradients can flow. This closes the "fluid Fibonacci" question: making Fibonacci continuous (phi_power) doesn't help because the problem is on the weight side, not the tier-basis side.
1 parent b66bae7 commit 7940a2c

1 file changed

Lines changed: 498 additions & 0 deletions

File tree

0 commit comments

Comments
 (0)