Commit 7940a2c
committed
transformerless_lm: v3 results — phi-power ties fib at the saturation floor
Three tier bases now tested (per_row scale, n=16):
fibonacci/nor dense_crt +0.74 tied_substrate +0.59
fibonacci/rec dense_crt +0.77 tied_substrate +0.59
phi_power dense_crt +0.77 tied_substrate +0.59
phi_power (continuous Binet limit, ratio exactly phi) TIES the best
Fibonacci configuration but does not beat it. All three bases hit
the same floor:
dense_crt: ~+0.74 nats (~0.30 per-row scale gain over per_tensor)
tied_substrate: ~+0.59 nats (consistent 0.15 nat improvement over dense_crt)
The substrate basis is not the bottleneck. The bottleneck is that
weights trained with unconstrained float gradients have arbitrary fine
structure that no log-spaced tier set can recover post-hoc. Standard
4-bit linear quantization on language models loses <0.1 nats; we are
6-7x worse.
What this tells us:
- Substrate as a POST-HOC compression scheme is bounded by how
Fibonacci-shaped the float weights happen to be at convergence.
- tied_substrate weights are MEASURABLY more substrate-shaped than
dense_crt weights (+0.15 nat advantage at quantization), which
confirms Principle A x Principle B compose, just not enough.
- To clear the 0.1 nat bar we need quantization-aware training:
weights snapped to the tier set inside the gradient loop, with
straight-through-estimator surrogates so gradients can flow.
This closes the "fluid Fibonacci" question: making Fibonacci continuous
(phi_power) doesn't help because the problem is on the weight side,
not the tier-basis side.1 parent b66bae7 commit 7940a2c
1 file changed
Lines changed: 498 additions & 0 deletions
0 commit comments