Commit 91c484f
committed
transformerless_lm: omniweight -- shared log-pressure ledger
Architectural refactor. Previously each primitive chained probs->probs
transforms, with no awareness of other primitives' effects. Stacked
multipliers caused 'no medium of argument' (user term) -- conflicting
suppressions compounded multiplicatively, peaks softened, mid-cycle
drops became routine.
OMNIWEIGHT model (user named, ported from earlier robotics work):
single shared delta_log_p accumulator. Each primitive contributes
log-space delta on the SAME base distribution. Total accumulator
clamped to [-pi*log(phi), +pi*log(phi)] (substrate-bounded), then
applied once via exp().
Benefits:
- No cascading conflicts (primitives don't see each other's effects)
- Bounded total influence (model's raw distribution preserved)
- All primitives negotiate through one currency
Wired into both autoregressive_generate and _single_stage_refine.
Vocab curriculum still applied as a hard mask post-omniweight (since
it's a constraint, not a soft preference).
Reverted v70 K-shrink slowdown and bigram threshold loosening --
those were symptom fixes for the underlying composition problem.1 parent 5aae99b commit 91c484f
1 file changed
Lines changed: 148 additions & 117 deletions
0 commit comments