Skip to content

Commit 91c484f

Browse files
committed
transformerless_lm: omniweight -- shared log-pressure ledger
Architectural refactor. Previously each primitive chained probs->probs transforms, with no awareness of other primitives' effects. Stacked multipliers caused 'no medium of argument' (user term) -- conflicting suppressions compounded multiplicatively, peaks softened, mid-cycle drops became routine. OMNIWEIGHT model (user named, ported from earlier robotics work): single shared delta_log_p accumulator. Each primitive contributes log-space delta on the SAME base distribution. Total accumulator clamped to [-pi*log(phi), +pi*log(phi)] (substrate-bounded), then applied once via exp(). Benefits: - No cascading conflicts (primitives don't see each other's effects) - Bounded total influence (model's raw distribution preserved) - All primitives negotiate through one currency Wired into both autoregressive_generate and _single_stage_refine. Vocab curriculum still applied as a hard mask post-omniweight (since it's a constraint, not a soft preference). Reverted v70 K-shrink slowdown and bigram threshold loosening -- those were symptom fixes for the underlying composition problem.
1 parent 5aae99b commit 91c484f

1 file changed

Lines changed: 148 additions & 117 deletions

File tree

0 commit comments

Comments
 (0)