Skip to content

Commit c2bca6b

Browse files
committed
transformerless_lm: split-brain resonance-aware mixer
v73 geometric mean: too restrictive (both must consent). v74 golden weighted: too lax (one hemisphere can override). v75 resonance-aware: per-token sign coherence gates the push. coherence = sign(math_fluid) * sign(lang_fluid) in {-1, 0, +1} gate = (1 + coherence) / 2 in {0, 0.5, 1} combined = (math_fluid + lang_fluid) * gate Agreement (both +/+ or -/-) -> full sum applied (resonance). Conflict (+/- or -/+) -> zero (dissonance cancels back to base). One silent -> half effect (single-hemisphere push damped). Models the corpus-callosum gate from split-brain neuroscience: hemispheres only push through when they agree. Cognitive resonance amplifies, cognitive dissonance suppresses.
1 parent e1269d7 commit c2bca6b

1 file changed

Lines changed: 15 additions & 16 deletions

File tree

experiments/transformerless_lm/train_self_recursive.py

Lines changed: 15 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1327,7 +1327,7 @@ def _omniweight_apply(base_probs: torch.Tensor,
13271327
def _omniweight_apply_split(base_probs: torch.Tensor,
13281328
math_delta: torch.Tensor,
13291329
lang_delta: torch.Tensor) -> torch.Tensor:
1330-
"""SPLIT-BRAIN omniweight: two registers, golden-weighted mixer.
1330+
"""SPLIT-BRAIN omniweight: RESONANCE-AWARE mixer.
13311331
13321332
Math hemisphere: bigram, recency, substrate sampling, anti-stag,
13331333
bigram-saturation. Frequency / decay primitives.
@@ -1336,25 +1336,24 @@ def _omniweight_apply_split(base_probs: torch.Tensor,
13361336
rhyme, agreement, word-spacing, char-cascade, pronounceability,
13371337
subject-threading, theme. Purpose / structure primitives.
13381338
1339-
Each hemisphere builds its own fluid delta via tanh-scaled
1340-
substrate reserve. Final distribution = golden-weighted arithmetic
1341-
mean: (phi * p_math + p_lang) / (phi + 1).
1339+
Per-token coherence gate (sign agreement of math_fluid x lang_fluid):
1340+
agree -> push through full sum (resonance)
1341+
conflict -> cancel back toward base (dissonance)
13421342
1343-
Math gets phi=1.618 weight (older substrate foundation, primary
1344-
signal). Lang gets 1.0 weight (modulator). Both contribute --
1345-
high-confidence proposals from either come through. Less
1346-
restrictive than geometric mean which required both-consent
1347-
(v73 was over-conservative).
1343+
Substrate-canonical: coherence = sign(math) * sign(lang) in
1344+
{-1, 0, +1}. Weight = (1 + coherence) / 2 in {0, 0.5, 1}.
1345+
1346+
This models the split-brain corpus-callosum gate: hemispheres
1347+
only push through when they agree.
13481348
"""
13491349
math_fluid = _OMNIWEIGHT_RESERVE * torch.tanh(math_delta / _OMNIWEIGHT_RESERVE)
13501350
lang_fluid = _OMNIWEIGHT_RESERVE * torch.tanh(lang_delta / _OMNIWEIGHT_RESERVE)
1351-
p_math = base_probs * torch.exp(math_fluid)
1352-
p_lang = base_probs * torch.exp(lang_fluid)
1353-
p_math = p_math / (p_math.sum() + 1e-8)
1354-
p_lang = p_lang / (p_lang.sum() + 1e-8)
1355-
phi = _PHI_FOR_SAMPLING
1356-
p_final = (phi * p_math + p_lang) / (phi + 1.0)
1357-
return p_final / (p_final.sum() + 1e-8)
1351+
# Per-token sign coherence.
1352+
coherence = torch.sign(math_fluid) * torch.sign(lang_fluid)
1353+
gate = (1.0 + coherence) / 2.0 # in {0, 0.5, 1}
1354+
combined_fluid = (math_fluid + lang_fluid) * gate
1355+
out = base_probs * torch.exp(combined_fluid)
1356+
return out / (out.sum() + 1e-8)
13581357

13591358

13601359
def autoregressive_generate(model, prompt: torch.Tensor, n_new: int,

0 commit comments

Comments
 (0)