Commit 2873364
docs(tier): Qwen3.6-27B element-level diff — outlier channel pattern
Continued investigation this session:
VALIDATED additional ruled-outs:
TQ_DELTANET_FP32=1 → same Tier 3, max 3.87
→ Quantization NOT the cause
→ Forward pass arithmetic bug confirmed
ELEMENT-LEVEL L0 diff (BOS-aligned, both engines prefilled BOS+Hello):
pos 0 (BOS), first 3 elements:
ours: [-0.055, 0.355, -0.790]
llama: [-0.110, -0.039, 0.036]
elem 0: 2× magnitude, same sign
elem 1: SIGN FLIP + 9× magnitude
elem 2: SIGN FLIP + 22× magnitude
PATTERN: outlier channels — specific dimensions blown up while
overall sum stays manageable (23.5 vs 6.8). Classic signature of:
- Mis-aligned norm weight (boundary issue at hidden=5120?)
- Missing embed_scale = sqrt(hidden_dim)
- QKV/conv1d channel split shifted by some offset
Updated docs/tier_benchmark_2026_04_25.md with concrete next
investigation step: dump first 20 elements of each named tensor
at L0 (post_embed, attn_norm_out, qkv_proj, conv1d, q/k/v split,
l2norm, decay, delta, state, output, ssm_norm). First materially
divergent named tensor identifies the bug location.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 0fa33cb commit 2873364
1 file changed
Lines changed: 18 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
81 | 81 | | |
82 | 82 | | |
83 | 83 | | |
84 | | - | |
| 84 | + | |
85 | 85 | | |
86 | 86 | | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
87 | 104 | | |
88 | 105 | | |
89 | 106 | | |
| |||
0 commit comments