Commit 0fa33cb
docs(tier): Qwen3.6-27B investigation log — narrow remaining scope
Updated tier benchmark doc with what we validated and ruled out
in this session, narrowing the unknown surface for next session:
VALIDATED (no issue here):
- Tensor names match A3B exactly
- All shapes consistent with config (qkv 10240 = 16×128×2 + 48×128)
- GGUF metadata correct (rope, eps, conv_kernel)
- Layer pattern matches llama-debug (16 attn at L3,7,...,63)
- ssm_a values sensible (-0.34 to -0.004 at L0)
- is_moe correctly false (num_experts=0)
- QK-norm not the issue (TQ_FORCE_QK_NORM=1 didn't help)
- DN_LLAMACPP_PORT correctly auto-enabled
REMAINING (next session needs):
- Element-level sub-op trace at L0 pos=0
- Compare ours vs llama-debug per named tensor
- First materially divergent sub-op = the bug
QUICK PATHS TO TRY FIRST:
- Q4_K dequant validation on hidden_dim 5120 (unusual size)
- TQ_DELTANET_FP32=1 bypass to localize quant vs forward bug
- Smaller quant (UD-IQ2_M) to test Tier 3 persistence
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 1a3807f commit 0fa33cb
1 file changed
Lines changed: 20 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
67 | 87 | | |
68 | 88 | | |
69 | 89 | | |
| |||
0 commit comments