KakeyaLattice

KakeyaLattice — v1.4 KV-cache compression codec.

Rigorous evaluation (n=32 passages + 95% CI + in-forward + K/V/KV + NIAH + ablation) is now the canonical data source for this release. Full report in reports/v1_4_release/rigorous_eval/RIGOROUS_EVAL_REPORT.md.

Key findings (n=32, Student t 95% CI)

iso-PPL compression ratio vs TurboQuant

Measured on 3 models × 2 modes (snapshot + in-forward). "CR gap" = (v1.4 CR / TQ CR − 1) × 100%.

Snapshot mode, tight quality thresholds (mean + CI95 ≤ T):

Model	|Δppl| ≤ 1%	|Δppl| ≤ 2%	|Δppl| ≤ 5%
Qwen3-4B	TQ wins by +2.4%	TQ wins by +2.9%	v1.4 wins by +26.9%
DeepSeek-1.5B	TQ wins by +2.7%	TQ wins by +2.7%	TQ wins by +3.8%
GLM-4-9B-Chat	both out-of-range	TQ wins by +2.4%	TQ wins by +2.8%

In-forward (native production) mode:

Model	|Δppl| ≤ 1%	|Δppl| ≤ 2%	|Δppl| ≤ 5%
Qwen3-4B	TQ wins by +2.4%	TQ wins by +2.9%	TQ wins by +26.5%
DeepSeek-1.5B	TQ wins by +2.2%	TQ wins by +2.7%	v1.4 wins by +24.4%
GLM-4-9B-Chat	both out-of-range	TQ wins by +2.4%	TQ wins by +2.8%

Honest read: at tight quality budgets TQ has a consistent 2-3% CR edge (fixed 32-bit overhead gap from v1.4's fp32 qmax). At aggressive 5% thresholds, v1.4 occasionally opens a +25% CR lead on specific model×mode cells. At matched |Δppl|, the two codecs are statistically indistinguishable on most cells.

K-only / V-only / K+V

V is 2-3× easier to compress than K at matched bits across all tested models. A V-only compression scheme at ~1.36× CR keeps |Δppl| < 1% even on the hardest model (GLM-4-9B).

NIAH long-context retrieval

Model	v1.4 Q=38	v1.4 Q=152	TQ b=6	TQ b=8
Qwen3-4B (ctx 4k-32k)	100%	100%	100%	100%
GLM-4-9B (ctx 4k-16k)	100%	100%	88.9%	88.9%
DeepSeek-1.5B (ctx 16k, depth=0.9)	0%	100%	0%	0%

On GLM and DeepSeek, v1.4 preserves long-context retrieval better than TQ. On DeepSeek at 16k ctx, v1.4 Q=152 is the only codec tested that keeps needle-in-a-haystack retrieval working.

Ablation (Qwen3-4B, Q=38 K+V, n=32)

variant	|Δppl|	vs v14_full
v14_full (baseline)	0.98% ± 0.24%	1.00×
no_hadamard	2.60% ± 0.66%	2.65× worse
no_per_vec_qmax	1.64% ± 0.65%	1.68× worse
no_unit_norm	0.89% ± 0.23%	0.91× (no effect)
per_block_qmax	0.80% ± 0.23%	0.81× (slightly better)
scalar_quantise (no D4)	0.84% ± 0.25%	0.85× (D4 lattice itself contributes almost nothing)

The load-bearing factors in v1.4 are Hadamard rotation and per-vector qmax. The D4 lattice shaping gain, which was the research motivation, contributes <20% and is nearly invisible at n=32 CI.

Boundary-layer ablation

codec	with boundary (CR 2.12×)	no boundary (CR 2.46×, +16%)	safe to drop?
v14_full	0.98% ± 0.24%	0.98% ± 0.27%	✓
scalar_quantise	0.84% ± 0.25%	0.88% ± 0.23%	✓

Production recipe: deploy v1.4 with no_boundary=True for +16% CR at zero quality cost.

Retractions from earlier reports

Explicitly retracted (previous claims were n=4 sampling artifacts):

• ❌ "v1.4 wins 12/12 K-MSE + 10/12 |Δppl|"
• ❌ "v1.4 +37.8% CR advantage on GLM at |Δppl| ≤ 2%"
• ❌ "D4 lattice shaping gain is the key to v1.4"

The rigorous n=32 story is less dramatic but credible: v1.4 is competitive with TQ at matched bits, has wins and losses depending on model/mode/threshold, and owes its performance primarily to Hadamard + per-vec qmax rather than the D4 lattice.

Model coverage

Model	Layers	head_dim	KV heads	Tested?
Qwen/Qwen3-4B	36	128	8	✓
deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B	28	128	2	✓
zai-org/GLM-4-9B-Chat	40	128	2	✓
google/gemma-4-E4B	24	256	2	✗ (deferred — per-layer variable head_dim in MatFormer / kv-shared arch)

Audit evidence

Every number in this release is backed by committed raw log + measurement JSON:

• reports/v1_4_release/rigorous_eval/snapshot/ — 4 *.log + 4 *.json
• reports/v1_4_release/rigorous_eval/inforward/ — 4 *.log + 4 *.json
• reports/v1_4_release/rigorous_eval/ablation/ — 2 *.log + 2 *.json (Qwen3-4B, with + without boundary)
• reports/v1_4_release/niah/ — 4 *.log + 4 *.json
• reports/v1_4_release/streaming/logs/ — 3 *.log
• reports/v1_4_release/audit/ — full end-to-end audit run with GPU trace CSV

All runs on vast.ai NVIDIA H200, CUDA 13.0, vLLM 0.19.2rc1.dev100+gf946659ff, PyTorch 2.11, transformers 5.5.2.

Compliance

Built under strict ban list: no mock / no simplification / no fallback / no overfit.

• head_dim % 4 != 0 → raise ValueError (by design)
• In-forward without codec_fn → RuntimeError
• Fire-count guard aborts silent-passthrough channels
• iso-PPL winners are raw empirical argmax-CR at mean + CI95 ≤ T, no curve fitting
• Ablation v14_full parity-checked against canonical V14KakeyaZamirLatticeGPU at max_abs_diff = 0.000e+00

Codec

kakeyaturbo_py.V14KakeyaZamirLatticeGPU:

• Zamir-Feder D4 nested lattice (Conway-Sloane 1982 closest-lattice-point)
• Sylvester Hadamard rotation
• per-vector qmax adaptive scaling (fp16 stored)
• unit-normalisation (fp16 stored)
• bf16 fidelity, strict-GPU, calibration-free, streaming-capable (per-vector stateless)

Reproducibility

pip install -e kakeyaturbo-py
pip install -e vllm_backend
export VLLM_ENABLE_V1_MULTIPROCESSING=0 KAKEYA_SNAPSHOT_QWEN3=1

python benchmarks/rigorous_eval.py \
    --model-path Qwen/Qwen3-4B --model-name qwen3_4b \
    --mode inforward \
    --q-values 10,38,152 --tq-b-values 4,6,8 \
    --kv-modes K,V,KV \
    --ctx-len 2048 --n-eval 64 --n-passages 32 \
    --gpu-mem-util 0.40 \
    --out-dir reports/v1_4_release/rigorous_eval/inforward

python benchmarks/niah_eval.py \
    --model-path Qwen/Qwen3-4B --model-name qwen3_4b \
    --mode inforward --n-trials 3 \
    --ctx-lengths 4096,8192,16384 --depths 0.1,0.5,0.9 \
    --q-values 38,152 --tq-b-values 6,8 \
    --out-dir reports/v1_4_release/niah

See RIGOROUS_EVAL_REPORT.md section 8 for full per-phase commands.