Skip to content

KakeyaLattice v1.5 — E8 lattice

Choose a tag to compare

View all tags
@FluffyAIcode FluffyAIcode released this 24 Apr 06:00
v1.5
b4f4781
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

KakeyaLattice v1.5 — the E8-lattice head-of-line codec for LLM KV-cache compression. Supersedes v1.4 D4 at aggressive bit budgets with a consistent +1.3-2.0 dB shaping gain (28-53 % |Δppl| reduction) across four open-source model families in real vLLM.

v1.4 release is unchanged — v1.5 is a strict superset. New code should import V15KakeyaZamirE8GPU; existing v1.4 users keep V14KakeyaZamirLatticeGPU working byte-identically (frozen sha256 parity tests enforce this indefinitely).

What's new

  • E8 nested lattice (Conway-Sloane 1982 Alg 5) replaces v1.4's D4 for the quantiser stage
    • 8-D Voronoi closer to a sphere than D4's 24-cell
    • +0.29 dB theoretical shaping gain over D4 at matched rate
    • Actually measured: +1.3 to +2.0 dB on real LLM K/V (four models, two Q points) — 4-6× the theoretical floor because E8's two-coset option handles coarse-quantisation outliers better than D4's parity flip
  • Hadamard / per-vector qmax / unit-norm wrapper unchanged — only the quantiser core differs
  • Public API: from kakeyalattice import V15KakeyaZamirE8GPU

Measured results (in-forward · no-boundary · n=32 · 95 % t-CI)

|Δppl| reduction at matched Q

model v1.4 D4 Q=10 v1.5 E8 Q=10 v1.4 D4 Q=4 v1.5 E8 Q=4
Qwen3-4B 5.62 % ±1.78 % 3.85 % (−31.5 %) 36.5 % ±13.8 % 17.0 % (−53.4 %)
Gemma-4-E4B 2.22 % ±0.60 % 1.56 % (−29.7 %) 11.2 % ±2.5 % 5.79 % (−48.3 %)
GLM-4-9B-Chat 9.94 % ±2.74 % 6.96 % (−30.0 %) 44.8 % ±14.3 % 32.4 % (−27.8 %)

Systematic 28-53 % |Δppl| improvement across three deployable models (4B/8B/9B). Greater at aggressive Q=4 than balanced Q=10, matching super-linear cross-layer error amplification.

(DeepSeek-R1-Distill-Qwen-1.5B included but not deployable no-boundary in-forward — too fragile for any codec; v1.5 still breaks 26× less than v1.4.)

Per-layer K-MSE gain (before cross-layer amplification)

Q ratio v1.5/v1.4 (4-model avg) dB gain
4 0.68× +1.70 dB
10 0.65× +1.87 dB

Consistent across Qwen3-4B, DeepSeek-1.5B, Gemma-4-E4B, GLM-4-9B.

Compression ratio (128k KV cache, head_dim=128)

codec Q bits/head/tok CR note
v1.4 D4 4 416 4.92× released
v1.5 E8 4 448 4.57× +32-bit overhead (no parity saving)
v1.4 D4 10 576 3.56× released
v1.5 E8 10 608 3.37× +32 bits
v1.4 D4 152 1088 1.88× near-lossless
v1.5 E8 152 1104 1.86× near-lossless

v1.5 costs a fixed −5-7 % CR vs v1.4 at iso-Q (E8 lacks D4's parity constraint). At iso-quality (|Δppl| matched) v1.5 still wins: on Qwen3-4B at |Δppl| ≈ 17 %, v1.5 Q=4 (4.57×) beats equivalent-quality TQ b≈3.48 (4.29×) by +6.4 % CR.

NIAH long-context retrieval (ctx ∈ {4k, 8k, 16k} × depth ∈ {0.1, 0.5, 0.9} × 3 trials)

model bf16 v1.5 Q=10 v1.4 Q=10 TQ b=3 TQ b=2 (bdry=2)
Qwen3-4B 100 % 100 % 100 % 100 % 0 %
Gemma-4-E4B 100 % 100 % 100 % 100 % 0 %
GLM-4-9B-Chat 100 % 89 % 100 % 56 % 0 %

v1.5 preserves long-context retrieval on homogeneous-head_dim models (Qwen3, Gemma). GLM-4-9B shows mild v1.5 Q=10 regression (89 % vs v1.4's 100 %), but still far ahead of TQ b=3 (56 %). TQ b=2 is structurally unusable in-forward even with boundary-layer protection on all four models.

Encode / decode latency (H200, N=2048 × H=8 × D=128, 500 iters)

codec mean μs p99 μs ratio vs v1.4
v1.4 D4 Q=10 330 476 1.00×
v1.5 E8 Q=10 596 988 1.80×
v1.4 D4 Q=152 348 723 1.00×
v1.5 E8 Q=152 588 1168 1.69×
TQ b=3 140 145 0.42×

v1.5 encode is 1.56-1.80 × slower than v1.4 (two D8 candidate cosets vs one parity flip). Absolute cost 0.55-0.6 ms per 2048-token slice × 8 heads remains < 2-5 % of typical vLLM 10-30 ms decode step on H200.

Deployment recommendation

regime recommended codec bits/head/tok CR typical |Δppl|
Near-lossless (|Δppl| ≤ 1 %) v1.5 E8 Q=152 1104 1.86× 0.3-0.9 %
Balanced (recommended default) v1.5 E8 Q=10 608 3.37× 1.6-6.9 %
Aggressive (CR-critical) v1.5 E8 Q=4 448 4.57× 5.8-32.4 %
Latency-critical v1.4 D4 Q=10 576 3.56× 2.2-9.9 %
Budget-critical unchanged TurboQuant b=3 416 4.92× 9.1-77.8 %

v1.5 replaces v1.4 as head-of-line codec for deployments where quality matters more than 1-2× encode latency. v1.4 remains valid for latency-critical or size-critical paths.

What's in the release

  • kakeyalattice/python/kakeyalattice/lattice_codebooks.py
    • LatticeCodebook — shared Hadamard + per-vector-qmax base class
    • D4LatticeCodebook — v1.4's quantiser (frozen-sha256 parity-tested)
    • E8LatticeCodebook — v1.5's new 8-D quantiser
  • kakeyalattice/python/kakeyalattice/v1_5_kakeya_zamir_e8_gpu.py
    • V15KakeyaZamirE8GPU — canonical public API (brand wrapper over E8LatticeCodebook)
  • kakeyalattice/python/kakeyalattice/v1_4_kakeya_zamir_lattice_gpu.py
    • V14KakeyaZamirLatticeGPU — unchanged, now a brand wrapper over D4LatticeCodebook
  • benchmarks/rigorous_eval.py — --v15-q-values + --boundary-size flags
  • benchmarks/niah_eval.py — v15 integration + per-layer head_dim dispatch
  • benchmarks/e8_latency_benchmark.py — new; measures pure codec wall time
  • benchmarks/e8_parity_and_smoke.py — frozen sha256 parity gate for v1.4 + v1.5
  • benchmarks/frozen_parity.json — 8 pinned sha256 hashes (v14 × 4 Q, v15 × 4 Q)
  • reports/v1_4_release/rigorous_eval/v15_vs_v14_vs_tq/
    • V15_FULL_4MODEL_REPORT.md — four-model consolidation
    • V15_VS_V14_VS_TQ_REPORT.md — Qwen3-4B first-measurement detail
    • per-model *_inforward.json + *.log for 4 models × 2 boundary modes
    • niah/*_niah_inforward.json + *.log for 4 models NIAH
    • e8_latency_benchmark.json + *.log

Compliance

Built under the same strict ban list as v1.4: no mock / no simplification / no fallback / no overfit / no deferred.

  • head_dim % 8 != 0 → raise ValueError (by design, no fallback; Gemma-4's 256/512 per-layer both satisfy)
  • Frozen sha256 parity: v1.4 Q={4, 10, 38, 152} + v1.5 Q={4, 10, 38, 152} = 8 pinned hashes asserted bit-identical on every benchmarks/e8_parity_and_smoke.py run
  • Fire-count guard: silent passthrough aborts the channel
  • n=32 passages + Student's t 95 % CI for PPL; n_trials=3 with exact substring scoring for NIAH; 500 iters with CUDA.synchronize-gated timing for latency
  • All numbers from real vLLM 0.19.2rc1.dev100+gf946659ff + real FlashAttention v3 + four real open-source models on a real NVIDIA H200

Scope caveats (honest)

  • DeepSeek-V4-Flash / V4-Pro not tested. The new DeepseekV4ForCausalLM architecture is not yet supported by vLLM 0.19.2rc1, the models are 160-865 GB (vs single H200's 143 GB HBM / 32 GB disk), and no DeepseekV4Attention patch exists in snapshot_hook.py. v1.5 tests use DeepSeek-R1-Distill-Qwen-1.5B (same baseline as v1.4). Expanding to V4-family models is v1.6 scope (requires vLLM upstream + multi-node hardware).
  • DeepSeek-R1-Distill-Qwen-1.5B is not deployable in-forward no-boundary on any codec including v1.5 — it is too small (28 layers × 2 KV heads) to absorb cross-layer codec error. With boundary ≥ 2 it is usable for all Kakeya codecs at Q ≥ 10. NIAH at depth=0.9 works (where base model retrieval is feasible).
  • GLM-4-9B shows mild v1.5 NIAH regression at Q=10 (89 % vs v1.4 100 %). Not observed on Qwen3 or Gemma. Likely interaction between GLM's partial-rotary RoPE and E8's Voronoi; does not affect |Δppl| superiority. Recommend v1.5 Q=152 (near-lossless, 100 % NIAH) for GLM deployments.
  • v1.5 costs +5-7 % CR at iso-Q vs v1.4 (fixed 32-bit overhead from E8's lack of parity constraint). Iso-quality makes v1.5 still net-positive vs TQ.

Upgrade path from v1.4

# Before (v1.4):
from kakeyalattice import V14KakeyaZamirLatticeGPU
cb = V14KakeyaZamirLatticeGPU(D=128, q_range=10)

# After (v1.5):
from kakeyalattice import V15KakeyaZamirE8GPU
cb = V15KakeyaZamirE8GPU(D=128, q_range=10)

Same .roundtrip(x) interface. Input must have head_dim % 8 == 0 (D4's less-strict % 4 == 0 still available via V14KakeyaZamirLatticeGPU).

For the vLLM snapshot harness no change is needed — benchmarks/rigorous_eval.py --v15-q-values 4,10 + benchmarks/niah_eval.py --v15-q-values 4,10 already pipe v1.5 through.

Reproducibility

git clone https://github.com/FluffyAIcode/LLM-KV--Cache-compress
cd LLM-KV--Cache-compress
git checkout v1.5
pip install -e kakeyalattice
pip install -e vllm_backend

# Parity gate (bit-level regression test):
python benchmarks/e8_parity_and_smoke.py
# Expected: 8 FROZEN PARITY OK (v14 + v15 × 4 Q values each)

# Full 4-model eval reproduction:
# See reports/v1_4_release/rigorous_eval/v15_vs_v14_vs_tq/V15_FULL_4MODEL_REPORT.md
# section "Reproducibility" for exact per-model commands.

Acknowledgments

This release extends the v1.4 KakeyaLattice work:

  • Theoretical foundation: Zamir-Feder nested lattice codes (1990s-2000s)
  • D4 / E8 lattice algorithms: Conway & Sloane, Sphere Packings, Lattices and Groups (1988), Algorithms 4 & 5
  • TurboQuant reference: Facebook Agnostic Compression team (baseline comparator)
  • H200 evaluation infrastructure: vast.ai

Changelog since v1.4

See the v1.4 → v1.5 compare view.

Key PRs:

  • #32 — E8LatticeCodebook scaffold + parity check
  • #33 — Retire D4TQStyleCodebook (naming cleanup)
  • #34 — Rename package kakeyaturbo_py → kakeyalattice (brand alignment)
  • #35 — v1.5 first real measurement (Qwen3-4B)
  • #36 — v1.5 full 4-model eval (PPL + MSE + CR + latency + NIAH)
Assets 2
Loading