You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+21Lines changed: 21 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,27 @@
2
2
3
3
## [Unreleased]
4
4
5
+
## [0.20.0] - 2026-04-24
6
+
7
+
### Added
8
+
9
+
#### Quantized matmul (Q4_K / Q6_K on CPU)
10
+
-**Q6_K Native Matmul**: New `Q6_KTensorData` / `Q6_KBlockTensorData` in `skainet-lang-core` stores 210-byte ggml Q6_K blocks verbatim (128 `ql` + 64 `qh` + 16 scales + 2 f16 `d`), row-major by default, with a `dequantizeBlock` path matching the `DequantOps` reference line-for-line. `DefaultCpuOpsJvm.chooseQuantizedMatmul` dispatches to a new `JvmQuantizedVectorKernels.matmulQ6_KVec` SIMD kernel (Kotlin Vector API, same `floatSpecies` as the Q4_K / Q8_0 kernels) using a dequant-one-block-to-scratch-then-SIMD-dot pattern. New `TensorEncoding.Q6_K` variant. Unblocks running Gemma 4 E2B Q4_K_M (and any mostly-Q4_K + Q6_K checkpoint) through the DSL path without a ~12 GB FP32 dequant blow-up at load.
11
+
-**Q4_K Lazy Shape-Swap Transpose**: `DefaultCpuOpsJvm.transpose(Q4_KTensorData)` now returns a new `Q4_KBlockTensorData` wrapping the *same* packed byte array with swapped shape — mirroring the existing Q4/Q8 MemorySegment lazy-transpose path. `matmulQ4_KVec`'s input-block-major layout produces correct values under the swapped shape without any physical data reordering, so `linearProject(x, W)` can run `matmul(x, transpose(Q4_K_W))` without round-tripping through FP32. Validated at the DSL level by `GemmaDslQ4KTest` in the transformers repo (Δ logits = 4.29e-6 vs the FP32 baseline).
12
+
-**Q6_K Lazy Transpose**: Same shape-swap specialization extended to `Q6_KTensorData`, enabling the same DSL path for Q6_K weights.
13
+
-**Lazy-Transpose Invariant Tests**: New `QuantizedMemSegMatmulTest` cases pin the two load-bearing properties of the Q4_K and Q6_K transpose specializations — (1) shape is swapped; (2) `packedData` is the SAME byte-array reference, not a copy — so the path cannot silently regress to the generic element-wise transpose (which would `ClassCastException` on packed nibbles).
14
+
15
+
#### StableHLO → IREE compilation
16
+
-**SDPA Recording + StableHLO Emission**: `scaledDotProductAttention` is now recorded by `RecordingExecution` (was silently delegating without recording, like `conv1d` before #532) and lowered to StableHLO by `NeuralNetOperationsConverter`. The decomposition is `dot_general(Q, K.T)` (batching dims `[0,1]`, contracting dims `[3]×[3]`) → scale → optional mask → softmax (max-subtract-exp-sum-div) → `dot_general(weights, V)` (contracting dims `[3]×[2]`). New `ScaledDotProductAttentionOperation` in `TensorOperations` with output-shape inference (output shape = query shape). New `SdpaHloExportTest` verifies tape → graph → MLIR with `dot_general`; `TapeAttentionPermuteBugTest` pins a regression around raw array permute producing zero constants. `ShapeOperationsConverter.concatenate` input-type annotation fix. (#543)
17
+
18
+
### Fixed
19
+
-**SDPA Q/K/V Shape Validation**: `scaledDotProductAttention` previously required only rank-4 inputs, so a mismatch in `head_dim` (e.g. Q=512 vs K=256, as seen in real Gemma 4 E2B where mixed-head-dim layers share a KV cache) surfaced as an `ArrayIndexOutOfBoundsException` buried 2000+ lines deep in the dot-product loop. Added `require()` preconditions on matching batch, head count, Q/K head_dim, Q/V head_dim, and K/V `seqKV`, each with a message naming the offending dimensions. New `SDPAShapeValidationTest` (5 cases, `commonTest`) pins the contract.
20
+
21
+
### Dependencies
22
+
- Kotlin: 2.3.20 → 2.3.21 (including JVM toolchain and `plugin.serialization`).
@@ -137,14 +137,13 @@ SKaiNET is a modular ecosystem. While this repository contains the core engine,
137
137
138
138
---
139
139
140
-
## What's New in 0.19.0
140
+
## What's New in 0.20.0
141
141
142
-
-**Qwen / GPT-2 Byte-Level BPE Tokenizer** — Full GPT-2-style pipeline (byte-to-unicode, pretokenization regex, merge-rank BPE, atomic special-token splitting). Builds from GGUF metadata or HuggingFace `tokenizer.json`; verified against Qwen2.5-0.5B reference token IDs.
143
-
-**LLaMA / SentencePiece Tokenizer** — llama.cpp SPM pipeline with whitespace escape, **score-priority** BPE (SPM rule, opposite of GPT-2 merge-rank), and `<0xNN>` byte fallback. Builds from GGUF (`tokenizer.ggml.model == "llama"`) and HuggingFace Unigram `tokenizer.json`.
144
-
-**`TokenizerFactory` Per-Architecture Dispatch** — Tokenizer selection is now per-architecture, not per file format. Qwen/GPT-2 → byte-level BPE, LLaMA/Gemma/TinyLlama → SentencePiece, regardless of whether weights come from GGUF or SafeTensors.
145
-
-**Byte-Level BPE Fix for Qwen/GPT-2** — Previously these models encoded text into garbage tokens because `GgufModelMetadata` ignored `tokenizer.ggml.merges` entirely, blocking chat mode and tool calling. (#463)
146
-
-**LLaMA GGUF Tokenization Fix** — `TokenizerFactory` previously threw `UnsupportedTokenizerException` for LLaMA-family GGUFs; the new SentencePiece path closes that gap. (#464)
147
-
-**GGUF UInt Field Fix** — UINT32 fields (e.g. `tokenizer.ggml.bos_token_id`) are Kotlin `UInt` value classes, not subclasses of `Number`, and were silently dropped by `as? Number` casts. Fixed via a `toIntFlexible` helper that handles every signed and unsigned numeric type GGUF can produce.
142
+
-**Q6_K Native Matmul** — New `Q6_KTensorData` stores 210-byte ggml blocks verbatim and a Vector-API SIMD kernel (`matmulQ6_KVec`) dispatches from `DefaultCpuOpsJvm.chooseQuantizedMatmul`. Together with the existing Q4_K infra, this unblocks running Gemma 4 E2B Q4_K_M (and any mostly-Q4_K + Q6_K checkpoint) through the DSL path without a ~12 GB FP32 dequant blow-up at load.
143
+
-**Q4_K / Q6_K Lazy Shape-Swap Transpose** — `ops.transpose` on `Q4_KTensorData` / `Q6_KTensorData` now returns a new tensor wrapping the *same* packed byte array with swapped shape, matching the existing Q4/Q8 MemorySegment path. `linearProject(x, W)` can run `matmul(x, transpose(W))` on Q4_K/Q6_K weights without round-tripping through FP32 (Δ logits = 4.29e-6 vs FP32 baseline on Gemma).
144
+
-**SDPA → StableHLO / IREE** — `scaledDotProductAttention` is now recorded by `RecordingExecution` and lowered to StableHLO as `dot_general(Q, K.T)` → scale → optional mask → softmax → `dot_general(weights, V)`, so attention blocks compile end-to-end through the SKaiNET → StableHLO → IREE path. (#543)
145
+
-**SDPA Q/K/V Shape Validation** — Mismatched `head_dim` between Q/K or Q/V (seen in real Gemma 4 E2B with mixed-head-dim layers sharing a KV cache) used to surface as an `ArrayIndexOutOfBoundsException` deep in the dot-product loop; `scaledDotProductAttention` now fails fast with `require()` messages naming the offending dimensions.
0 commit comments