Enhance Gemma 4 performance and features across multiple areas

.gitignore

@@ -17,3 +17,10 @@ go/lem
 
 # task build:embed output
 /bin/
+bench/last-run.json
+
+# generated at build:embed time — never commit compiled metallibs
+cli/*.metallib.gz
+
+# built example binaries — never commit build output (the 783MiB HEAD ballast, #51)
+examples/bin/

.gitlab-ci.yml

@@ -0,0 +1,77 @@
+# CI for go-inference on the git.lthn.sh homelab runner (shell executor, AMD box,
+# concurrent = 1 → one job at a time). The junctures come from the shared gates in
+# dappcore/devops/build — the gate owns WHEN, this file owns WHAT:
+#
+#   push    (gate/push.yml)     → build: compile bin/lem for linux/amd64 (engine/hip cgo).
+#   pr      (gate/pr.yml)       → lint → test: gofmt+vet, then the portable suite. Test is
+#                                 skipped if lint fails; "Pipelines must succeed" gates the merge.
+#   release (gate/release.yml)  → per-backend CLI binaries, on a git tag only.
+#   graphify (gitlab/graphify.yml) → auxiliary code-graph refresh (dev/main, allow_failure).
+include:
+  - { project: 'dappcore/devops/build', file: '/gate/push.yml',      ref: dev }
+  - { project: 'dappcore/devops/build', file: '/gate/pr.yml',        ref: dev }
+  - { project: 'dappcore/devops/build', file: '/gate/release.yml',   ref: dev }
+  - { project: 'dappcore/devops/build', file: '/gitlab/graphify.yml', ref: dev }
+
+# Run one pipeline per change: MR pipelines for MRs, tag pipelines for releases,
+# branch pipelines otherwise — and never a redundant branch pipeline while an MR
+# is open for that branch.
+workflow:
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
+    - if: '$CI_COMMIT_TAG'
+    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
+      when: never
+    - if: '$CI_COMMIT_BRANCH'
+
+variables:
+  GOTOOLCHAIN: "auto"                 # box go is 1.22; go.mod's 1.26 directive auto-fetches
+  GIT_SUBMODULE_STRATEGY: none        # a plain Go compile/test needs no vendored C sources
+
+# --- PUSH: standard flow — does it still build? ----------------------------
+build:
+  extends: .gate-push
+  script:
+    - go version
+    - make build
+
+# --- PR gate: lint THEN test (the test stage is skipped if lint fails) ------
+lint:
+  extends: .gate-lint
+  script:
+    - cd go
+    - test -z "$(gofmt -l .)"
+    - go vet ./...
+
+test:
+  extends: .gate-test
+  script:
+    - cd go
+    - go test -count=1 ./...
+
+# --- RELEASE: per-backend CLI binaries, on a tag ---------------------------
+.release-cli:
+  artifacts:
+    name: "$CI_JOB_NAME-$CI_COMMIT_SHORT_SHA"
+    paths: [ build/bin/ ]
+    expire_in: 1 week
+
+lem-cpu-x86:
+  extends: [.gate-release, .release-cli]
+  script: [ make lem-cpu-x86 ]
+
+lem-cpu-aarch64:
+  extends: [.gate-release, .release-cli]
+  script: [ make lem-cpu-aarch64 ]
+
+lem-amd:
+  extends: [.gate-release, .release-cli]
+  variables:
+    GIT_SUBMODULE_STRATEGY: normal    # GitLab inits the ROCm sources for the static archives
+  script: [ make lem-amd ]
+
+lem-cuda:
+  extends: [.gate-release, .release-cli]
+  variables:
+    GIT_SUBMODULE_STRATEGY: normal
+  script: [ make lem-cuda ]

.graphifyignore

@@ -0,0 +1,13 @@
+# .graphifyignore — scope the go-inference knowledge graph to OUR code.
+#
+# Same syntax as .gitignore (incl. ! negation). graphify already auto-respects
+# .gitignore, so /build/, /bin/, .core/, *.test etc. are skipped. This file
+# only ADDS exclusions.
+#
+# external/ is vendored mlx + ROCm C/C++ (git submodules — therefore NOT in
+# .gitignore). It's a dependency, not go-inference's own code; indexing it
+# roughly doubled the graph with headers like amd_hip_fp6.h / OCLBuffer*.h.
+external/
+
+# JS deps under the GUI (normally gitignored already; explicit for safety).
+gui/node_modules/

CLAUDE.md

@@ -9,7 +9,7 @@ The sovereign inference engine for the Core Go ecosystem. Module: `dappco.re/go/
 1. **The shared contract** (`go/*.go`) — `TextModel`/`Backend` interfaces, options, registry, discovery. Backends import this; never the reverse.
 2. **The engines** (`go/engine/`) — `engine/metal` (package `native`): the **no-cgo Apple GPU engine** (darwin/arm64, objc bridge via `github.com/tmc/apple`, ICB-replayed decode, MTP speculative pairs, paged SDPA, q8 KV); `engine/hip`: AMD ROCm (linux/amd64). Plus `cmd/lem` (the serving/bench binary), OpenAI/Anthropic/Ollama compat handlers, and conversation state (`-state`, no-prompt-replay).
 
-`docs/` at repo root is the manual: `architecture.md`, `backends.md` (registry + **engine runtime levers**), `cmd-lem.md`, `build.md`, and **`handover.md` — read that one first; it is the working handover from the engine-perf campaigns.**
+`docs/` at repo root is the manual: `architecture.md`, `backends.md` (registry + **engine runtime levers**), `cmd-lem.md`, `build.md`, `history.md`. Working state (handovers, design drafts, receipts, campaign perf notes) lives with the orchestrating agent, not here.
 
 ## Commands
 
@@ -22,7 +22,8 @@ go test -count=1 ...        # ALWAYS -count=1 for benchmarks — go test caches
 
 # kernel changes (from repo root)
 task metallib:kernels       # 31 .metal files -> build/dist/lib/lthn_kernels.metallib
-gzip -9 -c build/dist/lib/lthn_kernels.metallib > go/cmd/lem/lthn_kernels.metallib.gz   # then rebuild lem
+# metallibs are NEVER committed — cli/*.metallib.gz are generated by `task build:embed`
+# (gitignored); the plain build resolves them externally via MLX_METALLIB_PATH
 
 # bench shape (model path is POSITIONAL, last)
 ../bin/lem generate -draft <assistant-path> -temp 0 -max-tokens 400 [-context 20480 -prompt-file <f>] <model-path>
@@ -38,6 +39,17 @@ gzip -9 -c build/dist/lib/lthn_kernels.metallib > go/cmd/lem/lthn_kernels.metall
 - **No bulk perl/sed refactors** — one site at a time, vet after each.
 - Branch **dev**; origin `github.com/dAppCore/go-inference` (push non-force). Task board via the session task tools; git log is the history of record.
 
+## Qwen hybrid — the FACTORY is the ONLY route (#18/#50)
+
+Since 2026-07-19 (`ae095829`) **qwen3_5 / qwen3_5_moe load through the factory** (`model.Assemble` + `arch_session`) with the fused whole-token chain decode (`engine/metal/arch_qwen_fused.go`): every layer (gated-delta / gated-attention, dense or MoE tail) encodes into ONE command buffer with resident state, CB-recorded and replayed per token where servable. It **beat the composed lane on both local hybrids** — 0.8B 217 vs 195 tok/s greedy, 35B-A3B 30 vs 19 — and #50 (2026-07-20) deleted `model/composed` outright: the factory is no longer the default, it is the only route, for every arch.
+
+- **Routing** (`engine/metal/load.go`): every registered arch through `model.Load`; mamba2/rwkv7 by name to their own loaders. Typed declines name the retired-composed capability gaps: sub-2-bit packs (Bonsai 1-bit — no factory qmv width, #24), the qwen MTP `-draft` pairing (`LoadSpeculativePair` declines hybrids; factory pair route pending), and image turns on the vision-towered 35B (factory serves the TEXT stack; the engine's not-a-vision-model refusal answers). `LTHN_QWEN_FUSED=0` forces the factory's host halves (the correctness reference). `LTHN_QWEN_COMPOSED` is gone with the lane.
+- **The submit-ahead decode tails decline recurrent sessions** (`hasRecurrentLayers`): a speculative link past a stop cannot be unwound from a gated-delta recurrence. Serial tail only.
+- **Replay bookkeeping**: `composedChainReplay` (historic name — it is the factory chain's recorder) advances the `attnKVDeviceState.n` counters ITSELF — bumping them again host-side double-advances the position and silently corrupts KV slot addressing (coherent-but-wrong text; caught by the live-walk A/B).
+- **Activation is per-arch, not per-kernel.** `MoEExpertsQuant` is gemma's **GELU**; the qwen lane is **SiLU** (`MoEExpertsQuantSiLU`). Wrong binding = *coherent-but-wrong* text below the greedy argmax threshold. Gate on the byte-level test (`TestMoEExpertsQuantSiLU`), never on "the output reads fine".
+- **A/B recipe** (from a worktree, GPU free): `task build` → `bin/lem`; `MLX_METALLIB_PATH=build/dist/lib/mlx.metallib bin/lem generate -temp 0 -max-tokens N <snapshot>`; greedy output is the correctness oracle. Run default vs `LTHN_QWEN_FUSED=0`.
+- **#50 residue** (see `go/docs/composed-strip-inventory.md` for the full classification): the `composed_*`-named files in engine/metal ARE the factory chain machinery (rename = follow-up); `examples/pkg/packed-moe` retired with the composed API it demonstrated (a factory packed-MoE example is the follow-up); `examples/pkg/hybrid-quant` serves through the factory unchanged. Next gap to close: mlx-lm ~110 tok/s on the 35B vs our 30 — a unified-engine campaign.
+
 ## Stability Rules (root contract)
 
 Changes to `go/*.go` interfaces affect every consumer simultaneously.

Makefile

@@ -55,9 +55,13 @@ AMD_GOARCH ?= amd64
 CUDA_GOARCH ?= amd64
 CPU_X86_GOARCH ?= amd64
 CPU_AARCH64_GOARCH ?= arm64
+CPU_AARCH64_CC ?= aarch64-linux-gnu-gcc
+CPU_AARCH64_CXX ?= aarch64-linux-gnu-g++
 AMD_CGO_ENABLED ?= 1
 CUDA_CGO_ENABLED ?= 1
-CPU_CGO_ENABLED ?= 0
+# DuckDB (the go-store/chathistory driver) requires cgo — like engine/hip,
+# the linux lanes build with cgo on. Only the Apple metal engine is no-cgo.
+CPU_CGO_ENABLED ?= 1
 RELEASE_BINS := $(CLI_NAME)-amd $(CLI_NAME)-cuda $(CLI_NAME)-cpu-x86 $(CLI_NAME)-cpu-aarch64
 RELEASE_ARCHIVES := $(addsuffix -linux.tar.gz,$(RELEASE_BINS))
 RELEASE_SIDECARS = $(AMD_KERNEL_MODULE_NAME) $(CUDA_KERNEL_MODULE_NAME) $(CPU_X86_KERNEL_MODULE_NAME) $(CPU_AARCH64_KERNEL_MODULE_NAME)
@@ -112,10 +116,13 @@ HIP_CPU_INCLUDE ?= /opt/hip-cpu/include
 HIP_CPU_CXX ?= g++
 HIP_CPU_AARCH64_CXX ?= aarch64-linux-gnu-g++
 HIP_CPU_STD ?= c++20
+HIP_PRODUCTION_GGUF ?=
+HIP_PRODUCTION_HSACO ?= $(AMD_KERNEL_MODULE)
+HIP_PRODUCTION_MOE ?= 0
 
 .PHONY: all help build build-cli lem-rocm lthn-rocm named-binaries release-binaries release-dependency-guard release-artifacts dist static-hip-binaries rocr-cmake-shims hsa-static-archive hip-static-archive require-static-hip-archive hip-link-info lem-amd lem-cuda lem-cpu-x86 lem-cpu-aarch64 lthn-amd lthn-cuda lthn-cpu-x86 lthn-cpu-aarch64 test test-cli test-all clean \
 	hip hip-amd hip-nvidia hip-cpu hip-cpu-x86_64 hip-cpu-aarch64 \
-	test-hip-amd test-hip-nvidia test-hip-cpu test-hip-cpu-runtime test-hip-cpu-kernel-runtime test-zluda-cuda \
+	test-hip-amd test-hip-nvidia test-hip-cpu test-hip-cpu-runtime test-hip-cpu-kernel-runtime test-zluda-cuda test-hip-production \
 	compile-matrix test-matrix
 
 all: build
@@ -131,6 +138,7 @@ help:
 		'  named-binaries     build all named release binaries' \
 		'  release-artifacts  build archives and checksums under $(DIST_DIR)' \
 		'  test               run the Go module test suite' \
+		'  test-hip-production run the AMD Gemma4 release receipt (set HIP_PRODUCTION_GGUF)' \
 		'  test-matrix        run the host suite plus every hip toolchain smoke, skipping what is absent' \
 		'  clean              remove $(BUILD_DIR)'
 
@@ -230,11 +238,12 @@ rocr-cmake-shims:
 		'  set_target_properties(llvm-objcopy PROPERTIES IMPORTED_LOCATION "$(ROCR_LLVM_OBJCOPY)")' \
 		'endif()' > "$(ROCR_CMAKE_SHIM_DIR_ABS)/llvm/LLVMConfig.cmake"
 
-hsa-static-archive: rocr-cmake-shims
-	@test -d "$(ROCR_RUNTIME_SOURCE_DIR_ABS)" || { echo "missing ROCr runtime source submodule: $(ROCR_RUNTIME_SOURCE_DIR)"; exit 1; }
+hsa-static-archive:
+	@test -f "$(ROCR_RUNTIME_SOURCE_DIR_ABS)/CMakeLists.txt" || { echo "ROCr runtime source submodule is not initialized: $(ROCR_RUNTIME_SOURCE_DIR)"; exit 1; }
 	@test -n "$(DRM_AMDGPU_STATIC_ARCHIVE)" || { echo "missing static libdrm_amdgpu.a; install libdrm-amdgpu-dev"; exit 1; }
 	@test -n "$(DRM_STATIC_ARCHIVE)" || { echo "missing static libdrm.a; install libdrm-dev"; exit 1; }
 	@test -n "$(ELF_STATIC_ARCHIVE)" || { echo "missing static libelf.a; install libelf-dev"; exit 1; }
+	$(MAKE) --no-print-directory rocr-cmake-shims
 	$(CMAKE) -S "$(ROCR_RUNTIME_SOURCE_DIR_ABS)" -B "$(ROCR_RUNTIME_BUILD_DIR_ABS)" -G "$(CMAKE_GENERATOR)" \
 		-DBUILD_SHARED_LIBS=OFF \
 		-DClang_DIR="$(ROCR_CMAKE_SHIM_DIR_ABS)/clang" \
@@ -247,8 +256,8 @@ hsa-static-archive: rocr-cmake-shims
 	@test -s "$(ROCR_HSAKMT_STATIC_ARCHIVE)" || { echo "expected static HSAKMT archive was not produced: $(ROCR_HSAKMT_STATIC_ARCHIVE)"; exit 1; }
 
 hip-static-archive:
-	@test -d "$(HIP_API_SOURCE_DIR_ABS)" || { echo "missing HIP API source submodule: $(HIP_API_SOURCE_DIR)"; exit 1; }
-	@test -d "$(HIP_RUNTIME_SOURCE_DIR_ABS)" || { echo "missing HIP runtime source submodule: $(HIP_RUNTIME_SOURCE_DIR)"; exit 1; }
+	@test -d "$(HIP_API_SOURCE_DIR_ABS)/include" || { echo "HIP API source submodule not checked out: $(HIP_API_SOURCE_DIR) (ROCm/HIP is the HIP_COMMON_DIR headers repo — it has no CMakeLists.txt)"; exit 1; }
+	@test -f "$(HIP_RUNTIME_SOURCE_DIR_ABS)/CMakeLists.txt" || { echo "HIP runtime source submodule is not initialized: $(HIP_RUNTIME_SOURCE_DIR)"; exit 1; }
 	$(CMAKE) -S "$(HIP_RUNTIME_SOURCE_DIR_ABS)" -B "$(HIP_RUNTIME_BUILD_DIR_ABS)" -G "$(CMAKE_GENERATOR)" \
 		-DCLR_BUILD_HIP=ON \
 		-DCLR_BUILD_OCL=OFF \
@@ -294,7 +303,7 @@ lem-cpu-x86: hip-cpu-x86_64
 
 lem-cpu-aarch64: hip-cpu-aarch64
 	mkdir -p "$(BIN_DIR_ABS)"
-	CGO_ENABLED=$(CPU_CGO_ENABLED) GOOS=$(TARGET_GOOS) GOARCH=$(CPU_AARCH64_GOARCH) $(GO) -C "$(CLI_SUBTREE)" build -ldflags "$(CLI_GO_LDFLAGS)" -o "$(BIN_DIR_ABS)/$(CLI_NAME)-cpu-aarch64" "$(CLI_CMD)"
+	CGO_ENABLED=$(CPU_CGO_ENABLED) CC=$(CPU_AARCH64_CC) CXX=$(CPU_AARCH64_CXX) GOOS=$(TARGET_GOOS) GOARCH=$(CPU_AARCH64_GOARCH) $(GO) -C "$(CLI_SUBTREE)" build -ldflags "$(CLI_GO_LDFLAGS)" -o "$(BIN_DIR_ABS)/$(CLI_NAME)-cpu-aarch64" "$(CLI_CMD)"
 
 test:
 	$(GO) -C "$(GO_SUBTREE)" test ./... -count=1
@@ -344,6 +353,34 @@ test-hip-cpu-kernel-runtime:
 test-zluda-cuda:
 	GO_ROCM_RUN_ZLUDA_CUDA_TESTS=1 CUDA_PATH="$(CUDA_PATH)" CUDA_HOME="$(CUDA_HOME)" $(GO) -C "$(GO_SUBTREE)" test ./engine/hip -run TestHIPKernelSource_ZLUDACUDARuntimeSmoke_Good -count=1
 
+test-hip-production:
+	@test -n "$(strip $(HIP_PRODUCTION_GGUF))" || { echo "HIP_PRODUCTION_GGUF must name a local Gemma4 GGUF"; exit 1; }
+	@test -f "$(HIP_PRODUCTION_GGUF)" || { echo "HIP production GGUF not found: $(HIP_PRODUCTION_GGUF)"; exit 1; }
+	$(MAKE) --no-print-directory hip-amd
+	@test -f "$(HIP_PRODUCTION_HSACO)" || { echo "HIP production HSACO not found: $(HIP_PRODUCTION_HSACO)"; exit 1; }
+	$(GO) -C "$(GO_SUBTREE)" test ./engine/hip -count=1 -run '^TestScheduler_'
+	@set -o pipefail; \
+		gguf="$$(realpath "$(HIP_PRODUCTION_GGUF)")"; \
+		hsaco="$$(realpath "$(HIP_PRODUCTION_HSACO)")"; \
+		tests='TestNativeDecodeSmokeKernelStatus_Good|TestHIPGemma4ExactStateContinuityHardware_Good|TestHIPLaneSetE2BHardwareMatchesSingleLanes_Good'; \
+		if [ "$(HIP_PRODUCTION_MOE)" = "1" ]; then \
+			tests="$$tests|TestHIPLaneSet26BMoEHardwareMatchesSingleLanes_Good"; \
+		fi; \
+		log="$$(mktemp)"; \
+		trap 'rm -f "$$log"' EXIT; \
+		HIP_VISIBLE_DEVICES="$${HIP_VISIBLE_DEVICES:-0}" \
+		GO_ROCM_RUN_MODEL_TESTS=1 \
+		GO_ROCM_RUN_MOE_LANE_TESTS="$(HIP_PRODUCTION_MOE)" \
+		GO_ROCM_MODEL_PATH="$$gguf" \
+		GO_ROCM_KERNEL_HSACO="$$hsaco" \
+		$(GO) -C "$(GO_SUBTREE)" test ./engine/hip -count=1 -v -run "^($$tests)$$" | tee "$$log"; \
+		status="$${PIPESTATUS[0]}"; \
+		if grep -q -- '--- SKIP:' "$$log"; then \
+			echo "HIP production receipt skipped a required hardware test"; \
+			exit 1; \
+		fi; \
+		exit "$$status"
+
 # test-matrix aggregates the host suite plus every hip toolchain smoke behind
 # one command. Each leg probes its own toolchain first and skips with a
 # one-line reason when it is absent, so the verb runs unattended on any box

Taskfile.yml

@@ -152,6 +152,12 @@ tasks:
     cmds:
       - go test -tags metal_runtime -count=1 ./...
 
+  bench:grid:
+    desc: "Run the locked gemma4 benchmark grid (bench/gemma4.json) via the lem bench verb. Needs build/dist/lib/*.metallib — run task metallib first."
+    dir: cli
+    cmds:
+      - MLX_METALLIB_PATH={{.ROOT_DIR}}/build/dist/lib/mlx.metallib go run -tags metal_runtime . bench --config {{.ROOT_DIR}}/bench/gemma4.json --json {{.ROOT_DIR}}/bench/last-run.json
+
   cover:
     desc: "Portable coverage -> coverage.out (the profile codecov.yml reads; 95% target). Prints the total on the last line."
     dir: go

bench/README.md

@@ -0,0 +1,34 @@
+<!-- SPDX-Licence-Identifier: EUPL-1.2 -->
+
+# bench/ — the reproducible benchmark grids
+
+`lem bench --config bench/gemma4.json --json out.json` runs the locked house
+grid: per model × lane (plain, mtp), one untimed warmup then one timed greedy
+tg-512, decode tok/s from the engine's own metrics. Model refs are HF cache
+repo names, so the same config reproduces the grid on any box sharing the
+cache convention (the metal Mac, the hip AMD box) — edit the JSON to shift
+focus, never the code.
+
+Notes baked into gemma4.json:
+- Drafters are EXPLICIT (auto-detection ambiguity broke the shell sweep this
+  verb replaces) and quant-matched where possible — the 2026-07-16 matrix
+  measured E4B's matched QAT draft at +47% vs +11% for the same draft
+  cross-paired onto the non-QAT base. e4b-4bit therefore runs plain-only.
+- The 26B rows keep their mtp lane deliberately: the −70% MoE-verify loss is
+  #372's open tracking number, and the grid should show it until it's fixed.
+- The default prompt (counting to 800) is the house tg-N workload: maximally
+  predictable output, so MTP acceptance reads at its CEILING — right for
+  comparability with the reference tables, wrong for estimating real-workload
+  gains (creative text measured 8-20% acceptance on the same pair, i.e. MTP
+  roughly break-even there). Bench a workload by shifting the prompt in a
+  config copy.
+- QAT rows decode SLOWER than plain 4bit by design, not by engine defect: the
+  mlx-community `qat-4bit` checkpoints are mixed-precision — every
+  `mlp.{gate,up,down}_proj` carries a per-tensor 8-bit override (config.json
+  `quantization`), attention/embed/head stay 4-bit, group size 64 throughout.
+  Decode is weight-bandwidth-bound, so the extra bytes ARE the slowdown:
+  E2B 4.33 vs 3.55 GB (+22%) → measured −16%, E4B 6.80 vs 5.15 GB (+32%) →
+  −21%, 31B 28.8 vs 18.4 GB (+57%) → −32% — each at or slightly better than
+  the pure byte-ratio prediction (−18/−24/−36%), which is the receipt that the
+  q8 tensors ride the fast qmv path. Pick QAT for quality, plain 4bit for
+  speed; a grid row comparing them measures checkpoint mass, not the engine.

bench/gemma4.json

@@ -0,0 +1,16 @@
+{
+  "prompt": "Write the integers from 1 to 800 separated by single spaces. Output only the numbers, nothing else.",
+  "tokens": 512,
+  "warmup": 16,
+  "runs": [
+    {"name": "e2b-4bit",  "model": "mlx-community/gemma-4-e2b-it-4bit",        "draft": "mlx-community/gemma-4-E2B-it-assistant-bf16"},
+    {"name": "e2b-qat4",  "model": "mlx-community/gemma-4-E2B-it-qat-4bit",    "draft": "mlx-community/gemma-4-E2B-it-assistant-bf16"},
+    {"name": "e4b-4bit",  "model": "mlx-community/gemma-4-e4b-it-4bit"},
+    {"name": "e4b-qat4",  "model": "mlx-community/gemma-4-E4B-it-qat-4bit",    "draft": "mlx-community/gemma-4-E4B-it-qat-assistant-4bit"},
+    {"name": "12b-4bit",  "model": "mlx-community/gemma-4-12B-it-4bit",        "draft": "mlx-community/gemma-4-12B-it-assistant-bf16"},
+    {"name": "26b-4bit",  "model": "mlx-community/gemma-4-26b-a4b-it-4bit",    "draft": "mlx-community/gemma-4-26B-A4B-it-qat-assistant-4bit"},
+    {"name": "26b-qat4",  "model": "mlx-community/gemma-4-26B-A4B-it-qat-4bit","draft": "mlx-community/gemma-4-26B-A4B-it-qat-assistant-4bit"},
+    {"name": "31b-4bit",  "model": "mlx-community/gemma-4-31B-it-4bit",        "draft": "mlx-community/gemma-4-31B-it-assistant-bf16"},
+    {"name": "31b-qat4",  "model": "mlx-community/gemma-4-31B-it-qat-4bit",    "draft": "mlx-community/gemma-4-31B-it-assistant-bf16"}
+  ]
+}