Feature/api parity roadmap#5

Merged

solderzzc merged 92 commits into

mainfrom

feature/api-parity-roadmap

Mar 31, 2026

Commits on Mar 23, 2026

feat: Phase 1 API parity with mlx-lm

simba
authored and
simba
committed
feat: Phase 2 — JSON mode, VLM vision support, multipart content, extra sampling params

simba
authored and
simba
committed
feat: Phase 3 — Memory limit, /metrics, enhanced /health, graceful shutdown, stats

simba
authored and
simba
committed
feat: API key authentication (--api-key flag)

simba
authored and
simba
committed

Commits on Mar 24, 2026

feat: Prompt caching — reduce TTFT by reusing system prompt KV state

simba
authored and
simba
committed
fix: CI — install mlx.metallib from Python mlx package

simba
authored and
simba
committed
feat: GPU yield — prevent Metal from starving macOS WindowServer

simba
authored and
simba
committed

Commits on Mar 27, 2026

feat: add memory-aware model partitioning framework

simba
authored and
simba
committed
feat: wire GPU/CPU layer partitioning to --gpu-layers flag

simba
authored and
simba
committed
fix: restore Package.resolved and add CI retry for HF downloads

simba
authored and
simba
committed

Commits on Mar 28, 2026

chore: update fork to f8f315b (20 model architectures with LayerPartitionable)

simba
authored and
simba
committed
feat: add auto-calibration 'Wisdom' system

simba
authored and
simba
committed
feat(moe): Expose --stream-experts flag to enable SSD inference streaming for large MoE models

simba
authored and
simba
committed
feat: add download speed and progress bar UX

simba
authored and
simba
committed
fix(ux): progress bar fraction handling for non-byte counts

simba
authored and
simba
committed
feat(ux): add robust caching bandwidth speedometer

simba
authored and
simba
committed
fix(ux): add autonomous task-driven progress bar and restore MB counts

simba
authored and
simba
committed
feat: localize MLX frameworks to write C++ turboquant and ssd streamer scaffolds

simba
authored and
simba
committed

Commits on Mar 29, 2026

Commits on Mar 30, 2026

Commits on Mar 31, 2026