Add KV-q4 long-context smoke benchmark

Author: apstenku123

docs/mlx_port_master_plan.md

@@ -891,7 +891,7 @@ Excluded as Hopper-only dead-end on Metal: cppmega/megatron/cute_dsl_mimo/ (sm_9
 173. **SCOPED DONE (local token-id serving only)**: add `create_local_generation_app(...)` in `cppmega_mlx/inference/serve.py`, exported from `cppmega_mlx.inference`. The FastAPI adapter exposes `/health` and `/generate` for already-tokenized prompts, threads eager generation options, supports an explicit `model_kwargs_builder` hook for caller-owned side-channel tensors, validates invalid token-id/options requests as HTTP 400, and can return optional generated text via a caller-supplied token decoder. This follows the resolved local research topology; it is not an OpenAI-compatible chat/completions API, tokenizer service, cloud fleet endpoint, paged-attention serving stack, or vllm-mlx parity claim.
 174. **SCOPED DONE (allocation-safe smoke throughput harness)**: add `scripts/bench_inference_throughput.py` for current MLX inference prefill/decode timing. The default run covers `qwen3_4b_class` and `nam56r_class` smoke-scale route profiles, records prefill tokens/s, decode tokens/s, decode mode, actual config, parameter-memory estimate, hardware metadata, and explicit local-only/no-GB10/no-full-model-throughput claims. Full multi-billion-parameter receipts remain gated behind a future large-run workflow; `--scale full` fails closed unless explicitly allowed and is not produced by the default CI-safe path.
 175. **SCOPED DONE (q4 token-id quality harness)**: add `scripts/bench_inference_quality.py` for local q4 inference quality smoke checks. The harness quantizes eligible `HybridTinyLM` linear layers with the repo-local q4 affine helper, evaluates ARC/MMLU-style multiple-choice and HumanEval-style generation tasks over token-id rows, ships built-in smoke tasks for CI-safe coverage, accepts external JSONL token-id tasks, validates malformed rows fail-closed, and records per-suite metrics plus explicit local-only/no-leaderboard/no-GB10 claims. This is not a real ARC/MMLU/HumanEval leaderboard run, not a full q4 NAM56R/Qwen receipt, and not tokenizer/dataset download wiring.
-176. Add long-context benchmark (NIAH, RULER) on KV-q4 path.
+176. **SCOPED DONE (KV-q4 long-context smoke harness)**: add `scripts/bench_inference_long_context.py` for local token-id NIAH and RULER-style long-context checks on the repo-local contiguous `QuantizedKVCache` path. The default run is allocation-safe, uses built-in smoke tasks, supports external JSONL token-id rows, records actual context length, q4 KV bits/group/start, prefill+decode timing, exact-token-match metrics, memory-safety metadata, and explicit local-only/no-leaderboard/no-GB10/no-full-model claims. This is not a real NIAH/RULER leaderboard run, not a full-model long-context receipt, and not mixed bf16-to-q4 `quantized_kv_start > 0` transition coverage.
 177. Document inference modes in docs/inference.md.
 178. Add inference-only quantize_for_inference.py helper script.
 179. Add JSON-mode constrained decoding (logits processor).

docs/porting_plan.md

@@ -172,6 +172,7 @@ The current collected test files are:
 - tests/test_bench_dtype_cast_cost.py
 - tests/test_bench_dtype_long_horizon.py
 - tests/test_bench_inference_latency.py
+- tests/test_bench_inference_long_context.py
 - tests/test_bench_inference_quality.py
 - tests/test_bench_inference_throughput.py
 - tests/test_bench_matrix.py