|
1 | 1 | # Change log |
2 | 2 |
|
| 3 | +## [v1.2.0] 2026-06-08 |
| 4 | + |
| 5 | +### Save/Load Support for JointQ, RTN, and OneBit Quantizers |
| 6 | + |
| 7 | +- **JointQ**: Added `get_quant_config()`, `finalize_quant_config_for_save()`, and `create_inference_layer()` to `JointQ` class (`onecomp/quantizer/jointq/_jointq.py`) |
| 8 | + - Emits `quant_method="gptq"` to reuse `GPTQLinear` and vLLM GPTQ plugin (JointQ uses the same scale/zero/assignment structure as GPTQ) |
| 9 | + - `create_inference_layer()` converts JointQ's 3D assignment `(out_features, num_groups, group_size)` to 2D qweight `(out_features, in_features)` matching GPTQ format, with scale/zero transposition |
| 10 | + - Handles `actorder` permutation: restores original column order before passing to `GPTQLinear` so `g_idx` is constructed correctly |
| 11 | + - Symmetric quantization: shifts signed integers `[-2^(n-1), 2^(n-1)-1]` to unsigned `[0, 2^n - 1]` for GPTQLinear bit packing |
| 12 | + - Added `bits == 1` warning in `validate_params()`: GPTQLinear weight packing does not support 1-bit; inference layer must be built with `pack_weights=False` |
| 13 | + - Added `_build_quantization_bits()` static method to emit per-layer `quantization_bits` metadata for mixed-precision save |
| 14 | +- **RTN**: Added `get_quant_config()`, `finalize_quant_config_for_save()`, `create_inference_layer()`, and `RTNResult.compute_dequantized_weight()` to `RTN` class (`onecomp/quantizer/rtn/_rtn.py`) |
| 15 | + - Emits `quant_method="gptq"` to reuse `GPTQLinear` and vLLM GPTQ plugin (RTN uses the same qweight/scales/qzeros tensor format) |
| 16 | + - `compute_dequantized_weight()` implements `W = (quantized_weight - zero) * scale` with per-channel and group-wise paths |
| 17 | + - `create_inference_layer()` transposes scale/zero from `(out_features, num_groups)` to `(num_groups, out_features)` for GPTQLinear compatibility |
| 18 | + - Added `_build_quantization_bits()` static method for per-layer metadata |
| 19 | +- **OneBit**: Added `get_quant_config()`, `finalize_quant_config_for_save()`, `create_inference_layer()`, and `OnebitResult.compute_dequantized_weight()` to `Onebit` class (`onecomp/quantizer/onebit/_onebit.py`) |
| 20 | + - Emits `quant_method="onebit"` with OneBit-specific parameters (`iters`, `use_importance_scaling`, `use_balancing`, `balance_iters`, `balance_alpha`) |
| 21 | + - `compute_dequantized_weight()` implements `W ≈ a[:, None] * sign * b[None, :]` |
| 22 | + - `create_inference_layer()` builds `OneBitLinear` via `OneBitLinear.from_quantization_result()` |
| 23 | + - Added `_build_quantization_bits()` static method for per-layer metadata |
| 24 | + |
| 25 | +### Apple Silicon / macOS support |
| 26 | + |
| 27 | +- **MPS quantization**: GPTQ (and AutoBit with GPTQ-only candidates) on `device="mps"`; cross-platform `empty_cache()` via new `onecomp/utils/device.py` (`runner.py`, `quantizer/gptq/_gptq.py`, `quantizer/_quantizer.py`) |
| 28 | +- **MPS device placement (GPTQ on CPU, QEP correction on MPS)**: With `device="mps"`, `run_gptq` moves the Hessian and weights to **CPU** for the full column-wise GPTQ loop (including inverse-Hessian Cholesky). The main reason is not absent Cholesky kernels on MPS (recent PyTorch supports them); if the GPTQ loop stayed on MPS, `maxq.item()` inside `quantize()` would run once per column—each call waits for pending MPS work to finish and read back a single scalar to the host (per-column host sync), not a full matrix copy per column—and that overhead is often several times slower than CPU on Apple Silicon (~4× in internal benchmarks with PyTorch 2.12). When QEP weight correction runs (`adjust_weight`, typically under `qep=True`), per-layer work stays on **MPS** (e.g. `weight @ delta_hatX`); only the Cholesky solve uses CPU via `_safe_cholesky_and_solve` (one solve per layer). A full CPU fallback for QEP does not materially improve speed. Calibration forwards may still use MPS. Details: README (macOS / MPS). |
| 29 | +- **MPS inference**: load saved quantized models on Mac with `QuantizedModelLoader` + Transformers `generate()` (GemLite/vLLM remain Linux + CUDA) |
| 30 | +- **macOS `uv sync`**: added `darwin` to `tool.uv.environments`, `--extra mps` for MPS-enabled PyTorch from PyPI; `--extra cpu` is Linux-only (pytorch-cpu index); Linux-only markers on CUDA extras (`cu118`–`cu130`) |
| 31 | + |
| 32 | +## New Feature : Dashboard |
| 33 | + |
| 34 | +- Added `dashboard/`, a browser-based web app for OneCompression on **SLURM-managed HPC GPU nodes without Docker**: pick a Hugging Face model and quantization settings in the UI, run jobs on the GPU, deploy the quantized checkpoint, and validate inference via chat |
| 35 | +- **Stack**: React + Vite frontend (local PC), FastAPI API, Celery worker + user-built Redis, SQLite job DB, per-job output under `backend/tmp/quantized/`; CUDA quantization via `onecomp` and chat deploy via a separate **vLLM** subprocess from the same `backend/.venv` (`onecomp` + `vllm>=0.21` in `pyproject.toml`) |
| 36 | +- **Quantization methods exposed in the UI**: `gptq`, `autobit`, `jointq`, and `auto_run` (VRAM-based bitwidth / group size); optional QEP (not with JointQ); fractional bit widths for `autobit` / `auto_run` |
| 37 | + |
| 38 | +### New Feature: Global PTQ (Post-Training Quantization) |
| 39 | + |
| 40 | +- Added `GlobalPTQ` and `GlobalPTQDistributed` post-process classes for KL-distillation-based global optimisation of continuous quantization parameters (scales and zeros for GPTQ; scaling factors for DBF) |
| 41 | +- `GlobalPTQ`: Single-GPU implementation with cosine-warmup LR scheduling, early stopping, mixed-precision support, and gradient accumulation |
| 42 | +- `GlobalPTQDistributed`: Multi-GPU implementation using HuggingFace Trainer + DeepSpeed ZeRO-2, supporting KL divergence and/or NTP loss with automatic best-state rollback |
| 43 | + |
| 44 | +### Evaluation: |
| 45 | +- Added `onecomp.eval` and the `onecomp-eval` CLI: one vLLM server, subprocess evaluators, aggregated `summary.json` / `summary.csv` |
| 46 | +- Added `mt_bench` (Japanese MT-Bench) and opt-in `throughput` (TTFT / decode tok/s) evaluators |
| 47 | + |
| 48 | +### for Developer: pre-commit |
| 49 | + |
| 50 | +- Added `.pre-commit-config.yaml` with `black`, `isort`, and local hooks (`no-japanese`, `copyright-header`, `no-email-address`); install with `uv sync --extra dev` then `pre-commit install` (see README) |
| 51 | + |
| 52 | +### OneBitLinear Inference Layer Improvements |
| 53 | + |
| 54 | +- Added `OneBitLinear.from_quantization_result()` class method: builds `OneBitLinear` from `OnebitResult` (mirrors the pattern used by `GPTQLinear` and `DoubleBinaryLinear`) (`onecomp/quantizer/onebit/onebit_layer.py`) |
| 55 | +- Added `OneBitLinear.from_saved_state()` class method: reconstructs `OneBitLinear` from saved `state_dict` tensors (`a`, `b`, `sign_packed`, optional `bias`), using the same `cls.__new__` pattern as `DoubleBinaryLinear` (`onecomp/quantizer/onebit/onebit_layer.py`) |
| 56 | +- Removed `preunpack` parameter from `OneBitLinear.__init__()` and `replace_linear_with_onebit_layer()`: sign matrix is now always stored as packed `uint8` and unpacked on demand during `forward()`, matching the DBF inference layer pattern (`onecomp/quantizer/onebit/onebit_layer.py`) |
| 57 | +- Normalized buffers to FP16 with `detach()` in `OneBitLinear.__init__()` to drop autograd graph |
| 58 | +- Added `_load_from_state_dict()` override to clear `sign_matrix` cache when loading from checkpoint |
| 59 | +- Extracted `_unpack_sign_matrix()` helper for sign matrix unpacking logic |
| 60 | +- Removed unreferenced functions `replace_linear_with_onebit_layer()` and `extract_onebit_weights_for_save()` from `onebit_layer.py`: layer construction is now handled by `OneBitLinear.from_quantization_result()` / `OneBitLinear.from_saved_state()`, and save-time weight extraction is covered by the unified `create_inference_layer()` / `state_dict()` path (`onecomp/quantizer/onebit/onebit_layer.py`) |
| 61 | + |
| 62 | +### QuantizedModelLoader: OneBit Support |
| 63 | + |
| 64 | +- `QuantizedModelLoader` now supports `quant_method="onebit"` (`onecomp/quantized_model_loader.py`) |
| 65 | + - Added `OneBitLinear` to import and layer replacement logic |
| 66 | + - Added `OneBitLinear.from_saved_state()` call path for creating empty OneBit layers during model loading |
| 67 | + - Hadamard hook registration now recognizes `OneBitLinear` as a quantized layer class |
| 68 | + |
| 69 | +### BlockWisePTQ / CBQ OneBit Optimizer Compatibility |
| 70 | + |
| 71 | +- Updated OneBit block-wise and cross-block quantization (CBQ) optimizers to work with packed-only `OneBitLinear` (`onecomp/post_process/_blockwise/onebit_block_optimizer.py`, `onecomp/post_process/_blockwise/onebit_cbq_optimizer.py`) |
| 72 | + - Reads current sign matrices from `sign_packed` via `my_unpack()` when `sign_matrix` is not present, while still allowing `sign_matrix` as a temporary optimization override |
| 73 | + - Writes sign updates back to `sign_packed` with `my_pack()` and clears `sign_matrix` so packed signs remain the single source of truth after hard evaluation, best-state restore, and final updates |
| 74 | + - Hoisted `my_pack` / `my_unpack` imports in the OneBit CBQ optimizer |
| 75 | +- Clarified `OneBitLinear.sign_matrix` as a non-persistent temporary override used by optimization flows such as BlockWisePTQ and CBQ (`onecomp/quantizer/onebit/onebit_layer.py`) |
| 76 | + |
| 77 | +### Bug Fix |
| 78 | + |
| 79 | +- Fixed `GPTQLinear.from_saved_state()`: `_weight_is_packed` now defaults to `False` when `wbits == 1` (JointQ `wbits=1` checkpoints are saved with `pack_weights=False` because GPTQLinear packing does not support 1-bit) (`onecomp/quantizer/gptq/gptq_layer.py`) |
| 80 | +- Fixed redundant symmetric shift in RTN inference layer (`onecomp/quantizer/rtn/_rtn.py`) |
| 81 | +- Fixed `run_onebit()` returning `False` on NaN/Inf detection; now raises `ValueError` with proper GPU tensor cleanup to prevent OOM cascading (`onecomp/quantizer/onebit/onebit_impl.py`) |
| 82 | +- Removed pre-computed `dequantized_weight` from `run_onebit()` return dict and `OnebitResult`; dequantized weight is now computed on demand via `compute_dequantized_weight()` (`onecomp/quantizer/onebit/onebit_impl.py`, `onecomp/quantizer/onebit/_onebit.py`) |
| 83 | +- `QuantizedModelLoader._cast_fp16_to_target_dtype()` now skips `OneBitLinear` in addition to `GPTQLinear` and `DoubleBinaryLinear`, so OneBit's fp16 scaling buffers (`a`, `b`, `bias`) are preserved when loading a OneBit-quantized model that requires `bfloat16` (e.g. Gemma 3 / Gemma 4 detected via `needs_bfloat16`). Without this, the post-load safety-net cast rewrote OneBit's stored fp16 metadata to `bfloat16`, breaking the dtype contract that `OneBitLinear.forward` relies on (`self.a.to(x.dtype)` / `self.b.to(x.dtype)` casts to the activation dtype at compute time). Updated the function's docstring to list `OneBitLinear` alongside the other quantized layer types whose fp16 metadata is intentionally retained (`onecomp/quantized_model_loader.py`). |
| 84 | + |
| 85 | +### Tests |
| 86 | + |
| 87 | +- Enabled inherited `test_forward_error` tests for JointQ, OneBit, and RTN (previously skipped with `"does not support create_inference_layer"`) (`tests/onecomp/quantizer/jointq/test_jointq.py`, `tests/onecomp/quantizer/onebit/test_onebit.py`, `tests/onecomp/quantizer/rtn/test_rtn.py`) |
| 88 | +- Added `_forward_error_features` class attribute to `BaseQuantizeSpec` for parameterizing layer size in `test_forward_error`; JointQ overrides to `32` (requires `in_features` divisible by `pack_factor = 32 // wbits`) (`tests/onecomp/quantizer/test_module.py`) |
| 89 | +- Changed JointQ test default `bits` from `1` to `2` to match GPTQLinear packing constraints (`tests/onecomp/quantizer/jointq/test_jointq.py`) |
| 90 | +- Updated `check_equal_results` in RTN and OneBit tests to use `compute_dequantized_weight()` instead of direct `dequantized_weight` attribute access |
| 91 | +- Updated `apply_quantized_weights` in RTN and OneBit tests to use `compute_dequantized_weight()` with proper dtype preservation |
| 92 | +- Tightened GPTQ unit test tolerances in `tests/onecomp/quantizer/gptq/test_gptq.py` so regressions in dequantized-weight error are detected earlier (`error < 0.4`, `max_error < 1.71`; previously `0.6` / `2.5`) (`tests/onecomp/quantizer/gptq/test_gptq.py`) |
| 93 | +- Fixed `tests/onecomp/quantizer/test_module.py` to feed `y_replaced` consistently into `q_proj` / `k_proj` / `v_proj` after quantized weights are applied, aligning the replacement-path forward test with the intended residual update flow |
| 94 | +- Extracted the duplicated attention+MLP forward loop in `test_quantize_error` into `TestModel.forward()` (`tests/onecomp/quantizer/test_module.py`); both the pre-quantization and post-quantization inference paths now call `model(inp)` directly, eliminating 34 duplicate lines |
| 95 | + |
| 96 | +### Dependencies |
| 97 | + |
| 98 | +- Pinned the `vllm` optional dependency to `vllm>=0.10,<0.22` in `pyproject.toml` (and regenerated `uv.lock`). vLLM 0.22.0 removed the legacy Exllama GPTQ kernel that OneComp's GPTQ serving uses for low bit-widths (2-/3-bit, and 4-/8-bit models that are asymmetric or use `desc_act`), so **vLLM 0.22 and later are not supported** — serving affected models on 0.22+ fails at runtime. Documented in `docs/user-guide/vllm-inference.md` and `docs/getting-started/installation.md`. |
| 99 | + |
| 100 | +### Documentation |
| 101 | + |
| 102 | +- Documented save/load and vLLM compatibility for the newly-supported **JointQ**, **RTN**, and **OneBit** quantizers across the docs: |
| 103 | + - `docs/api/quantizers/base.md`: moved `JointQ`, `RTN`, and `Onebit` into the supported rows of the "Quantizer Feature Support" table (`get_quant_config` / `create_inference_layer` / Save / Quantized PPL/ACC all Yes), and added a new "Saved `quant_method` and vLLM compatibility" table mapping each quantizer to its emitted `quant_method` (`gptq` / `mixed_gptq` / `dbf` / `onebit`) and serving path |
| 104 | + - `docs/user-guide/basic-usage.md`: updated the quantized-model evaluation note and the "Quantizer feature support" table to include JointQ/RTN/OneBit, added a `quant_method` column, and clarified which saved models are vLLM-servable |
| 105 | + - `docs/user-guide/vllm-inference.md`: rewrote the "Supported Quantization Methods" table to distinguish vLLM's **built-in** GPTQ plugin (used for `gptq`: GPTQ uniform bits, JointQ, RTN) from the **OneComp** plugins (`mixed_gptq`, `dbf`), added a note that `Onebit` is not vLLM-servable, and listed the GPTQ/JointQ/AutoBit end-to-end examples; split the `gptq` row so GPTQ/RTN (`wbits` in {2, 3, 4, 8}) and JointQ (`bits` in {2, 3, 4}; `bits=1` is OneComp load-only with `pack_weights=False`) document their distinct supported bit-widths |
| 106 | + - `docs/algorithms/jointq.md`: added a "Save and Load" section (emits `quant_method="gptq"`, served by vLLM's built-in GPTQ plugin), a note that JointQ `bits` is limited to {2, 3, 4} for vLLM (the JointQ core quantizer rejects `bits > 4`) while `bits=1` requires an explicit `runner.save_quantized_model(..., pack_weights=False)` and is OneComp load-only / not vLLM-servable, and clarified that JointQ does not support QEP (`qep=False`) |
| 107 | + - `docs/algorithms/rtn.md`: added a "Save and Load" section, a note that vLLM serving uses `wbits` in {2, 3, 4, 8} (RTN itself accepts a wider range, but GPTQ-compatible bit packing and vLLM serving are limited to these), and a warning that rotation-preprocessed RTN models cannot be served with vLLM (no online Hadamard transform), though they remain loadable with `load_quantized_model()` |
| 108 | + - `docs/getting-started/quickstart.md`, `docs/index.md`, `README.md`: updated quantized-model evaluation and vLLM integration descriptions to include JointQ/RTN/OneBit and reference the GPTQ built-in plugin path |
| 109 | + |
| 110 | +### Examples |
| 111 | + |
| 112 | +- Added `example/vllm_inference/example_jointq_vllm_inference.py`: end-to-end JointQ quantization (4-bit, `group_size=128`) → save → vLLM offline inference. Mirrors the GPTQ vLLM example, uses `qep=False` (JointQ does not support QEP), and documents the `bits >= 2` requirement for vLLM bit-packing. Registered in the README example table. |
| 113 | + |
| 114 | + |
3 | 115 | ## [v1.1.1] 2026-05-21 |
4 | 116 |
|
5 | 117 | ### New Feature: Quantization progress logging |
|
0 commit comments