[ref]: Refeactor weight-only quantization. (#275)

C-KRSW · krizaltang · web-flow · commit 8f66b65df037 · 2026-03-25T17:54:13.000+08:00
Co-authored-by: krizaltang &lt;krizaltang@tencent.com&gt;
diff --git a/README.md b/README.md
@@ -22,6 +22,7 @@ A more accessible, comprehensive, and efficient toolkit for large model compress
 </p>
 
 ## 📣Latest News
+- [26/03/25] We have released **DAQ**, the quantization algorithm that preserves the knowledge acquired while the update of parameters is relatively small during post-training training.[[Paper]](https://arxiv.org/abs/2603.22324) | [[Docs]](docs/source/features/quantization/daq.md)
 - [26/02/09] We have released HY-1.8B-2Bit, 2bit on-device large language model,[[Huggingface]](https://huggingface.co/AngelSlim/HY-1.8B-2Bit).
 - [26/01/13] We have released v0.3. We support the training and deployment of Eagle3 for all-scale LLMs/VLMs/Audio models, as detailed in the [guidance documentation](https://angelslim.readthedocs.io/zh-cn/latest/features/speculative_decoding/eagle/index.html). And We released **Sherry**, the hardware-efficient 1.25 bit quantization algorithm [[Paper]](https://arxiv.org/abs/2601.07892) | [[Code]](https://github.com/Tencent/AngelSlim/tree/sherry/Sherry)🔥🔥🔥
 - [25/11/05] We have released v0.2. Quantization support for new models, such as `GLM-4.6`, `Qwen3-VL` and `Qwen3-Omni`, open-sources the Eagle3 speculative decoding training framework, and updates the Diffusion model quantization tools.
diff --git a/README_cn.md b/README_cn.md
@@ -22,8 +22,9 @@
 </p>
 
 ## 📣最新进展
+- [26/03/25] 我们发布了量化算法DAQ，该方法在后训练参数更新较小时，可保留量化后模型能力 [[论文]](https://arxiv.org/abs/2603.22324) | [[文档]](docs/source/features/quantization/daq.md)
 - [26/02/09] 我们发布了 HY-1.8B-2Bit, 2比特端侧大模型, 模型可见[[Huggingface]](https://huggingface.co/AngelSlim/HY-1.8B-2Bit).
-- [26/01/13]我们发布V0.3版本， 支持了全模态场景的投机采样训练及部署，文档：[Eagle3 for LLM/VLM/Audio](https://angelslim.readthedocs.io/zh-cn/latest/features/speculative_decoding/eagle/index.html)。并且我们发布了 **Sherry** 新的硬件高效的1.25bit三值量化算法 [[论文]](https://arxiv.org/abs/2601.07892) | [[代码]](https://github.com/Tencent/AngelSlim/tree/sherry/Sherry)🔥🔥🔥
+- [26/01/13] 我们发布V0.3版本， 支持了全模态场景的投机采样训练及部署，文档：[Eagle3 for LLM/VLM/Audio](https://angelslim.readthedocs.io/zh-cn/latest/features/speculative_decoding/eagle/index.html)。并且我们发布了 **Sherry** 新的硬件高效的1.25bit三值量化算法 [[论文]](https://arxiv.org/abs/2601.07892) | [[代码]](https://github.com/Tencent/AngelSlim/tree/sherry/Sherry)🔥🔥🔥
 - [25/11/05] 我们发布V0.2版本，支持了包括GLM-4.6/Qwen3-VL/Qwen3-Omni等更多模型的量化，开源投机采样Eagle3训练框架，更新Diffusion模型量化工具。
 - [25/09/30] 我们开源了思考早退新算法 **SpecExit** [[论文]](http://arxiv.org/abs/2509.24248) | [[文档]](https://angelslim.readthedocs.io/zh-cn/latest/features/speculative_decoding/spec_exit.html) | [[vLLM代码]](https://github.com/vllm-project/vllm/pull/27192)
 - [25/09/30] 我们发布了三值量化新算法 **Tequila** [[论文]](https://arxiv.org/abs/2509.23809) | [[代码]](https://github.com/Tencent/AngelSlim/tree/tequila/TernaryQuant)
diff --git a/angelslim/compressor/quant/core/kernels.py b/angelslim/compressor/quant/core/kernels.py
diff --git a/angelslim/compressor/quant/modules/daq/__init__.py b/angelslim/compressor/quant/modules/daq/__init__.py
@@ -12,8 +12,12 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from angelslim.compressor.quant.core.kernels import (  # noqa: F401
+    FP8_E4M3_SCHEME,
+    QuantScheme,
+)
+
 from .daq import DAQ  # noqa: F401
-from .kernels import FP8_E4M3_SCHEME, QuantScheme  # noqa: F401
 
 __all__ = [
     "DAQ",
diff --git a/angelslim/compressor/quant/modules/daq/daq.py b/angelslim/compressor/quant/modules/daq/daq.py
@@ -28,15 +28,16 @@
 from safetensors.torch import load_file, save_file
 from tqdm import tqdm
 
-from .kernels import FP8_E4M3_SCHEME, create_quant_kernel
+from angelslim.compressor.quant.core.kernels import FP8_E4M3_SCHEME, create_quant_kernel
+from angelslim.utils import print_info
+
 from .scale_search import scale_search_weight_quant
 from .utils import (
     compute_dynamic_cache_size,
     get_available_gpus,
     get_weight_map,
     load_base_weight,
     prefetch_base_shard,
-    print_info,
 )
 
 __all__ = ["DAQ"]
@@ -282,17 +283,6 @@ def run(self, save_path: str):
 
         print_info("DAQ quantization complete!")
 
-    def save(self, save_path: str):
-        """Run DAQ quantization and save the quantized model."""
-        self.run(save_path)
-
-        index_file = os.path.join(save_path, "model.safetensors.index.json")
-        config_file = os.path.join(save_path, "config.json")
-        if os.path.exists(index_file) and os.path.exists(config_file):
-            print_info(f"Quantized model saved to {save_path}")
-        else:
-            print_info(f"Warning: Expected output files not found in {save_path}")
-
     def _prepare_output_dir(self, save_path: str):
         # TODO: Currently we only support quantizing BF16 DeepSeek V3/R1 models to FP8.
         # To support all model architectures, the logic for determining which weights
diff --git a/angelslim/compressor/quant/modules/daq/scale_search.py b/angelslim/compressor/quant/modules/daq/scale_search.py
@@ -18,7 +18,8 @@
 
 import torch
 
-from .kernels import FP8E4M3Kernel, QuantKernel
+from angelslim.compressor.quant.core.kernels import FP8E4M3Kernel, QuantKernel
+
 from .utils import print_info
 
 
diff --git a/angelslim/compressor/quant/modules/daq/utils.py b/angelslim/compressor/quant/modules/daq/utils.py
@@ -22,12 +22,8 @@
 import torch
 from safetensors.torch import load_file
 
-from .kernels import weight_dequant
-
-
-def print_info(msg: str):
-    """Print info message with [DAQ] prefix."""
-    print(f"[DAQ] {msg}")
+from angelslim.compressor.quant.core.kernels import weight_dequant
+from angelslim.utils import print_info
 
 
 def get_available_gpus():
diff --git a/angelslim/compressor/quant/ptq.py b/angelslim/compressor/quant/ptq.py
@@ -24,7 +24,7 @@
 from ..compressor_factory import CompressorFactory
 from ..transform import TransformFactory
 from .core import PTQHook
-from .modules import AWQ, DAQ, FP8, GPTQ, INT8, NVFP4, W4A8INT8, LeptoFP8, SmoothQuant
+from .modules import AWQ, FP8, GPTQ, INT8, NVFP4, W4A8INT8, LeptoFP8, SmoothQuant
 
 __all__ = ["PTQ"]
 
@@ -117,10 +117,6 @@ def __init__(self, model, slim_config=None):
             )
         elif "nvfp4" in self.quant_algo:
             self.nvfp4 = NVFP4(self.quant_model)
-        elif "daq" in self.quant_algo:
-            self.daq = DAQ(
-                self.quant_model.quant_config._quantization_config, self.absolute_model_path
-            )
         else:
             raise NotImplementedError(f"[AngelSlim Error] algo {self.quant_algo} is not support")
 
@@ -144,8 +140,6 @@ def calibrate(self, dataloader):
             self.int8.run(dataloader)
         elif "nvfp4" in self.quant_algo:
             self.nvfp4.run(dataloader)
-        elif "daq" in self.quant_algo:
-            pass
         else:
             raise AssertionError(
                 f"[AngelSlim Error] algo {self.quant_algo} is not support calibrate"
@@ -155,10 +149,6 @@ def convert(self):
         """
         Saves scales and inserts QDQ modules.
         """
-        if "daq" in self.quant_algo:
-            print_info("DAQ: skipping convert (quantization is performed during save)")
-            return
-
         print_info("Start convert model...")
         if "gptq" in self.quant_algo or "gptaq" in self.quant_algo:
             self.gptq.convert()
@@ -209,8 +199,6 @@ def save(self, save_path: str):
             self.w4a8i8.save(save_path)
         elif "awq" in self.quant_algo:
             self.awq.save(save_path)
-        elif "daq" in self.quant_algo:
-            self.daq.save(save_path)
         else:
             save_func = self.quant_model.get_save_func()(self.quant_model)
             save_func.save(save_path)
diff --git a/angelslim/engine.py b/angelslim/engine.py
@@ -78,7 +78,6 @@ def prepare_model(
         using_multi_nodes=False,
         use_audio_in_video=False,
         attn_implementation="default",
-        build_model=True,
     ) -> Any:
         """Load pretrained model and tokenizer
         Args:
@@ -114,28 +113,26 @@ def prepare_model(
                 assert tokenizer, " If model is set, tokenizer must be also set."
                 self.slim_model.tokenizer = tokenizer
             else:
-                if build_model:
-                    self.slim_model.from_pretrained(
-                        model_path,
-                        torch_dtype=torch_dtype,
-                        device_map=device_map,
-                        trust_remote_code=trust_remote_code,
-                        low_cpu_mem_usage=low_cpu_mem_usage,
-                        use_cache=use_cache,
-                        using_multi_nodes=using_multi_nodes,
-                    )
+                self.slim_model.from_pretrained(
+                    model_path,
+                    torch_dtype=torch_dtype,
+                    device_map=device_map,
+                    trust_remote_code=trust_remote_code,
+                    low_cpu_mem_usage=low_cpu_mem_usage,
+                    use_cache=use_cache,
+                    using_multi_nodes=using_multi_nodes,
+                )
                 self.model_path = model_path
         elif self.series in ["Omni"]:
             if not model:
-                if build_model:
-                    self.slim_model.from_pretrained(
-                        model_path,
-                        torch_dtype=torch_dtype,
-                        device_map=device_map,
-                        trust_remote_code=trust_remote_code,
-                        use_audio_in_video=use_audio_in_video,
-                        attn_implementation=attn_implementation,
-                    )
+                self.slim_model.from_pretrained(
+                    model_path,
+                    torch_dtype=torch_dtype,
+                    device_map=device_map,
+                    trust_remote_code=trust_remote_code,
+                    use_audio_in_video=use_audio_in_video,
+                    attn_implementation=attn_implementation,
+                )
                 self.model_path = model_path
         else:
             raise ValueError(f"Unsupported series: {self.series}")
diff --git a/angelslim/utils/config_parser.py b/angelslim/utils/config_parser.py
@@ -320,20 +320,6 @@ def need_dataset(self) -> bool:
                 return True
         return False
 
-    @property
-    def need_build_model(self) -> bool:
-        """Check if any of the methods requires building the model."""
-        if not self.name:
-            return True
-
-        for method in self.name:
-            # PTQ/QAT usually need calibration dataset
-            if method in ["PTQ"]:
-                # DAQ is data-free, no calibration dataset needed
-                if self.quantization and self.quantization.name == "daq":
-                    return False
-        return True
-
     @property
     def only_inference(self) -> Union[bool, List[bool]]:
         """
diff --git a/configs/deepseek_r1/fp8_daq/deepseek_r1_daq_fp8_w8a8_block.yaml b/configs/deepseek_r1/fp8_daq/deepseek_r1_daq_fp8_w8a8_block.yaml
@@ -14,7 +14,7 @@ model:
 
 # Compression configuration
 compression:
-  name: PTQ
+  name: PTQWeightOnly
   quantization:
     name: daq
     bits: 8
diff --git a/configs/deepseek_r1/fp8_daq/deepseek_r1_daq_fp8_w8a8_channel.yaml b/configs/deepseek_r1/fp8_daq/deepseek_r1_daq_fp8_w8a8_channel.yaml
@@ -14,7 +14,7 @@ model:
 
 # Compression configuration
 compression:
-  name: PTQ
+  name: PTQWeightOnly
   quantization:
     name: daq
     bits: 8
diff --git a/docs/source/features/quantization/daq.md b/docs/source/features/quantization/daq.md
@@ -0,0 +1,153 @@
+(daq)=
+
+# DAQ量化
+
+## DAQ Dynamic量化简介
+
+**DAQ (Delta-Aware Quantization)** 是一个面向大语言模型（LLM）的免标定数据的后训练量化方法。标准量化方法以最小化重建误差为目标，但对基座模型无感知，导致量化噪声会不成比例地破坏编码后训练行为（如 SFT、RLHF、DPO）的小幅度参数增量（ΔW）。DAQ 通过将基于重建的优化目标替换为两个 delta-aware 指标——**Sign Preservation Rate（符号保持率）** 和 **Cosine Similarity（余弦相似度）**——来直接优化 ΔW 的方向保真度，仅需**基座模型**和**后训练模型**的权重矩阵，无需校准数据、激活统计量或 Hessian 估计。完整的方法论和实验细节请参阅[技术报告](https://arxiv.org/abs/2603.22324)。
+
+## 核心特性
+
+- **Delta-Aware 指标**：使用 Sign Preservation Rate 和 Cosine Similarity 优化 ΔW 的方向保真度，而非传统的重建误差
+- **免数据**：仅需基座模型和后训练模型的权重矩阵——无需校准数据、激活统计量和 Hessian 估计
+- **粗-精两阶段搜索**：先进行大范围粗搜索，再在最优候选附近精细搜索，实现高效的 scale 优化
+- **多 GPU 支持**：支持多 GPU 并行处理
+- **灵活的量化方式**：支持 block-wise 和 per-channel 两种量化方式
+
+## 技术报告
+
+关于 DAQ 方法论的完整描述——包括问题定义、指标定义、粗-精两阶段 scale 搜索算法以及详细的实验结果——请参阅[技术报告](https://arxiv.org/abs/2603.22324)。报告中还分析了为什么标准的基于重建的优化目标（如 MSE）会主动破坏后训练知识，并将 DAQ 与 AbsMax、SmoothQuant、AWQ 等基线方法进行了对比。
+
+
+## 环境要求
+
+- Python 3.8+
+- PyTorch 2.0+
+- Triton 2.0+
+- safetensors
+- huggingface_hub
+- tqdm
+
+## 使用方法
+
+运行示例如下
+
+```bash
+python3 tools/run.py -c configs/deepseek_r1/fp8_daq/deepseek_r1_daq_fp8_w8a8_block.yaml
+```
+
+该配置文件中，量化相关参数如下：
+
+该配置文件中，量化相关参数如下：
+- `name`：压缩策略，选填量化`quantization`。
+- `quantization.name`：压缩算法选填`daq`。
+- `quantization.bits`：目标量化比特数，如fp8量化对应填写8bit。
+- `quantization.base_model_path`：基座模型路径。
+- `quantization.base_model_repo`：基座模型在huggingface的路径。
+- `quantization.base_is_fp8`：基座模型是否是FP8格式。
+- `quantization.metric`：优化指标，选填`sign`、`cosine`、`mse`。详细说明可参见[指标说明](#指标说明)或[技术报告](https://arxiv.org/abs/2603.22324)
+- `quantization.quantization_method`：量化方式，选填`blockwise`、`per_channel`。详细说明可参见[量化方式](#量化方式)
+- `quantization.scale_search.min_range`：scale 搜索最小倍率。
+- `quantization.scale_search.max_range`：scale 搜索最大倍率。
+- `quantization.scale_search.coarse_intervals`：粗搜索区间数。
+- `quantization.scale_search.fine_intervals`：精搜索区间数。
+- `quantization.scale_search.delta_threshold`：优化指标为`sign`时，判断Δweight是否有意义的阈值。小于该阈值的值不参与指标计算
+- `quantization.num_workers`：多进程并行处理数。
+- `quantization.gpus`：GPU ID 列表。
+- `quantization.ignore_layers`：忽略量化层列表。
+
+```yaml
+compression:
+  name: PTQ
+  quantization:
+    name: daq
+    bits: 8
+    base_model_path: deepseek-ai/DeepSeek-R1-Base # DAQ-specific: path to the base model
+    base_model_repo: deepseek-ai/DeepSeek-R1
+    base_is_fp8: true # Set to true if the base model is FP8 format
+    metric: cosine    # Optimization metric: "sign"，"cosine"，or "mse"
+    quantization_method: blockwise # Quantization method: "blockwise" or "per_channel"
+    # Scale search hyper-parameters (optional, shown with defaults)
+    scale_search:
+      min_range: 0.8
+      max_range: 1.5
+      coarse_intervals: 5
+      fine_intervals: 10
+      delta_threshold: 1.0e-5
+    # Multi-process parallelism
+    num_workers: 8
+    gpus: "0,1,2,3"
+    # Skip quantization for these layers
+    ignore_layers:
+      - "lm_head"
+
+```
+
+## 指标说明
+
+### 1. Sign Preservation Rate（符号保持率）
+
+- 衡量量化后 `sign(ΔW_quant) == sign(ΔW_post)` 的元素比例
+- 简单、可解释、对幅度差异鲁棒；值越高表示对微调方向的保持越好
+
+### 2. Cosine Similarity（余弦相似度）
+
+- 衡量原始 delta 向量与量化 delta 向量之间的方向对齐程度，归一化到 [-1, 1]
+- 同时捕捉方向和相对幅度信息；值越高表示对微调方向的保持越好
+
+### 3. Mean Squared Error（均方误差，MSE）
+
+- 标准重建损失，衡量量化权重与原始权重之间的平方距离
+- 注意：MSE **不是** delta-aware 的——它对基座模型无感知，并且可能主动破坏后训练知识（详见[技术报告](tech_report.pdf)）
+
+## 量化方式
+
+### 1. Block-wise 量化
+
+- 将权重划分为 128×128 的块
+- 为每个块计算最优 scale
+
+
+### 2. Per-channel 量化
+
+- 为每个通道（行）计算最优 scale
+- 更加内存高效
+
+## 使用场景
+
+- **模型压缩**：缩减模型体积以便部署
+- **边缘部署**：在资源受限的设备上实现 LLM 推理
+- **微调模型优化**：在量化过程中保留微调带来的能力提升
+
+## 量化后模型权重分析
+
+DAQ 内置了分析工具（`tools/`），用于评估量化模型对微调信号的保持程度。
+
+### 计算指标
+
+- **Sign Preservation** — 逐元素检查 `sign(ΔW_quant) == sign(ΔW_sft)` 是否成立
+- **Delta Cosine** — `ΔW_quant` 与 `ΔW_sft` 向量之间的余弦相似度
+- **Distance Metrics** — 模型对之间的 L2、MSE、MAE、余弦相似度
+- **Magnitude Breakdown** — 按 `|ΔW_sft|` 幅度分段统计符号保持率
+
+### 用法
+
+```bash
+python tools/daq_analyze.py \
+    --quantized-model /path/to/quantized/model \
+    --sft-path /path/to/sft/model \
+    --base-model /path/to/base/model \
+    --quant-type block_fp8 \
+    --output-report /path/to/report.json \
+    --gpus 0,1,2,3 \
+    --num-workers 4
+```
+
+### 分析选项
+
+- `--quant-type`：量化格式（`block_fp8`、`channel_fp8`、`channel_int8`、`bf16`）
+- `--output-report`：JSON 报告输出路径（可选）
+- `--num-workers`：并行工作进程数
+- `--gpus`：GPU ID 列表，逗号分隔（如 `0,1,2,3`）
+- `--verbose`：在报告中包含逐权重详细信息
+- `--block-size`：量化格式为 block_fp8时，block量化的块大小（默认：128）
diff --git a/docs/source/features/quantization/index.md b/docs/source/features/quantization/index.md
@@ -13,4 +13,5 @@ awq
 gptq
 fp8_lepto
 qat
+daq
 :::
diff --git a/tools/run.py b/tools/run.py
