Skip to content

Commit 449c7fc

Browse files
authored
[AMD] Document QuarkQuantization in quantization docs (microsoft#2533)
## Describe your changes Adds documentation for the `QuarkQuantization` pass (AMD Quark), which was previously absent from the Olive quantization docs. - Adds an **AMD Quark** section to the quantization feature guide (`docs/source/features/quantization.md`) covering both the ONNX and Torch LLM backends, with example configs for each use case. - Adds a `QuarkQuantization` entry to the pass API reference (`docs/source/reference/pass.rst`). This is a docs-only change, independent of microsoft#2532. ## Checklist before requesting a review - [ ] Add unit tests for this change. - [x] Make sure all tests can pass. - [x] Update documents if necessary. - [x] Lint and apply fixes to your code by running `lintrunner -a` - [x] Is this a user-facing change? If yes, give a description of this change to be included in the release notes. Users can now find `QuarkQuantization` in the quantization documentation at https://microsoft.github.io/Olive/features/quantization.html. ## (Optional) Issue link
1 parent 5957997 commit 449c7fc

2 files changed

Lines changed: 44 additions & 0 deletions

File tree

docs/source/features/quantization.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,43 @@ This pass only supports ONNX models, and will only quantize `MatMul` nodes to 4
8383
}
8484
```
8585

86+
## AMD Quark
87+
Olive integrates [AMD Quark](https://quark.docs.amd.com/), AMD's deep learning model quantization toolkit for both PyTorch and ONNX models.
88+
89+
Olive consolidates Quark quantization into a single pass called `QuarkQuantization` that routes to the appropriate Quark backend based on the input model type:
90+
91+
- **ONNX models** (`ONNXModelHandler`) are quantized through the `quark.onnx` API. This path supports static and dynamic quantization, a wide range of data types (Int8/UInt8, Int16/UInt16, BFP16, MX), and advanced algorithms such as CLE, SmoothQuant, GPTQ, AdaRound, AdaQuant, and BiasCorrection.
92+
- **HuggingFace PyTorch models** (`HfModelHandler`) are quantized through the `quark.torch` API for LLMs, supporting schemes such as `uint4_wo_128`, `int4_wo_128`, `int8`, `fp8`, and `mxfp4`, with AWQ/GPTQ/SmoothQuant/rotation algorithms and export to HF safetensors, ONNX, or GGUF formats.
93+
94+
`QuarkQuantization` requires `amd-quark>=0.12`.
95+
96+
Please refer to [QuarkQuantization](quark_quantization) for more details about the pass and its config parameters.
97+
98+
### Example Configuration
99+
100+
a. Quantize an ONNX model (static quantization with calibration data)
101+
```json
102+
{
103+
"type": "QuarkQuantization",
104+
"data_config": "calib_data_config",
105+
"global_config": {
106+
"activation": { "data_type": "UInt8", "calibration_method": "Percentile" },
107+
"weight": { "data_type": "Int8", "calibration_method": "MinMax" }
108+
}
109+
}
110+
```
111+
112+
b. Quantize a HuggingFace LLM (weight-only 4-bit with AWQ)
113+
```json
114+
{
115+
"type": "QuarkQuantization",
116+
"quant_scheme": "uint4_wo_128",
117+
"quant_algo": "awq",
118+
"dataset": "pileval_for_awq_benchmark",
119+
"model_export": ["hf_format"]
120+
}
121+
```
122+
86123
## Quantize with onnxruntime
87124
Quantization is a technique to compress deep learning models by reducing the precision of the model weights from 32 bits to 8 bits. This
88125
technique is used to reduce the memory footprint and improve the inference performance of the model. Quantization can be applied to the

docs/source/reference/pass.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,13 @@ AimetQuantization
213213

214214
.. autoconfigclass:: olive.passes.AimetQuantization
215215

216+
.. _quark_quantization:
217+
218+
QuarkQuantization
219+
-----------------
220+
221+
.. autoconfigclass:: olive.passes.QuarkQuantization
222+
216223
Pytorch
217224
=================================
218225

0 commit comments

Comments
 (0)