π The feature, motivation and pitch
ModelOpt's SmoothQuant calibration defaults to alpha=1.0 and TensorRT-LLM's
examples/quantization/quantize.py exposes no way to change it β the only override in the
tree is the hardcoded Gemma special case in quantize_by_modelopt.py
({"method": "smoothquant", "alpha": 0.5}). The SmoothQuant paper recommends 0.5β0.9
depending on the model.
In our sweeps on Qwen3-0.6B/1.7B (11 alphas Γ 2 calib sets, real TRT engines, GSM8K-1319),
alpha=1.0 was never optimal and cost up to 20 GSM8K points vs the best alpha in the same
setting (worst case: Qwen3-0.6B, 15.39% at alpha=1.0 vs 35.41% at alpha=0.85, against a
36.32% bf16 baseline).
Proposal: add --smoothquant_alpha (float, default None = current behavior) to quantize.py
and thread it to quantize_and_export, where it sets the same algorithm dict the Gemma
special case already uses. Backward compatible; ModelOpt already pydantic-validates the
field (bounds [0, 1]). Happy to submit the PR (change is ready).
Alternatives
ModelOpt's hf_ptq.py has a --recipe mechanism, so an alternative is to contribute a recipe
YAML on the Model-Optimizer side instead. But TRT-LLM's quantize.py has no recipe mechanism,
so exposing the flag directly is the minimal, self-contained fix for this CLI. The two
approaches don't conflict.
Additional context
Full sweep (11 alphas Γ 2 calib sets Γ 2 Qwen3 sizes):
| Model |
Calib |
alpha=1.0 (current) |
best alpha |
GSM8K cost of default |
| Qwen3-1.7B |
wikitext |
39.80% |
57.09% @ Ξ±=0.75 |
β17.3 pts |
| Qwen3-1.7B |
gsm8k-train |
61.87% |
63.76% @ Ξ±=0.85 |
β1.9 pts |
| Qwen3-0.6B |
wikitext |
21.76% |
31.54% @ Ξ±=0.75 |
β9.8 pts |
| Qwen3-0.6B |
gsm8k-train |
15.39% |
35.41% @ Ξ±=0.85 |
β20.0 pts |
The PR is ready: +121/β1, includes a GPU-free unit test
(tests/unittest/trt/quantization/test_smoothquant_alpha.py). No behavior change when the
flag is unset.
Before submitting a new issue...
π The feature, motivation and pitch
ModelOpt's SmoothQuant calibration defaults to alpha=1.0 and TensorRT-LLM's
examples/quantization/quantize.py exposes no way to change it β the only override in the
tree is the hardcoded Gemma special case in quantize_by_modelopt.py
({"method": "smoothquant", "alpha": 0.5}). The SmoothQuant paper recommends 0.5β0.9
depending on the model.
In our sweeps on Qwen3-0.6B/1.7B (11 alphas Γ 2 calib sets, real TRT engines, GSM8K-1319),
alpha=1.0 was never optimal and cost up to 20 GSM8K points vs the best alpha in the same
setting (worst case: Qwen3-0.6B, 15.39% at alpha=1.0 vs 35.41% at alpha=0.85, against a
36.32% bf16 baseline).
Proposal: add --smoothquant_alpha (float, default None = current behavior) to quantize.py
and thread it to quantize_and_export, where it sets the same algorithm dict the Gemma
special case already uses. Backward compatible; ModelOpt already pydantic-validates the
field (bounds [0, 1]). Happy to submit the PR (change is ready).
Alternatives
ModelOpt's hf_ptq.py has a --recipe mechanism, so an alternative is to contribute a recipe
YAML on the Model-Optimizer side instead. But TRT-LLM's quantize.py has no recipe mechanism,
so exposing the flag directly is the minimal, self-contained fix for this CLI. The two
approaches don't conflict.
Additional context
Full sweep (11 alphas Γ 2 calib sets Γ 2 Qwen3 sizes):
The PR is ready: +121/β1, includes a GPU-free unit test
(tests/unittest/trt/quantization/test_smoothquant_alpha.py). No behavior change when the
flag is unset.
Before submitting a new issue...