quantize.py vs convert_checkpoint.py

[DreamGenX]

I've noticed that the new convert_checkpoint.py scripts (such as the one for llama) have some quantization options built in, though some, like fp8, are missing. The readme for llama suggests to use quantize.py for fp8 post-training quantization instead.

Is there any reason these two are split?

[Advait251206]

Yes — the split is intentional, and it comes down to two very different stages of the TensorRT-LLM pipeline.

---

1. convert_checkpoint.py = format + structural conversion

This script is mainly responsible for:

• Converting Hugging Face / PyTorch weights → TensorRT-LLM internal format
• Mapping layer names and tensor layouts
• Applying structural transformations
  • tensor parallel sharding
  • embedding rearrangement
  • fused QKV layout changes
• Optional lightweight quantization (in some models)

Key idea:

👉 It prepares the model so TensorRT-LLM can build an engine

So when it supports quantization, it is usually:

• static / structural quantization
• tightly coupled to conversion
• limited to formats that are safe during graph transformation

Example:

• INT8 weight-only (in some pipelines)
• some AWQ-style folding (depending on version)

But it is not a full PTQ framework

---

2. quantize.py = standalone post-training quantization tool

This script is responsible for:

• Running actual calibration / statistics collection
• Producing quantized weights (especially FP8 / INT8 activation-aware cases)
• Handling formats that require dataset passes

Typical responsibilities:

• FP8 scaling computation
• activation distribution collection
• layer-wise calibration
• KV cache / activation-aware quantization prep (depending on version)

Key idea:

👉 It produces a quantized checkpoint, not an engine-ready layout

---

3. Why they are separated

Because they solve fundamentally different problems:

Stage	Tool	Responsibility
1	quantize.py	Compute quantization parameters (FP8, scaling, stats)
2	convert_checkpoint.py	Convert + shard + adapt model for TensorRT-LLM engine

---

4. Why FP8 is usually in quantize.py only

FP8 requires:

• runtime-aware scaling
• calibration or heuristics over activations
• per-tensor / per-channel scaling factors

That logic:

❌ is not part of model conversion
✔ is part of quantization pipeline

So FP8 is kept in quantize.py because:

• it depends on data / calibration
• it must run before engine build
• it is independent of tensor-parallel layout decisions

---

5. Why some quantization exists in convert_checkpoint.py

Some quantization types are:

• deterministic
• weight-only
• layout-safe

Example:

• INT8 weight-only (in some cases)
• simple scaling fold-ins

These can safely happen during conversion because:

• they don’t require calibration data
• they don’t depend on runtime activations

---

6. Mental model (important)

Think of the pipeline like this:

HF Model
   ↓
[quantize.py]   → compute scales / FP8 / calibration
   ↓
quantized HF checkpoint
   ↓
[convert_checkpoint.py] → shard + layout transform
   ↓
TensorRT-LLM engine build

---

7. Why this design matters

TensorRT-LLM separates them to:

• keep conversion deterministic and fast
• avoid mixing data-dependent logic with structural transformation
• allow reuse of quantization across multiple engine builds
• support multiple backends (FP8, INT8, AWQ, etc.)

---

Bottom line

• quantize.py = data-driven quantization stage (FP8, calibration, stats)
• convert_checkpoint.py = model restructuring + TensorRT-LLM compatibility layer
• They are split because they operate on different inputs and different assumptions

---

If this answer helped or pointed you in the right direction, I'd appreciate it if you could mark it as the accepted answer so it's easier for others with the same issue to find.

Also, if you found my contribution useful, I'd appreciate it if you could check out my GitHub profile, follow me, and star any repositories you find interesting.

GitHub: https://github.com/Advait251206