Skip to content

feat: Quantization Refactor#169

Merged
chichun-charlie-liu merged 48 commits intofoundation-model-stack:mainfrom
BrandonGroth:quant_refactor_perCh
Oct 1, 2025
Merged

feat: Quantization Refactor#169
chichun-charlie-liu merged 48 commits intofoundation-model-stack:mainfrom
BrandonGroth:quant_refactor_perCh

Conversation

@BrandonGroth
Copy link
Copy Markdown
Collaborator

Description of the change

This PR adds the quantization refactor for the INT8/4:

  • New file architecture per quantizer family
  • Each quantizer uses "round-clip" order of operations consistent with PyTorch quantizers
  • Each quantizer now has dequantize=False enabled
  • Each quantizer now has "native PT" enabled
  • All new functions come fully tested to a reference TorchQuantizer for the forward method

TODO:

  • Refactor Qmax+Qminmax Per Group quantizer (if necessary)
  • Find a way to ensure the backward pass is fully covered
  • Lower testing duration

Related issues or PRs

N/A

How to verify the PR

run tests/quantizers pytest

Checklist for passing CI/CD:

  • All commits are signed showing "Signed-off-by: Name <email@domain.com>" with git commit -signoff or equivalent
  • PR title and commit messages adhere to Conventional Commits
  • Contribution is formatted with tox -e fix
  • Contribution passes linting with tox -e lint
  • Contribution passes spellcheck with tox -e spellcheck
  • Contribution passes all unit tests with tox -e unit

Note: CI/CD performs unit tests on multiple versions of Python from a fresh install. There may be differences with your local environment and the test environment.

BrandonGroth and others added 30 commits July 15, 2025 17:11
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
…wb_params

Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
…ation_params

Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
…ize_per_channel

Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
Signed-off-by: Brandon Groth <brandon.m.groth@gmail.com>
@BrandonGroth BrandonGroth self-assigned this Jul 29, 2025
@github-actions github-actions Bot added the feat label Jul 29, 2025
@chichun-charlie-liu chichun-charlie-liu marked this pull request as ready for review October 1, 2025 14:27
@chichun-charlie-liu chichun-charlie-liu merged commit 47b8716 into foundation-model-stack:main Oct 1, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants