This repository was archived by the owner on Oct 9, 2024. It is now read-only.

Add configs to run int4 inference#37

Open

RezaYazdaniAminabadi wants to merge 5 commits into

huggingface:mainfrom

RezaYazdaniAminabadi:int4-bloom

Commits on Nov 18, 2022

Add configs to run int4 inference
Reza Yazdani
committed
fix quantization-bit config & turn off ds_sample
Reza Yazdani
committed

Commits on Nov 19, 2022

change the quantization config format to work with the new style at DeepSpeed
Reza Yazdani
committed

Commits on Nov 21, 2022