Is your feature request related to a problem? Please describe.
The checkpoint conversion for AIU could be improved with some validation to the correctness of the input parameters to save_for_aiu function.
For example, if the weight quantizer in the model to be converted has not been calibrated (as it may happen during a unit test), the clip values may retain their initialization value of 0, thus generating an incorrectly quantized weight tensor which will not trigger recomputation.
In addition, we should ensure that save_for_aiu is only called to convert an 8-bit model (the only precision supported during conversion for AIU, at this time).
cc: @BrandonGroth
Describe the solution you'd like
Gate for incorrect behavior, raise error if pre-recomputation clip values are 0.
Validate number of bits during conversion.
Is your feature request related to a problem? Please describe.
The checkpoint conversion for AIU could be improved with some validation to the correctness of the input parameters to
save_for_aiufunction.For example, if the weight quantizer in the model to be converted has not been calibrated (as it may happen during a unit test), the clip values may retain their initialization value of 0, thus generating an incorrectly quantized weight tensor which will not trigger recomputation.
In addition, we should ensure that
save_for_aiuis only called to convert an 8-bit model (the only precision supported during conversion for AIU, at this time).cc: @BrandonGroth
Describe the solution you'd like
Gate for incorrect behavior, raise error if pre-recomputation clip values are 0.
Validate number of bits during conversion.