Provide unit testing coverage for all new custom ops. - [x] Tests must support CUDA and, where applicable, CPU devices - [x] Must be device-agnostic, i.e. can add support for additional devices for new backends. - [x] Tests must pass the [`torch.library.opcheck` ](https://pytorch.org/docs/stable/library.html#testing-custom-ops) tests. - [x] Tests must use all combinations of supported data types and options for each op. - [ ] Operator tests should compare CPU outputs with accelerator outputs with low tolerance for divergence - [x] Int8 blockwise quant/dequant - [x] LLM.int8 quant/dequant - [x] LLM.int8 matmul - [x] 4bit blockwise quant/dequant - [x] 4bit matmul
Provide unit testing coverage for all new custom ops.
Tests must support CUDA and, where applicable, CPU devices
Must be device-agnostic, i.e. can add support for additional devices for new backends.
Tests must pass the
torch.library.opchecktests.Tests must use all combinations of supported data types and options for each op.
Operator tests should compare CPU outputs with accelerator outputs with low tolerance for divergence
Int8 blockwise quant/dequant
LLM.int8 quant/dequant
LLM.int8 matmul
4bit blockwise quant/dequant
4bit matmul