Commit d0b0f18
Refactor Megablox GMM kernels for quantization
* Rename gmm.py to backend.py to avoid the name conflict with gmm function.
* Make gmm() function a wrapper around the actual custom_vjp object.
* Move quantization of parameters from backend.py to ops.py for better consistency.
* In bwd pass, apply the scales from residuals to the gradient to avoid double quantization.
PiperOrigin-RevId: 8144367571 parent f817224 commit d0b0f18
5 files changed
Lines changed: 213 additions & 280 deletions
File tree
- src/MaxText
- kernels/megablox
- layers
- tests
0 commit comments