Commit 8b0c3b0
authored
feat(pt): add HybridMuonOptimizer (#5149)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added HybridMuon optimizer option with configurable hyperparameters
(momentum alias, lr_adjust, lr_adjust_coeff, min_2d_dim, muon_2d_only,
weight decay) and integrated training support (initialization, LR
tracking, scheduler compatibility, checkpoint resume).
* **Tests**
* Added comprehensive tests covering orthogonalization, optimizer step
behavior, weight-decay effects, fallback cases, LR-adjust modes, and
state save/load.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->1 parent 9553e6e commit 8b0c3b0
File tree
5 files changed
+1087
-13
lines changed- deepmd
- pt
- optimizer
- train
- utils
- source/tests/pt
5 files changed
+1087
-13
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
5 | 8 | | |
6 | 9 | | |
7 | 10 | | |
8 | 11 | | |
9 | 12 | | |
10 | 13 | | |
11 | 14 | | |
12 | | - | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
0 commit comments