Skip to content

v0.3.0 pre-release cleanup#208

Merged
skyw merged 4 commits into
mainfrom
skyw/v0.3.0_cleanup
May 26, 2026
Merged

v0.3.0 pre-release cleanup#208
skyw merged 4 commits into
mainfrom
skyw/v0.3.0_cleanup

Conversation

@skyw

@skyw skyw commented May 26, 2026

Copy link
Copy Markdown
Contributor

Clean up a lot of inconsistency.

skyw added 3 commits May 26, 2026 13:50
Signed-off-by: Hao Wu <skyw@nvidia.com>
Signed-off-by: Hao Wu <skyw@nvidia.com>
Signed-off-by: Hao Wu <skyw@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented May 26, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@skyw

skyw commented May 26, 2026

Copy link
Copy Markdown
Contributor Author

@greptile ignore breaking API change in review

@skyw skyw requested a review from mkhona-nvidia May 26, 2026 22:13
@greptile-apps

greptile-apps Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

Got it — I'll stop flagging breaking API changes in future reviews.

View rule

@skyw

skyw commented May 26, 2026

Copy link
Copy Markdown
Contributor Author

Got it — I'll stop flagging breaking API changes in future reviews.

View rule

@greptile only for this PR

@greptile-apps

greptile-apps Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

The rule was already saved as a persistent one. If you'd like it removed so it only applied to this PR, you can delete it from the custom instructions page.

Going forward, I'll flag breaking API changes in other PRs as usual.

@greptile-apps

greptile-apps Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR standardizes the optimizer API across the repo: every step now raises ValueError("closure is not supported") instead of silently calling the closure, and every calculate_*_update function gets a * keyword-only separator with arguments reordered to match the convention documented in the new CONTRIBUTING.md section. All call sites (optimizers and tests) are updated to use keyword arguments.

  • All seven optimizer step methods uniformly reject closures via raise ValueError, replacing the old conditional-call pattern; CONTRIBUTING.md now documents this contract and the argument-order convention.
  • calculate_laprop_update, calculate_adam_update, calculate_ademamix_update, calculate_sim_ademamix_update, calculate_lion_update, calculate_signum_update, and calculate_madam_update all gain the * separator and consistent betas, eps, correct_bias, step ordering.
  • The TYPE_CHECKING overload blocks in all optimizer classes still declare a second overload step(closure: Callable[[], float]) -> float, which is now a misleading contract — a type checker will accept closure calls that always crash at runtime.

Confidence Score: 4/5

Safe to merge with the overload fix addressed — the runtime behavior is correct, but the TYPE_CHECKING stubs still advertise a closure→float path that will always crash.

Every optimizer's runtime guard is now correct and consistent — a non-None closure raises immediately. The remaining gap is in the static type stubs: the second @overload in each class still promises step(closure: Callable[[], float]) -> float, so a type checker will silently accept code that crashes on the first call.

The TYPE_CHECKING overload blocks in all seven optimizer files (laprop.py, lion.py, soap.py, psgd.py, orthogonalized_optimizer.py, adaptive_muon.py, normalized_optimizer.py) need the second overload removed to match the enforced runtime contract.

Important Files Changed

Filename Overview
emerging_optimizers/scalar_optimizers/laprop.py Adds * keyword-only separator to __init__, changes closure from call to ValueError raise, converts update call to kwargs — but the TYPE_CHECKING overload still advertises Callable -> float.
emerging_optimizers/scalar_optimizers/lion.py Same closure fix as laprop.py; betas kwarg call updated — stale Callable -> float overload remains.
emerging_optimizers/soap/soap.py Closure guard switched to ValueError and all five calculate_adam_update arguments converted to kwargs — overload still incorrect.
emerging_optimizers/psgd/psgd.py Identical closure-guard cleanup; stale callable overload persists.
emerging_optimizers/orthogonalized_optimizers/orthogonalized_optimizer.py Closure replaced with ValueError; overload not updated.
emerging_optimizers/riemannian_optimizers/normalized_optimizer.py Both ObliqueSGD and ObliqueAdam get * separator and closure ValueError; both retain the incorrect second overload.
emerging_optimizers/scalar_optimizers/update_functions/adam.py Adds * separator, moves eps before correct_bias to match convention, improves docstring clarity.
emerging_optimizers/scalar_optimizers/update_functions/ademamix.py Adds * separator to both calculate_ademamix_update and calculate_sim_ademamix_update; warmup-step args moved after step per convention.
CONTRIBUTING.md New Optimizer API section documents the closure-not-supported contract and the full argument-order convention with a concrete example; accurate and complete.
tests/test_scalar_optimizers.py All update-function call sites updated to keyword arguments matching the new signatures; logic unchanged.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["optimizer.step(closure)"] --> B{closure is not None?}
    B -- Yes --> C["raise ValueError('closure is not supported')"]
    B -- No --> D["for group in param_groups"]
    D --> E["_init_group(group)"]
    E --> F["for p in group['params']"]
    F --> G{p.grad is None?}
    G -- Yes --> H["skip (pragma: no cover)"]
    G -- No --> I["apply weight decay"]
    I --> J["calculate_*_update(grad, buffers, *, betas=, eps=, ...)"]    
    J --> K["p.data.add_(update, alpha=-lr)"]
    K --> L["return None"]
Loading

Comments Outside Diff (1)

  1. emerging_optimizers/scalar_optimizers/laprop.py, line 124-130 (link)

    P1 Stale overload still advertises closure support

    The second overload (closure: Callable[[], float]) -> float) was accurate when step actually called closure(). Now that every step immediately raises ValueError, this overload is wrong: a type checker will accept loss = opt.step(lambda: compute_loss()) without complaint, but the code crashes at runtime. The same stale overload is present in all seven optimizer classes changed in this PR (Lion, SOAP, PSGDPro, OrthogonalizedOptimizer, AdaptiveMuon, ObliqueSGD, ObliqueAdam). The overload and the runtime annotation Callable[[], float] | None should both be narrowed to None-only.

Reviews (2): Last reviewed commit: "raise on closure" | Re-trigger Greptile

@skyw

skyw commented May 26, 2026

Copy link
Copy Markdown
Contributor Author

@greptile type checking part for step() is necessary and intentional.

@greptile-apps

greptile-apps Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

Understood — keeping the Callable[[], float] overload is intentional, likely to maintain compatibility with PyTorch's Optimizer type contract which requires that signature.

That said, the assert closure is None concern from comment #2 is a separate runtime issue: assert is stripped under python -O, so a non-None closure would silently proceed and return None with no error. Worth considering a if closure is not None: raise TypeError(...) guard regardless of the type stubs decision.

Tip: You can give me more context with [.greptile/rules.md](https://www.greptile.com/docs/code-review/custom-standards) so I get this right next time.

Signed-off-by: Hao Wu <skyw@nvidia.com>
@skyw

skyw commented May 26, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test a475f63

@github-actions

Copy link
Copy Markdown

Test Results

   54 files  ±0    116 suites  ±0   1m 29s ⏱️ -1s
1 090 tests ±0  1 090 ✅ ±0  0 💤 ±0  0 ❌ ±0 
2 431 runs  ±0  2 431 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit a475f63. ± Comparison against base commit bda5c74.

@codecov

codecov Bot commented May 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@skyw skyw merged commit b309e2f into main May 26, 2026
25 checks passed
@skyw skyw deleted the skyw/v0.3.0_cleanup branch May 26, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants