resume_training false with weights set loads weights but resets optimizer/scheduler, strict_weights_loading argument

[dtronmans]

Purpose

• New model-side config item strict_weights_loading that fails immediately if any part of the model state dict is different.
• If resume_training is set to False and weights are passed, then the weights are still loaded but the optimizer/scheduler states are taken from the config. Previously this combination of resume_training and weights being passed would create a fresh model instead of loading the weights.
• If resume_training is set to True, then the behavior is unchanged (from Fix: resume_training and passing weights should function correctly #326)
• Small bug fix: passing CLI weights would warn Weights provided on the command line, but config weights are set. " "Ignoring weights provided in config or during LuxonisModel initialization even if the config weights were not set, this has been patched. I had to add self._weights_provided_during_init and self._weights_provided_in_config attributes because one level lower (in resolve_weights) it has no indication of which weights came from the config or the CLI.

Specification

Dependencies & Potential Impact

Deployment Plan

Testing & Validation

• Training a model from the DetectionModel config then creating a new checkpoint from this model with an extra key repvgg_encoder.scale_layer.conv.weight.__unexpected__ crashed quickly on train and infer if strict_weights_loading is set to True:

• If strict_weights_loading is set to False (default), training and inference continue even with mis-matching state dicts:

• resume_training set to False and weights are passed: starts from epoch 0 again with only the checkpoint loaded but not the surrounding (optimizer/scheduler/epoch count etc) state.

AI Usage

Submitted code was reviewed by a human: YES

The author is taking the responsibility for the contribution: YES

[coderabbitai]

Warning

Review limit reached

@dtronmans, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 1 minute and 54 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3855011e-5d8e-422e-9b21-65a495c2828c

📥 Commits

Reviewing files that changed from the base of the PR and between 720d4a4 and 1d1ee8e.

📒 Files selected for processing (7)

• configs/README.md
• configs/defaults.yaml
• luxonis_train/config/config.py
• luxonis_train/core/core.py
• luxonis_train/lightning/luxonis_lightning.py
• tests/conftest.py
• tests/integration/test_resume_training.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/changes-to-weights-loading

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}