feat(model): add model support verification cards#4909
Conversation
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
…port-cards Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
/nvskills-ci |
| the deferred MCore post-init logic. | ||
| """ | ||
| if self.mamba_chunk_size < 1: | ||
| raise ValueError("mamba_chunk_size must be at least 1.") |
There was a problem hiding this comment.
The new mamba_chunk_size < 1 guard in finalize() has no test. test_hybrid_provider.py covers the default (128) and the applied non-default (256), but never asserts that a value < 1 raises ValueError. Consider adding a small pytest.raises(ValueError) case to lock this in.
|
Review — model support verification cards Light review focused on correctness, tests, and docs. Overall a large but well-structured PR with strong test coverage. No blocking bugs found. Verified as correct
Minor test gaps (non-blocking)
Docs
Suggested test cases
No perf tests impacted. The touched perf_recipes/qwen/h100/qwen3_moe.py only re-bases |
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
/nvskills-ci |
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
/nvskills-ci |
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
|
/nvskills-ci |
|
/ok to test 52c5214 |
What does this PR do?
Adds concise, agent-readable model verification cards under
examples/model_verification_cards/for four variants:It also adds the repo-local create-model-verification-card skill, its deterministic validator, and a helper for exact-length, byte-stable HF inference verification.
The cards distinguish bounded convergence evidence from benchmark-only performance evidence, pin immutable Hugging Face and dataset revisions, preserve recipe-owned GBS/MBS/LR contracts, require saved post-setup runtime configuration, and keep private executor wiring out of tracked cards and PR text.
Card contract
pretrain_performanceonly when the exact variant has a canonical public performance recipe.pretrain_performance.<hardware>leaves.work/...paths.pretrain.H100), with resume/export dependencies resolved only within the same hardware target.Current verification status
The branch remains a draft while the final clean-contract resume/export audits finish. Current head:
fbd1ad6a4.The three long-context items are verified from one coherent clean-run evidence cohort:
Each completed exactly 20 optimizer steps with finite losses, zero skipped/NaN iterations, a persisted runtime config, and independently reconstructed packed-sample accounting.
GPU export config correction
The final commit fixes GPU export for checkpoints whose training-time config differs from the original Hugging Face config. Export retains the existing state-backed reference wrapper and shard map, but replaces its config with the checkpoint-derived Hugging Face config before saving. This preserves strict/distributed streaming behavior while making serialized config shapes agree with the checkpoint tensors.
The focused GPU-export tests pass 10/10 on both macOS and the fixed Linux container. Repository-wide pre-commit passes in that Linux container. The current public docs, secrets, and DCO checks are green.
Revision hardening and validation
git diff --checkpasses, and independent evidence reviews reproduced every promoted metric and supervised-token count.Remaining work
The final bounded-pretrain direct resumes and corrected Moonlight export are queued from frozen, independently reviewed launchers. Execution state alone does not promote an item: each remains unverified until its exact optimizer-step set, finite-loss/no-skip gates, saved runtime config, checkpoint or adapter artifacts, packing/supervised-token evidence, resume sentinels, strict load, deterministic inference, and frozen evidence audit all pass. The cards, PR status, and Linear issues will be promoted only after those gates complete.