Commit ee420cc
authored
fix(evaluate): Remove ModelPackageConfig from EvaluateBaseModel steps (#5635)
When evaluate_base_model=True, the EvaluateBaseModel step in both
DETERMINISTIC_TEMPLATE and CUSTOM_SCORER_TEMPLATE incorrectly included
ModelPackageConfig with SourceModelPackageArn, causing the base model
evaluation to load fine-tuned model weights instead of using only the
base model from the public hub. This made both evaluations identical,
leading users to believe fine-tuning had no effect.
Remove ModelPackageConfig from the EvaluateBaseModel step in both
templates so it only uses BaseModelArn from ServerlessJobConfig. The
EvaluateCustomModel step retains ModelPackageConfig to correctly load
fine-tuned weights. This is consistent with the fix already applied to
the LLMAJ_TEMPLATE.
---
X-AI-Prompt: Fix BenchMarkEvaluator evaluate_base_model bug from D406780217
X-AI-Tool: Kiro
sim: https://t.corp.amazon.com/D4067802171 parent 73e10c1 commit ee420cc
File tree
1 file changed
+0
-8
lines changed- sagemaker-train/src/sagemaker/train/evaluate
1 file changed
+0
-8
lines changedLines changed: 0 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
93 | 93 | | |
94 | 94 | | |
95 | 95 | | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | 96 | | |
101 | 97 | | |
102 | 98 | | |
| |||
612 | 608 | | |
613 | 609 | | |
614 | 610 | | |
615 | | - | |
616 | | - | |
617 | | - | |
618 | | - | |
619 | 611 | | |
620 | 612 | | |
621 | 613 | | |
| |||
0 commit comments