use of no stop gneration in validation

Ssukriti · Ssukriti · commit 84c4d8778014 · 2026-01-20T12:50:26.000-07:00
Signed-off-by: Sukriti-Sharma4 &lt;sukriti.sharma4@ibm.com&gt;
diff --git a/aiu_fms_testing_utils/scripts/validation.py b/aiu_fms_testing_utils/scripts/validation.py
@@ -690,7 +690,7 @@ def print_result(result, result_idx: int = 0, file_prefix: str = ""):
         ids.to(validation_device),
         args.max_new_tokens,
         LogitsExtractorHook(),
-        eos_token_id=tokenizer.eos_token_id,
+        eos_token_id=None if args.no_early_termination else tokenizer.eos_token_id,
         attn_algorithm="math",
         **padding_kwargs,
     )

Original file line number	Diff line number	Diff line change
`@@ -690,7 +690,7 @@ def print_result(result, result_idx: int = 0, file_prefix: str = ""):`
`690`	`690`	`ids.to(validation_device),`
`691`	`691`	`args.max_new_tokens,`
`692`	`692`	`LogitsExtractorHook(),`
`693`		`- eos_token_id=tokenizer.eos_token_id,`
	`693`	`+ eos_token_id=None if args.no_early_termination else tokenizer.eos_token_id,`
`694`	`694`	`attn_algorithm="math",`
`695`	`695`	`**padding_kwargs,`
`696`	`696`	`)`