Fix error type for IterableDataset in GRPO and RLOO

[albertvillanova]

This PR fixes the error type raised when an IterableDataset is passed to GRPOTrainer or RLOOTrainer.

Motivation

NotImplementedError (a subclass of RuntimeError) is meant for abstract methods that subclasses must override, or as a development placeholder. Rejecting an invalid combination of constructor arguments is not that case: the user passed arguments of the right type whose combination is unsupported, which is idiomatically a ValueError.

This aligns with the fix already applied for a similar case in DPOTrainer in:

• Raise ValueError instead of NotImplementedError for Liger DPO with PEFT #6225

Related change: tests are implemented (and should be updated) in:

• Support DatasetDict and IterableDatasetDict as eval_dataset in trainers #6322

Solution

Replace NotImplementedError with ValueError for the iterable dataset check in both trainers.

Changes

• GRPOTrainer: raise ValueError instead of NotImplementedError when train_dataset or eval_dataset is an IterableDataset
• RLOOTrainer: same fix as above

---

Note

Low Risk
Constructor-only exception type change with no training logic or API surface changes beyond how callers catch the error.

Overview
GRPOTrainer and RLOOTrainer now raise ValueError (instead of NotImplementedError) when train_dataset or eval_dataset is an IterableDataset, including iterable splits nested in an eval dict. The error message is unchanged.

Matching tests in test_grpo_trainer.py and test_rloo_trainer.py expect ValueError, consistent with the same pattern already used for DPOTrainer.

^{Reviewed by Cursor Bugbot for commit 53ce2f2. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

shouldn't we use TypeError?

[albertvillanova]

TypeError in this codebase is reserved for when an argument doesn't match the type signature at all.

Here that's not the case: both trainers' signatures explicitly declare train_dataset: Dataset | IterableDataset | None. Therefore, IterableDataset is a documented, accepted type. The rejection isn't "wrong type"; it's "this valid type isn't supported by this trainer's current implementation" (per the linked issue #3213). That's a value/capability constraint, not a type mismatch; the same reasoning #6225 used (use_liger_kernel=True is not supported with PEFT models → ValueError, even though PEFT-ness is itself a type-like property).

So ValueError is the correct match for the existing convention. No change needed.

[qgallouedec]

thanks, interesting