Document that max_steps is required for iterable train datasets

[albertvillanova]

This PR documents, in the trainer docstrings, that max_steps must be set when train_dataset is an iterable (streaming) dataset.

Motivation

When train_dataset is an IterableDataset, it has no length, and Trainer raises ValueError: The train_dataset does not implement __len__, max_steps has to be specified. at initialization. The requirement was undocumented, so users hit the error with no upfront guidance. This is the TRL-side counterpart to the transformers PR that clarifies the same requirement:

• Clarify that max_steps is required for datasets without __len__ transformers#47155

Solution

Add a note to the train_dataset docstring of the trainers that actually accept iterable datasets, stating that max_steps must be set in the training arguments. GRPO and RLOO are intentionally excluded, since they reject iterable datasets with NotImplementedError, so the note would not apply there.

Changes

• Add the max_steps note to the train_dataset docstring in SFTTrainer, DPOTrainer, KTOTrainer and RewardTrainer

TODO

First merge:

• Support DatasetDict and IterableDatasetDict as eval_dataset in trainers #6322

---

Note

Low Risk
Documentation-only changes to trainer docstrings; no runtime or API behavior changes.

Overview
Documents that max_steps must be set in training arguments when train_dataset is an IterableDataset (e.g. streaming), because length cannot be inferred and Hugging Face Trainer needs a fixed step count for the loop and LR scheduler.

The same note is added under train_dataset in SFTTrainer, DPOTrainer, KTOTrainer, and RewardTrainer docstrings. GRPO and RLOO are unchanged—they reject iterable datasets at runtime, so the note would not apply. This mirrors the transformers-side documentation update.

^{Reviewed by Cursor Bugbot for commit 64e5cc0. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[AmineDiro]

lgtm 👍🏼