Fix crash when reward function is a functools.partial or callable instance

[qgallouedec]

Fixes #6133.

What

Reward-function display names were built with an unconditional func.__name__, which raises AttributeError for functools.partial objects and callable class instances. Exactly the picklable reward forms the Async GRPO docs recommend. AsyncGRPO therefore crashed at rollout-worker init on a documented API.

Fix

Added a small shared helper get_callable_name in trl/trainer/utils.py that unwraps functools.partial and falls back to the class name for callable instances, and used it everywhere reward-func names are resolved:

• async_rollout_worker.py (the reported crash)
• grpo_trainer.py, rloo_trainer.py (same latent bug in-process)
• experimental online_dpo_trainer.py, sdpo_trainer.py (kept consistent)

Plain functions and lambdas are unaffected.

---

Note

Low Risk
Narrow display-name fix at reward registration; behavior for plain functions and model-based rewards is unchanged.

Overview
Fixes init-time crashes when custom reward functions are functools.partial or callable class instances (patterns Async GRPO and related trainers document as picklable). Those callables lack a reliable __name__, so building reward_func_names with func.__name__ raised AttributeError.

Adds shared get_callable_name in trl/trainer/utils.py: unwrap nested partials, use the wrapped function’s name when present, otherwise the class name (lambdas still report "<lambda>"). GRPO, RLOO, async rollout worker, and experimental Online DPO / SDPO now use it instead of direct __name__ access for non-module reward funcs. Unit tests cover functions, partials, callable instances, and lambdas.

^{Reviewed by Cursor Bugbot for commit ace778a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[AmineDiro]

Thx for the fix 👍🏼

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.