Skip to content

Commit f7d840c

Browse files
abrichrclaude
andauthored
feat: TRL GRPOTrainer migration with drop-in Python wrapper (#229)
TRL integration: - Outlines constrained decoding ported to rollout_func - TelemetryCallback maps to our telemetry events - train_trl_grpo.py: --constrained-decoding, --weave-project, --no-telemetry - README: TRL training section with 4 usage examples Drop-in Python wrapper (trl_wrapper.py): - Same API as standalone trainer: TrainingConfig + 4 callback hooks - Internally uses TRL GRPOTrainer + rollout_func - Client code doesn't change: from openadapt_evals.training.trl_wrapper import GRPOTrainer trainer = GRPOTrainer(config, on_step_complete=my_logger) trainer.train() Standalone trainer: - Deprecated with warning (not removed) - Falls back if TRL not installed Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 50fdc33 commit f7d840c

7 files changed

Lines changed: 649 additions & 7 deletions

File tree

README.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -196,6 +196,44 @@ python scripts/run_full_eval.py \
196196

197197
The endpoint uses the UI-Venus native bounding-box prompt format (`[x1,y1,x2,y2]`) and is compatible with vLLM, Ollama, or any OpenAI-compatible server. Both `DemoExecutor` and `PlannerGrounderAgent` use the same prompt format for consistency.
198198

199+
### GRPO training with TRL (recommended)
200+
201+
The recommended path for RL training of VLM desktop agents uses TRL's `GRPOTrainer` with dense milestone rewards from WAA environments. This replaces the standalone GRPO trainer with a battle-tested implementation that supports Unsloth, vLLM, constrained decoding, and automatic telemetry.
202+
203+
```bash
204+
# Basic training against a live WAA VM
205+
python scripts/train_trl_grpo.py \
206+
--task-dir ./example_tasks \
207+
--server-url http://localhost:5001 \
208+
--model Qwen/Qwen2.5-VL-7B-Instruct \
209+
--output ./grpo_output
210+
211+
# With Unsloth (2x VRAM efficiency) + constrained decoding
212+
python scripts/train_trl_grpo.py \
213+
--task-dir ./example_tasks \
214+
--server-url http://localhost:5001 \
215+
--model Qwen/Qwen2.5-VL-7B-Instruct \
216+
--use-unsloth \
217+
--constrained-decoding \
218+
--output ./grpo_output
219+
220+
# Mock mode (validates full pipeline without VM or GPU)
221+
python scripts/train_trl_grpo.py \
222+
--task-dir ./example_tasks \
223+
--mock \
224+
--output ./grpo_output_mock
225+
226+
# With Weave tracing for experiment tracking
227+
python scripts/train_trl_grpo.py \
228+
--task-dir ./example_tasks \
229+
--server-url http://localhost:5001 \
230+
--model Qwen/Qwen2.5-VL-7B-Instruct \
231+
--weave-project openadapt-grpo \
232+
--output ./grpo_output
233+
```
234+
235+
Key flags: `--constrained-decoding` (Outlines regex, eliminates unparseable output), `--vision-loss-mode` (exclude/include/checkpoint), `--weave-project` (Weave tracing), `--use-vllm` (faster generation), `--loss-type` (grpo/dapo/dr_grpo).
236+
199237
### Parallel evaluation
200238

201239
```bash

openadapt_evals/integrations/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
- Weights & Biases (wandb) for experiment tracking and report generation
55
- W&B callback functions for the standalone GRPO trainer
66
- Weave (W&B) for LLM/agent execution tracing
7+
- TRL TrainerCallback for telemetry during GRPO training
78
"""
89

910
from openadapt_evals.integrations.wandb_logger import WandbLogger
@@ -19,6 +20,7 @@
1920
generate_median_case_data,
2021
Scenario,
2122
)
23+
from openadapt_evals.integrations.trl_callbacks import TelemetryCallback
2224

2325
# Import report generator (may fail if wandb reports API not available)
2426
try:
@@ -42,6 +44,7 @@
4244
weave_op = None
4345

4446
__all__ = [
47+
"TelemetryCallback",
4548
"WandbLogger",
4649
"WandbReportGenerator",
4750
"weave_init",
Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
"""TRL TrainerCallback implementations for telemetry and Weave tracing.
2+
3+
Provides callbacks that integrate with TRL's GRPOTrainer to automatically
4+
track training events via our telemetry system and optionally log to Weave.
5+
6+
Usage::
7+
8+
from trl import GRPOConfig, GRPOTrainer
9+
from openadapt_evals.integrations.trl_callbacks import TelemetryCallback
10+
11+
trainer = GRPOTrainer(
12+
model=model,
13+
args=config,
14+
callbacks=[TelemetryCallback(model_name="Qwen/Qwen2.5-VL-7B-Instruct")],
15+
...
16+
)
17+
trainer.train()
18+
"""
19+
20+
from __future__ import annotations
21+
22+
import logging
23+
import time
24+
from typing import Any
25+
26+
logger = logging.getLogger(__name__)
27+
28+
29+
class TelemetryCallback:
30+
"""TRL TrainerCallback that emits telemetry events for training lifecycle.
31+
32+
Calls our telemetry functions at key training milestones:
33+
- ``on_train_begin`` -> ``track_training_run(phase="start", ...)``
34+
- ``on_step_end`` -> ``track_training_step(step=..., reward_mean=..., loss=...)``
35+
- ``on_save`` -> ``track_checkpoint_saved(step=...)``
36+
- ``on_train_end`` -> ``track_training_run(phase="completed", ...)``
37+
38+
Inherits from ``transformers.TrainerCallback`` at runtime so it can be
39+
passed directly to TRL's ``GRPOTrainer(callbacks=[...])``.
40+
41+
All telemetry calls are wrapped in try/except so failures never interrupt
42+
training.
43+
"""
44+
45+
def __init__(
46+
self,
47+
model_name: str | None = None,
48+
task_count: int | None = None,
49+
constrained_decoding: bool = False,
50+
vision_loss_mode: str | None = None,
51+
) -> None:
52+
self.model_name = model_name
53+
self.task_count = task_count
54+
self.constrained_decoding = constrained_decoding
55+
self.vision_loss_mode = vision_loss_mode
56+
self._train_start_time: float | None = None
57+
58+
def on_train_begin(
59+
self,
60+
args: Any,
61+
state: Any,
62+
control: Any,
63+
**kwargs: Any,
64+
) -> None:
65+
"""Called at the start of training."""
66+
self._train_start_time = time.time()
67+
try:
68+
from openadapt_evals.telemetry import track_training_run
69+
70+
num_steps = getattr(args, "max_steps", None)
71+
num_rollouts = getattr(args, "num_generations", None)
72+
track_training_run(
73+
phase="start",
74+
model_name=self.model_name,
75+
num_steps=num_steps,
76+
num_rollouts_per_step=num_rollouts,
77+
task_count=self.task_count,
78+
constrained_decoding=self.constrained_decoding,
79+
vision_loss_mode=self.vision_loss_mode,
80+
)
81+
logger.debug("Telemetry: training_run start emitted")
82+
except Exception as exc:
83+
logger.debug("Telemetry on_train_begin failed: %s", exc)
84+
85+
def on_step_end(
86+
self,
87+
args: Any,
88+
state: Any,
89+
control: Any,
90+
**kwargs: Any,
91+
) -> None:
92+
"""Called at the end of each training step."""
93+
try:
94+
from openadapt_evals.telemetry import track_training_step
95+
96+
# TRL logs metrics in state.log_history
97+
reward_mean = None
98+
loss = None
99+
if state.log_history:
100+
last_log = state.log_history[-1]
101+
reward_mean = last_log.get("reward", last_log.get("reward_mean"))
102+
loss = last_log.get("loss")
103+
104+
track_training_step(
105+
step=state.global_step,
106+
reward_mean=reward_mean,
107+
loss=loss,
108+
)
109+
except Exception as exc:
110+
logger.debug("Telemetry on_step_end failed: %s", exc)
111+
112+
def on_save(
113+
self,
114+
args: Any,
115+
state: Any,
116+
control: Any,
117+
**kwargs: Any,
118+
) -> None:
119+
"""Called when a checkpoint is saved."""
120+
try:
121+
from openadapt_evals.telemetry import track_checkpoint_saved
122+
123+
track_checkpoint_saved(step=state.global_step)
124+
logger.debug("Telemetry: checkpoint_saved at step %d", state.global_step)
125+
except Exception as exc:
126+
logger.debug("Telemetry on_save failed: %s", exc)
127+
128+
def on_train_end(
129+
self,
130+
args: Any,
131+
state: Any,
132+
control: Any,
133+
**kwargs: Any,
134+
) -> None:
135+
"""Called at the end of training."""
136+
try:
137+
from openadapt_evals.telemetry import track_training_run
138+
139+
duration = None
140+
if self._train_start_time is not None:
141+
duration = time.time() - self._train_start_time
142+
143+
# Extract final reward from last log entry
144+
reward_mean = None
145+
loss = None
146+
if state.log_history:
147+
last_log = state.log_history[-1]
148+
reward_mean = last_log.get("reward", last_log.get("reward_mean"))
149+
loss = last_log.get("loss")
150+
151+
track_training_run(
152+
phase="completed",
153+
model_name=self.model_name,
154+
num_steps=state.global_step,
155+
duration_seconds=duration,
156+
reward_mean=reward_mean,
157+
loss=loss,
158+
)
159+
logger.debug("Telemetry: training_run completed emitted")
160+
except Exception as exc:
161+
logger.debug("Telemetry on_train_end failed: %s", exc)
162+
163+
164+
# Register as a TrainerCallback subclass at import time so TRL recognizes it.
165+
# If transformers is not installed, the class still works as a plain object
166+
# (the callback methods are called by name, not by inheritance check in recent
167+
# TRL versions).
168+
try:
169+
from transformers import TrainerCallback as _TrainerCallback
170+
171+
# Dynamically add TrainerCallback as a base class
172+
TelemetryCallback.__bases__ = (_TrainerCallback,) + TelemetryCallback.__bases__
173+
except ImportError:
174+
logger.debug(
175+
"transformers not installed; TelemetryCallback will work as a "
176+
"duck-typed callback but won't inherit from TrainerCallback"
177+
)

openadapt_evals/training/standalone/trainer.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -484,6 +484,12 @@ def _save_checkpoint(self, step: int) -> str:
484484
def train(self) -> str:
485485
"""Run GRPO training loop. Returns path to final checkpoint."""
486486
import torch
487+
488+
logger.warning(
489+
"The standalone GRPO trainer is deprecated. Use scripts/train_trl_grpo.py "
490+
"with TRL's GRPOTrainer instead. See docs/eval_results/ for migration guide."
491+
)
492+
487493
self._load_task_configs()
488494
if not self._config.task_ids:
489495
raise ValueError("No task IDs. Provide --task-dir with YAML configs or set task_ids.")

0 commit comments

Comments
 (0)