feat: add VAGEN/verl-agent environment adapter for VLM RL training

WAADesktopEnv implements the GymImageEnv protocol from VAGEN, enabling
desktop GUI automation training with verl-agent's multi-turn VLM RL
pipeline (GiGPO, GRPO, PPO).

The adapter translates between openadapt-evals BenchmarkObservation
(PNG bytes + a11y tree) and VAGEN's observation format (obs_str +
multi_modal_input with PIL images).

- Async interface (reset/step/close/system_prompt)
- Action DSL parsing (CLICK, TYPE, KEY, SCROLL, WAIT, DONE)
- Fractional coordinate support (0.0-1.0)
- Lazy adapter initialization
- 21 tests passing with mock adapter
- Example VAGEN training config included

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: abrichr

configs/train_waa_vagen.yaml

@@ -0,0 +1,66 @@
+# VAGEN training config for WAA desktop automation
+#
+# This trains a VLM (e.g., Qwen2.5-VL-3B) to automate Windows desktop tasks
+# using GRPO/GiGPO via the verl-agent framework.
+#
+# Prerequisites:
+#   1. WAA server running (via SSH tunnel): ssh -L 5001:localhost:5050 azureuser@<VM_IP>
+#   2. VAGEN installed: pip install vagen
+#   3. Register env: add to vagen's env_registry.yaml:
+#        WAADesktop: openadapt_evals.adapters.verl_env.WAADesktopEnv
+#
+# Usage:
+#   python -m vagen.train --config configs/train_waa_vagen.yaml
+#
+# For mock testing (no VM):
+#   Set server_url to "mock" and use WAAMockAdapter internally
+
+# --- Model ---
+model:
+  name: Qwen/Qwen2.5-VL-3B-Instruct
+  # For larger models with LoRA:
+  # name: Qwen/Qwen2.5-VL-7B-Instruct
+  # lora:
+  #   r: 16
+  #   alpha: 32
+  #   target_modules: [q_proj, k_proj, v_proj, o_proj]
+
+# --- Environment ---
+envs:
+  - name: WAADesktop
+    n_envs: 8                    # Number of parallel environments (= GRPO group size)
+    data_source: waa
+    seed: [1, 100, 1]            # [start, end, step] for task selection
+    max_turns: 15                # Max actions per episode
+    response_length_per_turn: 512
+    config:
+      server_url: "http://localhost:5001"
+      task_id: "REPLACE_WITH_WAA_TASK_UUID"
+      max_steps: 15
+      evaluate_at_done: true
+      action_type: fractional    # VLM outputs normalized 0-1 coordinates
+
+# --- Training (GRPO) ---
+algorithm:
+  name: grpo                     # or "gigpo" for step-level advantages
+  kl_coef: 0.0                   # No KL penalty (DAPO/Open-Reasoner-Zero style)
+  epsilon: 0.2                   # PPO clip range (inactive with single epoch)
+  gamma: 1.0                     # No discounting for episodic tasks
+
+trainer:
+  total_epochs: 100
+  n_gpus_per_node: 2             # Minimum for VLM training
+  micro_batch_size: 4
+  gradient_accumulation_steps: 2
+
+# --- Rollout ---
+rollout:
+  temperature: 0.7
+  top_p: 0.95
+  mode: async                    # async sglang rollout for throughput
+
+# --- Logging ---
+logging:
+  project: openadapt-waa-rl
+  log_interval: 1
+  save_interval: 10

openadapt_evals/adapters/__init__.py

@@ -39,6 +39,7 @@
     RLEnvironment,
     RolloutStep,
 )
+from openadapt_evals.adapters.verl_env import WAADesktopEnv
 from openadapt_evals.adapters.waa import (
     WAAAdapter,
     WAAConfig,
@@ -69,6 +70,8 @@
     "WAAMockAdapter",
     "WAALiveAdapter",
     "WAALiveConfig",
+    # verl-agent / VAGEN integration
+    "WAADesktopEnv",
     # Task ID validation
     "SyntheticTaskError",
     "is_real_waa_task_id",