ProCodeBench shows that proactive coding assistants trained or evaluated on synthetic traces can misread real development behavior.
The agent must infer coding intent and assistance opportunities from real IDE trajectories rather than explicit user prompts.
The paper uses real VS Code traces and compares them with simulated development trajectories to expose sim-to-real differences.
The benchmark is specific to software development and may not generalize to other computer-use domains without new trace data.
Use this for real-world proactive coding evaluation, sim-to-real analysis, and IDE trace-based benchmark design.