|
1 | 1 | --- |
2 | | -title: "Overview" |
3 | | -description: "" |
| 2 | +title: "NeMo Gym" |
| 3 | +description: "NeMo Gym is a library for evaluating and improving models and agents using environments." |
4 | 4 | position: 1 |
5 | 5 | --- |
6 | | -## Motivation |
7 | 6 |
|
8 | | -Building and scaling RL training environments for LLMs presents several key challenges: |
| 7 | +# NeMo Gym |
9 | 8 |
|
10 | | -- **Decoupling environments from training**: Many RL workflows tightly couple environment logic with the training pipeline, making it difficult to integrate complex agent loops, iterate on environment design, and run controlled ablations. |
11 | | -- **Representing agentic trajectories consistently**: The community widely uses Chat Completions today, but it was designed for stateless, single-turn interactions. Agentic rollouts include interleaved reasoning, tool calls, and text across multiple turns. Without a schema that natively represents this, custom parsing and serialization is required for every environment. |
12 | | -- **Managing resources**: Environments often depend on external resources such as sandboxed execution, databases, and APIs. Each rollout needs isolated instances that must be reliably initialized and cleaned up. |
13 | | -- **Scaling rollout collection**: Training may require thousands of parallel rollouts. Environment instances must scale accordingly with distribution, load balancing, and fault tolerance. |
| 9 | +NeMo Gym is a library for evaluating and improving models and agents using environments. NeMo Gym provides infrastructure to develop environments, scalably run evaluation and training, and a collection of popular benchmarks and training environments. |
14 | 10 |
|
15 | | -## NeMo Gym |
| 11 | +## When to Use NeMo Gym |
16 | 12 |
|
17 | | -[NeMo Gym](https://github.com/NVIDIA-NeMo/Gym) is an open-source library that provides infrastructure to build RL environments and scale rollout collection, enabling seamless integration with your preferred training framework. |
| 13 | +- You need to **evaluate models or agents** in stateful environments (for example, code execution, tool calling, sandboxes). |
| 14 | +- You want **reproducible evaluation** across teams using shared environments and verifiers. |
| 15 | +- You need to use environments **at scale** — multiple repeats per task, or thousands of concurrent requests for training. |
| 16 | +- You want to **seamlessly transition** between evaluation, agent optimization, and training. |
18 | 17 |
|
19 | | -NeMo Gym was designed to address these challenges and accelerate environment development: |
| 18 | +If you are scoring model outputs with a stateless check and do not need scale or training, a script is probably sufficient. |
20 | 19 |
|
21 | | -- **Decoupled architecture**: Environment development is fully separated from training, so teams can build, test, and iterate on environments independently of the RL training loop. Interoperable with existing environments, systems, and RL training frameworks. |
22 | | -- **Environment scaffolding**: Patterns and infrastructure to accelerate environment development for multi-step, multi-turn, and user modeling scenarios. |
23 | | -- **Standardized trajectories**: NeMo Gym uses the OpenAI Responses API as its native format, providing a schema that natively represents multi-turn, tool-calling agentic rollouts without custom serialization. |
24 | | -- **Managed resource lifecycles**: Resources servers handle initialization, isolation, and cleanup of external dependencies (sandboxes, APIs, databases) per rollout. |
25 | | -- **Scalable rollout collection**: Infrastructure for distributing thousands of parallel rollouts with load balancing and fault tolerance. |
26 | | -- **Growing environment hub**: NVIDIA and community-contributed environments and datasets for training and evaluation. |
| 20 | +## What NeMo Gym Provides |
27 | 21 |
|
28 | | -<Tip> |
29 | | -The name "NeMo Gym" comes from historical reinforcement learning literature, where the word "Gym" refers to a collection of RL training environments! |
| 22 | +- Modular, extensible interfaces for agents, environments, tasks, and verifiers |
| 23 | +- Environment hub of popular benchmarks and training environments |
| 24 | +- Use your own agents or choose from built-in harnesses |
| 25 | +- Scale to thousands of concurrent environments |
| 26 | +- Train with the RL framework of your choice |
| 27 | +- Battle-tested in production Nemotron training |
30 | 28 |
|
31 | | -</Tip> |
| 29 | + |
| 30 | + |
| 31 | +## Integrations |
| 32 | + |
| 33 | +NeMo Gym integrates with the broader agentic ecosystem: |
| 34 | + |
| 35 | +- **Environment libraries**: Seamlessly combine environments and benchmarks from other libraries alongside NeMo Gym environments. |
| 36 | +- **Training framework libraries**: Use environments for SFT and RL training. |
| 37 | +- **Agent harnesses**: Popular agent harnesses for evaluation and training available out of the box. |
| 38 | +- **Agent framework libraries**: Use your custom agent built with agent frameworks in NeMo Gym environments. |
| 39 | +- **Sandboxes**: Isolate agent runtime execution. |
0 commit comments