feat(cli): add 'gym env validate' pre-flight config check (#1205 friction #12)

[wprazuch]

What

Adds gym env validate (+ ng_validate / nemo_gym_validate deprecated shims) — runs the full config parse with no Ray and no server subprocesses, then exits 0 (valid) / 1 (invalid) with a clean, rich-escaped message (no traceback). Returns in well under a second instead of after a ~30–60s Ray bootstrap.

gym env validate --config resources_servers/<env>/configs/<env>.yaml --config responses_api_models/<model>/configs/<model>.yaml
gym env validate --benchmark gsm8k --model-type openai_model

How

validate() lives in cli/env.py and is registered as env validate in the gym router (cli/main.py COMMANDS) with the same config-selection flags as env start (--config, --benchmark, --environment, --resources-server, --model-type, --search-dir, --model*). It reuses the same get_global_config_dict() parse path the other commands use, so the validation checks stay in sync:

• config_paths resolution — missing/typo'd (#1488) and malformed (#1490)
• server cross-references — unknown name: refs (#1561)
• mandatory ??? values (#1575)
• schema (BaseNeMoGymCLIConfig)

Wrapped in exit_cleanly_on_config_error (from #1609) so any ConfigError becomes a clean message + exit 1. A dummy policy_model is injected (the NO_MODEL parser config, as in gym list / env compose) so model interpolations like ${policy_base_url} resolve without real creds — validation is about config well-formedness; the real model is supplied by the --model* flags at run time.

Targets main

Originally drafted on the unified-CLI epic branch; rebuilt directly on main now that #1630 (and #1637/#1609/#1635/#1671) have merged. The old branch contents (a snapshot of the CLI refactor + unrelated CI commits) were superseded and replaced.

Scope note

The zero-server check (#1489, "nothing configured to run") is intentionally not part of validate: NO_MODEL injects a dummy model server (which would defeat the check), and "is anything configured to run" is a start-time concern already enforced by gym env start before Ray init. validate focuses on config well-formedness.

Why

Epic #1205 friction #12 (no config validation tooling) — the M1 "fast failure triage" deliverable. Config errors otherwise only surface after Ray starts (~30–60s).

Tests

• test_cli_main.py: gym env validate --config X routes to nemo_gym.cli.env:validate with +config_paths=[X] (added to the parametrized config-command matrix).
• test_cli.py: validate() prints OK on a valid config; a raised ConfigError becomes exit 1 (no traceback).
• All test_cli + test_cli_main + test_cli_legacy pass (the only failures are the pre-existing Python-3.12 TestDidYouMean argparse issue on main); ruff + pre-commit clean. Smoke-tested end-to-end: ✓ Config is valid. on a real benchmark, clean error + exit 1 on a bad path, and the ng_validate deprecation shim.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[cmunley1]

This double config flag seems strange to me, i get that we used to do a list, is there any other option? Hmm

gym env validate --config resources_servers/<env>/configs/<env>.yaml --config responses_api_models/<model>/configs/<model>.yaml

[cmunley1]

i guess another option could be

gym env validate --environment workplace_assistant --model-type openai_model

not sure what will be most common for development experience tho

[cmunley1]

Given

A dummy policy_model is injected (the NO_MODEL parser config, as in gym list / env compose) so model interpolations like ${policy_base_url} resolve without real creds — validation is about config well-formedness; the real model is supplied by the --model* flags at run time.

Maybe dont need to even pass the model config here then?

[cmunley1]

lgtm, just docs comment