docs: fill CLI reference gaps for data prep and rollout collection#1675
Conversation
- ng_materialize_prompts: full parameter table, example, and note vs ng_prepare_data (closes #1347). - resume_from_cache: params-table entries plus Resume Interrupted Runs section (closes #1239). - Generation parameters: ++responses_create_params.<field> Hydra overrides with shallow-merge behavior documented (closes #637). - PyPI-first installation tab in installation.mdx (closes #1191). - Mirror CLI reference and installation changes into v0.3.0 stable docs. Signed-off-by: Lawrence Lane <llane@nvidia.com>
|
🌿 Preview your docs: https://nvidia-preview-lbliii-docs-cli-dco-fix.docs.buildwithfern.com/nemo/gym Here are the markdown pages you've updated: |
Resolve cli-commands.mdx conflict by adopting main's gym CLI rewrite and porting doc-gap content (resume interrupted runs, shallow-merge Hydra note, dataset render vs collate guidance) into the new structure. Signed-off-by: Lawrence Lane <llane@nvidia.com>
sephmard
left a comment
There was a problem hiding this comment.
Good additions overall — the Resume Interrupted Runs section and the render vs. collate Note are both genuinely useful. Two things to resolve before merge: the latest/ Generation Parameters example still uses old ng_collect_rollouts syntax (tracked in #1741 — suggest absorbing those fixes here before merging), and the about shallow merge reads better adjacent to the gym eval run section rather than at the top of the page before the commands are introduced.
| pip install nemo-gym | ||
| ``` |
There was a problem hiding this comment.
The PyPI tab shows pip install nemo-gym with no Python version constraint, then uv pip install nemo-gym with --python 3.12. The plain pip path could silently use the wrong Python version. Worth either adding a note ("requires Python 3.12") or showing pip with a version check first.
| +wandb_project=gym-dev | ||
| ``` | ||
|
|
||
| <Tip> |
There was a problem hiding this comment.
Nit: Generation Parameters Tip placement.
The about shallow merge is placed at the very top of the page in the quick-reference section (line 95 context), before any commands are defined. A reader scanning top-to-bottom hits this tip before they know what gym eval run or responses_create_params are. It reads better adjacent to the gym eval run section where it's acted on.
Signed-off-by: Lawrence Lane <llane@nvidia.com>
Signed-off-by: Lawrence Lane <llane@nvidia.com>
|
Updated this PR to cover the remaining review feedback and absorb the relevant latest-doc content from #1741. Summary:
Validation:
|
Summary
Fills documented gaps in the CLI reference and installation pages. All CLI additions were verified against
nemo_gym/prompt.pyandnemo_gym/rollout_collection.py.ng_materialize_prompts— full parameter table, example, and a note clarifying when to use it vs.ng_prepare_data. Closes docs: add missing CLI commands to reference and clarify when to use each #1347.resume_from_cache— params-table entries forng_collect_rolloutsandng_e2e_collect_rollouts, plus a Resume Interrupted Runs section covering the*_materialized_inputs.jsonlartifact, per-rollout incremental flush,(task_index, rollout_index)matching, fallback behavior, and the stale-cache footgun. Closes docs: document resume_from_cache for rollout collection #1239.temperature,top_p, andmax_output_tokensare passed via++responses_create_params.<field>rather than as standalone flags. Documents shallow (top-level) merge behavior for nested overrides. Closes bug: could not find temperature CLI params due to docs segmented control #637.installation.mdx. Closes docs: swap install path to pip install nemo-gym with 0.3.0 PyPI release #1191.Review feedback addressed
responses_create_paramsoverrides use a shallow merge (dict | overridesin_preprocess_rows_from_config), so nested Hydra overrides replace the entire nested dict rather than deep-merging fields.Verification
npm run check(fern check) passes with 0 errors.Note: Replaces #1498. That PR's branch (
docs/cli-reference-cleanup) could not be rewritten due to branch protection (no force-push/delete); an unsigned commit (91526fce) left DCO failing. This PR uses a single signed commit onlbliii/docs-cli-dco-fix.