feat(sc): NeMo-Gym path#3267
Draft
yuki-97 wants to merge 7 commits into
Draft
Conversation
yuki-97
marked this pull request as draft
July 17, 2026 13:07
yuki-97
force-pushed
the
yukih/sc-split-4
branch
from
July 18, 2026 10:02
6afeaae to
62947e5
Compare
yuki-97
force-pushed
the
yukih/sc-split-3
branch
2 times, most recently
from
July 18, 2026 13:31
8025950 to
5aba114
Compare
yuki-97
force-pushed
the
yukih/sc-split-4
branch
from
July 18, 2026 15:25
62947e5 to
7f1d6ce
Compare
Signed-off-by: Yuki Huang <yukih@nvidia.com>
Signed-off-by: Yuki Huang <yukih@nvidia.com>
This reverts commit 0eeab08. Signed-off-by: Yuki Huang <yukih@nvidia.com>
Signed-off-by: Yuki Huang <yukih@nvidia.com>
yuki-97
force-pushed
the
yukih/sc-split-4
branch
from
July 18, 2026 15:29
7f1d6ce to
4278c73
Compare
Signed-off-by: Yuki Huang <yukih@nvidia.com>
Signed-off-by: Yuki Huang <yukih@nvidia.com>
Signed-off-by: Yuki Huang <yukih@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires NeMo-Gym rollouts into the SC (Single Controller) entrypoint by extracting the gym-actor spinup out of
grpo.pyinto a reusable helper.spinup_nemo_gym_actor(nemo_rl/environments/nemo_gym.py): factors the NeMo-Gym actor construction (config assembly, uv/venv cache reuse, image-baked venv reuse, softNodeAffinitySchedulingStrategywhennum_gpu_nodes > 0) into a shared function so bothgrpo.pyand the SC setup can call it.grpo.setup(nemo_rl/algorithms/grpo.py): drop the inlined ~60-line spinup, callspinup_nemo_gym_actorinstead. No behavior change on the sync-GRPO path.examples/nemo_gym/run_{grpo,distillation}_nemo_gym.py: small entrypoint tweaks to match the new helper.tests/functional/grpo_async_gym_single_controller.shcovers the SC + NeMo-Gym combination end-to-end.Part 4/4 of the #2819 split. See that PR for the full code and context.