Skip to content

fix: set TIS type for Automodel GRPO release recipes#2544

Closed
zpqiu wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
zpqiu:zpqiu/fix-qwen35-dapo-tis-config
Closed

fix: set TIS type for Automodel GRPO release recipes#2544
zpqiu wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
zpqiu:zpqiu/fix-qwen35-dapo-tis-config

Conversation

@zpqiu

@zpqiu zpqiu commented May 22, 2026

Copy link
Copy Markdown
Contributor

What does this PR do ?

Fixes Automodel GRPO release recipes that can crash when truncated_importance_sampling_ratio is configured without an explicit truncated_importance_sampling_type.

The Qwen3.5 DAPO and GLM-4.7 Flash Automodel recipes already set truncated_importance_sampling_ratio: 2, but did not set the corresponding sampling type. This PR sets loss_fn.truncated_importance_sampling_type: tis in both recipes so the ratio is interpreted through the intended TIS path.

Additional release-test stabilization:

  • Keeps the GLM release test median train/token_mult_prob_error check.
  • Removes only the GLM step-30 point check for train/token_mult_prob_error, which has high tail variance.
  • Keeps train/gen_kl_error as the policy/generation mismatch check.

Issues

N/A

Usage

No user-facing usage change. This updates release recipe/test configuration only.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

Validation performed:

  • bash -n tests/test_suites/llm/grpo-glm47-flash-4n8g-automodel.sh
  • Qwen3.5 DAPO Automodel release rerun passed after setting truncated_importance_sampling_type: tis.
  • GLM-4.7 Flash Automodel rerun completed 30 steps. The retained checks passed:
    • median(train/token_mult_prob_error) = 1.059859812259674 < 1.1
    • mean(train/gen_kl_error) = 0.007180878799408675 < 0.01
    • train/reward[30] = 0.666015625 > 0.3
    • max(validation/accuracy) = 0.5291666388511658 > 0.2
    • mean(train/grad_norm) = 0.15234075884024303 < 0.6
  • The removed GLM single-step check was train/token_mult_prob_error[30] < 1.1; the rerun observed 17.038604736328125 at step 30 while the median and KL checks remained healthy.

@copy-pr-bot

copy-pr-bot Bot commented May 22, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

zpqiu added 2 commits May 22, 2026 21:56
Signed-off-by: Zhaopeng Qiu <alexq@nvidia.com>
Signed-off-by: Zhaopeng Qiu <alexq@nvidia.com>
@zpqiu zpqiu force-pushed the zpqiu/fix-qwen35-dapo-tis-config branch from ac996c8 to 4e0738b Compare May 23, 2026 04:57
@zpqiu zpqiu closed this May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant