Skip to content

docs: expand golden values update strategy in release process guide#3268

Merged
ko3n1g merged 3 commits intomainfrom
ko3n1g/docs/golden-values-update-strategy
Apr 22, 2026
Merged

docs: expand golden values update strategy in release process guide#3268
ko3n1g merged 3 commits intomainfrom
ko3n1g/docs/golden-values-update-strategy

Conversation

@ko3n1g
Copy link
Copy Markdown
Contributor

@ko3n1g ko3n1g commented Apr 10, 2026

Summary

Expands the Release Developer Guide with two additions:

  • Golden values update strategy: clarifies when and how to update golden values for PRs targeting main vs. the release branch, including the Week 5 bulk-update convention and individual engineer responsibility in the second half of code-freeze.
  • Patch release timeline: documents the post-release cadence — branches reopen on release day, lock on the first Monday of the following month (when XX.YY.01.RC0 ships internally), followed by ~2 weeks of stabilization (bug fixes + small improvements), then QA exit and patch release.

Example

## Patch Release

After the main release ships (Week 7, typically mid-month), the release branches are **reopened** for contributions targeting the patch release.

| Period | Approximate Timing | Key Activity |
|--------|--------------------|--------------|
| Reopening | Release day (Week 7) | Branches accept contributions; patch development begins |
| Lockdown | First Monday of the following month (~2 weeks later) | Release branches locked; patch RC0 (`XX.YY.01.RC0`) shipped internally |
| Stabilization | Week 1–2 after RC0 | Bug fixes and small improvements only |
| | End of Week 2 | QA exit, patch release |

ko3n1g and others added 2 commits April 10, 2026 10:15
Add explicit step-by-step instructions for when and how to update golden
values: paired internal PRs for metric-affecting changes, exact CI inputs
(container, MCore commit, MBridge commit) for both main and release
branches, and which internal CI repo branch to target in each case.

Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: oliver könig <okoenig@nvidia.com>
Specify that the last bulk golden values update happens in Week 5 of
code-freeze; after that, engineers are individually responsible for
keeping golden values current on the release branch.

Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: oliver könig <okoenig@nvidia.com>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 10, 2026

📝 Walkthrough

Walkthrough

Documentation update to the release process guide that expands and clarifies guidance on golden values management, including new rules for PR updates, step-by-step procedures for updating golden values for main and release branches, and revised release-branch code-freeze timelines.

Changes

Cohort / File(s) Summary
Release Process Documentation
docs/releases/release-process.md
Expanded "Golden Values" section with definitions and new guidance; added explicit procedures for updating golden values in PRs targeting main and during releases; clarified code-freeze timeline sections with specific week references (Weeks 3–5 and 6–7) and refined language on golden-value refresh responsibilities.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely summarizes the main change: expanding documentation on golden values update strategy in the release process guide.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Test Results For Major Changes ✅ Passed PR contains documentation-only changes to release procedures without code modifications, qualifying as minor changes that do not require numerical test results.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ko3n1g/docs/golden-values-update-strategy

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
docs/releases/release-process.md (2)

33-36: Add explicit requirement to record perf deltas + run context.

Line 35 mandates paired golden-values PRs, which is good. Please also require before/after performance numbers and the exact run configuration/context in the PR description so reviewers can validate why golden values changed.

Based on learnings, if a change could affect performance, the PR description should include before-and-after performance numbers, as well as the configuration and context in which they apply.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/releases/release-process.md` around lines 33 - 36, Update the "When to
update golden values" section to require that any PR triggering a golden-values
update must include before-and-after performance numbers and the exact run
configuration/context in the PR description; specifically, augment the guideline
under the "When to update golden values" header to explicitly state that PRs
must record performance deltas (numeric before/after metrics), the benchmark
commands, dataset/seed/environment details, and any relevant hardware/software
configuration so reviewers can validate the change.

37-46: Minor wording cleanup on Line 45.

“merged together” is a bit wordy; “merged in sync” (or similar) reads cleaner.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/releases/release-process.md` around lines 37 - 46, Update the wording in
the "Updating golden values for PRs targeting `main`" step 4: replace the phrase
"The MBridge PR and the internal golden-values PR should be merged together (or
the golden-values PR first)." with a cleaner alternative such as "The MBridge PR
and the internal golden-values PR should be merged in sync (or the golden-values
PR first)." Modify the text under that header/step to use "merged in sync" (or a
similar concise phrase) instead of "merged together" to improve clarity.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/releases/release-process.md`:
- Around line 47-57: The phrase "MBridge PR commit" defined as "head of the
MBridge release branch" is ambiguous; update the text so it explicitly instructs
to use the head commit of the PR branch (or the exact PR SHA) to avoid picking
the wrong commit when multiple PRs target the same release branch—replace the
current wording "MBridge PR commit (head of the MBridge release branch)" with
something like "MBridge PR commit — the head commit of the PR branch (use the
exact PR branch tip SHA or PR merge SHA)".

---

Nitpick comments:
In `@docs/releases/release-process.md`:
- Around line 33-36: Update the "When to update golden values" section to
require that any PR triggering a golden-values update must include
before-and-after performance numbers and the exact run configuration/context in
the PR description; specifically, augment the guideline under the "When to
update golden values" header to explicitly state that PRs must record
performance deltas (numeric before/after metrics), the benchmark commands,
dataset/seed/environment details, and any relevant hardware/software
configuration so reviewers can validate the change.
- Around line 37-46: Update the wording in the "Updating golden values for PRs
targeting `main`" step 4: replace the phrase "The MBridge PR and the internal
golden-values PR should be merged together (or the golden-values PR first)."
with a cleaner alternative such as "The MBridge PR and the internal
golden-values PR should be merged in sync (or the golden-values PR first)."
Modify the text under that header/step to use "merged in sync" (or a similar
concise phrase) instead of "merged together" to improve clarity.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d0ed51b4-22f2-4771-8999-699c40b889af

📥 Commits

Reviewing files that changed from the base of the PR and between fad15ab and 5ffc8c1.

📒 Files selected for processing (1)
  • docs/releases/release-process.md

Comment on lines +47 to 57
### Updating golden values during a release

When golden values need to be refreshed on the release branch (e.g. at the start of code-freeze or after an accepted regression):

1. **Rebase the MBridge PR against the MBridge release branch** so it is at the head of that branch.
2. **Launch an internal CI run** using:
- The **latest internal RC container** for the release.
- The **MCore commit pinned on the release branch**.
- The **MBridge PR commit** (head of the MBridge release branch).
3. Open a PR against the **internal CI repository's release branch** with the updated golden values.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Clarify the commit source on Line 55 to avoid wrong SHA selection.

Line 55 says “MBridge PR commit” and then defines it as “head of the MBridge release branch.” That can conflict when multiple PRs target the same release branch. Recommend explicitly saying “the head commit of the PR branch (or exact PR SHA).”

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/releases/release-process.md` around lines 47 - 57, The phrase "MBridge
PR commit" defined as "head of the MBridge release branch" is ambiguous; update
the text so it explicitly instructs to use the head commit of the PR branch (or
the exact PR SHA) to avoid picking the wrong commit when multiple PRs target the
same release branch—replace the current wording "MBridge PR commit (head of the
MBridge release branch)" with something like "MBridge PR commit — the head
commit of the PR branch (use the exact PR branch tip SHA or PR merge SHA)".

### When to update golden values

Any PR that can affect performance metrics (e.g. changes to model code, training loop, optimizer, or numerical kernels) **must be accompanied by a corresponding internal PR that updates the golden values** before merging. Do not wait until after the PR lands.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a statement for exceptions- "Exceptions can be made on rare occasion of issues with GPU availability- cluster is offline, compute availability is low, etc."?

@yaoyu-33 yaoyu-33 added docs-only With great power comes great responsibility. needs-review PR is ready for code review and waiting on a reviewer and removed needs-review PR is ready for code review and waiting on a reviewer labels Apr 13, 2026
yaoyu-33
yaoyu-33 previously approved these changes Apr 17, 2026
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g ko3n1g merged commit c6ae0ba into main Apr 22, 2026
32 checks passed
@ko3n1g ko3n1g deleted the ko3n1g/docs/golden-values-update-strategy branch April 22, 2026 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-only With great power comes great responsibility.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants