Skip to content

docs(guardrail-catalog): document running multiple self-check rails#2176

Merged
Pouyanpi merged 2 commits into
developfrom
pouyanpi/docs-self-check-multiple-rails
Jul 17, 2026
Merged

docs(guardrail-catalog): document running multiple self-check rails#2176
Pouyanpi merged 2 commits into
developfrom
pouyanpi/docs-self-check-multiple-rails

Conversation

@Pouyanpi

@Pouyanpi Pouyanpi commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Description

Add a "Running Multiple Self-Check Rails" section to the self-check guardrail reference. It covers the $variant= flow parameter: one flow per variant, a matching prompt task per variant, optional per-variant model routing via a model type that matches the variant name, and the fallback to the base self-check model type and then the main model. Notes that a missing variant prompt raises at config load and that a rail without $variant= uses the default prompt.

Related Issue(s)

#1874 #2175

Verification

fern passed locally

AI Assistance

  • No AI tools were used.
  • AI tools were used; a human reviewed and can explain every change (tool: ___).

Checklist

  • I've read the CONTRIBUTING guidelines.
  • This PR links to a triaged issue assigned to me.
  • My PR title follows the project commit convention.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • I've noted any verification beyond CI and any checks I couldn't run.
  • I did not update generated changelog files manually.
  • I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
  • @mentions of the person or team responsible for reviewing proposed changes.

Summary by CodeRabbit

  • Documentation
    • Added guidance for running multiple self-check input and output rails in a single configuration.
    • Documented variant-specific prompts, matching task names, and dedicated model routing.
    • Clarified configuration requirements, execution order, blocking behavior, and combining default and variant rails.

Add a "Running Multiple Self-Check Rails" section to the self-check
guardrail reference. It covers the $variant= flow parameter: one flow per
variant, a matching prompt task per variant, optional per-variant model
routing via a model type that matches the variant name, and the fallback
to the base self-check model type and then the main model. Notes that a
missing variant prompt raises at config load and that a rail without
$variant= uses the default prompt.
@Pouyanpi Pouyanpi changed the title docs(self-check): document running multiple self-check rails docs(guardrail-catalog): document running multiple self-check rails Jul 16, 2026
@Pouyanpi
Pouyanpi requested a review from miyoungc July 16, 2026 15:54
@github-actions github-actions Bot added size: S status: needs triage New issues that have not yet been reviewed or categorized. labels Jul 16, 2026
@Pouyanpi Pouyanpi added status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile). and removed status: needs triage New issues that have not yet been reviewed or categorized. labels Jul 16, 2026
@Pouyanpi Pouyanpi self-assigned this Jul 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

@greptile-apps

greptile-apps Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a "Running Multiple Self-Check Rails" section to the self-check guardrail reference documentation, explaining how to run several independent self-check rails of the same kind (input or output) using the $variant=<name> flow parameter.

  • Covers the three configuration pieces needed per variant: a flow entry in rails.{input,output}.flows, a matching prompts.yml task (using the self_check_{input,output} $variant=<name> format), and an optional model entry whose type equals the variant name.
  • Accurately reflects the implementation in utils.py (get_self_check_llm fallback chain: variant model → base self-check model type → main model) and the config-load validation (_validate_self_check_rail_prompts) that raises on a missing prompt.

Confidence Score: 5/5

Documentation-only change; all YAML examples and behavioral claims were verified against the implementation and are accurate.

The new section correctly describes the variant flow parameter, the prompt task naming convention, the three-tier model fallback chain, and the config-load validation — each cross-checked against the corresponding source files. No code paths are modified.

No files require special attention.

Important Files Changed

Filename Overview
docs/configure-rails/guardrail-catalog/self-check.mdx Adds a 65-line 'Running Multiple Self-Check Rails' section; prose and YAML examples are factually consistent with the implementation in utils.py and config.py.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Output Rail triggered] --> B{Iterate flows in order}
    B --> C[Next variant flow]
    C --> D{Model type=variant defined?}
    D -- Yes --> E[Use variant model]
    D -- No --> F{Base self_check_output model defined?}
    F -- Yes --> G[Use base self-check model]
    F -- No --> H[Use main model]
    E --> I[Render prompt task]
    G --> I
    H --> I
    I --> J{LLM returns Yes blocked?}
    J -- Yes --> K[bot refuse to respond stop processing]
    J -- No --> L{More variant flows?}
    L -- Yes --> C
    L -- No --> M[Allow output]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Output Rail triggered] --> B{Iterate flows in order}
    B --> C[Next variant flow]
    C --> D{Model type=variant defined?}
    D -- Yes --> E[Use variant model]
    D -- No --> F{Base self_check_output model defined?}
    F -- Yes --> G[Use base self-check model]
    F -- No --> H[Use main model]
    E --> I[Render prompt task]
    G --> I
    H --> I
    I --> J{LLM returns Yes blocked?}
    J -- Yes --> K[bot refuse to respond stop processing]
    J -- No --> L{More variant flows?}
    L -- Yes --> C
    L -- No --> M[Allow output]
Loading

Reviews (2): Last reviewed commit: "Apply suggestion from @miyoungc" | Re-trigger Greptile

Comment thread docs/configure-rails/guardrail-catalog/self-check.mdx
Comment thread docs/configure-rails/guardrail-catalog/self-check.mdx Outdated
Signed-off-by: Miyoung Choi <miyoungc@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Adds documentation for configuring and running multiple self-check rails with named variants, matching prompts, optional dedicated models, execution ordering, blocking behavior, and default rails.

Changes

Self-check rail configuration

Layer / File(s) Summary
Variant self-check rail guidance
docs/configure-rails/guardrail-catalog/self-check.mdx
Documents $variant=<name> flows, required per-variant prompts, optional model routing through models[].type, execution order, stop-on-first-block behavior, and combining default and variant rails.

Estimated code review effort: 1 (Trivial) | ~2 minutes

🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: documenting how to run multiple self-check rails.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes ✅ Passed This is a small documentation-only change; the PR description also includes verification notes (“fern passed locally”).
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch pouyanpi/docs-self-check-multiple-rails

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/configure-rails/guardrail-catalog/self-check.mdx`:
- Around line 298-313: Expand the model-routing example around the model
definitions and fallback description to state the routing mode, supported model
modes, and applicable limitations or access requirements. Include the required
model type, routing mode, supported modes, and known limitations while
preserving the existing variant-specific and fallback routing behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 083523e8-4065-4373-b7b0-cf01e17d8538

📥 Commits

Reviewing files that changed from the base of the PR and between 450594a and 22f1248.

📒 Files selected for processing (1)
  • docs/configure-rails/guardrail-catalog/self-check.mdx

Comment thread docs/configure-rails/guardrail-catalog/self-check.mdx
@Pouyanpi
Pouyanpi merged commit 1e86f75 into develop Jul 17, 2026
19 checks passed
@Pouyanpi
Pouyanpi deleted the pouyanpi/docs-self-check-multiple-rails branch July 17, 2026 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: S status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants