feat(server): add /v1/guardrail/checks endpoint

[m-misiura]

Description

This PR add /v1/guardrail/checks endpoint, wired through check_async()

Briefly,

• options.py: added log field to RailsResult to surface generation log from check_async
• llmrails.py: threaded log=response.log through all three RailsResult return sites in check_async()
• schemas/openai.py: added GuardrailCheckRequest, GuardrailCheckResponse, RailStatusEntry, GuardrailCheckDataOutput models matching upstream OpenAPI spec
• api.py: endpoint + helpers (_inject_model, _filter_log, _map_rail_status, _build_rails_status). Extracted _inject_model() from _get_rails to share with inline config path

This PR deals with the following issue

Test Plan

1. Added tests/server/test_guardrail_checks.py, tests seem to pass:

pytest tests/server/test_guardrail_checks.py
======================== test session starts =========================
platform darwin -- Python 3.12.0, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/mmisiura/repos/forked/NeMo-Guardrails
configfile: pytest.ini (WARNING: ignoring pytest config in pyproject.toml, tox.ini!)
plugins: httpx-0.36.0, recording-0.13.4, langsmith-0.4.30, anyio-4.11.0, asyncio-1.3.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
collected 16 items                                                   

tests/server/test_guardrail_checks.py ................  

2. Tested against live server; examples of requests (also tested with optional "options": {"log": {"activated_rails": true}}

• input to be blocked by self check rail

curl -s -X POST http://localhost:8000/v1/guardrail/checks \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.6-35B-A3B",
    "messages": [{"role": "user", "content": "my email is John.Doe@test.com"}],
    "guardrails": {"config_id": "checks_live_test"}
  }' | jq

{
  "status": "blocked",
  "rails_status": {
    "self check input": {
      "status": "blocked"
    }
  },
  "guardrails_data": {
    "config_ids": [
      "checks_live_test"
    ],
    "log": {
      "activated_rails": [],
      "stats": {
        "input_rails_duration": 3.91440486907959,
        "dialog_rails_duration": null,
        "generation_rails_duration": null,
        "output_rails_duration": null,
        "total_duration": 3.9169669151306152,
        "llm_calls_duration": 3.8865039348602295,
        "llm_calls_count": 1,
        "llm_calls_total_prompt_tokens": 144,
        "llm_calls_total_completion_tokens": 522,
        "llm_calls_total_tokens": 666
      }
    }
  }
}

• input not to be blocked by self check rail

curl -s -X POST http://localhost:8000/v1/guardrail/checks \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.6-35B-A3B",
    "messages": [{"role": "user", "content": "my email is my private matter"}],
    "guardrails": {"config_id": "checks_live_test"}
  }' | jq

{
  "status": "success",
  "rails_status": {
    "self check input": {
      "status": "success"
    },
    "detect sensitive data on input": {
      "status": "success"
    }
  },
  "guardrails_data": {
    "config_ids": [
      "checks_live_test"
    ],
    "log": {
      "activated_rails": [],
      "stats": {
        "input_rails_duration": 4.953992128372192,
        "dialog_rails_duration": null,
        "generation_rails_duration": null,
        "output_rails_duration": null,
        "total_duration": 4.959690093994141,
        "llm_calls_duration": 4.178013324737549,
        "llm_calls_count": 1,
        "llm_calls_total_prompt_tokens": 142,
        "llm_calls_total_completion_tokens": 590,
        "llm_calls_total_tokens": 732
      }
    }
  }
}

Checklist

• I've read the CONTRIBUTING guidelines.
• I've updated the documentation if applicable.
• I've added tests if applicable.
• @mentions of the person or team responsible for reviewing proposed changes.

cc @Pouyanpi @tgasser-nv

Summary by CodeRabbit

• New Features

  • Introduced /v1/guardrail/checks API endpoint for validating messages against guardrails
  • Added per-rail status reporting to track which guardrails are triggered
  • Enhanced guardrail check results with comprehensive logging information across all outcomes

• Improvements

  • Guardrail checks now provide consistent logging data regardless of result status (blocked, modified, or passed)

[codecov]

Codecov Report

❌ Patch coverage is 94.36620% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
nemoguardrails/server/api.py	92.85%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR adds a /v1/guardrail/checks endpoint wired through check_async(), surfaces the generation log via a new log field on RailsResult, and refactors model injection into a shared _inject_model helper.

• New endpoint (api.py): guardrail_check supports inline dict configs, named config_id/config_ids, and the new config string shorthand; helpers _filter_log, _map_rail_status, and _build_rails_status build the response.
• Schema additions (schemas/openai.py): GuardrailCheckRequest, GuardrailCheckResponse, GuardrailCheckDataInput (with config exclusivity validation), GuardrailCheckDataOutput, and RailStatusEntry all match the upstream OpenAPI spec.
• Test coverage (test_guardrail_checks.py): 16 tests covering all config resolution paths, validation, context forwarding, status mapping, and log filtering.

Confidence Score: 5/5

Safe to merge; the new endpoint is well-isolated and the surrounding generation and rail logic is unchanged.

The changes are purely additive: a new HTTP endpoint, new Pydantic schemas, a new field on RailsResult, and propagation of response.log at three existing return sites. No existing behavior is altered.

No files require special attention; the inline-config path in api.py is the most novel logic and is covered by tests.

Important Files Changed

Filename	Overview
nemoguardrails/server/api.py	Adds /v1/guardrail/checks endpoint with helpers _inject_model, _filter_log, _map_rail_status, _build_rails_status; refactors model injection into a shared _inject_model function.
nemoguardrails/server/schemas/openai.py	Adds GuardrailCheckRequest, GuardrailCheckResponse, GuardrailCheckDataInput, GuardrailCheckDataOutput, and RailStatusEntry schemas; config field exclusivity is validated correctly.
nemoguardrails/rails/llm/options.py	Adds optional log: GenerationLog field to RailsResult; minimal, backward-compatible change.
nemoguardrails/rails/llm/llmrails.py	Threads log=response.log into all three RailsResult return sites in check_async; no logic changes.
tests/server/test_guardrail_checks.py	16 tests covering status mapping, config resolution paths, validation errors, context forwarding, and log filtering; good coverage.

Sequence Diagram

sequenceDiagram
    participant C as Client
    participant A as api.py /v1/guardrail/checks
    participant R as _get_rails / LLMRails (inline)
    participant CA as llmrails.check_async
    participant GA as llmrails.generate_async

    C->>A: POST /v1/guardrail/checks (model, messages, guardrails)
    A->>A: validate messages non-empty
    alt inline dict config
        A->>R: RailsConfig.from_content(config) + _inject_model
        R-->>A: LLMRails instance (fresh)
    else config_id / config_ids / default
        A->>R: _get_rails(config_ids, model_name)
        R-->>A: LLMRails instance (cached)
    end
    A->>CA: check_async(messages)
    CA->>CA: _determine_rails_from_messages
    CA->>GA: generate_async(messages, options)
    GA-->>CA: GenerationResponse (with log)
    CA-->>A: RailsResult(status, content, rail, log)
    A->>A: _build_rails_status(result)
    A->>A: _filter_log(result.log.model_dump(), log_options)
    A-->>C: GuardrailCheckResponse(status, rails_status, guardrails_data)

Loading

Prompt To Fix All With AI

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
nemoguardrails/server/api.py:735
`model_dump()` without `mode="json"` may produce non-JSON-serializable objects

`result.log.model_dump()` uses the default `mode="python"`, which keeps Python objects (e.g. Enum instances, datetime values, or arbitrary `return_value: Any` objects inside `ExecutedAction`) in the resulting dict. That dict is then stored in `GuardrailCheckDataOutput.log: Optional[dict]` — a plain `dict` field with no further Pydantic coercion. FastAPI's `jsonable_encoder` will attempt standard JSON serialization on these raw Python objects, which can raise a `TypeError` at response time if any rail action returns a non-primitive value. Using `model_dump(mode="json")` ensures all values are converted to JSON-safe primitives before the dict is handed to the response model.

### Issue 2 of 2
nemoguardrails/server/api.py:669-673
`MODIFIED` is silently collapsed into `"success"`, making it impossible for callers to tell whether their content was sanitized (e.g. PII redacted) or passed unchanged. Both `_map_rail_status` and `_build_rails_status` derive blocked/success solely from `RailStatus.BLOCKED` / `rail.stop`, with no representation for the MODIFIED case. A caller who relies on a `"success"` response to treat the content as unmodified would silently forward sanitized content.

```suggestion
def _map_rail_status(status: RailStatus) -> str:
    """Map internal RailStatus to upstream StatusEnum values."""
    if status == RailStatus.BLOCKED:
        return "blocked"
    if status == RailStatus.MODIFIED:
        return "modified"
    return "success"
```

_{Reviews (3): Last reviewed commit: ":sparkles: implement `/v1/guardrail/chec..." | Re-trigger Greptile}

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR introduces a new /v1/guardrail/checks endpoint that accepts guardrail check requests, runs async rail checks against loaded configurations, and returns per-rail status and filtered logs. The implementation spans data model updates to carry logging information, OpenAI schema definitions for the new endpoint contract, server logic with model injection refactoring and request/response handling, and comprehensive test coverage.

Changes

Guardrail checks endpoint

Layer / File(s)	Summary
Data model: RailsResult log field nemoguardrails/rails/llm/options.py, nemoguardrails/rails/llm/llmrails.py	RailsResult Pydantic model gains an optional log field typed as GenerationLog. check_async() now populates this field on all return paths (BLOCKED, MODIFIED, PASSED) from the response object.
Endpoint schemas: check request and response models nemoguardrails/server/schemas/openai.py	OpenAI schema module adds new request/response types: GuardrailsDataInput extended with config (string or dict) and return_choice fields, and new models RailStatusEntry, GuardrailCheckRequest, GuardrailCheckDataOutput, and GuardrailCheckResponse for mapping rail status and returning check results.
Endpoint implementation: guardrail_check with model injection nemoguardrails/server/api.py	Server imports new schemas, adds _inject_model() helper to inject environment-based model configuration, refactors _get_rails() to use that helper, and implements POST /v1/guardrail/checks endpoint handler with log filtering, rail status mapping, config resolution from inline content or config ids, optional model injection, and error handling (422 for validation/config issues, 500 for unexpected failures).
Endpoint tests: status mapping, config resolution, validation, and log filtering tests/server/test_guardrail_checks.py	Test module provides FastAPI TestClient fixture and autouse reset, then validates status result mapping (PASSED/MODIFIED to success, BLOCKED to blocked), rails_status generation from activated rails, config resolution pathways (config_id list, config string, inline dict, default fallback), validation errors (empty messages, missing config), runtime errors (_get_rails and check_async failures), context message prepending, and log filtering behavior.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

• Pouyanpi

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 44.83% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: a new /v1/guardrail/checks endpoint is added to the API. This is the primary feature delivered across multiple files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes	✅ Passed	PR contains major changes (new endpoint) with documented test results: 16 passing tests and manual testing with example curl requests.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@nemoguardrails/server/api.py`:
- Line 713: The HTTPException re-raises in api.py should preserve exception
chaining; update the three raise sites (the existing raise
HTTPException(status_code=422, detail=f"Invalid inline config: {ex}") and the
similar raises around lines referenced) to use exception chaining by re-raising
with "from ex" (or "from None" where you intentionally want to suppress context)
so the original traceback is preserved; locate the raise calls in the inline
config parsing/validation handlers (the HTTPException raises at the spots shown
in the diff) and change them to raise HTTPException(...) from ex accordingly.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: adb4d588-3aab-48ac-976d-ab512af83b4f

📥 Commits

Reviewing files that changed from the base of the PR and between 7285f2c and 0ab21ac.

📒 Files selected for processing (5)

• nemoguardrails/rails/llm/llmrails.py
• nemoguardrails/rails/llm/options.py
• nemoguardrails/server/api.py
• nemoguardrails/server/schemas/openai.py
• tests/server/test_guardrail_checks.py

[Pouyanpi]

Thanks @m-misiura , some comments and questions for you.

I think we should remove the log. If check_async isn’t exposing the right contract for the server as a consumer, let’s discuss that separately (feel free to open an issue and assign it to me.)

there’s always exactly one blocking rail, which we currently report. the rest are either no-ops, passed, or modified.
we don't report modified because there can be multiple rails. the current check logic is intentionally minimal, but we can extend it to support additional scenarios if needed.

If we align on the desired output shape, I think we can simplify this considerably. Let's discuss that

[Pouyanpi]

@m-misiura also we should return a clear unsupported 4xx for colang 2.0 configs, check_async doesn't support Colang 2.0.

[m-misiura]

Thanks @m-misiura , some comments and questions for you.

I think we should remove the log. If check_async isn’t exposing the right contract for the server as a consumer, let’s discuss that separately (feel free to open an issue and assign it to me.)

there’s always exactly one blocking rail, which we currently report. the rest are either no-ops, passed, or modified. we don't report modified because there can be multiple rails. the current check logic is intentionally minimal, but we can extend it to support additional scenarios if needed.

If we align on the desired output shape, I think we can simplify this considerably. Let's discuss that

As usual, thanks for very constructive comments @Pouyanpi

WDYT about the following action plan.

I will strip this PR down to a thin HTTP surface for check_async(). No SDK changes, no log in the response. The endpoint returns what check_async() already provides:

{
  "status": "passed | modified | blocked",
  "content": "text after rails processing",
  "rail": "blocking rail name or null"
}

• status: maps directly to RailStatus enum values (fixing "success" → "passed", adding "modified")
• content: RailsResult.content (modified text, refusal text, or original)
• rail: RailsResult.rail (which rail blocked, null otherwise)

There should be no changes to llmrails.py or options.py. No rails_status, no guardrails_data, no log. Just the projection.

On the request side: the checks endpoint seem to need config (as a string ID or inline dict), which GuardrailsDataInput doesn't carry today. I'll define a checks-specific request schema (GuardrailCheckRequest) with its own guardrails input model rather than adding config to the shared GuardrailsDataInput used by chat/completions.

Follow-up PRs :

1. Per-rail status + opt-in log: add log to RailsResult, rails_status dict, and opt-in guardrails_data.log. Happy to open an issue so we can discuss the RailsResult contract separately.
2. Tool rail support: add RailType.TOOL_INPUT / TOOL_OUTPUT, extends _determine_rails_from_messages(). Purely additive.

The _inject_model refactor (extracting the model/engine/base_url resolution out of _get_rails() into a reusable helper) is a pure code-motion change with no behavior change. I'll pull it into its own PR so this one stays focused on the endpoint.

[Pouyanpi]

@m-misiura thank you. this sounds great! re follow-up PRs, l suggest we discuss them in our meeting. I really like the opportunistic refactoring that you did, let's keep that 👍🏻

[github-actions]

Open review comments need your response

@m-misiura this PR is waiting on you. Reply to each open review comment so a reviewer can confirm it is resolved. For every comment, leave a reply that either points to the change you made or explains why no change is needed.

Pushing a fix without replying is not enough: reviewers cannot tell which comments a commit addresses, so each thread needs an explicit reply.

Review readiness guide: https://github.com/NVIDIA-NeMo/Guardrails/blob/develop/CONTRIBUTING.md#review-readiness

[m-misiura]

@m-misiura also we should return a clear unsupported 4xx for colang 2.0 configs, check_async doesn't support Colang 2.0.

after resolving llm_rails, we check colang_version != "1.0" and return 422 with "check_async does not support Colang 2.0 configurations."

[Pouyanpi]

LGTM, thank you @m-misiura 🚀