Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -54,5 +54,9 @@ RUN python -c "from fastembed.embedding import FlagEmbedding; FlagEmbedding('sen

RUN nemoguardrails --help

ENV NEMO_GUARDRAILS_HEALTHCHECK_URL=http://localhost:8000/v1/health
HEALTHCHECK --interval=30s --timeout=5s --start-period=60s --retries=3 \
CMD python -c "import os, urllib.request, sys; opener = urllib.request.build_opener(urllib.request.ProxyHandler({})); sys.exit(0 if opener.open(os.environ['NEMO_GUARDRAILS_HEALTHCHECK_URL']).status == 200 else 1)"

ENTRYPOINT ["nemoguardrails"]
CMD ["server", "--verbose", "--config=/config"]
24 changes: 24 additions & 0 deletions docs/deployment/using-docker.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -106,11 +106,35 @@ To use the Chat CLI interface, run the Docker container in interactive mode:

```bash
docker run -it \
--no-healthcheck \
-e OPENAI_API_KEY=$OPENAI_API_KEY \
-v </path/to/local/config/>:/config \
nemoguardrails chat --config=/config --verbose
```

The image's built-in Docker `HEALTHCHECK` probes the server's liveness endpoint, so it is only meaningful when the container runs the default `server` command.
Other commands such as `chat`, `actions-server`, `convert`, or `eval` do not serve a health endpoint.
This means Docker would report the container as `unhealthy`.
Add `--no-healthcheck` to `docker run` whenever the container does not run the server, as shown in the `chat` example above.

## Container Health Check

The image defines a Docker `HEALTHCHECK` that probes the server's liveness endpoint, so `docker ps` and orchestrators can report container health.
By default it requests `http://localhost:8000/v1/health`, which matches the default `CMD` (the `server` command on port `8000` with no path prefix).

If you run the container on a different port or behind a path prefix, override the probe URL with the `NEMO_GUARDRAILS_HEALTHCHECK_URL` environment variable so the healthcheck targets the address the server actually serves:

```bash
docker run \
-p 9000:9000 \
-e OPENAI_API_KEY=$OPENAI_API_KEY \
-e NEMO_GUARDRAILS_HEALTHCHECK_URL=http://localhost:9000/v1/health \
nemoguardrails server --config=/config --port 9000
```

The health check applies only to the `server` application.
For other commands, disable it with `--no-healthcheck`.

## AlignScore Fact-Checking

If one of your configurations uses the AlignScore fact-checking model, run the AlignScore server in a separate container:
Expand Down
6 changes: 6 additions & 0 deletions docs/index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -602,6 +602,12 @@ navigation:
title: List Challenges
slug: list-challenges
- health:
- endpoint: GET /v1/health
title: Health Check
slug: get-health
- endpoint: GET /healthz
title: Health Check (Alias)
slug: get-healthz
- endpoint: GET /
title: Get Server Health or Chat UI
slug: get-root
Expand Down
17 changes: 17 additions & 0 deletions docs/run-rails/using-fastapi-server/run-guardrails-server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,23 @@ For production deployments, disable it using the `--disable-chat-ui` flag.

</Info>

## Health Check

The server exposes a liveness endpoint at `GET /v1/health` (also available as `GET /healthz`) for orchestrators, load balancers, and container runtimes.
It returns HTTP `200` with the `application/health+json` media type and a body of `{"status": "pass"}` whenever the server process is running and able to serve requests.

```bash
curl http://localhost:8000/v1/health
```

```json
{"status": "pass"}
```

The check is intentionally shallow: it reflects only that the server process is up, and does not verify guardrails configurations, the model provider, or any datastore.
Use this endpoint rather than `GET /` for health probes, because `GET /` redirects to the chat UI when the UI is enabled.
When the server runs behind a path prefix (`--prefix`), the endpoints are served at `<prefix>/v1/health` and `<prefix>/healthz`.

Comment thread
coderabbitai[bot] marked this conversation as resolved.
## Related Topics

- [Chat with a Guardrailed Model](/run-guardrailed-inference/using-fastapi-server/chat-with-guardrailed-model)
Expand Down
45 changes: 45 additions & 0 deletions fern/openapi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,51 @@ paths:
- id: jailbreak-1
description: Attempt to bypass safety guardrails.
category: jailbreak
/v1/health:
get:
operationId: getHealth
tags:
- Health
summary: Liveness health check
description: |
Shallow liveness check. Returns HTTP 200 while the server process is
running and able to serve requests. It does not verify guardrails
configurations, the model provider, or any datastore.
responses:
"200":
description: The server process is up and able to serve requests.
content:
application/health+json:
schema:
type: object
properties:
status:
type: string
example: pass
example:
status: pass
/healthz:
get:
operationId: getHealthz
tags:
- Health
summary: Liveness health check (alias)
description: |
Kubernetes-style alias for `GET /v1/health`. Returns the same shallow
liveness response.
responses:
"200":
description: The server process is up and able to serve requests.
content:
application/health+json:
schema:
type: object
properties:
status:
type: string
example: pass
example:
status: pass
/:
get:
operationId: getRoot
Expand Down
17 changes: 16 additions & 1 deletion nemoguardrails/server/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
from fastapi import FastAPI, HTTPException, Request
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel, ValidationError
from starlette.responses import RedirectResponse, StreamingResponse
from starlette.responses import JSONResponse, RedirectResponse, StreamingResponse

from nemoguardrails import LLMRails, RailsConfig, utils
from nemoguardrails.rails.llm.config import Model
Expand Down Expand Up @@ -253,6 +253,21 @@ async def get_rails_configs():
return [{"id": config_id} for config_id in config_ids]


@app.get(
"/v1/health",
summary="Liveness health check.",
tags=["Health"],
)
@app.get(
"/healthz",
summary="Liveness health check.",
tags=["Health"],
)
async def health():
"""Return HTTP 200 while the server process is running and able to serve requests."""
return JSONResponse(content={"status": "pass"}, media_type="application/health+json")


@app.get(
"/v1/models",
response_model=OpenAIModelsList,
Expand Down
67 changes: 67 additions & 0 deletions tests/server/test_health.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import pytest

pytest.importorskip("openai", reason="openai is required for server tests")
from fastapi.testclient import TestClient

from nemoguardrails.server import api

client = TestClient(api.app)

ENDPOINT = "/v1/health"


def test_health_returns_200_pass_status():
"""GET /v1/health returns HTTP 200 with a JSON body of {"status": "pass"}."""
response = client.get(ENDPOINT)

assert response.status_code == 200
assert response.json() == {"status": "pass"}


def test_health_uses_health_json_media_type():
"""GET /v1/health responds with exactly the application/health+json media type."""
response = client.get(ENDPOINT)

assert response.headers["content-type"] == "application/health+json"


def test_health_rejects_post_with_405():
"""POST /v1/health returns HTTP 405 because the endpoint is GET-only."""
response = client.post(ENDPOINT)

assert response.status_code == 405


def test_health_ok_without_config_or_cached_rails(monkeypatch):
"""GET /v1/health returns 200 with no default config and no cached rails instances."""
monkeypatch.setattr(api.app, "default_config_id", None)
monkeypatch.setattr(api, "llm_rails_instances", {})

response = client.get(ENDPOINT)

assert response.status_code == 200
assert response.json() == {"status": "pass"}


def test_healthz_alias_matches_health_contract():
"""GET /healthz returns the same 200 application/health+json {"status": "pass"} response as /v1/health."""
response = client.get("/healthz")

assert response.status_code == 200
assert response.headers["content-type"] == "application/health+json"
assert response.json() == {"status": "pass"}