Did you check docs and existing issues?
Python version (python --version)
3.12
Operating system/version
26.4
NeMo-Guardrails version (if you must use a specific version and not the latest
No response
Describe the bug
ModelEngine._resolve_base_url() returns the user-provided base_url as-is, and _prepare_request() appends /v1/chat/completions to it. When the user sets base_url to an OpenAI-compatible endpoint that already includes /v1 (the standard convention for vLLM, LiteLLM, etc.), the constructed URL becomes /v1/v1/chat/completions, resulting in a 404.
This only affects IORails — LLMRails uses the LangChain OpenAI client which appends only /chat/completions to the base URL, so the same config works fine under LLMRails.
_CHAT_COMPLETIONS_ENDPOINT = "/v1/chat/completions", while the default _ENGINE_BASE_URLS intentionally omit /v1. The mismatch breaks user-provided URLs that follow the LLMRails / OpenAI SDK convention of including /v1.
Steps To Reproduce
- Create a config with
base_url ending in /v1 and flows that trigger IORails
- Use the Guardrails Python API:
import asyncio
from nemoguardrails import RailsConfig
from nemoguardrails.guardrails.guardrails import Guardrails
async def main():
config = RailsConfig.from_path("path/to/config")
g = Guardrails(config=config)
print(f"Engine: {type(g.rails_engine).__name__}") # IORails
await g.startup()
result = await g.generate_async(
messages=[{"role": "user", "content": "Hello"}]
)
print(result)
asyncio.run(main())
- Observe the log output showing the doubled path:
HTTP POST https://my-openai-compatible-server.com/v1/v1/chat/completions model='my-model'
HTTP 404 from model 'my-model': {"detail":"Not Found"}
Expected Behavior
The URL should be https://my-openai-compatible-server.com/v1/chat/completions — a single /v1 prefix — regardless of whether the user includes /v1 in base_url or not.
Actual Behavior
IORails constructs URL with double /v1: https://my-openai-compatible-server.com/v1/v1/chat/completions, which returns 404 from the upstream server.
The same config works correctly under LLMRails because the LangChain OpenAI client only appends /chat/completions.
Did you check docs and existing issues?
Python version (python --version)
3.12
Operating system/version
26.4
NeMo-Guardrails version (if you must use a specific version and not the latest
No response
Describe the bug
ModelEngine._resolve_base_url()returns the user-providedbase_urlas-is, and_prepare_request()appends/v1/chat/completionsto it. When the user setsbase_urlto an OpenAI-compatible endpoint that already includes/v1(the standard convention for vLLM, LiteLLM, etc.), the constructed URL becomes/v1/v1/chat/completions, resulting in a 404.This only affects IORails — LLMRails uses the LangChain OpenAI client which appends only
/chat/completionsto the base URL, so the same config works fine under LLMRails._CHAT_COMPLETIONS_ENDPOINT = "/v1/chat/completions", while the default_ENGINE_BASE_URLSintentionally omit/v1. The mismatch breaks user-provided URLs that follow the LLMRails / OpenAI SDK convention of including/v1.Steps To Reproduce
base_urlending in/v1and flows that trigger IORailsExpected Behavior
The URL should be
https://my-openai-compatible-server.com/v1/chat/completions— a single/v1prefix — regardless of whether the user includes/v1inbase_urlor not.Actual Behavior
IORails constructs URL with double
/v1:https://my-openai-compatible-server.com/v1/v1/chat/completions, which returns 404 from the upstream server.The same config works correctly under LLMRails because the LangChain OpenAI client only appends
/chat/completions.