Skip to content

bug: IORails ModelEngine double /v1 in URL when base_url includes /v1 #1861

@m-misiura

Description

@m-misiura

Did you check docs and existing issues?

  • I have read all the NeMo-Guardrails docs
  • I have updated the package to the latest version before submitting this issue
  • (optional) I have used the develop branch
  • I have searched the existing issues of NeMo-Guardrails

Python version (python --version)

3.12

Operating system/version

26.4

NeMo-Guardrails version (if you must use a specific version and not the latest

No response

Describe the bug

ModelEngine._resolve_base_url() returns the user-provided base_url as-is, and _prepare_request() appends /v1/chat/completions to it. When the user sets base_url to an OpenAI-compatible endpoint that already includes /v1 (the standard convention for vLLM, LiteLLM, etc.), the constructed URL becomes /v1/v1/chat/completions, resulting in a 404.

This only affects IORails — LLMRails uses the LangChain OpenAI client which appends only /chat/completions to the base URL, so the same config works fine under LLMRails.

_CHAT_COMPLETIONS_ENDPOINT = "/v1/chat/completions", while the default _ENGINE_BASE_URLS intentionally omit /v1. The mismatch breaks user-provided URLs that follow the LLMRails / OpenAI SDK convention of including /v1.

Steps To Reproduce

  1. Create a config with base_url ending in /v1 and flows that trigger IORails
  2. Use the Guardrails Python API:
import asyncio
from nemoguardrails import RailsConfig
from nemoguardrails.guardrails.guardrails import Guardrails

async def main():
config = RailsConfig.from_path("path/to/config")
g = Guardrails(config=config)
print(f"Engine: {type(g.rails_engine).__name__}") # IORails
await g.startup()
result = await g.generate_async(
messages=[{"role": "user", "content": "Hello"}]
)
print(result)

asyncio.run(main())
  1. Observe the log output showing the doubled path:
HTTP POST https://my-openai-compatible-server.com/v1/v1/chat/completions model='my-model'
HTTP 404 from model 'my-model': {"detail":"Not Found"}

Expected Behavior

The URL should be https://my-openai-compatible-server.com/v1/chat/completions — a single /v1 prefix — regardless of whether the user includes /v1 in base_url or not.

Actual Behavior

IORails constructs URL with double /v1: https://my-openai-compatible-server.com/v1/v1/chat/completions, which returns 404 from the upstream server.

The same config works correctly under LLMRails because the LangChain OpenAI client only appends /chat/completions.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingstatus: needs triageNew issues that have not yet been reviewed or categorized.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions