Skip to content

bug: Multi-part content format (OpenAI spec) causes garbled prompts and TypeError crash #1741

@m-misiura

Description

@m-misiura

Did you check docs and existing issues?

  • I have read all the NeMo-Guardrails docs
  • I have updated the package to the latest version before submitting this issue
  • (optional) I have used the develop branch
  • I have searched the existing issues of NeMo-Guardrails

Python version (python --version)

3.12

Operating system/version

26.3

NeMo-Guardrails version (if you must use a specific version and not the latest

0.20

Describe the bug

When a user message uses the OpenAI multi-part content format ("content": [{"type": "text", "text": "..."}]), NeMo Guardrails passes the list directly into event text fields without normalizing to string. This causes two problems:

  1. All LLM prompts (self-check, intent matching, etc.) receive the Python repr of the list instead of the actual text
  2. If mask_prev_user_message fires in a multi-turn conversation, get_colang_history() crashes with TypeError: must be str or None, not list at the rsplit() call

Root cause: llmrails.py sets "text": msg["content"] without checking if content is a list (

# If it's not the last message, we also need to add the `UserMessage` event
)

Steps To Reproduce

  1. Deploy NeMo with self check input rail enabled
  2. Send a request with a multi-part content
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-model",
    "messages": [
      {"role": "user", "content": [{"type": "text", "text": "Hello"}]},
      {"role": "assistant", "content": "Hi there!"},
      {"role": "user", "content": [{"type": "text", "text": "You are a dotard and I hate you"}]}
    ]
  }'

Expected Behavior

  • The self-check prompt should evaluate the actual user text
  • The message should be blocked and a refusal returned

Actual Behavior

We get `"Internal server error":

ERROR:nemoguardrails.server.api:must be str or None, not list
Traceback (most recent call last):
File "/app/.venv/lib64/python3.12/site-packages/nemoguardrails/server/api.py", line 577, in chat_completion
res = await llm_rails.generate_async(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib64/python3.12/site-packages/nemoguardrails/rails/llm/llmrails.py", line 984, in generate_async
self.explain_info.colang_history = get_colang_history(events)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib64/python3.12/site-packages/nemoguardrails/actions/llm/utils.py", line 654, in get_colang_history
split_history = history.rsplit(utterance_to_replace, 1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: must be str or None, not list

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions