Skip to content

[BUG] System instruction persistence doesn't normalize structured Content::Raw (parity with add_message) #791

Description

@skovy

Basic checks

  • I searched existing issues - this hasn't been reported
  • I can reproduce this consistently
  • This is a RubyLLM bug, not my application code

What's broken?

When persisting system instructions via acts_as_chat, structured content (RubyLLM::Content::Raw) is not normalized the way message content is. It gets written straight into the content text column, where ActiveRecord coerces it with Object#to_s — producing "#<RubyLLM::Content::Raw:0x...>".

The message persistence path already handles this correctly. add_message and persist_message_completion route content through prepare_content_for_storage, which knows about Content::Raw (stores the structure in content_raw and leaves content nil):

# active_record/chat_methods.rb#prepare_content_for_storage
when RubyLLM::Content::Raw
  content_raw = content.value
  content_text = nil

But the instruction persistence path does not — it writes content: instructions directly:

# active_record/chat_methods.rb
def persist_system_instruction(instructions, append:)
  # ...
  messages_association.create!(role: :system, content: instructions)
end

def replace_persisted_system_instructions(instructions)
  # ...
  messages_association.create!(role: :system, content: instructions)   # create branch
  primary_message.update!(content: instructions) if primary_message.content != instructions
end

RubyLLM::Content::Raw (in content.rb) defines value, format, and to_h, but no to_s, so the text column ends up with the object's inspect string.

Why this matters: structured Content::Raw instructions are how you put a cache_control breakpoint inside the system prompt — e.g. a static, cacheable prefix block followed by a dynamic tail block, for Anthropic prompt caching. The request itself works (the in-memory render carries the blocks to the wire), but the persisted system message is corrupted, so anything that reads it back (inspection/admin/debug tooling) gets an unusable inspect string instead of the prompt.

How to reproduce

# Any model configured with `acts_as_chat`
chat = Chat.create!

raw = RubyLLM::Content::Raw.new([
  { type: "text", text: "static prefix", cache_control: { type: "ephemeral" } },
  { type: "text", text: "dynamic tail" }
])

chat.with_instructions(raw)

chat.messages.find_by(role: "system").content
# => "#<RubyLLM::Content::Raw:0x0000000123456789>"

(with_instructions on the persisted chat → persist_system_instructioncreate!(role: :system, content: raw).)

Expected behavior

System-instruction persistence should normalize Content / Content::Raw the same way add_message does — routing through prepare_content_for_storage so the structure lands in content_raw (and the text column is left nil / holds readable text), instead of stringifying the object. The persisted system message should round-trip to something usable, consistent with how message content is stored.

What actually happened

The content text column holds "#<RubyLLM::Content::Raw:0x...>". The structured blocks (and their cache_control) are dropped from persistence — only the runtime render carries them to the API. content_raw is never populated for the system message.

Environment

  • Ruby version: 3.4.7
  • RubyLLM version: 1.15.0
  • Provider (OpenAI, Anthropic, etc.): Anthropic
  • OS: macOS (arm64-darwin24)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions