Skip to content

Commit 2843e81

Browse files
newstlerclaude
newstler
and
claude
authored
fix(message): render messages/_message partial in broadcasts (#143)
RubyLLM 1.14.1 added a `to_partial_path` override returning "messages/<role>" (user/assistant/system/tool). With the bump from 1.11.0 in #102, turbo-rails' `broadcasts_to` stopped finding a partial because the app only has a single role-aware `_message.html.erb`. Every message save crashed Turbo::Streams::ActionBroadcastJob with ActionView::MissingTemplate, silently breaking live chat UI updates for subscribers (real chats, summaries, testimonials, translations). Pin the broadcast partial explicitly. Added a regression test that saves a Message with every role and asserts the enqueued broadcast job renders cleanly. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 6a04042 commit 2843e81

3 files changed

Lines changed: 195 additions & 1 deletion

File tree

‎app/models/message.rb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
class Message < ApplicationRecord
22
acts_as_message tool_calls_foreign_key: :message_id
33
has_many_attached :attachments
4-
broadcasts_to ->(message) { "chat_#{message.chat_id}" }, inserts_by: :append, target: "messages"
4+
broadcasts_to ->(message) { "chat_#{message.chat_id}" }, inserts_by: :append, target: "messages", partial: "messages/message"
55

66
after_update_commit :broadcast_message_replacement, if: :assistant?
77
before_save :calculate_cost, if: :should_calculate_cost?

‎docs/ideas/rag-chatbot.md‎

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
# Idea: RAG Chatbot ("Ask about Ruby")
2+
3+
A small chatbot on the homepage where visitors can learn about Ruby, augmented with our own content (Posts, Testimonials, Projects) via vector search.
4+
5+
## Goals
6+
7+
- Add a simple "Ask about Ruby" experience to the homepage
8+
- Gain experience with vector search using SQLite (no separate DB)
9+
- Reuse existing chat infrastructure (Chat/Message models, views, streaming)
10+
- Keep costs bounded and prevent abuse as a free general-purpose AI
11+
12+
## UX
13+
14+
- **Homepage**: a single input field ("Ask about Ruby...") with a send button — no full chat widget
15+
- **Homepage nav**: new "Chat" link alongside existing category buttons and "Community"
16+
- **`/chat` page**: reuses the existing beautiful chat view (from the remote template), requires login
17+
- **Single chat per user** — no chat list, no "new chat" button, no multi-tenant URLs exposed to users
18+
- **Auth gate**: hitting send on the homepage, or navigating to `/chat`, triggers GitHub OAuth if not signed in, then lands on `/chat` with the question submitted
19+
20+
## Cost control — single rotating chat
21+
22+
A single ever-growing chat would get expensive fast because RubyLLM's `acts_as_chat` sends full conversation history on every request.
23+
24+
**Strategy:** one active chat per user, silently rotated:
25+
26+
- Always show "the user's current chat" at `/chat`
27+
- If the current chat exceeds ~30 messages or is older than 24h, start a new one behind the scenes
28+
- Old chats stay in the DB for cost tracking but the user never sees them
29+
- From the user's perspective it feels like one continuous conversation
30+
31+
## Topic restriction
32+
33+
Restrict conversations to Ruby / our site content via the system prompt. No code-level topic detection — the LLM enforces this. If a question isn't about Ruby, the Ruby ecosystem, or content on our site, politely decline and redirect.
34+
35+
## Architecture: RAG with SQLite vector search
36+
37+
### Stack
38+
39+
| Layer | Tool | Notes |
40+
|-------|------|-------|
41+
| Vector storage | `sqlite-vec` gem | SQLite extension, no separate DB |
42+
| Rails integration | `neighbor` gem | Andrew Kane — `has_neighbors`, `nearest_neighbors` |
43+
| Embeddings | `RubyLLM.embed` | Already have `ruby_llm` |
44+
| Chat + streaming | Existing `Chat`/`Message` + `ChatResponseJob` + Turbo Streams | Full reuse |
45+
46+
Two new gems: `sqlite-vec`, `neighbor`. Everything else already in the project.
47+
48+
### Flow
49+
50+
```
51+
User question
52+
│
53+
▼
54+
① RubyLLM.embed(question) → query vector (1536 floats)
55+
│
56+
▼
57+
② ContentChunk.nearest_neighbors(:embedding, vector, distance: "cosine").first(8)
58+
│
59+
▼
60+
③ Build system prompt with retrieved chunks + Ruby-only instructions
61+
│
62+
▼
63+
④ chat.ask(question) → streamed response via existing Turbo Streams
64+
```
65+
66+
### Data model
67+
68+
**Two new tables:**
69+
70+
```ruby
71+
# vec0 virtual table — stores only vectors + IDs
72+
create_virtual_table :content_embeddings, :vec0, [
73+
"id text PRIMARY KEY NOT NULL",
74+
"embedding float[1536] distance_metric=cosine"
75+
]
76+
77+
# Metadata table — stores text chunks + polymorphic source reference
78+
create_table :content_chunks, id: { type: :string, default: -> { "uuid7()" } } do |t|
79+
t.string :source_type, null: false # "Post", "Testimonial", "Project"
80+
t.string :source_id, null: false
81+
t.text :content, null: false
82+
t.string :title
83+
t.timestamps
84+
end
85+
```
86+
87+
### Content indexed
88+
89+
| Model | Fields embedded |
90+
|-------|----------------|
91+
| **Post** (published) | `title + summary + content_or_url` |
92+
| **Testimonial** (published) | `heading + quote + body_text` |
93+
| **Project** (visible) | `name + description + topics` |
94+
95+
### New concerns
96+
97+
**`Embeddable`** — shared by Post, Testimonial, Project. Each model defines `embeddable_text`. Callbacks trigger `GenerateEmbeddingJob` on save.
98+
99+
**`Chat::RagAugmentable`** — added to Chat. Provides `ask_with_rag(question, &block)`:
100+
1. Generates query embedding via `RubyLLM.embed`
101+
2. Finds nearest 8 `ContentChunk` records
102+
3. Builds system prompt with retrieved context + Ruby-only instructions
103+
4. Calls `ask(question, with_instructions: system_prompt, &block)`
104+
105+
### Chat purpose
106+
107+
Add `"rag"` to existing purposes (`conversation`, `summary`, `testimonial_generation`, `testimonial_validation`). `ChatResponseJob` detects `purpose == "rag"` and calls `ask_with_rag`.
108+
109+
### Settings (no hardcoded models)
110+
111+
Add `embedding_model` to `Setting::ALLOWED_KEYS`, default `"text-embedding-3-small"` (1536 dims). Configurable via the existing Madmin settings page, same pattern as `summary_model`, `testimonial_model`, etc.
112+
113+
## Routes
114+
115+
```ruby
116+
# Top-level, no team slug exposed
117+
resource :chat, only: [:show, :create], controller: "chats"
118+
```
119+
120+
- `GET /chat` — shows user's current chat (creates one if none), requires auth
121+
- `POST /chat` — sends a message, requires auth
122+
- Homepage input form posts to `/chat`
123+
124+
## Auth flow
125+
126+
The existing `authenticate_user!` in `ApplicationController` already handles the redirect:
127+
128+
```ruby
129+
def authenticate_user!
130+
unless user_signed_in?
131+
session[:return_to] = request.original_url if request.get?
132+
redirect_to github_auth_with_return_path, alert: "Please sign in with GitHub to continue."
133+
end
134+
end
135+
```
136+
137+
For POST from the homepage input, store the pending question in the session before redirecting to GitHub OAuth, then replay it after sign-in.
138+
139+
## Files to create/modify
140+
141+
| Action | File |
142+
|--------|------|
143+
| **Gemfile** | +`sqlite-vec`, +`neighbor` |
144+
| **database.yml** | Load vec0 extension alongside existing uuid extension |
145+
| **Migration** | `create_content_chunks` + `create_content_embeddings` (vec0) |
146+
| **Model** | `app/models/content_chunk.rb` (with `has_neighbors`) |
147+
| **Concern** | `app/models/concerns/embeddable.rb` |
148+
| **Concern** | `app/models/concerns/chat/rag_augmentable.rb` |
149+
| **Model mods** | Include `Embeddable` in Post, Testimonial, Project |
150+
| **Model mod** | Include `Chat::RagAugmentable` in Chat |
151+
| **Setting** | Add `embedding_model` to `ALLOWED_KEYS` |
152+
| **Controller** | New top-level `/chat` controller (or reuse existing with new route) |
153+
| **Job** | `generate_embedding_job.rb` |
154+
| **Job** | `backfill_embeddings_job.rb` (one-time for existing content) |
155+
| **Job mod** | `chat_response_job.rb` — detect `purpose == "rag"`, call `ask_with_rag` |
156+
| **View mod** | `home/index.html.erb` — add input + "Chat" nav link |
157+
| **Routes** | `config/routes.rb` — add top-level `/chat` |
158+
| **i18n** | Locale files for new strings |
159+
160+
## Design decisions
161+
162+
- **No hardcoded AI models** — everything via `Setting.get(...)`, same pattern as existing AI features
163+
- **No new chat views** — reuse the existing beautiful templates from the remote template
164+
- **No team slug in URLs** — multi-tenancy is internal, users see clean `/chat`
165+
- **No chat list, no multi-chat UI** — single rotating chat keeps UX simple and costs bounded
166+
- **No vector DB** — sqlite-vec keeps infrastructure footprint zero
167+
- **Auth required** — connects chats to users for cost tracking via existing `Costable`
168+
169+
## Migration path if corpus grows
170+
171+
If content grows to tens of thousands of documents or semantic quality becomes insufficient:
172+
- Switch embedding model via the setting (e.g., to `text-embedding-3-large`, 3072 dims)
173+
- Add chunking (split long posts into overlapping windows) in `GenerateEmbeddingJob`
174+
- Everything else (controller, concern, UI) stays the same
175+
176+
## Open questions for implementation
177+
178+
- Auto-rotation threshold: 30 messages? 24h? Both?
179+
- Should old rotated chats be visible to admins in Madmin? (Probably yes for debugging)
180+
- Rate limit per user: e.g., 20 messages/hour to prevent abuse even among logged-in users
181+
- Do we index draft/unpublished content? (No — only published)
182+
- Should failed embeddings retry or silently skip? (Retry with backoff via SolidQueue defaults)

‎test/models/message_test.rb‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
require "test_helper"
22

33
class MessageTest < ActiveSupport::TestCase
4+
include ActiveJob::TestHelper
5+
46
test "calculates cost based on token usage" do
57
message = Message.new(
68
chat: chats(:one),
@@ -15,4 +17,14 @@ class MessageTest < ActiveSupport::TestCase
1517

1618
assert message.cost > 0
1719
end
20+
21+
test "broadcasts create using messages/message partial regardless of role" do
22+
%w[user assistant system tool].each do |role|
23+
message = Message.new(chat: chats(:one), role: role, content: "hi")
24+
25+
assert_nothing_raised do
26+
perform_enqueued_jobs { message.save! }
27+
end
28+
end
29+
end
1830
end

0 commit comments

Comments
 (0)