feat(chat): origin-tag & collapse SAM-injected prompt text (persist + broadcast) (#1569)

* feat(vm-agent): thread origin marker from injected prompt block to message reporter

- startAgentWithPrompt emits SAM-injected instructions as a separate ACP block
  carrying _meta{sam.origin:system}; create-session body accepts injectedInstructions
- ExtractMessages reads the in-memory marker and sets ExtractedMessage.Origin
- Origin flows through MessageReportEntry -> outbox (additive origin column +
  idempotent migration) -> batch payload to the control plane
- Go tests for marker extraction, outbox persist/forward, migration idempotency

* feat(api,web): persist + collapse origin=system injected messages

- producer: buildInjectedInstructions() splits the get_instructions reminder out
  of the visible prompt; startAgentSessionOnNode sends it as injectedInstructions
- API: MessageEntrySchema.origin; toProjectDataMessages forwards it
- DO: additive migration 024 (ALTER TABLE chat_messages ADD COLUMN origin, no DROP);
  persistMessageBatch INSERT + getMessages SELECT + row parsers carry origin
- web: ChatMessageResponse.origin + UserMessage.origin; chatMessagesToConversationItems
  maps it; AcpConversationItemView collapses origin=system behind a chevron disclosure
- tests: producer split, DO origin persist/read (workers), web collapse component;
  updated task-runner prompt tests for the split

Persisted path only; live-broadcast hiding deferred (SDK strips _meta outbound).

* feat: finish injected message origin propagation

* test: audit injected context disclosure

* test: account for origin migration

* test: preserve prompt origin contracts

* fix(origin): gate system-origin marker to trusted prompts; address review findings

Specialist review fixes for the injected-message origin feature:
- security: only honor _meta[sam.origin]=system from the SAM control-plane
  initial prompt; strip it from untrusted browser/gateway + follow-up prompts
  so a viewer cannot hide their own content from search/dedup/topic/attention.
  HandlePrompt gains trustedSource; regression test added.
- ui: fix invisible focus ring (ring-accent-primary -> ring-focus-ring);
  harden Playwright audit (mount wait, boolean open, robust selector).
- api: emit origin:null on the single-message message.new broadcast for a
  stable contract vs messages.batch.
- tests: add dedup-bypass, conversion origin-mapping, DO topic-exclusion, and
  attention-exclusion regressions.
- docs: CLAUDE.md Recent Changes + architecture overview.md; migration comment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove dead buildAgentStartPromptPayload

task-completion-validator flagged the old combined-prompt helper as dead code
(rule 01): no production caller remains after the visible/injected prompt split.
Removed the function and its test; retest the visible-prompt helper asserts the
reminder is NOT concatenated into the visible prompt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: drop unrelated prettier reflow from project-data/index + node-agent

The origin commit reformatted the entire ProjectData DO index (799->1309 lines)
and node-agent.ts as a side effect, which (a) tripped the 800-line file-size hard
gate and (b) buried a 1-line functional change under ~490 lines of churn. The repo
does not enforce prettier in CI (main ships the dense style), so this restores the
status-quo formatting and keeps only the real origin changes:
- index.ts: persistMessageBatch input type gains `origin?`
- node-agent.ts: startAgentSessionOnNode gains `injectedInstructions?` forwarded
  to the vm-agent create-session body

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: re-trigger checks with corrected preflight classification

Reclassified as cross-component (not external-api-change); no code change.

* fix(origin): forward origin through project-data service to the DO

Staging E2E revealed the injected origin=system message persisted with
origin=null: the service-layer persistMessageBatch (project-data.ts) re-mapped
batch messages before the DO RPC and dropped `origin` — a missing-propagation
bug at a boundary no test covered (vm-agent tests, DO worker tests, and the
un-run CI worker tests each mock only one side). Add origin to the param type
and the re-map, plus a service-layer unit regression test (runs in CI, unlike
the DO worker tests) asserting origin reaches the stub.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: post-mortem for origin service-boundary drop + CI worker-test gap backlog

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Raphaël Titsworth-Morin <raphael@raphaeltm.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Author: 3 people

CLAUDE.md

@@ -241,6 +241,7 @@ Tasks tracked as markdown in `tasks/` (backlog -> active -> archive). See `tasks
 
 Use the `/changelog` skill for structured queries.
 
+- origin-tag-injected-messages: SAM-injected prompt text (e.g. the `get_instructions` reminder) is no longer concatenated into the visible user prompt. It is passed separately as `injectedInstructions` on the vm-agent create-session HTTP body (`startAgentSessionOnNode` → `buildInitialPromptParams`), emitted as a second ACP prompt block tagged `_meta["sam.origin"]="system"`. The vm-agent honors that marker ONLY from a trusted control-plane prompt (`HandlePrompt(..., trustedSource)`) — browser viewer prompts have it stripped so a user cannot hide their own content. Origin propagates through the messagereport outbox (new `origin` column), the ProjectData DO `chat_messages` table (additive migration `024-chat-message-origin`, NULL/absent = `user`), `ChatMessageResponse`, and the live `session/update` broadcast. The web collapses `origin=system` user messages behind a `<details>` "Show system context" disclosure (`AcpConversationItemView`). `origin=system` content is excluded from user-content dedup, LIKE/FTS search (materialization filters it out of `chat_messages_grouped`), topic auto-capture, and attention resolution.
 - chat-full-load-timeline-jump: Project chat loads the FULL conversation on open (`getSessionMessageLimit` split into a small default page size `CHAT_SESSION_MESSAGE_LIMIT` for the 3s poll/load-more and a large `CHAT_SESSION_MESSAGE_MAX` ceiling the initial load requests; new `DEFAULT_CHAT_SESSION_MESSAGE_MAX`). The 30 MiB RPC size guard + `hasMore` "Load earlier" remain the fallback for oversized sessions. Timeline jump-to-message now anchors by messageId+timestamp with a `loadUntil` fallback (no more dead clicks), every entry kind (user message, status update, activity) is clickable, and the jumped-to message flashes (`.sam-message-highlight`). Removed the now-dead `messageIndex`/`messageIndexMap` from the timeline.
 - skills-profile-override-layer: Skills as a first-class profile-override layer (skill → profile → project → platform); additive migration `0063_skills.sql` adds `skills`, `skill_runtime_env_vars`, `skill_runtime_files` and `tasks.skill_id`/`tasks.skill_hint`/`triggers.skill_id`; shared mappers in `profile-fields.ts`/`profile-runtime-assets.ts`; CRUD + runtime routes under `/api/projects/:projectId/skills`; resolution wired through task/trigger/MCP/dispatch + workspace runtime asset injection; Skills UI + chat/task skill selectors in `apps/web`
 - explicit-sam-provider-selection: Require explicit opt-in to SAM as AI provider via `providerMode: 'sam'`; three-mode agent auth (user-api-key, oauth, sam); AI proxy auth gate on all endpoints including /models

apps/api/src/durable-objects/migrations.ts

@@ -636,6 +636,15 @@ export const MIGRATIONS: Migration[] = [
       sql.exec(`CREATE INDEX IF NOT EXISTS idx_chat_sessions_created_by ON chat_sessions(created_by_user_id)`);
     },
   },
+  {
+    // Origin tag for SAM-injected messages (e.g. the get_instructions reminder)
+    // so the UI can collapse them. Additive column; NULL/absent = normal user
+    // message. No DROP/recreate (rule 31 — chat_messages is a CASCADE parent).
+    name: '024-chat-message-origin',
+    run: (sql) => {
+      sql.exec(`ALTER TABLE chat_messages ADD COLUMN origin TEXT`);
+    },
+  },
 ];
 
 /**

apps/api/src/durable-objects/project-data/index.ts

@@ -115,7 +115,7 @@ export class ProjectData extends DurableObject<Env> {
 
   async persistMessageBatch(
     sessionId: string,
-    batchMessages: Array<{ messageId: string; role: string; content: string; toolMetadata: string | null; timestamp: string; sequence?: number }>
+    batchMessages: Array<{ messageId: string; role: string; content: string; toolMetadata: string | null; timestamp: string; sequence?: number; origin?: string | null }>
   ): Promise<messagePersistence.MessageBatchPersistenceResult> {
     return messagePersistence.persistMessageBatchWithSideEffects(this.sql, this.env, this.messagePersistenceHooks(), sessionId, batchMessages);
   }

apps/api/src/durable-objects/project-data/materialization.ts

@@ -32,33 +32,24 @@ export function materializeSession(sql: SqlStorage, sessionId: string): void {
   if (!session) return;
   const { materializedAt } = parseMaterializationCheck(session);
   if (materializedAt !== null) return;
-
   const tokens = sql
     .exec(
-      'SELECT id, role, content, created_at FROM chat_messages WHERE session_id = ? ORDER BY created_at ASC, sequence ASC',
+      "SELECT id, role, content, created_at FROM chat_messages WHERE session_id = ? AND COALESCE(origin, 'user') != 'system' ORDER BY created_at ASC, sequence ASC",
       sessionId
     )
     .toArray()
     .map((row) => parseMaterializationToken(row));
 
   if (tokens.length === 0) {
-    sql.exec(
-      'UPDATE chat_sessions SET materialized_at = ? WHERE id = ?',
-      Date.now(),
-      sessionId
-    );
+    sql.exec('UPDATE chat_sessions SET materialized_at = ? WHERE id = ?', Date.now(), sessionId);
     return;
   }
 
   // Group consecutive same-role tokens
   const grouped: Array<{ id: string; role: string; content: string; createdAt: number }> = [];
   for (const token of tokens) {
     const last = grouped[grouped.length - 1];
-    if (
-      last &&
-      last.role === token.role &&
-      GROUPABLE_ROLES.has(token.role)
-    ) {
+    if (last && last.role === token.role && GROUPABLE_ROLES.has(token.role)) {
       last.content += token.content;
     } else {
       grouped.push({
@@ -97,11 +88,7 @@ export function materializeSession(sql: SqlStorage, sessionId: string): void {
     }
   }
 
-  sql.exec(
-    'UPDATE chat_sessions SET materialized_at = ? WHERE id = ?',
-    Date.now(),
-    sessionId
-  );
+  sql.exec('UPDATE chat_sessions SET materialized_at = ? WHERE id = ?', Date.now(), sessionId);
 }
 
 /**
@@ -135,7 +122,9 @@ export function materializeAllStopped(
   }
 
   const remainingRow = sql
-    .exec(`SELECT COUNT(*) as count FROM chat_sessions WHERE status = 'stopped' AND materialized_at IS NULL`)
+    .exec(
+      `SELECT COUNT(*) as count FROM chat_sessions WHERE status = 'stopped' AND materialized_at IS NULL`
+    )
     .toArray()[0];
   const remaining = remainingRow ? parseCount(remainingRow, 'materialization.remaining') : 0;
 

apps/api/src/durable-objects/project-data/message-persistence.ts

@@ -15,6 +15,7 @@ export type BatchMessageInput = {
   toolMetadata: string | null;
   timestamp: string;
   sequence?: number;
+  origin?: string | null;
 };
 
 export type MessagePersistenceHooks = {
@@ -79,6 +80,11 @@ export async function persistMessageWithSideEffects(
       toolMetadata: parseToolMetadata(toolMetadata, sessionId),
       createdAt: result.now,
       sequence: result.sequence,
+      // The single-message path only persists browser/RPC user messages, which
+      // are never system-injected — origin is always null here. Emitting it
+      // keeps the message.new payload shape aligned with messages.batch (which
+      // carries origin) so live consumers see a stable contract.
+      origin: null,
     },
     sessionId
   );
@@ -118,7 +124,7 @@ export async function persistMessageBatchWithSideEffects(
 
   const latestMessageAt = result.persistedMessages.reduce(
     (latest, message) => Math.max(latest, message.createdAt),
-    0,
+    0
   );
   if (latestMessageAt > 0) {
     sessionState.refreshWorkingActivityForChatSession(sql, sessionId, latestMessageAt);
@@ -146,9 +152,9 @@ async function resolveAttentionForRoles(
   sql: SqlStorage,
   hooks: MessagePersistenceHooks,
   sessionId: string,
-  persistedMessages: Array<{ id: string; role: string }>
+  persistedMessages: Array<{ id: string; role: string; origin?: string | null }>
 ): Promise<void> {
-  const firstUserMsg = persistedMessages.find((m) => m.role === 'user');
+  const firstUserMsg = persistedMessages.find((m) => m.role === 'user' && m.origin !== 'system');
   const firstAssistantMsg = persistedMessages.find((m) => m.role === 'assistant');
   let resolved = 0;
   let reason: string | null = null;

apps/api/src/durable-objects/project-data/messages.ts

@@ -41,9 +41,7 @@ export function resolveCompactMessageOptions(env: Env): CompactMessageOptions {
   const parsed = Number.parseInt(env.DOCUMENT_CARD_RAW_OUTPUT_MAX_BYTES || '', 10);
   return {
     documentCardRawOutputMaxBytes:
-      Number.isFinite(parsed) && parsed > 0
-        ? parsed
-        : DEFAULT_DOCUMENT_CARD_RAW_OUTPUT_MAX_BYTES,
+      Number.isFinite(parsed) && parsed > 0 ? parsed : DEFAULT_DOCUMENT_CARD_RAW_OUTPUT_MAX_BYTES,
   };
 }
 
@@ -150,6 +148,7 @@ export function persistMessageBatch(
     toolMetadata: string | null;
     timestamp: string;
     sequence?: number;
+    origin?: string | null;
   }>
 ): {
   persisted: number;
@@ -161,6 +160,7 @@ export function persistMessageBatch(
     toolMetadata: unknown;
     createdAt: number;
     sequence: number;
+    origin: string | null;
   }>;
   workspaceId: string | null;
   firstUserContent: string | null;
@@ -195,13 +195,15 @@ export function persistMessageBatch(
     toolMetadata: unknown;
     createdAt: number;
     sequence: number;
+    origin: string | null;
   }> = [];
 
   // Track user message content seen within this batch to avoid redundant
   // DB queries when the same user content appears multiple times in one batch.
   const seenUserContent = new Set<string>();
 
   for (const msg of messages) {
+    const origin = msg.origin ?? null;
     const existing = sql
       .exec('SELECT id FROM chat_messages WHERE id = ?', msg.messageId)
       .toArray()[0];
@@ -221,7 +223,7 @@ export function persistMessageBatch(
     // persistMessageBatch (VM agent batch path) because the WebSocket handler
     // persists synchronously on receipt, while the batch arrives after the VM
     // agent processes the prompt, extracts messages, and flushes (~2-5s later).
-    if (msg.role === 'user') {
+    if (msg.role === 'user' && origin !== 'system') {
       if (seenUserContent.has(msg.content)) {
         duplicates++;
         continue;
@@ -250,15 +252,16 @@ export function persistMessageBatch(
     const createdAt = new Date(msg.timestamp).getTime() || now;
     const sequence = msg.sequence ?? nextSeq++;
     sql.exec(
-      `INSERT INTO chat_messages (id, session_id, role, content, tool_metadata, created_at, sequence)
-       VALUES (?, ?, ?, ?, ?, ?, ?)`,
+      `INSERT INTO chat_messages (id, session_id, role, content, tool_metadata, created_at, sequence, origin)
+       VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
       msg.messageId,
       sessionId,
       msg.role,
       msg.content,
       msg.toolMetadata,
       createdAt,
-      sequence
+      sequence,
+      origin
     );
     persisted++;
     persistedMessages.push({
@@ -268,6 +271,7 @@ export function persistMessageBatch(
       toolMetadata: msg.toolMetadata ? JSON.parse(msg.toolMetadata) : null,
       createdAt,
       sequence,
+      origin,
     });
   }
 
@@ -284,7 +288,9 @@ export function persistMessageBatch(
     );
 
     if (!session.topic) {
-      const firstUserMsg = messages.find((m) => m.role === 'user');
+      const firstUserMsg = messages.find(
+        (m) => m.role === 'user' && (m.origin ?? null) !== 'system'
+      );
       if (firstUserMsg) {
         firstUserContent = firstUserMsg.content;
         const truncatedTopic =
@@ -351,7 +357,7 @@ export function getMessages(
   compactOptions?: CompactMessageOptions
 ): { messages: Record<string, unknown>[]; hasMore: boolean } {
   let query =
-    'SELECT id, session_id, role, content, tool_metadata, created_at, sequence FROM chat_messages WHERE session_id = ?';
+    'SELECT id, session_id, role, content, tool_metadata, created_at, sequence, origin FROM chat_messages WHERE session_id = ?';
   const params: (string | number)[] = [sessionId];
 
   if (before !== null) {
@@ -526,7 +532,10 @@ function searchMessagesLike(
   onlyNonMaterialized: boolean = false
 ): SearchResult[] {
   const escapedQuery = query.replace(/[%_\\]/g, '\\$&');
-  const conditions: string[] = ["m.content LIKE ? ESCAPE '\\'"];
+  const conditions: string[] = [
+    "m.content LIKE ? ESCAPE '\\'",
+    "COALESCE(m.origin, 'user') != 'system'",
+  ];
   const params: (string | number)[] = [`%${escapedQuery}%`];
 
   if (sessionId) {

apps/api/src/durable-objects/project-data/row-schemas/messages.ts

@@ -16,6 +16,9 @@ const ChatMessageRowSchema = v.object({
   tool_metadata: v.nullable(v.string()),
   created_at: v.number(),
   sequence: v.nullable(v.number()),
+  // Added by migration 024-chat-message-origin; optional so pre-migration rows
+  // and SELECTs that omit the column still parse (defaults to null → 'user').
+  origin: v.optional(v.nullable(v.string())),
 });
 
 export function parseChatMessageRow(row: unknown): {
@@ -26,6 +29,7 @@ export function parseChatMessageRow(row: unknown): {
   toolMetadata: unknown;
   createdAt: number;
   sequence: number | null;
+  origin: string | null;
 } {
   const r = parseRow(ChatMessageRowSchema, row, 'chat_message');
   return {
@@ -36,6 +40,7 @@ export function parseChatMessageRow(row: unknown): {
     toolMetadata: safeParseJson(r.tool_metadata),
     createdAt: r.created_at,
     sequence: r.sequence,
+    origin: r.origin ?? null,
   };
 }
 
@@ -58,6 +63,7 @@ export function parseChatMessageRowCompact(row: unknown, options?: CompactMessag
   toolMetadata: unknown;
   createdAt: number;
   sequence: number | null;
+  origin: string | null;
 } {
   const r = parseRow(ChatMessageRowSchema, row, 'chat_message');
   const parsed = safeParseJson(r.tool_metadata);
@@ -70,6 +76,7 @@ export function parseChatMessageRowCompact(row: unknown, options?: CompactMessag
     toolMetadata,
     createdAt: r.created_at,
     sequence: r.sequence,
+    origin: r.origin ?? null,
   };
 }
 

apps/api/src/durable-objects/task-runner/agent-session-step.ts

@@ -6,7 +6,7 @@
  */
 import { log } from '../../lib/logger';
 import {
-  buildAgentStartPromptPayload,
+  buildSamBootstrapInstructions,
   buildVisibleInitialPrompt,
 } from '../../services/agent-bootstrap-prompt';
 import { transitionToInProgress } from './state-machine';
@@ -17,12 +17,11 @@ export function buildTaskAgentSessionLabel(taskTitle: string): string {
 }
 
 export function buildTaskInitialPrompt(state: TaskRunnerState): string {
-  return buildAgentStartPromptPayload({
-    message: state.config.taskDescription || state.config.taskTitle,
-    attachments: state.config.attachments,
-    systemPromptAppend: state.config.systemPromptAppend,
-    contextType: 'task',
-  });
+  return buildVisibleTaskInitialPrompt(state);
+}
+
+export function buildInjectedInstructions(): string {
+  return buildSamBootstrapInstructions({ contextType: 'task' });
 }
 
 function buildVisibleTaskInitialPrompt(state: TaskRunnerState): string {
@@ -35,7 +34,7 @@ function buildVisibleTaskInitialPrompt(state: TaskRunnerState): string {
 
 export async function handleAgentSession(
   state: TaskRunnerState,
-  rc: TaskRunnerContext,
+  rc: TaskRunnerContext
 ): Promise<void> {
   await rc.updateD1ExecutionStep(state.taskId, 'agent_session');
 
@@ -47,9 +46,9 @@ export async function handleAgentSession(
 
   // Step 1: Create the agent session (skip if already created on a previous attempt)
   if (sessionId) {
-    const existing = await rc.env.DATABASE.prepare(
-      `SELECT id FROM agent_sessions WHERE id = ?`
-    ).bind(sessionId).first<{ id: string }>();
+    const existing = await rc.env.DATABASE.prepare(`SELECT id FROM agent_sessions WHERE id = ?`)
+      .bind(sessionId)
+      .first<{ id: string }>();
 
     if (!existing) {
       // StepResults had a sessionId but it's gone from D1 — reset and recreate

apps/api/src/routes/workspaces/runtime.ts

@@ -401,6 +401,7 @@ function toProjectDataMessages(body: MessageBatchBody) {
     toolMetadata: m.toolMetadata ? safeParseJson(m.toolMetadata) : null,
     timestamp: m.timestamp,
     sequence: m.sequence,
+    origin: m.origin ?? null,
   }));
 }
 

apps/api/src/schemas/workspaces.ts

@@ -68,6 +68,8 @@ const MessageEntrySchema = v.object({
   toolMetadata: v.optional(v.nullable(v.string())),
   timestamp: v.string(),
   sequence: v.optional(v.number()),
+  // "system" for SAM-injected messages the UI collapses; absent for normal messages.
+  origin: v.optional(v.nullable(v.picklist(['user', 'system']))),
 });
 
 export const MessageBatchSchema = v.object({