Scope check
Due diligence
What problem does this solve?
Anthropic shipped a server-side context-compaction feature (compact_20260112, docs) that summarizes earlier conversation turns when input tokens cross a configurable threshold. The summary is returned in the response as a dedicated compaction content block, separate from the assistant's normal text reply.
When streaming, the response stream contains a new event type:
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"compaction"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"compaction_delta","summary":[...]}}
lib/ruby_llm/providers/anthropic/streaming.rb#build_chunk only recognizes text_delta, thinking_delta, and signature_delta:
def extract_content_delta(data, delta_type)
return data.dig('delta', 'text') if delta_type == 'text_delta'
nil
end
So compaction_delta events are silently dropped.
Result: using RubyLLM::Chat#ask against claude-sonnet-4-6 or claude-opus-4-7 with context_management.edits = [{ type: 'compact_20260112', ... }] have no way to:
- Detect that compaction actually fired server-side
- Read the compaction summary content (needed for follow-up turns per Anthropic's recommended pattern: "Keep the original messages in your list and let the API handle removing the compacted content". But the summary itself is what needs to round-trip back to keep Anthropic from re-running compaction every turn)
- Persist the summary for observability
Proposed solution
Mirror how thinking is handled. Concretely:
1.
- Recognize
compaction_delta (and any compaction_* deltas) in build_chunk / extract_*_delta helpers
- Add
compaction: accumulation alongside content/thinking
- Add
extract_compaction_content(content_blocks) that selects c['type'] == 'compaction'
- Pass to
Message.new via a new keyword
- Add
compaction attribute similar to thinking
- Default
nil for non-Anthropic providers
4: Update format_message to round-trip compaction blocks back to Anthropic on subsequent turns (so the gem also handles the cost-saving "skip re-compaction" path Anthropic's docs describe).
This is symmetric with the thinking-block plumbing already present, so the change should be local and additive.
Why this belongs in RubyLLM
- It's a first-class feature of the Anthropic Messages API.
- The data is only available at the streaming layer:
compaction_delta events live in the SSE stream; without gem support, an application has to either fork the gem or re-implement Anthropic SSE parsing externally.
- It's not actually Anthropic-specific in concept. OpenAI shipped an equivalent server-side compaction feature on its Responses API (docs) with the same shape: a
context_management config, a token threshold, and a server-returned compaction item meant to round-trip on subsequent turns. So a generic compaction accessor on RubyLLM::Message will eventually be populated by multiple providers, same evolution thinking went through.
Scope check
Due diligence
What problem does this solve?
Anthropic shipped a server-side context-compaction feature (
compact_20260112, docs) that summarizes earlier conversation turns when input tokens cross a configurable threshold. The summary is returned in the response as a dedicatedcompactioncontent block, separate from the assistant's normal text reply.When streaming, the response stream contains a new event type:
lib/ruby_llm/providers/anthropic/streaming.rb#build_chunkonly recognizestext_delta,thinking_delta, andsignature_delta:So
compaction_deltaevents are silently dropped.Result: using
RubyLLM::Chat#askagainstclaude-sonnet-4-6orclaude-opus-4-7withcontext_management.edits = [{ type: 'compact_20260112', ... }]have no way to:Proposed solution
Mirror how
thinkingis handled. Concretely:1.
compaction_delta(and anycompaction_*deltas) inbuild_chunk/extract_*_deltahelperscompaction:accumulation alongsidecontent/thinkingextract_compaction_content(content_blocks)that selectsc['type'] == 'compaction'Message.newvia a new keywordcompactionattribute similar tothinkingnilfor non-Anthropic providers4: Update
format_messageto round-trip compaction blocks back to Anthropic on subsequent turns (so the gem also handles the cost-saving "skip re-compaction" path Anthropic's docs describe).This is symmetric with the thinking-block plumbing already present, so the change should be local and additive.
Why this belongs in RubyLLM
compaction_deltaevents live in the SSE stream; without gem support, an application has to either fork the gem or re-implement Anthropic SSE parsing externally.context_managementconfig, a token threshold, and a server-returned compaction item meant to round-trip on subsequent turns. So a genericcompactionaccessor onRubyLLM::Messagewill eventually be populated by multiple providers, same evolutionthinkingwent through.