Skip to content

Commit 56a478c

Browse files
author
molty3000
committed
feat: prompt caching — Anthropic cache_control, OpenAI/DeepSeek prefix caching, max_tokens, cache metrics
1 parent 42375d0 commit 56a478c

7 files changed

Lines changed: 488 additions & 33 deletions

File tree

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,7 @@ odek run "@README.md what does this project do?"
135135
| `--no-learn` | Disable skill learning mode |
136136
| `--system <prompt>` | Override system prompt |
137137
| `--max-iter <n>` | Max think→act cycles (default 90) |
138+
| `--prompt-caching` | Enable Anthropic/OpenAI/DeepSeek prompt caching markers |
138139
| `--no-color` | Disable colored output |
139140
| `--ctx <files>` / `-c` | Attach files as context blocks (comma-separated) |
140141
| `--no-agents` | Skip AGENTS.md project file |
@@ -149,6 +150,7 @@ odek run "@README.md what does this project do?"
149150
| [Configuration](docs/CONFIG.md) | Config files, env vars, priority chain, all sections |
150151
| [Programmatic API](docs/API.md) | **SDK Guide**: import, Agent lifecycle, Tool interface, multi-turn sessions, memory system, model profiles, complete examples |
151152
| [Providers & Models](docs/PROVIDERS.md) | Supported providers, thinking config, context windows |
153+
| [Prompt Caching](docs/CACHING.md) | Anthropic/OpenAI/DeepSeek caching support, config, metrics |
152154
| [Memory](docs/MEMORY.md) | Three-tier design, go-vector merge-on-write, `memory` tool |
153155
| [Sessions](docs/SESSIONS.md) | Multi-turn conversations, save/resume/trim/cleanup |
154156
| [Sandboxing](docs/SANDBOXING.md) | Docker isolation model, config, security hardening |

docs/CACHING.md

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Prompt Caching
2+
3+
odek supports prompt caching for supported LLM providers. When enabled, the system prompt and first user message are annotated with cache markers, reducing both latency and cost on repeated interactions.
4+
5+
## Supported Providers
6+
7+
| Provider | Mechanism | Cached tokens cost | TTFT improvement |
8+
|---|---|---|---|
9+
| **Anthropic** (Claude) | Explicit `cache_control: ephemeral` markers | ~90% reduction | ~60-80% |
10+
| **DeepSeek** | Automatic prefix caching (no client markers needed) | ~50-80% reduction | ~50% |
11+
| **OpenAI** | Automatic prefix caching (GPT-4o, GPT-4o-mini) | ~50% reduction ||
12+
13+
When caching is enabled, odek:
14+
15+
1. Moves the system prompt from the `messages[]` array into a dedicated `system` field with `cache_control: {"type": "ephemeral"}` (Anthropic format — silently ignored by other providers)
16+
2. Marks the first user message with `cache_control: {"type": "ephemeral"}`
17+
3. Sends the `anthropic-version: 2023-06-01` header (required by Anthropic for caching; ignored by others)
18+
19+
## Enabling
20+
21+
### CLI
22+
```bash
23+
odek run --prompt-caching "Does this work with caching?"
24+
```
25+
26+
The `--prompt-caching` flag is available on `odek run`, `odek repl`, and `odek serve`.
27+
28+
### Config file (`~/.odek/config.json` or `./odek.json`)
29+
```json
30+
{
31+
"prompt_caching": true
32+
}
33+
```
34+
35+
### Environment variable
36+
```bash
37+
export ODEK_PROMPT_CACHING=true
38+
```
39+
40+
### Programmatic API
41+
```go
42+
agent, err := odek.New(odek.Config{
43+
Model: "claude-sonnet-4",
44+
APIKey: os.Getenv("ANTHROPIC_API_KEY"),
45+
BaseURL: "https://api.anthropic.com/v1",
46+
PromptCaching: true,
47+
})
48+
```
49+
50+
## When to Enable
51+
52+
**Enable for:**
53+
- Anthropic models (Claude 3.5 Sonnet, Claude 3 Opus, Claude 4 Sonnet) — explicit cache markers provide the largest benefit
54+
- DeepSeek models — automatic prefix caching works best when the conversation prefix is stable; caching markers don't hurt
55+
- Any multi-turn session where the system prompt is large (e.g., AGENTS.md files, loaded skills) — the system prompt is cached after the first iteration
56+
57+
**Disable for:**
58+
- One-shot tasks where the agent runs exactly one iteration
59+
- Providers that don't support caching and have unusual request parsing (safety: unknown fields are ignored by all major providers, but caching is always opt-in via `--prompt-caching`)
60+
61+
## Cache Metrics
62+
63+
When caching is active, odek tracks cache metrics and exposes them through the agent API:
64+
65+
```go
66+
agent.TotalCacheCreationTokens() // Anthropic: tokens written to cache
67+
agent.TotalCacheReadTokens() // Anthropic: tokens read from cache hit
68+
agent.TotalCachedTokens() // OpenAI: cached prompt tokens
69+
```
70+
71+
These are accumulated across all iterations of the most recent run. Zero values mean the provider didn't return cache metrics (either caching is not enabled, or the provider doesn't report them).
72+
73+
## How It Works
74+
75+
1. **Before each LLM call**, if `PromptCaching` is enabled, the loop calls `llm.ApplyCacheMarkers(messages)` which:
76+
- Extracts the first system message and converts it to an Anthropic `SystemBlock` with `cache_control: ephemeral`
77+
- Marks the first user message with `cache_control: ephemeral`
78+
79+
2. **The request is sent** with the system in the `system` field (not `messages[]`) and the cache markers in place. Providers that don't support these fields silently ignore them.
80+
81+
3. **The response is parsed** for cache metrics from both Anthropic (`cache_creation_input_tokens`, `cache_read_input_tokens`) and OpenAI (`prompt_tokens_details.cached_tokens`).
82+
83+
4. **Metrics are accumulated** across iterations and exposed via the agent API.
84+
85+
## Implementation Details
86+
87+
- The `anthropic-version: 2023-06-01` header is always sent when caching is enabled. This header is required by Anthropic for prompt caching and is ignored by OpenAI and DeepSeek.
88+
- Cache markers are applied **per iteration** — the system prompt and first user message are marked on every LLM call. This is safe because the markers reference the same content each time, so the cache is populated on the first iteration and read on subsequent ones.
89+
- The `max_tokens` field is now included in all requests when set via `odek.Config.MaxTokens` or model profile defaults. Some providers (Anthropic) tie caching behavior to this field being present.
90+
- The system prompt is moved out of `messages[]` into a separate `system` field only when caching is enabled. When disabled, it stays in `messages[]` for maximum provider compatibility.

docs/CLI.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@
3535
| `--thinking <level>` | string | profile default | Reasoning depth: `enabled`/`disabled`/`low`/`medium`/`high` |
3636
| `--sandbox` | bool | false | Execute shell commands inside Docker container |
3737
| `--no-color` | bool | false | Disable colored terminal output |
38+
| `--prompt-caching` | bool | false | Enable Anthropic/OpenAI/DeepSeek prompt caching markers |
3839
| `--no-agents` | bool | false | Skip loading AGENTS.md |
3940
| `--session` | bool | false | Save conversation as a multi-turn session |
4041
| `--learn` | bool | `true` | Enable skill learning mode (detects patterns, saves skills). On by default |

internal/llm/client.go

Lines changed: 122 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -14,11 +14,12 @@ import (
1414

1515
// Client sends chat completion requests to any OpenAI-compatible endpoint.
1616
type Client struct {
17-
BaseURL string
18-
APIKey string
19-
Model string
20-
Thinking string // "enabled", "disabled", "low", "medium", "high", or empty
21-
http *http.Client
17+
BaseURL string
18+
APIKey string
19+
Model string
20+
Thinking string // "enabled", "disabled", "low", "medium", "high", or empty
21+
MaxTokens int // max output tokens (0 = provider default)
22+
http *http.Client
2223
}
2324

2425
// maxResponseSize limits the LLM response body read to prevent DoS/OOM.
@@ -28,25 +29,48 @@ const maxResponseSize = 50 * 1024 * 1024 // 50 MB
2829
// (120s). The timeout applies per HTTP request — the agent loop may have
2930
// multiple requests; set a generous timeout for deep-reasoning models.
3031
func New(baseURL, apiKey, model, thinking string, timeout time.Duration) *Client {
32+
return NewWithMaxTokens(baseURL, apiKey, model, thinking, 0, timeout)
33+
}
34+
35+
// NewWithMaxTokens creates a Client with a specific max_tokens setting.
36+
// maxTokens=0 means no limit (provider default).
37+
func NewWithMaxTokens(baseURL, apiKey, model, thinking string, maxTokens int, timeout time.Duration) *Client {
3138
if timeout <= 0 {
3239
timeout = 120 * time.Second
3340
}
3441
return &Client{
35-
BaseURL: strings.TrimRight(baseURL, "/"),
36-
APIKey: apiKey,
37-
Model: model,
38-
Thinking: thinking,
39-
http: &http.Client{Timeout: timeout},
42+
BaseURL: strings.TrimRight(baseURL, "/"),
43+
APIKey: apiKey,
44+
Model: model,
45+
Thinking: thinking,
46+
MaxTokens: maxTokens,
47+
http: &http.Client{Timeout: timeout},
4048
}
4149
}
4250

51+
// CacheControl marks a message or system block as cacheable by Anthropic.
52+
// Providers that don't support it (OpenAI, DeepSeek) silently ignore the field.
53+
type CacheControl struct {
54+
Type string `json:"type"` // "ephemeral"
55+
}
56+
57+
// SystemBlock represents an Anthropic-style system prompt block with optional
58+
// cache control. OpenAI-compatible endpoints that don't support this format
59+
// silently ignore the field.
60+
type SystemBlock struct {
61+
Type string `json:"type"` // "text"
62+
Text string `json:"text"`
63+
CacheControl *CacheControl `json:"cache_control,omitempty"`
64+
}
65+
4366
// Message represents a chat message.
4467
type Message struct {
45-
Role string `json:"role"` // "system", "user", "assistant", "tool"
46-
Content string `json:"content"` // text content
47-
Name string `json:"name,omitempty"` // tool name (for tool role)
48-
ToolCallID string `json:"tool_call_id,omitempty"` // required for tool role
49-
ToolCalls []ToolCall `json:"tool_calls,omitempty"` // required for assistant role with tool calls
68+
Role string `json:"role"` // "system", "user", "assistant", "tool"
69+
Content string `json:"content"` // text content
70+
Name string `json:"name,omitempty"` // tool name (for tool role)
71+
ToolCallID string `json:"tool_call_id,omitempty"`
72+
ToolCalls []ToolCall `json:"tool_calls,omitempty"` // required for assistant role with tool calls
73+
CacheControl *CacheControl `json:"cache_control,omitempty"` // Anthropic prompt caching marker
5074
}
5175

5276
// ToolCall represents a single tool invocation requested by the model.
@@ -77,8 +101,10 @@ type FunctionDef struct {
77101
type CallParams struct {
78102
Model string `json:"model"`
79103
Messages []Message `json:"messages"`
104+
System []SystemBlock `json:"system,omitempty"` // Anthropic-style system blocks
80105
Tools []ToolDef `json:"tools,omitempty"`
81106
Stream bool `json:"stream"`
107+
MaxTokens int `json:"max_tokens,omitempty"` // max output tokens (0 = omit/provider default)
82108
Thinking *ThinkingConfig `json:"thinking,omitempty"`
83109
ReasoningEffort string `json:"reasoning_effort,omitempty"`
84110
}
@@ -94,11 +120,65 @@ type CallResult struct {
94120
ToolCalls []ToolCall // tool calls requested by the model
95121
InputTokens int // prompt_tokens from API usage (0 = not reported)
96122
OutputTokens int // completion_tokens from API usage (0 = not reported)
123+
124+
// Cache metrics. Only populated when the provider returns them.
125+
// Anthropic: cache_creation_input_tokens, cache_read_input_tokens
126+
// OpenAI: prompt_tokens_details.cached_tokens
127+
CacheCreationTokens int // Anthropic — tokens written to cache
128+
CacheReadTokens int // Anthropic — tokens read from cache hit
129+
CachedTokens int // OpenAI — cached tokens in prompt
97130
}
98131

99132
// toolChoiceNone forces the model to not call tools.
100133
var toolChoiceNone = "none"
101134

135+
// ApplyCacheMarkers annotates messages with Anthropic-style cache_control
136+
// markers to enable prompt caching. It:
137+
// 1. Marks the first system message (if present) with cache_control: ephemeral
138+
// 2. Marks the first user message with cache_control: ephemeral
139+
//
140+
// Returns the updated messages and a System field (populated if the system
141+
// message was moved out of the messages array for Anthropic compatibility).
142+
// Providers that don't support prompt caching silently ignore these fields.
143+
func ApplyCacheMarkers(messages []Message) ([]Message, []SystemBlock) {
144+
var systemBlocks []SystemBlock
145+
annotated := make([]Message, 0, len(messages))
146+
147+
// Track whether we've marked the first user message
148+
markedUser := false
149+
150+
for i, m := range messages {
151+
// If this is the first system message, move it to System field
152+
// (Anthropic format) with cache_control
153+
if m.Role == "system" && len(systemBlocks) == 0 {
154+
systemBlocks = append(systemBlocks, SystemBlock{
155+
Type: "text",
156+
Text: m.Content,
157+
CacheControl: &CacheControl{Type: "ephemeral"},
158+
})
159+
continue // don't add to messages — it's now in System
160+
}
161+
162+
// Mark the first user message with cache_control
163+
if m.Role == "user" && !markedUser {
164+
m.CacheControl = &CacheControl{Type: "ephemeral"}
165+
markedUser = true
166+
}
167+
168+
// For assistant messages with preceded by system (now in System field),
169+
// mark them too if they're the first non-system assistant response
170+
// (This helps cache the initial turn in multi-turn conversations)
171+
if i > 0 && m.Role == "assistant" && len(m.ToolCalls) == 0 && !markedUser {
172+
// This is the final assistant message from a previous run —
173+
// keep going, we already marked the first user above
174+
}
175+
176+
annotated = append(annotated, m)
177+
}
178+
179+
return annotated, systemBlocks
180+
}
181+
102182
// SimpleCall sends a single-turn chat completion request and returns the
103183
// text response. No tools, no streaming, no thinking config. Used for
104184
// lightweight LLM calls like skill risk assessment.
@@ -162,12 +242,18 @@ func (c *Client) SimpleCall(ctx context.Context, systemPrompt, userPrompt string
162242
}
163243

164244
// Call sends a chat completion request and returns the result.
165-
func (c *Client) Call(ctx context.Context, messages []Message, tools []ToolDef) (*CallResult, error) {
245+
// systemBlocks is optional — pass nil for providers that don't support
246+
// the separate System field (OpenAI, DeepSeek). When non-nil, the system
247+
// prompt is sent in the "system" field instead of as a system message in
248+
// the messages array (Anthropic format for prompt caching).
249+
func (c *Client) Call(ctx context.Context, messages []Message, systemBlocks []SystemBlock, tools []ToolDef) (*CallResult, error) {
166250
body := CallParams{
167-
Model: c.Model,
168-
Messages: messages,
169-
Tools: tools,
170-
Stream: false,
251+
Model: c.Model,
252+
Messages: messages,
253+
System: systemBlocks,
254+
Tools: tools,
255+
Stream: false,
256+
MaxTokens: c.MaxTokens,
171257
}
172258

173259
switch c.Thinking {
@@ -190,6 +276,10 @@ func (c *Client) Call(ctx context.Context, messages []Message, tools []ToolDef)
190276
req.Header.Set("Content-Type", "application/json")
191277
req.Header.Set("Authorization", "Bearer "+c.APIKey)
192278

279+
// Add Anthropic-specific API version header (required for prompt caching)
280+
// Safe to always send — other providers ignore unknown headers.
281+
req.Header.Set("anthropic-version", "2023-06-01")
282+
193283
resp, err := c.http.Do(req)
194284
if err != nil {
195285
return nil, fmt.Errorf("llm: %w", err)
@@ -228,6 +318,13 @@ func parseResponse(data []byte) (*CallResult, error) {
228318
Usage *struct {
229319
PromptTokens int `json:"prompt_tokens"`
230320
CompletionTokens int `json:"completion_tokens"`
321+
// Anthropic prompt caching
322+
CacheCreationTokens int `json:"cache_creation_input_tokens"`
323+
CacheReadTokens int `json:"cache_read_input_tokens"`
324+
// OpenAI prompt caching (nested details)
325+
PromptTokensDetails *struct {
326+
CachedTokens int `json:"cached_tokens"`
327+
} `json:"prompt_tokens_details"`
231328
} `json:"usage"`
232329
}
233330
if err := json.Unmarshal(data, &raw); err != nil {
@@ -244,6 +341,11 @@ func parseResponse(data []byte) (*CallResult, error) {
244341
if raw.Usage != nil {
245342
result.InputTokens = raw.Usage.PromptTokens
246343
result.OutputTokens = raw.Usage.CompletionTokens
344+
result.CacheCreationTokens = raw.Usage.CacheCreationTokens
345+
result.CacheReadTokens = raw.Usage.CacheReadTokens
346+
if raw.Usage.PromptTokensDetails != nil {
347+
result.CachedTokens = raw.Usage.PromptTokensDetails.CachedTokens
348+
}
247349
}
248350
for _, tc := range msg.ToolCalls {
249351
result.ToolCalls = append(result.ToolCalls, ToolCall{

0 commit comments

Comments
 (0)