Skip to content

Commit 7b77c99

Browse files
committed
feat(memory): inherit unset memory.extended.llm fields from main LLM
memory.extended.llm was all-or-nothing: without base_url AND model it fell back to the main client entirely, so operators could not override a single knob (e.g. thinking) without duplicating the parent connection settings. ResolveLLM now inherits empty fields (base_url, api_key, model, thinking, max_tokens, temperature) from the main *llm.Client before building the dedicated client, enabling partial overrides such as: "llm": {"thinking": "disabled"} // same model, no reasoning for memory "llm": {"model": "qwen2.5:7b"} // cheaper model, same backend Fallback semantics are preserved: if base_url/model are still empty after inheritance, it warns and reuses the main client as before. Also patch the docker configs (godmode + restricted) to disable thinking for extended memory calls via the new partial override, and document the inheritance behavior in CONFIG.md and EXTENDED_MEMORY.md.
1 parent 27c6299 commit 7b77c99

6 files changed

Lines changed: 94 additions & 5 deletions

File tree

docker/config.godmode.json

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,10 @@
6262
"proactive_nudges_enabled": true,
6363
"nudge_max_per_day": 5,
6464
"nudge_cooldown_hours": 24,
65-
"nudge_stale_goal_days": 7
65+
"nudge_stale_goal_days": 7,
66+
"llm": {
67+
"thinking": "disabled"
68+
}
6669
}
6770
},
6871
"dangerous": {

docker/config.restricted.json

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,10 @@
6262
"proactive_nudges_enabled": true,
6363
"nudge_max_per_day": 5,
6464
"nudge_cooldown_hours": 24,
65-
"nudge_stale_goal_days": 7
65+
"nudge_stale_goal_days": 7,
66+
"llm": {
67+
"thinking": "disabled"
68+
}
6669
}
6770
},
6871
"dangerous": {

docs/CONFIG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -434,7 +434,7 @@ The `memory` section controls the persistent memory system (see [docs/MEMORY.md]
434434
| `nudge_cooldown_hours` | `24` | `ODEK_MEMORY_EXTENDED_NUDGE_COOLDOWN_HOURS` || Per-kind cooldown before a nudge of the same kind can fire again. |
435435
| `nudge_stale_goal_days` | `7` | `ODEK_MEMORY_EXTENDED_NUDGE_STALE_GOAL_DAYS` || Days without activity before a goal/intent atom counts as stale for nudges. |
436436
| `nudge_open_question_min_age_hours` | `24` | `ODEK_MEMORY_EXTENDED_NUDGE_OPEN_QUESTION_MIN_AGE_HOURS` || Minimum age before a `question` atom may become a nudge candidate (younger questions are usually about to be answered). |
437-
| `llm` | omitted ||| Dedicated memory LLM. If omitted, the main agent LLM is reused. A warning is emitted if that model has thinking enabled. |
437+
| `llm` | omitted ||| Dedicated memory LLM. If omitted, the main agent LLM is reused. A warning is emitted if that model has thinking enabled. Fields left empty inside `llm` are inherited from the main agent LLM, so a partial override such as `"llm": {"thinking": "disabled"}` reuses the parent `base_url`/`api_key`/`model`/`max_tokens`/`temperature` while disabling reasoning for memory calls. |
438438
| `embedding` | omitted ||| Dedicated embedding backend for atoms. If omitted, inherits `memory.embedding` or the shared top-level `embedding`. |
439439

440440
### `embedding` — real semantic embeddings (optional)

docs/EXTENDED_MEMORY.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,8 @@ Extended Memory can use its own LLM, separate from the main agent. This is ideal
113113

114114
If `memory.extended.llm` is omitted, the module **MUST** use the default global model. The default global model is the fully resolved main agent LLM after all config layers have been merged: top-level `model`, `base_url`, `api_key`, `thinking`, `max_tokens`, `temperature`, and `timeout` from `~/.odek/config.json`, `ODEK_*` environment variables, and CLI flags. Extended Memory does not read any of those values again; it reuses the exact `llm.Client` instance constructed for the main agent loop.
115115

116+
When `memory.extended.llm` is present, any field left empty (`base_url`, `api_key`, `model`, `thinking`, `max_tokens`, `temperature`) is inherited from that same main agent client, so a partial override reuses the parent connection while changing only what it sets. For example, `"llm": {"thinking": "disabled"}` keeps the main model and endpoint but disables reasoning for memory calls, and `"llm": {"model": "qwen2.5:7b"}` switches to a cheaper model on the same backend.
117+
116118
If the default global model has reasoning/thinking enabled, memory extraction and reranking may be expensive. In that case the operator should configure a dedicated `memory.extended.llm` for cost isolation; a warning is emitted when thinking is enabled and no dedicated memory LLM is configured.
117119

118120
### Memory LLM Responsibilities

internal/memory/extended/config.go

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,10 @@ func Resolve(cfg Config) Config {
214214

215215
// ResolveLLM returns the LLM client to use for Extended Memory. If cfg.LLM is
216216
// set, it builds a dedicated client; otherwise it returns the provided main
217-
// client unchanged. When falling back to the main client and the main model
217+
// client unchanged. Fields left empty in cfg.LLM are inherited from the main
218+
// client (when it is an *llm.Client), so operators can override a single knob
219+
// — e.g. `"llm": {"thinking": "disabled"}` — without duplicating base_url,
220+
// api_key, and model. When falling back to the main client and the main model
218221
// has thinking enabled, a warning is logged because reasoning tokens are
219222
// wasted on memory-only calls.
220223
func ResolveLLM(cfg Config, mainLLM LLMClient, thinking string) LLMClient {
@@ -224,7 +227,27 @@ func ResolveLLM(cfg Config, mainLLM LLMClient, thinking string) LLMClient {
224227
}
225228
return mainLLM
226229
}
227-
lmc := cfg.LLM
230+
lmc := *cfg.LLM
231+
if main, ok := mainLLM.(*llm.Client); ok {
232+
if lmc.BaseURL == "" {
233+
lmc.BaseURL = main.BaseURL
234+
}
235+
if lmc.APIKey == "" {
236+
lmc.APIKey = main.APIKey
237+
}
238+
if lmc.Model == "" {
239+
lmc.Model = main.Model
240+
}
241+
if lmc.Thinking == "" {
242+
lmc.Thinking = main.Thinking
243+
}
244+
if lmc.MaxTokens == 0 {
245+
lmc.MaxTokens = main.MaxTokens
246+
}
247+
if lmc.Temperature == 0 {
248+
lmc.Temperature = main.Temperature
249+
}
250+
}
228251
if lmc.BaseURL == "" || lmc.Model == "" {
229252
fmt.Fprintf(os.Stderr, "odek: warning: extended memory llm requires base_url and model; falling back to main LLM\n")
230253
return mainLLM

internal/memory/extended/config_test.go

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,9 @@ package extended
33
import (
44
"context"
55
"testing"
6+
"time"
7+
8+
"github.com/BackendStack21/odek/internal/llm"
69
)
710

811
type dummyLLM struct{}
@@ -69,3 +72,58 @@ func TestResolveLLMWithTimeout(t *testing.T) {
6972
t.Error("expected dedicated LLM client")
7073
}
7174
}
75+
76+
func TestResolveLLMInheritsMainSettings(t *testing.T) {
77+
main := llm.NewWithMaxTokens("https://api.example.com/v1/", "main-key", "main-model", "enabled", 0, 4096, 10*time.Second)
78+
main.Temperature = 0.7
79+
cfg := Config{
80+
LLM: &LLMConfig{Thinking: "disabled"}, // override thinking only
81+
}
82+
got := ResolveLLM(cfg, main, "enabled")
83+
client, ok := got.(*llm.Client)
84+
if !ok {
85+
t.Fatalf("expected *llm.Client, got %T", got)
86+
}
87+
if client == main {
88+
t.Fatal("expected a dedicated client, got the main client")
89+
}
90+
if client.BaseURL != main.BaseURL {
91+
t.Errorf("BaseURL = %q, want inherited %q", client.BaseURL, main.BaseURL)
92+
}
93+
if client.APIKey != main.APIKey {
94+
t.Errorf("APIKey = %q, want inherited %q", client.APIKey, main.APIKey)
95+
}
96+
if client.Model != main.Model {
97+
t.Errorf("Model = %q, want inherited %q", client.Model, main.Model)
98+
}
99+
if client.Thinking != "disabled" {
100+
t.Errorf("Thinking = %q, want %q", client.Thinking, "disabled")
101+
}
102+
if client.MaxTokens != main.MaxTokens {
103+
t.Errorf("MaxTokens = %d, want inherited %d", client.MaxTokens, main.MaxTokens)
104+
}
105+
if client.Temperature != main.Temperature {
106+
t.Errorf("Temperature = %v, want inherited %v", client.Temperature, main.Temperature)
107+
}
108+
}
109+
110+
func TestResolveLLMPartialOverrideKeepsInheritedRest(t *testing.T) {
111+
main := llm.New("https://api.example.com/v1", "main-key", "main-model", "enabled", 0, 0)
112+
cfg := Config{
113+
LLM: &LLMConfig{Model: "cheap-model"}, // different model, same backend
114+
}
115+
got := ResolveLLM(cfg, main, "enabled")
116+
client, ok := got.(*llm.Client)
117+
if !ok {
118+
t.Fatalf("expected *llm.Client, got %T", got)
119+
}
120+
if client.Model != "cheap-model" {
121+
t.Errorf("Model = %q, want %q", client.Model, "cheap-model")
122+
}
123+
if client.BaseURL != main.BaseURL || client.APIKey != main.APIKey {
124+
t.Errorf("expected inherited BaseURL/APIKey, got %q/%q", client.BaseURL, client.APIKey)
125+
}
126+
if client.Thinking != main.Thinking {
127+
t.Errorf("Thinking = %q, want inherited %q", client.Thinking, main.Thinking)
128+
}
129+
}

0 commit comments

Comments
 (0)