Skip to content

Commit ec540d2

Browse files
jkyberneeesclaude
andcommitted
feat(web_search): add SearXNG-backed web_search tool + compose sidecar
Gives the agent local-first web search with no cloud API or keys, matching the transcribe/vision zero-setup pattern. Tool (cmd/odek/web_search_tool.go): - Native Go tool querying a self-hosted SearXNG JSON API; returns ranked results (title/url/snippet/engine) + direct answers, capped by max_results. - Output wrapped as untrusted content (SERP snippets can carry injection). - Gated as network_egress (prompt in restricted, allow in godmode), consistent with browser/http_batch. The backend URL is fixed config, not agent-supplied, so the tool has no SSRF surface (only a query string is accepted). - Registered only when web_search.base_url is set, so plain installs without a SearXNG instance don't see a dead tool. Config (internal/config): - WebSearchConfig{BaseURL, Categories, Language, MaxResults, Timeout} threaded end-to-end (FileConfig, ResolvedConfig, resolveWebSearch, overlayFile). Wiring (cmd/odek): - builtinTools' growing positional config params (Transcription, Vision) are bundled into a toolConfig struct to stop per-tool signature churn; all ~10 call sites updated. web_search is threaded into run/serve/repl/telegram/ schedule/subagent/mcp. Docker: - New `searxng` compose sidecar (pinned image), co-starting with every profile, internal-only (no host port), with depends_on wired on each odek service. - docker/searxng/settings.yml enables the JSON API and disables the anti-bot limiter, so no Redis/Valkey is needed. SEARXNG_SECRET added to .env.example. - Both bundled configs set web_search.base_url=http://searxng:8080. Tests: hermetic httptest SearXNG mock covering happy path, max_results override vs config cap, untrusted wrapping, JSON-disabled 403, unreachable backend, policy denial, empty query, schema; resolveWebSearch defaults/merge. Full suite green under -race. Docs: README, SECURITY, CHEATSHEET, CONFIG, TELEGRAM, docker/README, DOCKER_COMPOSE_USER_GUIDE. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 903e453 commit ec540d2

26 files changed

Lines changed: 689 additions & 21 deletions

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ odek is not a framework. It's a **runtime** — the smallest possible surface ar
3636
Every session can run in an isolated Docker container: no network, no host mounts beyond the working directory, zero capabilities, destroyed on exit. `odek serve` enables the sandbox **by default**; `odek run` keeps it opt-in but warns when running unsandboxed. `--ctx` files are auto-injected into the container at `/workspace/`. Full security model in [docs/SANDBOXING.md](docs/SANDBOXING.md).
3737

3838
### 🛡️ Prompt-Injection-Aware
39-
External content the agent ingests (`browser`, `read_file`, `shell`, `search_files`, `multi_grep`, `transcribe`, `vision`, `session_search`, MCP tools) is wrapped in per-call nonce'd `<untrusted_content>` boundaries so the model can distinguish data from instructions. Redirect hops are re-classified (`browser`/`http_batch`), MCP tool descriptions are scanned for injection at registration, and the MCP error channel is wrapped too. The danger classifier resists 8 known shell-evasion tricks (`$()`, backticks, `$IFS`, `command`/`exec`, `\rm`, basenamed absolute paths). Approvers engage friction mode after 3 same-class approvals in 60 s. Memory episodes from tainted sessions are stored but never auto-replayed. Skill auto-save tracks provenance and pins untrusted suggestions for explicit `odek skill promote`. `odek audit <session-id>` surfaces every ingest + per-turn divergence heuristic. Full threat model in [docs/SECURITY.md](docs/SECURITY.md).
39+
External content the agent ingests (`browser`, `read_file`, `shell`, `search_files`, `multi_grep`, `transcribe`, `vision`, `web_search`, `session_search`, MCP tools) is wrapped in per-call nonce'd `<untrusted_content>` boundaries so the model can distinguish data from instructions. Redirect hops are re-classified (`browser`/`http_batch`), MCP tool descriptions are scanned for injection at registration, and the MCP error channel is wrapped too. The danger classifier resists 8 known shell-evasion tricks (`$()`, backticks, `$IFS`, `command`/`exec`, `\rm`, basenamed absolute paths). Approvers engage friction mode after 3 same-class approvals in 60 s. Memory episodes from tainted sessions are stored but never auto-replayed. Skill auto-save tracks provenance and pins untrusted suggestions for explicit `odek skill promote`. `odek audit <session-id>` surfaces every ingest + per-turn divergence heuristic. Full threat model in [docs/SECURITY.md](docs/SECURITY.md).
4040

4141
### 🧩 Sub-Agent Delegation
4242
Parallel OS-process sub-agents via `delegate_tasks`. True isolation — each sub-agent is a fresh `odek subagent` process with its own config, tools, and termination timeout. Up to 8 concurrent workers. [docs/SUBAGENTS.md](docs/SUBAGENTS.md)

cmd/odek/injection_hardening_test.go

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@ import (
99
"testing"
1010

1111
"github.com/BackendStack21/odek"
12-
"github.com/BackendStack21/odek/internal/config"
1312
"github.com/BackendStack21/odek/internal/danger"
1413
)
1514

@@ -244,7 +243,7 @@ func TestBuiltinTools_SessionSearchWrappedAsUntrusted(t *testing.T) {
244243
store, cleanup := seedSessionStore(t)
245244
defer cleanup()
246245

247-
tools := builtinTools(danger.DangerousConfig{}, nil, nil, 4, "", config.TranscriptionConfig{}, config.VisionConfig{}, store)
246+
tools := builtinTools(danger.DangerousConfig{}, nil, nil, 4, "", toolConfig{}, store)
248247

249248
var ss odek.Tool
250249
for _, tool := range tools {

cmd/odek/main.go

Lines changed: 22 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -779,7 +779,7 @@ func run(args []string) error {
779779

780780
// Sandbox setup
781781
var sandboxCleanup func() error
782-
tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, resolved.Transcription, resolved.Vision, nil)
782+
tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, toolConfig{Transcription: resolved.Transcription, Vision: resolved.Vision, WebSearch: resolved.WebSearch}, nil)
783783

784784
// MCP server tools
785785
var mcpCleanup func()
@@ -1054,7 +1054,17 @@ func setupSandbox(tools []odek.Tool, cfg sandboxConfig) (containerName string, c
10541054
return containerName, cleanup, nil
10551055
}
10561056

1057-
func builtinTools(dc danger.DangerousConfig, sm *skills.SkillManager, approver danger.Approver, maxConcurrency int, apiKey string, tc config.TranscriptionConfig, vc config.VisionConfig, store *session.Store) []odek.Tool {
1057+
// toolConfig bundles the per-tool configuration sections threaded into
1058+
// builtinTools. Grouping them keeps the builtinTools signature stable as new
1059+
// configurable tools are added (rather than growing a positional parameter
1060+
// per tool).
1061+
type toolConfig struct {
1062+
Transcription config.TranscriptionConfig
1063+
Vision config.VisionConfig
1064+
WebSearch config.WebSearchConfig
1065+
}
1066+
1067+
func builtinTools(dc danger.DangerousConfig, sm *skills.SkillManager, approver danger.Approver, maxConcurrency int, apiKey string, tcfg toolConfig, store *session.Store) []odek.Tool {
10581068
tools := []odek.Tool{
10591069
&shellTool{
10601070
dangerousConfig: dc,
@@ -1088,8 +1098,8 @@ func builtinTools(dc danger.DangerousConfig, sm *skills.SkillManager, approver d
10881098
&base64Tool{dangerousConfig: dc},
10891099
&trTool{dangerousConfig: dc},
10901100
&wordCountTool{dangerousConfig: dc},
1091-
newTranscribeTool(dc, tc),
1092-
newVisionTool(dc, vc),
1101+
newTranscribeTool(dc, tcfg.Transcription),
1102+
newVisionTool(dc, tcfg.Vision),
10931103
// session_search returns content from arbitrary past sessions —
10941104
// including sessions that ingested untrusted content. That path
10951105
// otherwise bypasses the memory taint gate and the audit log, so
@@ -1098,6 +1108,13 @@ func builtinTools(dc danger.DangerousConfig, sm *skills.SkillManager, approver d
10981108
newBrowserTool(dc),
10991109
}
11001110

1111+
// web_search is registered only when a SearXNG backend is configured —
1112+
// without a base_url there is no instance to query, so the tool would just
1113+
// confuse the agent. The Docker compose setup sets this automatically.
1114+
if tcfg.WebSearch.BaseURL != "" {
1115+
tools = append(tools, newWebSearchTool(dc, tcfg.WebSearch))
1116+
}
1117+
11011118
if sm != nil {
11021119
tools = append(tools,
11031120
&skills.SkillLoadTool{Manager: sm},
@@ -1599,7 +1616,7 @@ func continueCmd(args []string) error {
15991616
"./.odek/skills",
16001617
)
16011618
}
1602-
tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, resolved.Transcription, resolved.Vision, store)
1619+
tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, toolConfig{Transcription: resolved.Transcription, Vision: resolved.Vision, WebSearch: resolved.WebSearch}, store)
16031620
var sandboxCleanup func() error
16041621

16051622
// MCP server tools

cmd/odek/main_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -203,7 +203,7 @@ func TestRun_NoAPIKey(t *testing.T) {
203203
}
204204

205205
func TestBuiltinTools(t *testing.T) {
206-
tools := builtinTools(danger.DangerousConfig{}, nil, nil, 3, "", config.TranscriptionConfig{}, config.VisionConfig{}, nil)
206+
tools := builtinTools(danger.DangerousConfig{}, nil, nil, 3, "", toolConfig{}, nil)
207207
if len(tools) == 0 {
208208
t.Fatal("builtinTools() returned empty slice")
209209
}

cmd/odek/mcp.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ Flags:
7373
}
7474

7575
// Build tools
76-
toolSet := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, config.TranscriptionConfig{}, config.VisionConfig{}, nil)
76+
toolSet := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, toolConfig{WebSearch: resolved.WebSearch}, nil)
7777

7878
// MCP server tools — connect and discover before sandbox
7979
var mcpCleanup func()

cmd/odek/repl.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ func replCmd(args []string) error {
7777
"./.odek/skills",
7878
)
7979
}
80-
tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, config.TranscriptionConfig{}, config.VisionConfig{}, nil)
80+
tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, toolConfig{WebSearch: resolved.WebSearch}, nil)
8181
var sandboxCleanup func() error
8282

8383
// MCP server tools

cmd/odek/schedule.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -570,7 +570,7 @@ func runTaskHeadless(ctx context.Context, resolved config.ResolvedConfig, system
570570
resolved.Dangerous.NonInteractive = &deny
571571
}
572572

573-
tools := builtinTools(resolved.Dangerous, nil, nil, resolved.MaxConcurrency, resolved.APIKey, resolved.Transcription, resolved.Vision, nil)
573+
tools := builtinTools(resolved.Dangerous, nil, nil, resolved.MaxConcurrency, resolved.APIKey, toolConfig{Transcription: resolved.Transcription, Vision: resolved.Vision, WebSearch: resolved.WebSearch}, nil)
574574
tools = append(tools, mcpTools...)
575575

576576
// Capture cumulative token usage from the final iteration so the Runner

cmd/odek/serve.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -267,7 +267,7 @@ func newServeAgent(resolved config.ResolvedConfig, system string, sendFn func(v
267267
approver := newWSApprover(sendFn)
268268
resolved.Dangerous.Approver = approver
269269

270-
tools := builtinTools(resolved.Dangerous, sm, approver, resolved.MaxConcurrency, resolved.APIKey, config.TranscriptionConfig{}, config.VisionConfig{}, nil)
270+
tools := builtinTools(resolved.Dangerous, sm, approver, resolved.MaxConcurrency, resolved.APIKey, toolConfig{WebSearch: resolved.WebSearch}, nil)
271271

272272
// Find the delegateTasksTool to wire up sub-agent log streaming
273273
var subagentTool *delegateTasksTool

cmd/odek/subagent.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -291,7 +291,7 @@ func subagentCmd(args []string) error {
291291
"./.odek/skills",
292292
)
293293
}
294-
tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, config.TranscriptionConfig{}, config.VisionConfig{}, nil)
294+
tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, toolConfig{WebSearch: resolved.WebSearch}, nil)
295295
var sandboxCleanup func() error
296296

297297
// MCP server tools

cmd/odek/subagent_contract_test.go

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@ import (
1212
"time"
1313

1414
"github.com/BackendStack21/odek"
15-
"github.com/BackendStack21/odek/internal/config"
1615
"github.com/BackendStack21/odek/internal/danger"
1716
"github.com/BackendStack21/odek/internal/llm"
1817
)
@@ -320,7 +319,7 @@ func TestSubagent_ExitCodeThree(t *testing.T) {
320319
// ── 4. delegate_tasks Tool Schema ───────────────────────────────────
321320

322321
func TestDelegateTasksTool_Exists(t *testing.T) {
323-
tools := builtinTools(danger.DangerousConfig{}, nil, nil, 3, "", config.TranscriptionConfig{}, config.VisionConfig{}, nil)
322+
tools := builtinTools(danger.DangerousConfig{}, nil, nil, 3, "", toolConfig{}, nil)
324323
if len(tools) == 0 {
325324
t.Fatal("builtinTools() returned empty slice")
326325
}
@@ -338,7 +337,7 @@ func TestDelegateTasksTool_Exists(t *testing.T) {
338337
}
339338

340339
func TestDelegateTasksTool_HasSchema(t *testing.T) {
341-
tools := builtinTools(danger.DangerousConfig{}, nil, nil, 3, "", config.TranscriptionConfig{}, config.VisionConfig{}, nil)
340+
tools := builtinTools(danger.DangerousConfig{}, nil, nil, 3, "", toolConfig{}, nil)
342341

343342
var tool odek.Tool
344343
for _, t2 := range tools {
@@ -432,7 +431,7 @@ func TestDelegateTasksTool_HasSchema(t *testing.T) {
432431
}
433432

434433
func TestDelegateTasksTool_Description(t *testing.T) {
435-
tools := builtinTools(danger.DangerousConfig{}, nil, nil, 3, "", config.TranscriptionConfig{}, config.VisionConfig{}, nil)
434+
tools := builtinTools(danger.DangerousConfig{}, nil, nil, 3, "", toolConfig{}, nil)
436435

437436
var tool odek.Tool
438437
for _, t2 := range tools {

0 commit comments

Comments
 (0)