Skip to content

Commit d0c2854

Browse files
committed
v0.52.1 — Concise defaultSystem + machine-specific IDENTITY.md
defaultSystem cut from 124 lines (~1200 tokens) to 34 lines (~350 tokens): - Removed self-promotion (website, GitHub, philosophy) — agent doesn't need its own brochure - Removed generic LLM boilerplate (output discipline, ReAct scaffold verbiage) - Removed redundant tool reminders (tool descriptions already handle this) - Kept only odek-specific rules: tool name mappings, TDD workflow, batch tools, safety ~/.odek/IDENTITY.md created with machine-specific context: - Stack (Ubuntu 24.04, Go 1.26, Python 3.11, Node 26, Docker 29, 6C/11GB) - Workflow rules (TDD, -race, docs lockstep, CI verify) - Code principles (no new deps, local models only, bounded goroutines, O_NOFOLLOW) AGENTS.md restored (was accidentally deleted by sibling subagent) with current v0.52.1 state.
1 parent 69f448d commit d0c2854

3 files changed

Lines changed: 84 additions & 363 deletions

File tree

AGENTS.md

Lines changed: 34 additions & 206 deletions
Original file line numberDiff line numberDiff line change
@@ -12,237 +12,65 @@ It provides context about the project's architecture, conventions, and how to up
1212
- **Binary:** `odek` — single static binary, ~12 MB, instant startup.
1313
- **Config:** Five-layer priority: `~/.odek/secrets.env``~/.odek/config.json``./odek.json``ODEK_*` env vars → CLI flags.
1414
- **Benchmark:** AIEB v2.0 — 80.3% (highest published agent score on the Autonomous Intelligence Engineering Benchmark).
15-
- **Version:** v0.44.1 — see latest tag at https://github.com/BackendStack21/odek/releases
15+
- **Version:** v0.52.1 — see latest tag at https://github.com/BackendStack21/odek/releases
1616

1717
## Source Layout
1818

1919
```
2020
odek.go Public API (Config, New, Run, Close, ModelProfile, KnownProfiles, Tool interface)
21-
odek_test.go Tests for public API
2221
cmd/odek/
23-
main.go CLI entry point, flag parsing, commands, sandbox setup
22+
main.go CLI entry point, flag parsing, commands, sandbox setup, system prompt
2423
shell.go Built-in shell tool (local or docker exec; danger-gated)
2524
serve.go Web UI server (HTTP + WebSocket; @-resource completion)
2625
repl.go Interactive REPL with multi-turn session support
27-
repl_editor.go REPL inline editor helpers
28-
refs.go @-reference resolver (files, sessions)
2926
telegram.go Telegram bot command — wires odek agent into Telegram poller
3027
subagent.go Sub-agent command (--goal, --context, --task)
3128
subagent_tool.go delegate_tasks built-in tool (sub-agent spawning)
3229
browser_tool.go Built-in browser tool (HTTP fetch + headless navigation)
33-
file_tool.go Built-in file tools (read_file, write_file, search_files, patch)
34-
perf_tools.go Performance/parallelism tools (batch_read, batch_patch, math_eval, diff, multi_grep, etc.)
30+
file_tool.go Built-in file tools (read_file, write_file, search_files, patch, batch_read, glob, file_info)
31+
perf_tools.go Performance/parallelism tools (batch_patch, parallel_shell, http_batch, math_eval, diff, count_lines, multi_grep, json_query, tree, checksum, sort, head_tail, base64, tr, word_count)
3532
mcp.go MCP server implementation (stdio + SSE transport)
36-
wsapprover.go WebSocket-based approval bridge
37-
ui/index.html Single-page Web UI (~770 LOC, vanilla JS + CSS)
38-
*_test.go CLI, subagent, contract, E2E, and integration tests
33+
transcribe_tool.go Whisper.cpp audio transcription
34+
session_search_tool.go Session search tool
35+
*_test.go 130+ unit tests covering all tools
3936
internal/
40-
auth/ Interactive approval system (confirm/cancel for dangerous tools)
41-
config/loader.go Config file loading, env vars, secrets.env, priority merge
42-
danger/classifier.go Command/URL classification for security gating
43-
display/display.go ANSI terminal output helpers
44-
llm/client.go OpenAI-compatible HTTP client with reasoning_content support
45-
loop/loop.go ReAct engine: observe → think → parallel-act → repeat
46-
mcp/server.go MCP server handler (tools/list, tools/call, SSE streaming)
47-
mcpclient/client.go MCP client (connect to external MCP servers)
37+
llm/ OpenAI-compatible HTTP client with reasoning_content support
38+
loop/ ReAct engine: observe → think → parallel-act → repeat
39+
tool/ Thread-safe tool registry, clarify.go
40+
danger/ Command/URL classification for security gating
41+
auth/ Interactive approval system
4842
memory/ MemoryManager (facts, buffer, episodes, merge, scan, LLM search)
49-
narrate/narrate.go Narrator — LLM-powered emoji-rich progress messages for engaging/enhance mode
50-
redact/redact.go Secret redaction (13-pattern scanner: API keys, tokens, credentials)
51-
render/render.go Terminal output with model label, color, narrator message support
52-
resource/resource.go @-reference resolver for file/session completion
53-
sandbox/manager.go Docker sandbox lifecycle (start, exec, stop, cleanup)
54-
session/session.go Session store (CRUD, trim, cleanup, compact JSON)
43+
session/ Session store (CRUD, trim, cleanup, compact JSON)
5544
skills/ Skill system (types, loader, triggers, self-improve, curator, import)
56-
telegram/ Telegram bot: bot.go, poller.go, handler.go, commands.go, session.go, plan.go, download.go
57-
tool/registry.go Thread-safe tool registry, clarify.go
58-
transport/http.go Shared HTTP transport with connection pooling
59-
ws/ws.go RFC 6455 WebSocket framing (~200 LOC)
45+
config/ Config file loading, env vars, secrets.env, priority merge
46+
telegram/ Telegram bot: bot.go, poller.go, handler.go, commands.go, session.go
47+
render/ Terminal output and narrator support
48+
narrate/ LLM-powered emoji-rich progress messages
49+
redact/ Secret redaction (13-pattern scanner)
50+
mcp/ MCP server handler (tools/list, tools/call, SSE streaming)
51+
mcpclient/ MCP client (connect to external MCP servers)
52+
sandbox/ Docker sandbox lifecycle
53+
transport/ Shared HTTP transport with connection pooling
54+
ws/ RFC 6455 WebSocket framing
6055
docs/ Documentation (CLI, API, CONFIG, MCP, MEMORY, TELEGRAM, etc.)
6156
benchmark/ AIEB v2.0 benchmark suite (9 tasks, 4 tiers, automated scoring)
6257
```
6358

6459
## How It Works
6560

66-
### 1. Agent Loop (`internal/loop/loop.go`)
61+
### Agent Loop (`internal/loop/loop.go`)
6762
ReAct cycle: observe → think → act → repeat.
6863
- LLM returns tool calls or a final answer.
69-
- **Parallel tool execution** — multiple independent tool calls run concurrently with bounded concurrency (`max_tool_parallel`, default: 4).
70-
- **Batch approval gate** — when multiple tools need approval, all risky tools are shown at once in a single inline keyboard.
71-
- **Interaction modes** — three modes control tool-call display:
72-
- `engaging` — LLM-narrated emoji-rich progress (default)
73-
- `enhance` — like engaging but narration persists after response
74-
- `verbose` — raw tool names, args, and results (debug-friendly)
75-
- **Per-tool trace messages** — each tool call generates a reasoning-backed trace in Telegram.
76-
- Max 90 iterations by default (`--max-iter`).
64+
- **Parallel tool execution** — multiple independent tool calls run concurrently (max_tool_parallel, default: 4).
65+
- **Batch approval gate** — multiple risky tools shown at once in a single prompt.
66+
- **Interaction modes** — engaging (narrated), enhance (persistent), verbose (raw), off.
67+
- Max 300 iterations by default.
7768

78-
### 2. Tools
69+
### Tools
70+
All built-in tools with zero subprocess forks: batch_read, batch_patch, parallel_shell, http_batch, math_eval, diff, count_lines, multi_grep, json_query, tree, checksum, sort, head_tail, base64, tr, word_count, transcribe, browser, read_file, write_file, search_files, patch, shell, delegate_tasks, session_search.
7971

80-
**Core** (always available):
81-
`read_file`, `write_file`, `search_files`, `patch`, `shell`, `browser`, `memory`, `clarify`, `delegate_tasks`, `send_message`
72+
### Identity
73+
System prompt is loaded by priority: `--system` flag > `~/.odek/IDENTITY.md` > compiled-in defaultSystem. The default is a concise identity focused on TDD workflow, tool discipline, and safety rules.
8274

83-
**Performance/parallelism** (added v0.38-v0.40):
84-
| Category | Tools |
85-
|----------|-------|
86-
| Parallel batch | `batch_read` — N files in 1 call, `batch_patch` — N edits atomically, `parallel_shell` — N commands true-parallel, `http_batch` — N URLs parallel fetch |
87-
| Zero-fork data | `math_eval` — native arithmetic, `diff` — LCS diff, `json_query` — dot-path JSON, `tr` — text transform, `base64` — encode/decode |
88-
| File analysis | `glob` — fast glob find, `file_info` — stat metadata, `count_lines` — streaming line count, `word_count` — streaming word count, `checksum` — SHA256/SHA1/MD5, `sort` — sort lines, `head_tail` — first/last N lines |
89-
| Multi-pattern | `multi_grep` — N regex patterns parallel, `tree` — structured directory tree |
90-
| Audio | `transcribe` — local whisper.cpp audio transcription with auto-OGG→WAV conversion via ffmpeg |
91-
92-
All gated by the `danger` security classifier with three actions: allow, deny, prompt.
93-
- `shell`: Classifies commands into risk classes (safe, local_write, system_write, destructive, network_egress, code_execution, install, blocked).
94-
- `send_message`: Sends text/photo/document/voice to the Telegram chat with inline keyboard support.
95-
- All file tools: `O_NOFOLLOW` on opens, `danger.ClassifyPath` per path, atomic temp+rename for writes.
96-
- All network tools: `danger.ClassifyURL` per URL, configurable network egress gate.
97-
98-
### 3. Skills
99-
Trigger-matched `SKILL.md` files loaded on-demand via lazy injection. Auto-learns from patterns every session.
100-
- Stored in `~/.odek/skills/` (user) and `skills/` (project).
101-
- `odek skill curate` analyzes quality, staleness, and trigger overlap.
102-
- Skills with `auto_load: true` are injected as passive reference; `auto_load: false` + strong triggers → lazy injection as system message (stronger signal).
103-
104-
### 4. Memory (`internal/memory/`)
105-
Three-tier persistence:
106-
- **Facts** — Markdown files on disk, merge-on-write via go-vector cosine similarity (~80% fewer LLM calls).
107-
- **Session buffer** — Full conversation history.
108-
- **Episodes** — Searchable summaries extracted by LLM on session end.
109-
- go-vector RP is ephemeral — rebuilt from text on every write, no embedding state to persist.
110-
111-
### 5. Config System (`internal/config/loader.go`)
112-
Five-layer priority chain:
113-
```
114-
0. ~/.odek/secrets.env ← Auto-loaded into process environment
115-
1. ~/.odek/config.json ← Global defaults
116-
2. ./odek.json ← Project overrides
117-
3. ODEK_* env vars ← Runtime overrides
118-
4. CLI flags ← Explicit invocation (highest priority)
119-
```
120-
- `${VAR}` substitution in JSON config files.
121-
- `interaction_mode` field: `engaging` | `verbose` | `enhance` | `off`.
122-
- `max_tool_parallel`: bounded concurrency for tool execution.
123-
- `--deliver` CLI flag delivers agent response to Telegram default chat (for cron).
124-
125-
### 6. MCP (Two-Way)
126-
- **Server:** `odek mcp` exposes native tools via stdio or SSE to any MCP client (Claude Code, Cursor).
127-
- **Client:** Connects to external MCP servers (Playwright, Fetch, databases) via `mcp_servers` config.
128-
129-
### 7. Telegram Bot (`cmd/odek/telegram.go` + `internal/telegram/`)
130-
Full-featured bot with long-polling:
131-
- Slash commands: `/start`, `/new`, `/mode`, `/plan`, `/status`, `/cancel`, `/reset`, `/budget`
132-
- Interaction mode-aware progress display (verbose trace edits or engaging narration)
133-
- Voice message transcription, photo analysis, file attachments
134-
- Inline keyboard for approvals, clarifications, and cancel
135-
- `send_message` tool — agent can send structured messages, files, and keyboards back to chat
136-
- `--deliver` flag delivers final response to configured default chat (cron integration)
137-
- Per-chat session management with TTL-based cleanup
138-
139-
### 8. Web UI
140-
`odek serve` — single-page browser interface with WebSocket streaming, `@` resource completion, token economics display, drag-and-drop file attachments, inline loading. Built from Go's embed — zero npm.
141-
142-
### 9. Dynamic Model Discovery
143-
On startup, odek queries the LLM provider's `GET /models` endpoint and auto-discovers model capabilities (max context, thinking support, parallel tool support). This replaces hardcoded model profiles — the agent adapts to whatever the provider exposes.
144-
145-
### 10. Secret Redaction (`internal/redact/`)
146-
Active at two layers:
147-
- Tool outputs are redacted before the LLM sees them (ReAct loop).
148-
- Session files are sanitized on save (defense-in-depth).
149-
- 13-pattern scanner covers OpenAI, GitHub, AWS, JWT, private keys, Slack, Stripe, Google, Twilio, generic API keys, Auth headers, and env var credentials.
150-
151-
## Key Conventions
152-
153-
- **No external frameworks.** Stdlib + four focused packages (go-vector, yaml.v3 for MCP, goja for JS, chroma for highlight).
154-
- **Single binary.** Everything compiles into one static executable (~12 MB).
155-
- **Tests live alongside code.** `*_test.go` files in the same package, never in a separate `test/` directory.
156-
- **Test data** uses `t.TempDir()` for isolation.
157-
- **CLI commands** follow the pattern: `cobra.Command` with `RunE` handler. Flag parsing is in `main.go`.
158-
- **New config fields** require 11-point wiring: FileConfig → CLIFlags → ResolvedConfig → loadFile → overlayFile → env var → CLI flag → resolved mapping → flag parsing → pass-through → call sites. Missing any step = silent failure.
159-
- **Build before test.** Always `go build ./...` first to catch compile errors before running tests.
160-
- **Use `odek run` for analysis.** Prefer delegating complex code questions to odek itself (Pattern 2 in odek-file-qa skill).
161-
- **The odek Tool interface** is `Call(args string) (string, error)` and `Schema() any` — NOT `Execute()` or `json.RawMessage`.
162-
- **MarkdownV2 reserved chars** must be escaped in Telegram messages. The escape function handles `_*[]()~>#+-=|{}.!`.
163-
- **Inline keyboard buttons** use `*telegram.InlineKeyboardMarkup`, not raw `map[string]any`.
164-
- **Skill auto_load: false** with strong trigger keywords is STRONGER than `auto_load: true` — lazy injection as a system message right before the user message beats passive reference injection.
165-
- **Patch old_string must be unique** in the file. For repeated patterns (class="card"), include enough surrounding context (e.g., the card's inner content).
166-
- **After modifying shared files in parallel subagents**, always run `go build ./... && go test ./...` to catch integration errors. Subagent patch conflicts produce `Could not find a match for old_string` warnings.
167-
168-
## Testing
169-
170-
```bash
171-
# Full test suite
172-
go test ./... -count=1
173-
174-
# With race detector (recommended for loop/tool/telegram changes)
175-
go test -race ./... -count=1
176-
177-
# Specific package with verbose output
178-
go test -v ./internal/loop/ -run TestTrimContext -count=1
179-
180-
# Benchmark
181-
go test -bench=. -benchmem ./internal/loop/
182-
183-
# Run integration/E2E tests (require Docker and/or network)
184-
go test -v -run "TestTelegram|TestMCP" ./cmd/odek/ -count=1
185-
```
186-
187-
### Test Conventions
188-
- Coverage targets: 70%+ for new packages, no regression on existing.
189-
- `httptest.Server` for mocking HTTP endpoints (LLM, Telegram, MCP).
190-
- `t.Setenv` for environment variable overrides (no global env mutation).
191-
- Parallel tests must use `-race` to catch data races on shared slices.
192-
- `timedTool` pattern for testing parallel execution with configurable delays.
193-
- Secret redaction tests must use runtime string concatenation to avoid triggering GitHub's push-time secret scanner.
194-
195-
## Common Pitfalls
196-
197-
- **Session files are JSON in `~/.odek/sessions/`.** Corrupt data is handled gracefully with fallback scan.
198-
- **The Telegram bot uses long-polling (no webhooks),** built on stdlib `net/http`.
199-
- **Background subprocesses don't inherit `ODEK_API_KEY`.** When running odek as a subprocess, the spawned process may not have the key. Pass it explicitly or save to `/tmp/.aieb_odek_key`.
200-
- **`go build .``go build ./cmd/odek`.** The bare `.` builds the library package (an ar archive, not an executable).
201-
- **Config partial overlay + Go zero values = missing defaults.** Always start from `DefaultConfig()` and overlay only non-zero user fields. Sentinel values or pointer fields are needed to distinguish "unset" from "explicitly zero."
202-
- **`delegate_tasks` sub-agents have a 120s default timeout.** Override via `subagent.timeout_seconds` config.
203-
- **Parallel tool tests** use `timedTool` with configurable delays — always run with `-race` to catch data races on the results slice.
204-
- **Batch approval gate** checks `e.approver != nil && len(result.ToolCalls) > 1` — single-tool responses skip the gate. When adding a new approver, implement `SetTrustAll(bool)` to benefit from batch trust cascade.
205-
- **SetTrustAll is bounded by `defer`** — it's enabled before Phase 2 and disabled when `runLoop` returns. The mockApprover's `trustAll` field will be `false` after `Run()` returns.
206-
- **Approval callbacks must route BEFORE `OnCallbackQuery`.** If it falls through to the generic handler, the approver's channel never receives the response → 120s timeout deadlock.
207-
- **Async dispatch prevents update loop deadlock.** Agent loop processing MUST run in a goroutine from the callback handler.
208-
- **`DownloadFile` hardcodes the production API URL.** Add `FileBaseURL` to Bot for testability; both `BaseURL` and `FileBaseURL` must be overridden in tests.
209-
- **Review every interface path for goroutine safety** after writing a new subsystem (Telegram, approvals, parallel tools). Checklist: mutable shared state, mutex protection, per-chat state isolation, `httptest.Server` URL configurability.
210-
- **`reasoning_content` echo for DeepSeek models.** If the API returns `reasoning_content`, it must be echoed back in subsequent assistant messages or the next call fails with `400 Bad Request`.
211-
- **`frame-ancestors` in `<meta>` produces a browser warning.** CSP meta tags ignore this directive — set it via HTTP headers or accept the cosmetic warning.
212-
- **Rebuild the binary after source changes.** `go build -o /usr/local/bin/odek ./cmd/odek`. Source changes don't take effect until rebuilt.
213-
214-
## Release Checklist
215-
216-
1. `go build ./...` — compile check
217-
2. `go test ./... -count=1` — full test suite
218-
3. `go test -race ./... -count=1` — race detection
219-
4. Update `docs/CHANGELOG.md` with notable changes
220-
5. Commit and push to `main`
221-
6. Tag: `git tag -a vX.Y.Z -m "vX.Y.Z: short description"`
222-
7. Push tag: `git push origin vX.Y.Z`
223-
8. Create GitHub release: `gh release create vX.Y.Z --title "vX.Y.Z" --notes "..."`
224-
9. Restart Telegram bot: `nohup bash build-and-restart-telegram.sh --restart-only > /tmp/odek-restart.log 2>&1 &`
225-
10. Verify: `sleep 3 && ps aux | grep "odek telegram"`
226-
227-
## Documentation Files
228-
229-
| File | Covers |
230-
|------|--------|
231-
| `docs/CLI.md` | All commands, flags, file attachments, @-references |
232-
| `docs/CONFIG.md` | Five-layer priority, secrets.env, all config fields |
233-
| `docs/API.md` | Go SDK: Agent, Tools, memory, multi-turn, custom tools |
234-
| `docs/TELEGRAM.md` | Bot architecture, config, API client, session management |
235-
| `docs/MEMORY.md` | Three-tier memory, episodes, merge-on-write |
236-
| `docs/MCP.md` | Two-way MCP (server + client), SSE transport |
237-
| `docs/SANDBOXING.md` | Docker isolation, config, security model |
238-
| `docs/SECURITY.md` | Prompt injection defense, danger classifier, redaction |
239-
| `docs/LEARNING.md` | Skill auto-learning, curation, trigger system |
240-
| `docs/SESSIONS.md` | Save, resume, list, trim, cleanup |
241-
| `docs/SUBAGENTS.md` | delegate_tasks architecture, parallel sub-agents |
242-
| `docs/PROVIDERS.md` | Model profiles, provider configs, model discovery |
243-
| `docs/WEBUI.md` | Web UI architecture, WebSocket streaming |
244-
| `docs/CACHING.md` | Response caching, cache control |
245-
| `docs/CHEATSHEET.md` | Quick reference: commands, config, memory, env vars |
246-
| `docs/DEVELOPMENT.md` | Building, testing, contributing |
247-
| `docs/DAILY-WORKER.md` | Cron/daily worker integration patterns |
248-
| `AGENTS.md` | This file — agent auto-load context |
75+
### Platform Support
76+
CLI, REPL, Web UI, Telegram bot — all in a single binary.

0 commit comments

Comments
 (0)