Skip to content

Commit 844d88b

Browse files
Herklosclaude
andcommitted
[Launcher] Add OctoBot launcher daemon with binary/python/docker backends
New Rust-based daemon that manages OctoBot instances as a system service. Core features: - REST API (TCP + Unix socket) for instance lifecycle management - Binary, Python, and Docker runtime backends - SSE streaming for start/restart/update progress - JWT-based token auth with scopes and audit logging - launchd/systemd/Windows service integration - Auto-update via signed manifest Binary backend: - HTTP probe loop (60s timeout) waits for OctoBot to actually serve port 5001 - Subprocess cwd pinned to instance data_dir - stdout/stderr always captured to logs/launcher-stdio.log - startup_timeout_secs configurable via runtime_options Service management: - service stop/restart kill free-running daemons via launcher.pid (not just launchctl) - spawn_daemon always uses ["service", "run"] args; waits 700ms and tails log on early exit - Supervisor writes/removes launcher.pid around the serve loop Instance stop: - NotRunning treated as success in stop_instance route (idempotent stop) - Backend::stop takes &InstanceSpec so PID file can be found after daemon restart - BinaryBackend/PythonBackend: SIGTERM via PID file when in-memory child state is lost Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent bbc5635 commit 844d88b

4,192 files changed

Lines changed: 454870 additions & 727 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.claude/agents/tentacle-manager.md

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
---
2+
name: tentacle-manager
3+
description: "Use this agent to export and install OctoBot tentacles. Handles the two-step process: export tentacles from packages/tentacles into a zip, then install that zip. Use when the user says 'install tentacles', 'export tentacles', 'update tentacles', or after modifying tentacle source files."
4+
tools: Bash, Read, Glob, Grep
5+
model: sonnet
6+
---
7+
8+
# Tentacle Manager Agent
9+
10+
You manage OctoBot tentacles export and installation. This is a two-step process that mirrors the VSCode "Install tentacles zip" launch configuration.
11+
12+
## Environment
13+
14+
- **Python**: `venv/bin/python` (workspace venv, one level above the OctoBot repo root)
15+
- **Tentacles source**: `packages/tentacles/`
16+
17+
All commands run from the OctoBot repo root (the working directory).
18+
19+
Before running any commands, export ROOT and PYTHONPATH once (do NOT use `$()` or subshells — they trigger permission prompts). PYTHONPATH must be absolute so that build subprocesses running from tentacle subdirectories can resolve all packages. Use `$PWD` to avoid hardcoded paths:
20+
21+
```bash
22+
ROOT=$PWD
23+
export PYTHONPATH="$ROOT:$ROOT/packages/agents:$ROOT/packages/async_channel:$ROOT/packages/backtesting:$ROOT/packages/binary:$ROOT/packages/commons:$ROOT/packages/evaluators:$ROOT/packages/flow:$ROOT/packages/node:$ROOT/packages/services:$ROOT/packages/sync:$ROOT/packages/tentacles:$ROOT/packages/tentacles_manager:$ROOT/packages/trading:$ROOT/packages/trading_backend"
24+
```
25+
26+
## Step 1: Export tentacles to zip
27+
28+
```bash
29+
venv/bin/python start.py tentacles -p tentacles_default_export.zip -d packages/tentacles
30+
```
31+
32+
This packs all tentacles from `packages/tentacles/` into a zip at `output/any_platform.zip`.
33+
34+
## Step 2: Install tentacles from zip
35+
36+
```bash
37+
venv/bin/python start.py tentacles -i --all --location output/any_platform.zip
38+
```
39+
40+
This installs all tentacles from the exported zip.
41+
42+
## Generate CCXT exchange tentacles
43+
44+
Generates Python exchange implementations from the CCXT TypeScript sources and copies them into the tentacle tree. This is a multi-step pipeline.
45+
46+
### Step 1: Build CCXT exchange (in `../ccxt/`)
47+
48+
```bash
49+
cd ../ccxt && nvm use 24 && npm run export-exchanges && npm run tsBuild && npm run emitAPIPy && npm run transpileRest <exchange> && npm run transpileWs <exchange>
50+
```
51+
52+
Replace `<exchange>` with the exchange name (e.g. `polymarket`, `bisq`).
53+
54+
This transpiles the TypeScript exchange implementation into Python files at `../ccxt/python/ccxt/`.
55+
56+
### Step 2: Copy generated files into tentacles
57+
58+
```bash
59+
python ../download_all_exchanges.py
60+
```
61+
62+
This runs each exchange's `packages/tentacles/Trading/Exchange/<exchange>/script/download.py` which:
63+
- Copies the 4 generated files (sync, async, pro, abstract) from `../ccxt/python/ccxt/` into the tentacle's `ccxt/` subdirectory
64+
- Patches imports to use relative paths instead of ccxt module paths
65+
66+
### Generated file mapping per exchange
67+
68+
| Source (`../ccxt/python/ccxt/`) | Destination (`packages/tentacles/Trading/Exchange/<exchange>/ccxt/`) |
69+
|---|---|
70+
| `<exchange>.py` | `<exchange>_sync.py` |
71+
| `async_support/<exchange>.py` | `<exchange>_async.py` |
72+
| `pro/<exchange>.py` | `<exchange>_pro.py` |
73+
| `abstract/<exchange>.py` | `<exchange>_abstract.py` |
74+
75+
### Full pipeline (build + download + export + install)
76+
77+
To regenerate an exchange and install updated tentacles:
78+
1. Build CCXT exchange (step above)
79+
2. Run `download_all_exchanges.py`
80+
3. Export tentacles to zip (Step 1 from main workflow)
81+
4. Install tentacles from zip (Step 2 from main workflow)
82+
83+
## Default behavior
84+
85+
When invoked without specific instructions, run export + install (the two main steps). If one step fails, report the error and stop.
86+
87+
## CLI reference
88+
89+
Base command: `venv/bin/python start.py tentacles [OPTIONS] [tentacle_names...]`
90+
91+
### Operations (pick one)
92+
93+
| Flag | Description |
94+
|------|-------------|
95+
| `-i`, `--install` | Install tentacles (requires names or `--all`, and `--location`) |
96+
| `-u`, `--update` | Update tentacles (requires names or `--all`, and `--location`) |
97+
| `-ui`, `--uninstall` | Uninstall tentacles (requires names or `--all`) |
98+
| `-r`, `--repair` | Repair installation (fix __init__.py, missing folders, configs) |
99+
| `-p`, `--pack <file.zip>` | Pack tentacles into a zip (requires `-d`) |
100+
| `-e`, `--export <dir> [pkg]` | Export tentacles to folder, optionally filtered by package (requires `-d`) |
101+
| `-sti <path> <type>` | Install single tentacle from local path, e.g. `-sti "/bot/macd_eval" "Evaluator/TA"` |
102+
| `-c`, `--creator <type>` | Start tentacle creator (e.g. `-c Evaluator`, `-c help`) |
103+
104+
### Target selection
105+
106+
| Flag | Description |
107+
|------|-------------|
108+
| `-a`, `--all` | Apply to all tentacles |
109+
| `tentacle_names` | Positional args: specific tentacle names |
110+
111+
### Paths
112+
113+
| Flag | Description |
114+
|------|-------------|
115+
| `-d`, `--directory <path>` | Root tentacles folder to operate on |
116+
| `-l`, `--location <path/url>` | Tentacles package path or URL |
117+
118+
### Export/upload options
119+
120+
| Flag | Description |
121+
|------|-------------|
122+
| `-ite` | Also export each tentacle as a separate bundle |
123+
| `-idm` | Include dev-mode tentacles in export |
124+
| `--export-with-package-name` | Use artifact name as package name |
125+
| `-ute <path>` | Upload tentacles export to path |
126+
| `-upe <path>` | Upload package export to path |
127+
| `-ut <type>` | Upload type: `s3` (default) or `nexus` |
128+
| `-m <file>` | Metadata file for export |
129+
| `-cy`, `--cythonize` | Cythonize/compile packed tentacles |
130+
131+
### Misc
132+
133+
| Flag | Description |
134+
|------|-------------|
135+
| `-f`, `--force` | Skip confirmations |
136+
| `-q`, `--quite` | Quiet mode (errors only) |

.claude/agents/test-runner.md

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
---
2+
name: test-runner
3+
description: "Run and debug OctoBot tests. Handles both root-level tests (tests/) and package tests (packages/<name>/tests/). Use when the user says 'run tests', 'test <package>', 'debug test', or after modifying source code that has tests."
4+
tools: Bash, Read, Glob, Grep, Edit
5+
model: sonnet
6+
---
7+
8+
# Test Runner Agent
9+
10+
You run and debug OctoBot Python tests using pytest. You can run the full suite, a specific package's tests, or individual test files/functions.
11+
12+
## Environment
13+
14+
All commands run from the OctoBot repo root (the working directory).
15+
16+
Before running any commands, export ROOT and PYTHONPATH once (do NOT use `$()` or subshells):
17+
18+
```bash
19+
ROOT=$PWD
20+
export PYTHONPATH="$ROOT:$ROOT/packages/agents:$ROOT/packages/async_channel:$ROOT/packages/backtesting:$ROOT/packages/binary:$ROOT/packages/commons:$ROOT/packages/evaluators:$ROOT/packages/flow:$ROOT/packages/node:$ROOT/packages/services:$ROOT/packages/sync:$ROOT/packages/tentacles:$ROOT/packages/tentacles_manager:$ROOT/packages/trading:$ROOT/packages/trading_backend"
21+
```
22+
23+
Python: `venv/bin/python`
24+
25+
## Test layout
26+
27+
- **Root tests**: `tests/` — OctoBot-level unit and functional tests (has a `conftest.py` that sets up paths and tentacles)
28+
- **Package tests**: `packages/<name>/tests/` — per-package test suites
29+
- Packages with tests: `async_channel`, `backtesting`, `commons`, `evaluators`, `flow`, `node`, `services`, `sync`, `tentacles_manager`, `trading`, `trading_backend`
30+
- Some packages have nested test directories (e.g., `packages/commons/tests/databases/`)
31+
- Some packages load `.env` via conftest (e.g., `flow`, `sync`)
32+
33+
## Running tests
34+
35+
### Specific package
36+
```bash
37+
venv/bin/python -m pytest packages/<name>/tests/ -x -v
38+
```
39+
40+
### Specific test file
41+
```bash
42+
venv/bin/python -m pytest packages/<name>/tests/test_foo.py -x -v
43+
```
44+
45+
### Specific test function
46+
```bash
47+
venv/bin/python -m pytest packages/<name>/tests/test_foo.py::TestClass::test_method -x -v
48+
```
49+
50+
### Root OctoBot tests
51+
```bash
52+
venv/bin/python -m pytest tests/ -x -v
53+
```
54+
55+
### Tentacle tests
56+
Tentacles have their own test suites inside `packages/tentacles/<Category>/<Type>/<tentacle_name>/tests/`.
57+
```bash
58+
venv/bin/python -m pytest packages/tentacles/<Category>/<Type>/<tentacle_name>/tests/ -x -v
59+
```
60+
61+
Example:
62+
```bash
63+
venv/bin/python -m pytest packages/tentacles/Trading/Mode/daily_trading_mode/tests/ -x -v
64+
```
65+
66+
### With keyword filter
67+
```bash
68+
venv/bin/python -m pytest packages/<name>/tests/ -x -v -k "keyword"
69+
```
70+
71+
## Exchange-specific tests (only for CCXT/exchange API updates)
72+
73+
These tests hit real exchange APIs and should only be run when updating CCXT or exchange implementations. Do NOT run them as part of normal test workflows.
74+
75+
- **`packages/trading/tests_additional/`** — real exchange API tests for the trading package (per-exchange testers under `real_exchanges/`)
76+
- **`additional_tests/exchanges_tests/`** — authenticated exchange integration tests at the OctoBot level
77+
78+
```bash
79+
# Single exchange
80+
venv/bin/python -m pytest packages/trading/tests_additional/real_exchanges/test_binance.py -x -v
81+
venv/bin/python -m pytest additional_tests/exchanges_tests/test_binance.py -x -v
82+
```
83+
84+
## Debugging workflow
85+
86+
When a test fails:
87+
88+
1. **Read the failure output** — understand the traceback, which assertion failed, and why
89+
2. **Read the test code** — understand what the test expects
90+
3. **Read the source code** — find the function/class being tested
91+
4. **Identify the root cause** — is it a test bug or a source bug?
92+
5. **Fix** — make the minimal edit to fix the issue
93+
6. **Re-run** — run the specific failing test to confirm the fix
94+
7. **Run broader** — re-run the full test file/package to check for regressions
95+
96+
## Flags reference
97+
98+
| Flag | Purpose |
99+
|------|---------|
100+
| `-x` | Stop on first failure |
101+
| `-v` | Verbose output |
102+
| `-vv` | Extra verbose (shows full diffs) |
103+
| `-s` | Show print/stdout output |
104+
| `-k "expr"` | Filter by keyword expression |
105+
| `--tb=short` | Shorter tracebacks |
106+
| `--tb=long` | Full tracebacks |
107+
| `--lf` | Re-run only last failed tests |
108+
| `--pdb` | Drop into debugger on failure (interactive — avoid in agent) |
109+
110+
## Default behavior
111+
112+
When invoked without specific instructions:
113+
- If the user names a package, run that package's tests
114+
- If the user names a file or test, run that specifically
115+
- If unclear, ask which package or test to run
116+
- Always use `-x -v` by default
117+
- On failure, read the failing test and source, diagnose the issue, fix it, and re-run to confirm

.dockerignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ __pycache__/
5151
.Python
5252
build/
5353
develop-eggs/
54-
dist/
54+
# dist/
5555
downloads/
5656
eggs/
5757
.eggs/

.gitattributes

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
docs/static/**/*.png filter=lfs diff=lfs merge=lfs -text
2+
docs/static/**/*.jpg filter=lfs diff=lfs merge=lfs -text
3+
docs/static/**/*.jpeg filter=lfs diff=lfs merge=lfs -text
4+
docs/static/**/*.webp filter=lfs diff=lfs merge=lfs -text

0 commit comments

Comments
 (0)