|
| 1 | +--- |
| 2 | +description: Architecture overview and module map for contributors and AI coding agents working on ralphify. |
| 3 | +--- |
| 4 | + |
| 5 | +# Codebase Map |
| 6 | + |
| 7 | +Quick orientation guide for anyone working on this codebase — human contributors and AI coding agents alike. |
| 8 | + |
| 9 | +## What this project is |
| 10 | + |
| 11 | +Ralphify is a CLI tool (`ralph`) that runs AI coding agents in autonomous loops. It reads a prompt file, pipes it to an agent command (e.g. `claude -p`), waits for it to finish, then repeats. Each iteration gets a fresh context window. Progress is tracked through git commits. |
| 12 | + |
| 13 | +The core loop is simple. The complexity lives in **prompt assembly** — resolving contexts, instructions, and check failures into the prompt before each iteration. |
| 14 | + |
| 15 | +## Directory structure |
| 16 | + |
| 17 | +``` |
| 18 | +src/ralphify/ # All source code |
| 19 | +├── __init__.py # Version detection + app entry point |
| 20 | +├── cli.py # CLI commands (init, run, status, new, prompts) — delegates to engine for the loop |
| 21 | +├── engine.py # Core run loop with structured event emission (extracted from cli.py) |
| 22 | +├── manager.py # Multi-run orchestration for the UI layer (concurrent runs via threads) |
| 23 | +├── checks.py # Discover and run validation checks, format failures |
| 24 | +├── contexts.py # Discover and run dynamic data contexts, resolve into prompt |
| 25 | +├── instructions.py # Discover and resolve static text instructions |
| 26 | +├── prompts.py # Named prompt discovery and resolution |
| 27 | +├── resolver.py # Template placeholder resolution (shared by contexts + instructions) |
| 28 | +├── detector.py # Auto-detect project type from manifest files |
| 29 | +├── _runner.py # Execute shell commands with timeout and capture output |
| 30 | +├── _frontmatter.py # Parse YAML frontmatter from markdown primitives, discover primitives |
| 31 | +├── _templates.py # Scaffold templates for init and new commands |
| 32 | +├── _console_emitter.py # Rich console renderer for run-loop events (ConsoleEmitter) |
| 33 | +├── _events.py # Event types and emitter protocol (NullEmitter, QueueEmitter) |
| 34 | +├── _output.py # Combine/truncate stdout+stderr |
| 35 | +└── ui/ # Web UI layer (optional — not part of the core CLI) |
| 36 | + ├── app.py # FastAPI application setup |
| 37 | + ├── api/ # REST API endpoints |
| 38 | + ├── models.py # Pydantic models for API |
| 39 | + ├── persistence.py # SQLite persistence via aiosqlite |
| 40 | + ├── frontend/ # Frontend assets (HTML, JS, CSS) |
| 41 | + └── static/ # Static files served by the UI |
| 42 | +
|
| 43 | +tests/ # Pytest tests — one test file per module |
| 44 | +docs/ # MkDocs site (Material theme) — user-facing documentation |
| 45 | +docs/contributing/ # Contributor documentation (this section) |
| 46 | +.github/workflows/ |
| 47 | +├── test.yml # Run tests on push to main and PRs (Python 3.11–3.13) |
| 48 | +├── docs.yml # Deploy docs to GitHub Pages on push to main |
| 49 | +└── publish.yml # Publish to PyPI on release (with test gate) |
| 50 | +``` |
| 51 | + |
| 52 | +## Architecture: how the pieces connect |
| 53 | + |
| 54 | +The CLI entry point is `cli.py:run()`, which parses options, resolves the prompt via the priority chain, and delegates to `engine.py:run_loop()` for the actual iteration cycle. The engine emits structured events via an `EventEmitter`, making the same loop reusable from both CLI and web UI contexts. |
| 55 | + |
| 56 | +``` |
| 57 | +ralph run |
| 58 | + │ |
| 59 | + ├── cli.py:run() — parse options, resolve prompt, print banner |
| 60 | + │ ├── Load config from ralph.toml |
| 61 | + │ ├── Resolve prompt via priority chain (--prompt > name > --prompt-file > toml > root) |
| 62 | + │ └── Build RunConfig and call engine.run_loop() |
| 63 | + │ |
| 64 | + └── engine.py:run_loop(config, state, emitter) |
| 65 | + ├── Discover checks, contexts, instructions from .ralph/ |
| 66 | + └── Loop: |
| 67 | + ├── Read PROMPT.md (or use ad-hoc text) |
| 68 | + ├── Run contexts → resolve {{ contexts.* }} placeholders |
| 69 | + ├── Resolve {{ instructions.* }} placeholders |
| 70 | + ├── Append check failures from previous iteration (if any) |
| 71 | + ├── Pipe assembled prompt to agent command via subprocess |
| 72 | + ├── Emit iteration events (started, completed, failed, timed_out) |
| 73 | + ├── Run checks → emit check events → format failures for next iteration |
| 74 | + ├── Handle pause/resume/stop/reload requests via RunState |
| 75 | + └── Repeat |
| 76 | +``` |
| 77 | + |
| 78 | +### The four primitives |
| 79 | + |
| 80 | +All four follow the same pattern: a directory under `.ralph/` with a marker markdown file containing YAML frontmatter. |
| 81 | + |
| 82 | +| Primitive | Marker file | Runs | Injects into prompt | |
| 83 | +|---|---|---|---| |
| 84 | +| Check | `CHECK.md` | After iteration | Failures appended to next prompt | |
| 85 | +| Context | `CONTEXT.md` | Before iteration | Output replaces `{{ contexts.name }}` | |
| 86 | +| Instruction | `INSTRUCTION.md` | Before iteration | Content replaces `{{ instructions.name }}` | |
| 87 | +| Prompt | `PROMPT.md` | At run start | Replaces root PROMPT.md when selected by name | |
| 88 | + |
| 89 | +Discovery is handled by `_frontmatter.py:discover_primitives()` which scans `.ralph/{kind}/*/` for marker files. |
| 90 | + |
| 91 | +### Placeholder resolution |
| 92 | + |
| 93 | +Both contexts and instructions use the same resolver (`resolver.py:resolve_placeholders()`): |
| 94 | + |
| 95 | +- `{{ contexts.git-log }}` — named placement for a specific primitive |
| 96 | +- `{{ contexts }}` — bulk placement for all remaining primitives |
| 97 | +- No placeholders at all — everything appended to the end of the prompt |
| 98 | + |
| 99 | +### Event system |
| 100 | + |
| 101 | +The run loop communicates via structured events (`_events.py`). Each event has a type (`EventType` enum), run ID, data dict, and UTC timestamp. |
| 102 | + |
| 103 | +- **`EventEmitter`** — protocol that any listener implements (just an `emit(event)` method) |
| 104 | +- **`NullEmitter`** — discards events (used in tests) |
| 105 | +- **`QueueEmitter`** — pushes events into a `queue.Queue` for async consumption (used by the UI) |
| 106 | +- **`FanoutEmitter`** — broadcasts events to multiple emitters (used by the manager for fan-out to queue + persistence) |
| 107 | + |
| 108 | +The CLI uses a `ConsoleEmitter` (defined in `_console_emitter.py`) that renders events to the terminal with Rich formatting. |
| 109 | + |
| 110 | +### Multi-run management (UI layer) |
| 111 | + |
| 112 | +`manager.py:RunManager` orchestrates concurrent runs for the web UI: |
| 113 | + |
| 114 | +- Creates runs with unique IDs and wraps them in `ManagedRun` (config + state + emitter + thread) |
| 115 | +- Starts each run in a daemon thread via `engine.run_loop()` |
| 116 | +- Supports pause/resume/stop per run via `RunState` thread-safe control methods |
| 117 | +- Uses `FanoutEmitter` to broadcast events to multiple listeners (e.g., queue + persistence) |
| 118 | + |
| 119 | +## Key files to understand first |
| 120 | + |
| 121 | +1. **`engine.py`** — The core run loop. Understands `RunConfig`, `RunState`, and `EventEmitter`. This is where iteration logic lives. |
| 122 | +2. **`cli.py`** — All CLI commands and prompt resolution. Delegates to `engine.run_loop()` for the actual loop. Scaffold templates live in `_templates.py`. Terminal event rendering lives in `_console_emitter.py`. |
| 123 | +3. **`_frontmatter.py`** — The primitive discovery system. Understanding `discover_primitives()` and `parse_frontmatter()` is essential for working on checks/contexts/instructions/prompts. |
| 124 | +4. **`resolver.py`** — Template placeholder logic shared by contexts and instructions. Small file but critical — changes here affect both. |
| 125 | + |
| 126 | +## Traps and gotchas |
| 127 | + |
| 128 | +### If you change the primitive marker filenames... |
| 129 | + |
| 130 | +The marker file names (`CHECK.md`, `CONTEXT.md`, `INSTRUCTION.md`, `PROMPT.md`) are defined as constants in `_frontmatter.py` (`CHECK_MARKER`, `CONTEXT_MARKER`, `INSTRUCTION_MARKER`, `PROMPT_MARKER`). All modules — `checks.py`, `contexts.py`, `instructions.py`, `prompts.py`, `cli.py`, and the UI layer — import from there. Change the constant to rename everywhere. |
| 131 | + |
| 132 | +### If you change frontmatter fields... |
| 133 | + |
| 134 | +Frontmatter parsing is in `_frontmatter.py:parse_frontmatter()` but the field names are consumed in each module's `discover_*()` function. The `timeout` and `enabled` fields get special type coercion in `parse_frontmatter()` — adding a new typed field requires updating the coercion logic there. |
| 135 | + |
| 136 | +### If you add a new CLI command... |
| 137 | + |
| 138 | +Add it in `cli.py`. The CLI uses Typer. The `new` subcommand group uses `app.add_typer()`. Update `docs/cli.md` to document the new command. |
| 139 | + |
| 140 | +### If you add a new primitive type... |
| 141 | + |
| 142 | +You need to: |
| 143 | + |
| 144 | +1. Create a new module (like `prompts.py`) with dataclass, discover, and resolve functions |
| 145 | +2. Add a scaffold template in `_templates.py` and a `new` subcommand in `cli.py` |
| 146 | +3. Wire it into `engine.py:run_loop()` if it affects the iteration cycle |
| 147 | +4. Add tests |
| 148 | +5. Update `docs/primitives.md` |
| 149 | + |
| 150 | +### If you change the event system... |
| 151 | + |
| 152 | +Events are defined in `_events.py:EventType`. The `ConsoleEmitter` in `_console_emitter.py` renders them to the terminal. The UI layer consumes them via `QueueEmitter`. Adding a new event type requires handling it in both places. |
| 153 | + |
| 154 | +### Output truncation |
| 155 | + |
| 156 | +`_output.py:truncate_output()` caps output at 5000 chars. This affects check failure output injected into prompts. If agents complain about missing error details, this is why. |
| 157 | + |
| 158 | +### The `run.*` script convention |
| 159 | + |
| 160 | +Checks and contexts can use either a `command` in frontmatter or a `run.*` script file in the primitive directory. If both exist, the script wins. This is handled by `_frontmatter.py:find_run_script()`. |
| 161 | + |
| 162 | +## Testing |
| 163 | + |
| 164 | +```bash |
| 165 | +uv run pytest # Run all tests |
| 166 | +uv run pytest -x # Stop on first failure |
| 167 | +``` |
| 168 | + |
| 169 | +Tests are in `tests/` with one file per module. All tests use temporary directories and don't require any external services. |
| 170 | + |
| 171 | +## Dependencies |
| 172 | + |
| 173 | +Minimal by design: |
| 174 | + |
| 175 | +- **typer** — CLI framework |
| 176 | +- **rich** — Terminal formatting (used via typer's console) |
| 177 | +- No other runtime dependencies |
| 178 | + |
| 179 | +Dev dependencies: pytest, mkdocs, mkdocs-material. |
| 180 | + |
| 181 | +Optional UI dependencies: fastapi, uvicorn, aiosqlite, websockets. |
0 commit comments