You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> For an overview of all available workflows, see the [main README](../README.md).
4
+
5
+
**Iterative optimization agent inspired by [Autoresearch](https://github.com/karpathy/autoresearch) and Claude Code's `/loop`**
6
+
7
+
The [Autoloop workflow](../workflows/autoloop.md?plain=1) runs on a schedule to autonomously improve target artifacts toward measurable goals. Each iteration proposes a change, evaluates it against a metric, and keeps only improvements. Supports **multiple independent loops** in the same repository.
8
+
9
+
## Installation
10
+
11
+
```bash
12
+
# Install the 'gh aw' extension
13
+
gh extension install github/gh-aw
14
+
15
+
# Add the workflow to your repository
16
+
gh aw add-wizard githubnext/agentics/autoloop
17
+
```
18
+
19
+
This walks you through adding the workflow to your repository.
20
+
21
+
## How It Works
22
+
23
+
```mermaid
24
+
graph LR
25
+
A[Scheduled Run] --> B[Discover Programs]
26
+
B --> C[For Each Program]
27
+
C --> D[Review History]
28
+
D --> E[Propose Change]
29
+
E --> F[Implement on Branch]
30
+
F --> G[Run Evaluation]
31
+
G --> H{Metric Improved?}
32
+
H -->|Yes| I[Create Draft PR]
33
+
H -->|No| J[Record & Reject]
34
+
I --> K[Update Experiment Log]
35
+
J --> K
36
+
```
37
+
38
+
## Getting Started
39
+
40
+
When you install Autoloop, a **template program file** is added at `.github/autoloop/programs/example.md`. This template has placeholder sections you must fill in — the workflow **will not run** until you do.
41
+
42
+
### Setup flow
43
+
44
+
```mermaid
45
+
graph LR
46
+
A[Install Workflow] --> B[Rename & Edit Program]
47
+
B --> C[Define Goal, Targets, Evaluation]
48
+
C --> D[Remove UNCONFIGURED sentinel]
49
+
D --> E[Commit & Push]
50
+
E --> F[Loop Begins]
51
+
```
52
+
53
+
1.**Install** — `gh aw add-wizard githubnext/agentics/autoloop`
54
+
2.**Rename** — Rename `.github/autoloop/programs/example.md` to something meaningful (e.g., `training.md`, `coverage.md`). The filename becomes the program name.
55
+
3.**Edit** — Replace the placeholders with your project's goal, target files, and evaluation command. The template includes three complete examples for inspiration.
56
+
4.**Activate** — Remove the `<!-- AUTOLOOP:UNCONFIGURED -->` line at the top.
If you forget to edit the template, the first scheduled run will create a GitHub issue reminding you, with a direct link to edit the file.
60
+
61
+
### Adding more loops
62
+
63
+
To run multiple optimization loops in parallel, just add more `.md` files to `.github/autoloop/programs/`:
64
+
65
+
```
66
+
.github/autoloop/programs/
67
+
├── training.md ← optimize model training loss
68
+
├── coverage.md ← maximize test coverage
69
+
└── build-perf.md ← minimize build time
70
+
```
71
+
72
+
Each program runs independently with its own metric tracking, experiment log issue, and PR namespace. Copy the template, fill it in, and push — the next scheduled run picks it up automatically.
73
+
74
+
## Configuration
75
+
76
+
Each program file in `.github/autoloop/programs/` has three sections:
77
+
78
+
### 1. Goal — What to optimize
79
+
80
+
Describe the objective in natural language. Be specific about what "better" means.
81
+
82
+
### 2. Target — What files can be changed
83
+
84
+
List the files the agent is allowed to modify. Everything else is off-limits.
85
+
86
+
### 3. Evaluation — How to measure success
87
+
88
+
Provide a command to run and a metric to extract. Specify whether higher or lower is better.
89
+
90
+
### Example program file
91
+
92
+
````markdown
93
+
# Autoloop Program
94
+
95
+
## Goal
96
+
97
+
Optimize the training script to minimize validation loss on CIFAR-10
Metric: `validation_loss` from `results.json`. Lower is better.
113
+
````
114
+
115
+
### Customizing the Schedule
116
+
117
+
Edit the workflow's `schedule` field. Examples:
118
+
-`every 6h` — 4 times a day (default)
119
+
-`every 1h` — hourly iterations
120
+
-`daily` — once a day
121
+
-`0 */2 * * *` — every 2 hours (cron syntax)
122
+
123
+
After editing, run `gh aw compile` to update the workflow.
124
+
125
+
Note: The schedule applies to the workflow as a whole — all programs iterate on the same schedule. To run programs at different frequencies, you can install the workflow multiple times with different schedules, each pointing to a subset of programs.
126
+
127
+
## Usage
128
+
129
+
### Automatic mode
130
+
131
+
Once at least one configured program exists, iterations run automatically on schedule. Each run processes every configured program:
132
+
133
+
1. Reads the program definition and past history
134
+
2. Proposes a single targeted change
135
+
3. Runs the evaluation command
136
+
4. Accepts (creates draft PR) or rejects (logs the attempt)
137
+
138
+
### Manual trigger
139
+
140
+
```bash
141
+
# Run all programs now
142
+
gh aw run autoloop
143
+
144
+
# Target a specific program
145
+
gh aw run autoloop -- "training: try using cosine annealing"
146
+
147
+
# If only one program exists, no prefix needed
148
+
gh aw run autoloop -- "try batch size 64 instead of 32"
149
+
```
150
+
151
+
### Slash command
152
+
153
+
Comment on any issue or PR:
154
+
```
155
+
/autoloop training: try batch size 64 instead of 32
156
+
```
157
+
158
+
## Experiment Tracking
159
+
160
+
Each program gets its own monthly experiment log issue titled `[Autoloop: {program-name}] Experiment Log {YYYY-MM}`. The issue tracks:
161
+
162
+
- Current best metric value
163
+
- Full iteration history with accept/reject status
164
+
- Links to PRs for accepted changes
165
+
- Links to GitHub Actions runs
166
+
167
+
## Human in the Loop
168
+
169
+
-**Review draft PRs** — accepted improvements appear as draft PRs for human review
170
+
-**Merge or close** — you decide which optimizations to keep
171
+
-**Adjust programs** — edit any program file to change the goal, targets, or evaluation
172
+
-**Add/remove loops** — add or delete files in `.github/autoloop/programs/`
173
+
-**Steer via slash command** — use `/autoloop {program}: {instructions}` to direct experiments
174
+
-**Pause** — disable the workflow schedule to stop all loops, or add the sentinel back to a single program file to pause just that loop
175
+
176
+
## Security
177
+
178
+
- Runs with read-only GitHub permissions
179
+
- Only modifies files listed in each program's Target section
180
+
- Never modifies evaluation scripts
181
+
- All changes go through draft PRs requiring human approval
182
+
- Uses "safe outputs" to constrain what the agent can create
@@ -239,12 +238,21 @@ If there is no `source-map.json` (first run), regenerate all pages.
239
238
240
239
### 3d. Build context and generate content
241
240
241
+
**MANDATORY CONSTRAINTS — read carefully before generating any content:**
242
+
243
+
-**Never generate more than 4 pages per `push-wiki` call.** If there are more than 4 pages to generate, process them in sequential batches of up to 4, calling `push-wiki` once per batch.
244
+
-**Never spawn a sub-agent or background agent to generate pages.** Generate all pages directly in the main conversation loop.
245
+
-**Each page must be kept under 3 KB of markdown.** Keep pages focused and concise.
246
+
-**Each `push-wiki` JSON payload must stay under 30 KB total.** If a batch would exceed 30 KB (including the sidebar), split it into a smaller batch.
247
+
-**If a `push-wiki` call fails with an API error**, it is likely a timeout caused by a large payload. Retry up to 2 times with progressively smaller batches (halving the batch size each retry, minimum 1 page per call). If a single-page call also fails, the error is unrecoverable — report it and stop.
248
+
242
249
For each page that needs regeneration:
243
250
244
251
1. Check MEMORY_DIR for cached summaries of the relevant source files (files named `summary--{path}.md`). If a file's hash matches (stored on the first line as `<!-- hash: ... -->`), use the cached summary to save context window space. If not, read the full file.
245
-
2. For files you read in full, write a condensed summary to MEMORY_DIR as `summary--{path}.md` (replace `/` with `--`). The summary should capture: exports, key types/interfaces, function signatures, class structure, and important constants. Keep summaries under 2KB each. Include the file's content hash on the first line: `<!-- hash: abc123 -->`.
246
-
3. Generate the content for each `*{ ... }*` instruction block, following the **Content Generation Guidelines** below.
247
-
4. Assemble the page: combine static text with generated content, normalizing heading levels (H4→H2, H5→H3, H6→H4 in the output).
252
+
2.**For source files longer than 500 lines**, do not read the entire file. Instead, use `head` to read the first 100 lines (for imports, exports, and top-level types), then use `grep` to find lines containing keywords from the page's instruction block (e.g., function names, class names, config keys), and use `head`/`tail` to read only those surrounding sections. For example: `grep -n "functionName\|ClassName" src/foo.ts | head -20` to locate relevant line numbers, then `head -n 150 src/foo.ts | tail -50` to extract that region.
253
+
3. For files you read in full, write a condensed summary to MEMORY_DIR as `summary--{path}.md` (replace `/` with `--`). The summary should capture: exports, key types/interfaces, function signatures, class structure, and important constants. Keep summaries under 2KB each. Include the file's content hash on the first line: `<!-- hash: abc123 -->`.
254
+
4. Generate the content for each `*{ ... }*` instruction block, following the **Content Generation Guidelines** below.
255
+
5. Assemble the page: combine static text with generated content, normalizing heading levels (H4→H2, H5→H3, H6→H4 in the output).
248
256
249
257
### 3e. Self-review
250
258
@@ -254,22 +262,34 @@ Before finalizing each page, review your generated content against the **Self-Re
254
262
255
263
**Do NOT write wiki page files to disk.** Do NOT create output directories. Do NOT use shell commands to write files.
256
264
257
-
Instead, construct ALL wiki page content as strings and pass them directly to the `push-wiki` safe-output as a single JSON object.
265
+
**Do NOT use sub-agents or background agents for page generation.** Generate all pages directly in the main conversation loop.
266
+
267
+
Construct wiki page content as strings and pass them to the `push-wiki` safe-output as JSON objects. **Push in batches of at most 4 pages per call** to avoid API timeouts:
258
268
259
-
1. Build a JSON object mapping filenames to markdown content for every page.
260
-
2. Generate `_Sidebar.md` content following the **Sidebar Generation** rules below and include it in the same JSON object.
261
-
3. Pass the complete JSON object to the `push-wiki` safe-output.
269
+
1. Divide the full list of pages into batches of up to 4 pages each.
270
+
2. For each batch, build a JSON object mapping filenames to markdown content.
271
+
3. Include `_Sidebar.md` (generated following the **Sidebar Generation** rules below) **only in the final batch**.
272
+
4. Before calling `push-wiki`, estimate the total JSON payload size. **If the payload exceeds 30 KB, reduce the batch size** (use 2 pages per call or fewer) until it fits.
273
+
5. Call `push-wiki` once per batch. Proceed to the next batch only after the current call succeeds.
274
+
6.**If a `push-wiki` call fails with an API or timeout error**, halve the current batch size (minimum 1 page per call) and retry up to 2 times. API errors during generation are most often caused by large response payloads, not transient network issues. If a single-page call still fails, the error is unrecoverable — report it and stop.
262
275
263
-
The JSON object should look like:
276
+
A single-batch JSON object looks like:
264
277
```json
265
278
{
266
279
"Home.md": "Welcome to the project...\n\n## Overview\n...",
Every wiki page and the sidebar must be included in this single JSON object. Pages use the slug as their filename (e.g., `Getting-Started.md`).
292
+
Pages use the slug as their filename (e.g., `Getting-Started.md`).
273
293
274
294
### 3g. Save memory
275
295
@@ -494,6 +514,13 @@ Diagram type by use case: `flowchart LR` for architecture/data flow; `sequenceDi
494
514
495
515
Syntax rules: Always specify direction (`LR` or `TD`). Wrap labels containing special characters in double quotes: `A["MyClass::method()"]`. One relationship per line. Use subgraphs sparingly (max one level deep). Add a brief sentence before the diagram explaining what it shows.
496
516
517
+
**Critical mermaid restrictions** — GitHub's renderer is strict. Violating these causes "Unable to render rich display" errors:
518
+
519
+
-**No backtick strings** — Do NOT use the backtick/markdown-string syntax inside node labels: `` A["`label`"] `` is invalid. Use plain text or double-quoted strings only: `A["label"]`.
520
+
-**No `\n` in labels** — Do NOT use `\n` escape sequences inside node labels. They are not rendered as newlines and cause lexer errors. Keep labels to a single line. If a label is too long, shorten it or split the node into two nodes.
521
+
-**No special characters unquoted** — Any label containing `@`, `(`, `)`, `:`, `/`, `<`, `>`, or other non-alphanumeric characters must be wrapped in double quotes.
522
+
-**Test mentally before writing** — Before including a diagram, verify each node label is either plain alphanumeric text or a properly double-quoted string with no escape sequences.
523
+
497
524
```mermaid
498
525
flowchart LR
499
526
A[Input] --> B[Process] --> C[Output]
@@ -577,7 +604,13 @@ Before finalizing each page, check for these issues and fix them:
577
604
578
605
5.**Accuracy** — Content matches what the source code actually does. No fabricated features or APIs.
579
606
580
-
6.**Structural consistency** — Similar sections across pages use the same structure and formatting patterns.
607
+
6.**Mermaid diagram syntax** — For every mermaid diagram, verify:
0 commit comments