Skip to content

Commit 7fd73c8

Browse files
Copilotjmbish04
Copilot
and
jmbish04
committed
Add tests and documentation for flows API
Co-authored-by: jmbish04 <26469722+jmbish04@users.noreply.github.com>
1 parent 6b18434 commit 7fd73c8

3 files changed

Lines changed: 428 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,22 @@ This is a modular, extensible Cloudflare Worker that proxies the GitHub API, bui
44

55
## 🚀 Usage
66

7-
The worker exposes three main sets of endpoints:
7+
The worker exposes four main sets of endpoints:
88

9+
- `/api/flows`: High-level flows for repository setup and bulk operations.
910
- `/api/tools`: High-level tools for common agent workflows, such as creating files and opening pull requests.
1011
- `/api/octokit/rest`: A generic proxy for the GitHub REST API.
1112
- `/api/octokit/graphql`: A proxy for the GitHub GraphQL API.
1213

14+
### Flows API
15+
16+
The Flows API provides high-level operations for managing GitHub repositories at scale.
17+
18+
- `POST /api/flows/create-new-repo`: Create a new repository with default workflows.
19+
- `POST /api/flows/retrofit-workflows`: Add workflows to existing repositories.
20+
21+
📖 **[Full Flows API Documentation](./docs/FLOWS.md)**
22+
1323
### Tools API
1424

1525
The Tools API is the recommended way for agents to interact with this worker. It provides a simplified interface for common tasks.

‎docs/FLOWS.md‎

Lines changed: 334 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,334 @@
1+
# Flows API Documentation
2+
3+
The Flows API provides high-level operations for managing GitHub repositories and workflows at scale.
4+
5+
## Overview
6+
7+
The Flows API is designed to automate the setup and maintenance of GitHub repositories with standard configurations, particularly for Cloudflare Worker projects.
8+
9+
## Endpoints
10+
11+
### POST /api/flows/create-new-repo
12+
13+
Creates a new GitHub repository with default GitHub Actions workflows pre-configured.
14+
15+
**Request Body:**
16+
```json
17+
{
18+
"owner": "string", // Required: Organization or user name
19+
"name": "string", // Required: Repository name
20+
"description": "string", // Optional: Repository description
21+
"private": boolean, // Optional: Whether repo is private (default: false)
22+
"auto_init": boolean // Optional: Initialize with README (default: true)
23+
}
24+
```
25+
26+
**Response:**
27+
```json
28+
{
29+
"repo": {
30+
"id": 123456,
31+
"name": "my-worker",
32+
"full_name": "my-org/my-worker",
33+
"html_url": "https://github.com/my-org/my-worker",
34+
"private": false
35+
},
36+
"workflows": [
37+
{
38+
"path": ".github/workflows/pr-comment-extractor.yml",
39+
"status": "success",
40+
"message": "Created"
41+
},
42+
{
43+
"path": ".github/workflows/deploy-worker.yml",
44+
"status": "success",
45+
"message": "Created"
46+
}
47+
],
48+
"secrets": [
49+
{
50+
"name": "CLOUDFLARE_API_TOKEN",
51+
"status": "skipped",
52+
"message": "Secret encryption not implemented - use GitHub CLI or manual setup"
53+
},
54+
{
55+
"name": "CLOUDFLARE_ACCOUNT_ID",
56+
"status": "skipped",
57+
"message": "Secret encryption not implemented - use GitHub CLI or manual setup"
58+
}
59+
]
60+
}
61+
```
62+
63+
**Default Workflows:**
64+
65+
1. **PR Comment Extractor** (`.github/workflows/pr-comment-extractor.yml`)
66+
- Automatically extracts and summarizes all PR comments
67+
- Cleans and formats feedback for AI bot consumption
68+
- Posts consolidated summary as a PR comment
69+
- Triggers on: `issue_comment`, `pull_request_review_comment`
70+
71+
2. **Cloudflare Worker Deployment** (`.github/workflows/deploy-worker.yml`)
72+
- Automatically deploys to Cloudflare Workers on push to main
73+
- Uses `cloudflare/wrangler-action@v3`
74+
- Requires: `CLOUDFLARE_API_TOKEN`, `CLOUDFLARE_ACCOUNT_ID` secrets
75+
- Triggers on: push to `main` branch
76+
77+
**Example:**
78+
```bash
79+
curl -X POST https://your-worker.workers.dev/api/flows/create-new-repo \
80+
-H "Content-Type: application/json" \
81+
-H "x-api-key: YOUR_API_KEY" \
82+
-d '{
83+
"owner": "my-org",
84+
"name": "my-new-worker",
85+
"description": "A new Cloudflare Worker project",
86+
"private": false,
87+
"auto_init": true
88+
}'
89+
```
90+
91+
---
92+
93+
### POST /api/flows/retrofit-workflows
94+
95+
Adds or updates default workflows in existing repositories based on filters.
96+
97+
**Request Body:**
98+
```json
99+
{
100+
"owner": "string", // Required: Organization name to filter by
101+
"repos": ["string"], // Optional: Specific repo names (if empty, uses date filters)
102+
"date_active_gt": "string", // Optional: Filter by last activity > date (ISO 8601)
103+
"date_active_lt": "string", // Optional: Filter by last activity < date (ISO 8601)
104+
"date_added_gt": "string", // Optional: Filter by creation date > date (ISO 8601)
105+
"date_added_lt": "string", // Optional: Filter by creation date < date (ISO 8601)
106+
"force": boolean // Optional: Overwrite existing files (default: false)
107+
}
108+
```
109+
110+
**Response:**
111+
```json
112+
{
113+
"summary": {
114+
"total_repos_processed": 10,
115+
"successful": 7,
116+
"skipped": 2,
117+
"failed": 1
118+
},
119+
"results": [
120+
{
121+
"repo_name": "my-org/repo1",
122+
"status": "success",
123+
"workflows_added": [
124+
".github/workflows/pr-comment-extractor.yml",
125+
".github/workflows/deploy-worker.yml"
126+
],
127+
"message": "Added 2 workflow(s)"
128+
},
129+
{
130+
"repo_name": "my-org/repo2",
131+
"status": "skipped",
132+
"workflows_added": [],
133+
"message": "All workflows already exist"
134+
}
135+
]
136+
}
137+
```
138+
139+
**Filtering Logic:**
140+
141+
- If `repos` array is provided, only those repositories are processed
142+
- If `repos` is empty, all repositories in the organization are fetched
143+
- Date filters are applied using AND logic:
144+
- `date_active_gt`: Repositories with last push after this date
145+
- `date_active_lt`: Repositories with last push before this date
146+
- `date_added_gt`: Repositories created after this date
147+
- `date_added_lt`: Repositories created before this date
148+
- Cloudflare deployment workflow is only added if `wrangler.toml`, `wrangler.jsonc`, or `wrangler.json` exists
149+
- If `force` is `false`, existing workflow files are not overwritten (status: `skipped`)
150+
- If `force` is `true`, existing workflow files are updated
151+
152+
**Example: Retrofit specific repos**
153+
```bash
154+
curl -X POST https://your-worker.workers.dev/api/flows/retrofit-workflows \
155+
-H "Content-Type: application/json" \
156+
-H "x-api-key: YOUR_API_KEY" \
157+
-d '{
158+
"owner": "my-org",
159+
"repos": ["repo1", "repo2", "repo3"],
160+
"force": false
161+
}'
162+
```
163+
164+
**Example: Retrofit all active repos from 2024**
165+
```bash
166+
curl -X POST https://your-worker.workers.dev/api/flows/retrofit-workflows \
167+
-H "Content-Type: application/json" \
168+
-H "x-api-key: YOUR_API_KEY" \
169+
-d '{
170+
"owner": "my-org",
171+
"date_active_gt": "2024-01-01",
172+
"date_added_gt": "2024-01-01",
173+
"force": false
174+
}'
175+
```
176+
177+
---
178+
179+
## Audit Logging
180+
181+
All flow operations are logged to the `gh_management_config` D1 table for tracking and auditing:
182+
183+
**Table Schema:**
184+
```sql
185+
CREATE TABLE gh_management_config (
186+
id INTEGER PRIMARY KEY AUTOINCREMENT,
187+
timestamp DATETIME NOT NULL,
188+
repo_name TEXT NOT NULL,
189+
action TEXT NOT NULL, -- e.g., 'create_new_repo', 'retrofit_workflows'
190+
status TEXT NOT NULL, -- 'success', 'skipped', 'failure'
191+
status_details TEXT, -- JSON with additional context
192+
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
193+
);
194+
```
195+
196+
**Query Example:**
197+
```sql
198+
-- Get all retrofit operations in the last 7 days
199+
SELECT * FROM gh_management_config
200+
WHERE action = 'retrofit_workflows'
201+
AND timestamp > datetime('now', '-7 days')
202+
ORDER BY timestamp DESC;
203+
204+
-- Get failed operations
205+
SELECT * FROM gh_management_config
206+
WHERE status = 'failure'
207+
ORDER BY timestamp DESC;
208+
```
209+
210+
---
211+
212+
## Environment Variables
213+
214+
The Flows API requires the following environment variables to be configured:
215+
216+
### Required (inherited from worker)
217+
- `GITHUB_TOKEN`: GitHub Personal Access Token with repo permissions
218+
- `WORKER_API_KEY`: API key for authenticating requests to the worker
219+
220+
### Optional (for automated secret management)
221+
- `CLOUDFLARE_API_TOKEN`: Used to set repository secrets (not currently functional)
222+
- `GITHUB_ACTION_CLOUDFLARE_ACCOUNT_ID`: Used to set repository secrets (not currently functional)
223+
224+
**Note:** Secret encryption is not currently implemented due to missing libsodium library in Cloudflare Workers. Secrets should be set manually via GitHub CLI or UI:
225+
226+
```bash
227+
# Set secrets manually using GitHub CLI
228+
gh secret set CLOUDFLARE_API_TOKEN -b "your-token" -R owner/repo
229+
gh secret set CLOUDFLARE_ACCOUNT_ID -b "your-account-id" -R owner/repo
230+
```
231+
232+
---
233+
234+
## Workflow Templates
235+
236+
### PR Comment Extractor Workflow
237+
238+
**Purpose:** Aggregates all PR comments into a single summary for AI consumption.
239+
240+
**Features:**
241+
- Waits 30 seconds to batch comments
242+
- Fetches both issue comments and inline review comments
243+
- Filters out specific users and bot comments
244+
- Removes badges, slash commands, and other noise
245+
- Posts cleaned summary as a new PR comment
246+
247+
**Permissions Required:**
248+
- `pull-requests: write`
249+
- `issues: read`
250+
- `contents: read`
251+
252+
---
253+
254+
### Cloudflare Worker Deployment Workflow
255+
256+
**Purpose:** Automatically deploys worker to Cloudflare on push to main.
257+
258+
**Features:**
259+
- Triggers on push to `main` branch
260+
- Uses official `cloudflare/wrangler-action@v3`
261+
- 60-minute timeout for long builds
262+
- Requires Cloudflare secrets to be configured
263+
264+
**Permissions Required:**
265+
- None (uses repository secrets)
266+
267+
---
268+
269+
## Error Handling
270+
271+
All endpoints provide detailed error information:
272+
273+
**Success (200):**
274+
```json
275+
{
276+
"summary": { ... },
277+
"results": [ ... ]
278+
}
279+
```
280+
281+
**Validation Error (400/422):**
282+
```json
283+
{
284+
"error": "Validation failed",
285+
"details": { ... }
286+
}
287+
```
288+
289+
**Authentication Error (401):**
290+
```json
291+
{
292+
"error": "Unauthorized"
293+
}
294+
```
295+
296+
**GitHub API Error (500):**
297+
```json
298+
{
299+
"error": "Failed to create repository",
300+
"message": "Repository already exists"
301+
}
302+
```
303+
304+
---
305+
306+
## Best Practices
307+
308+
1. **Test with a single repo first:** Use `repos: ["test-repo"]` to validate behavior
309+
2. **Use date filters carefully:** Narrow down repositories before running bulk operations
310+
3. **Check logs:** Query `gh_management_config` table to verify operations
311+
4. **Use force sparingly:** Only set `force: true` when you need to update existing workflows
312+
5. **Monitor rate limits:** GitHub API has rate limits; large retrofits may take time
313+
6. **Set secrets manually:** Until encryption is implemented, configure secrets via GitHub CLI
314+
315+
---
316+
317+
## Limitations
318+
319+
1. **Secret Encryption:** The API cannot currently set repository secrets due to missing libsodium library. Secrets must be configured manually.
320+
2. **Rate Limits:** Large-scale retrofits are subject to GitHub API rate limits.
321+
3. **Organization Scope:** Currently only supports organization repositories, not user repositories.
322+
4. **Pagination:** Repository listing is limited to 100 repos per request.
323+
324+
---
325+
326+
## Future Enhancements
327+
328+
- [ ] Implement secret encryption with libsodium-wrappers
329+
- [ ] Add support for custom workflow templates
330+
- [ ] Implement pagination for large organizations
331+
- [ ] Add workflow validation before deployment
332+
- [ ] Support for user repositories (not just orgs)
333+
- [ ] Dry-run mode for testing without making changes
334+
- [ ] Webhook integration for automated workflow management

0 commit comments

Comments
 (0)