Skip to content

Commit e8187c0

Browse files
study8677study8677
committed
docs: add AgentCoder product architecture
0 parents  commit e8187c0

1 file changed

Lines changed: 344 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 344 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,344 @@
1+
# AgentCoder
2+
3+
AgentCoder is a training platform for real engineering work in the AI agent era.
4+
5+
It is not another LeetCode clone focused on handwritten algorithm problems. AgentCoder trains the two skills that matter when AI coding tools become part of everyday software work:
6+
7+
1. Using AI to complete real development tasks until they are shippable.
8+
2. Reviewing AI-generated code and deciding whether it can be merged.
9+
10+
Chinese positioning:
11+
12+
> 练习 AI 时代真正需要的工程能力。
13+
14+
Direct comparison:
15+
16+
> LeetCode 训练你写代码。AgentCoder 训练你交付代码，并审核 AI 写的代码。
17+
18+
## Product Direction
19+
20+
AgentCoder asks a different question from traditional coding platforms.
21+
22+
Traditional platforms ask:
23+
24+
> Can you write the correct code yourself?
25+
26+
AgentCoder asks:
27+
28+
> Can you use AI to deliver a correct, tested, mergeable change?
29+
30+
And:
31+
32+
> Can you tell whether AI-generated code is actually safe to merge?
33+
34+
The V0 product has two entry points:
35+
36+
- **Task Mode**: complete a realistic engineering task with help from AI tools.
37+
- **Review Mode**: review an AI-generated PR or diff and judge whether it can be merged.
38+
39+
V0 does not need algorithm Hot100, contests, complex community features, ranking systems, or a built-in cloud IDE. The first goal is to produce 20 high-quality exercises that make the product direction obvious.
40+
41+
The quality bar for V0 is not feature count. It is whether the first 20 challenges feel real, reproducible, scoreable, and obviously different from traditional coding exercises.
42+
43+
## V0 Architecture Decision
44+
45+
AgentCoder should be built as a **challenge asset + evaluation platform**, not as a full online IDE.
46+
47+
Users may use Cursor, Claude Code, Codex, Copilot, ChatGPT, or any other AI coding tool outside the platform. AgentCoder owns the challenge definition, repository assets, submission flow, evaluation rules, and review scoring.
48+
49+
The product should start as a focused monolith. The hard boundary is not between services, but between trusted platform code and untrusted user submissions.
50+
51+
The recommended V0 architecture has three core layers:
52+
53+
1. **Challenge Asset Layer**
54+
- Stores Task Mode and Review Mode exercises.
55+
- Each challenge is versioned and reproducible.
56+
- Task Mode challenges contain a seed repo, issue brief, tests, hidden checks, and reference solution.
57+
- Review Mode challenges contain an AI-generated PR/diff, answer key, rubric, and explanation.
58+
59+
2. **Product Experience Layer**
60+
- Shows challenge catalog, challenge detail, workspace instructions, submission flow, and result pages.
61+
- Task Mode accepts a patch, GitHub PR URL, or repository URL.
62+
- Review Mode provides a diff-reading interface and structured answer form.
63+
64+
3. **Evaluation Layer**
65+
- Runs Task Mode submissions in isolated containers.
66+
- Applies the user change to the seed repo.
67+
- Runs install, lint, tests, hidden tests, and challenge-specific validation.
68+
- Scores Review Mode answers against a rubric with required findings, severity, merge decision, and false-positive penalties.
69+
70+
This keeps the first version buildable while preserving the core product insight: AgentCoder evaluates delivery and judgment, not typing speed.
71+
72+
## Core Modes
73+
74+
### Task Mode
75+
76+
Task Mode gives the user a realistic development task.
77+
78+
Typical flow:
79+
80+
1. User opens a challenge.
81+
2. Platform provides a repo, issue, constraints, and validation instructions.
82+
3. User works locally or in their preferred AI coding tool.
83+
4. User submits a patch, PR URL, or repo URL.
84+
5. AgentCoder runs automated evaluation and returns a result.
85+
86+
Task Mode evaluates whether the user can:
87+
88+
- Understand the requirement.
89+
- Drive AI tools effectively.
90+
- Modify the codebase safely.
91+
- Add or adjust tests.
92+
- Catch and fix AI-generated mistakes.
93+
- Deliver a mergeable result.
94+
95+
Example Task Mode topics:
96+
97+
- Fix a real bug.
98+
- Implement a small feature.
99+
- Optimize a slow query.
100+
- Fix cache inconsistency.
101+
- Add missing tests.
102+
- Refactor complex logic.
103+
- Prevent duplicate async job execution.
104+
- Implement rate limiting.
105+
- Add parameter validation.
106+
- Fix pagination boundary behavior.
107+
108+
### Review Mode
109+
110+
Review Mode gives the user an AI-generated PR or diff.
111+
112+
The user must answer:
113+
114+
- Can this PR be merged?
115+
- If not, what exactly is wrong?
116+
- Did the AI only make a superficial fix?
117+
- Are there hidden edge cases?
118+
- Did it break compatibility?
119+
- Are there security, performance, or concurrency risks?
120+
- Do the tests cover the real risk?
121+
- Is the implementation maintainable?
122+
123+
Review Mode evaluates whether the user can judge AI-generated code, not whether they can rewrite it from scratch.
124+
125+
Example Review Mode topics:
126+
127+
- AI PR appears to fix a bug but misses an edge case.
128+
- AI PR passes existing tests but breaks compatibility.
129+
- AI PR adds a feature but misses permission checks.
130+
- AI PR has many tests but misses the core risk.
131+
- AI PR fixes performance but introduces data inconsistency.
132+
- AI PR changes too much and becomes hard to merge safely.
133+
- AI PR duplicates logic and hurts maintainability.
134+
- AI PR fixes frontend display but leaves backend data wrong.
135+
- AI PR introduces a concurrency bug.
136+
- AI PR is actually good, and the correct answer is to merge it.
137+
138+
## Core Data Model
139+
140+
The initial domain model can stay small:
141+
142+
- **User**
143+
- Identity, profile, progress, submissions.
144+
145+
- **Challenge**
146+
- Shared entity for both modes.
147+
- Fields: title, slug, mode, difficulty, tags, status, version, estimated time.
148+
149+
- **ChallengeAsset**
150+
- Points to the seed repo, diff, tests, rubric, fixtures, and solution materials.
151+
152+
- **TaskSubmission**
153+
- User submission for Task Mode.
154+
- Stores submitted patch, repo URL, commit SHA, run status, score, and result summary.
155+
156+
- **EvaluationRun**
157+
- One execution attempt for a Task Mode submission.
158+
- Stores runner image, commands, logs, test results, timeout, and final verdict.
159+
160+
- **ReviewSubmission**
161+
- User answer for Review Mode.
162+
- Stores merge decision, findings, severity labels, affected files, explanation, and score.
163+
164+
- **ReviewRubric**
165+
- Required findings, acceptable alternatives, severity weights, false-positive rules, and final explanation.
166+
167+
- **Progress**
168+
- Tracks solved challenges, attempts, best score, and review accuracy.
169+
170+
## Challenge Asset Structure
171+
172+
Challenges should live in the repository as versioned content, not only in the database.
173+
174+
Recommended structure:
175+
176+
```text
177+
content/
178+
challenges/
179+
task/
180+
fix-pagination-boundary/
181+
challenge.yaml
182+
prompt.md
183+
repo/
184+
tests/
185+
public/
186+
hidden/
187+
solution.patch
188+
explanation.md
189+
review/
190+
ai-pr-missing-permission-check/
191+
challenge.yaml
192+
prompt.md
193+
base.diff
194+
ai-pr.diff
195+
rubric.yaml
196+
explanation.md
197+
```
198+
199+
`challenge.yaml` should define metadata and execution settings:
200+
201+
```yaml
202+
id: fix-pagination-boundary
203+
mode: task
204+
title: Fix pagination boundary behavior
205+
difficulty: medium
206+
tags:
207+
- backend
208+
- testing
209+
- edge-case
210+
runtime:
211+
image: node:22
212+
install: npm install
213+
test: npm test
214+
limits:
215+
timeoutSeconds: 120
216+
```
217+
218+
This makes exercises reviewable, portable, and reproducible.
219+
220+
## Evaluation Design
221+
222+
### Task Mode Evaluation
223+
224+
Task Mode should combine deterministic checks:
225+
226+
- Patch applies cleanly.
227+
- Project installs successfully.
228+
- Lint/typecheck passes when configured.
229+
- Existing tests pass.
230+
- Public tests pass.
231+
- Hidden tests pass.
232+
- Challenge-specific assertions pass.
233+
- Forbidden shortcuts or hardcoded outputs are rejected when needed.
234+
235+
The result should be transparent:
236+
237+
- `accepted`: change passes the required checks.
238+
- `failed`: tests or validation failed.
239+
- `needs_review`: automated checks pass but challenge requires manual or rubric-based inspection.
240+
241+
LLM-based inspection can be added as an auxiliary feedback layer for suspicious patches, hardcoded fixes, or maintainability concerns. It should support deterministic checks, not replace them.
242+
243+
### Review Mode Evaluation
244+
245+
Review Mode should use structured scoring:
246+
247+
- Correct merge decision.
248+
- Required issues found.
249+
- Correct severity.
250+
- Correct affected area.
251+
- Quality of explanation.
252+
- Penalty for false positives.
253+
- Penalty for missing the central risk.
254+
255+
For V0, the rubric should be the source of truth. LLM scoring can help normalize free-form answers later, but it should not be the only authority.
256+
257+
## Recommended Technical Architecture
258+
259+
V0 can use a pragmatic monorepo:
260+
261+
```text
262+
apps/
263+
web/ # Next.js product UI
264+
worker/ # Evaluation runner worker
265+
packages/
266+
db/ # Prisma schema and database access
267+
evaluator/ # Shared evaluation logic
268+
ui/ # Shared UI components
269+
challenge/ # Challenge loading and validation helpers
270+
content/
271+
challenges/ # Versioned challenge assets
272+
```
273+
274+
Recommended stack:
275+
276+
- **Web**: Next.js + TypeScript.
277+
- **Database**: Postgres.
278+
- **ORM**: Prisma.
279+
- **Auth**: GitHub OAuth first, with Clerk, Auth.js, or another simple hosted auth provider.
280+
- **Queue**: BullMQ, Inngest, Trigger.dev, or a managed job queue.
281+
- **Runner**: Docker-based isolated workers for V0.
282+
- **Storage**: S3-compatible object storage for patches, logs, and artifacts.
283+
- **Diff UI**: Monaco Editor, CodeMirror, or a dedicated diff viewer.
284+
285+
The most important technical boundary is the runner. User submissions are untrusted code, so execution must be isolated with timeouts, network restrictions, resource limits, and clean workspaces.
286+
287+
V0 should also keep the challenge stack narrow. TypeScript, React, and Node.js are enough to prove the loop before expanding into many languages and frameworks.
288+
289+
## MVP Scope
290+
291+
V0 should include:
292+
293+
- Challenge catalog.
294+
- Task Mode challenge page.
295+
- Review Mode challenge page.
296+
- Patch or PR URL submission.
297+
- Automated Task Mode evaluation.
298+
- Structured Review Mode scoring.
299+
- Result page with actionable feedback.
300+
- Admin/content workflow for adding challenges.
301+
- 20 high-quality seed challenges.
302+
303+
V0 should not include:
304+
305+
- Traditional algorithm problem sets.
306+
- Full online IDE.
307+
- Browser-based terminal.
308+
- Social feed.
309+
- Complex discussion system.
310+
- Contests.
311+
- Public leaderboard.
312+
- Company interview marketplace.
313+
- Heavy AI tutor workflow.
314+
- Full plagiarism detection.
315+
- Broad multi-language support.
316+
317+
These can come later, after the core loop proves useful.
318+
319+
## First 20 Challenges
320+
321+
Initial content target:
322+
323+
- 10 Task Mode challenges.
324+
- 10 Review Mode challenges.
325+
326+
The quality bar is more important than breadth. Each challenge should have a concrete engineering lesson, realistic failure mode, deterministic assets, and a clear explanation.
327+
328+
## Brand
329+
330+
Domain:
331+
332+
- `agentcoder.codes`
333+
334+
One-line English positioning:
335+
336+
> Practice real coding work in the AI agent era.
337+
338+
Chinese positioning:
339+
340+
> 练习 AI 时代真正需要的工程能力。
341+
342+
Core belief:
343+
344+
> As AI makes writing code cheaper, judging whether code can safely ship becomes more valuable.

0 commit comments

Comments
 (0)