Skip to content

Commit 4a061f2

Browse files
yunhao-jclaude
andcommitted
docs: project name is CoderCup; codearena- stays only as resource prefix
CodeArena was the working title. Prose now says CoderCup; literal identifiers (codearena-* AWS resources, CodeArena*Stack stack names, the Project=CodeArena tag) keep the historical prefix, with a naming note in design.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent e52b5c0 commit 4a061f2

13 files changed

Lines changed: 46 additions & 44 deletions

app/changelog/ChangelogClient.tsx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -229,7 +229,7 @@ const ENTRIES: Entry[] = [
229229
{
230230
date: '2026-05-26',
231231
tag: 'platform',
232-
title: 'Rebrand: CodeArena → CoderCup',
232+
title: 'Rebrand: CoderCup → CoderCup',
233233
body: (
234234
<>
235235
Picked <code>codercup.ai</code> as the launch brand after rejecting

docs/design.md

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,28 @@
1-
# CodeArena — High-Level Design
1+
# CoderCup — High-Level Design
22

33
Status: v0.1 — first articulation of the architecture that the v1 release (codename `codearena-v1`) builds toward.
44
Owner: Claude Code (autonomous build, Yunhao Jiao as product owner)
55
Last updated: 2026-05-25
66

7-
This document is the cross-cutting architecture reference for CodeArena. It is **not** the project's strategic intent and it is **not** the week-by-week schedule (those live in internal planning docs outside this repo). It is the single source of truth for **how the system is shaped** — components, contracts, data flow, security boundaries.
7+
This document is the cross-cutting architecture reference for CoderCup. It is **not** the project's strategic intent and it is **not** the week-by-week schedule (those live in internal planning docs outside this repo). It is the single source of truth for **how the system is shaped** — components, contracts, data flow, security boundaries.
88

9-
If you only read one doc to understand CodeArena's architecture, read this one.
9+
If you only read one doc to understand CoderCup's architecture, read this one.
10+
11+
> **Naming note:** CoderCup began under the working title *CodeArena*. AWS resources (`codearena-*` buckets/tables/Lambdas), CDK stack names (`CodeArena*Stack`), and the `Project=CodeArena` tag keep the original prefix — renaming live infrastructure isn't worth the churn. Prose uses the product name, CoderCup.
1012
1113
---
1214

1315
## 1. Vision and operating principle
1416

15-
### 1.1 What CodeArena is
17+
### 1.1 What CoderCup is
1618

17-
CodeArena is **the public leaderboard for AI coding agents**, hosted by TestSprite as the neutral verifier. Each event poses one standardized task; the headline frontier-lab agents (Claude Code, Codex, Anti-Gravity) all ship a deployable solution under identical prompts, time budgets, and environments. TestSprite verifies the deployable, scores it on three dimensions (correctness, bugs caught, cost-efficiency), and publishes everything — raw transcripts, deployed apps, scores — to the public leaderboard.
19+
CoderCup is **the public leaderboard for AI coding agents**, hosted by TestSprite as the neutral verifier. Each event poses one standardized task; the headline frontier-lab agents (Claude Code, Codex, Anti-Gravity) all ship a deployable solution under identical prompts, time budgets, and environments. TestSprite verifies the deployable, scores it on three dimensions (correctness, bugs caught, cost-efficiency), and publishes everything — raw transcripts, deployed apps, scores — to the public leaderboard.
1820

1921
The inaugural event is the **World Cup Code Battle 2026**, with the leaderboard going public on 2026-06-22, three days before the start of the championship's knockout rounds. The task each agent ships is a public web app for predicting championship outcomes. After launch, prediction-accuracy of the deployed apps becomes a live side-metric that updates every 15 minutes during knockout matches.
2022

21-
### 1.2 What CodeArena is not
23+
### 1.2 What CoderCup is not
2224

23-
- Not a benchmark for academic AI research. SWE-bench fills that role. CodeArena is positioned for **public-facing** comparison, not laboratory comparison.
25+
- Not a benchmark for academic AI research. SWE-bench fills that role. CoderCup is positioned for **public-facing** comparison, not laboratory comparison.
2426
- Not a TestSprite product marketing surface. The TestSprite brand sits in the footer; the leaderboard's job is to be a credible scoreboard, not a funnel. The marketing value accrues to TestSprite by being *the* trusted referee for the AI coding agent comparison.
2527
- Not a betting site. No monetary stakes, no real-money predictions. Users predict outcomes in the agent-built apps for cosmetic points only.
2628
- Not a runner-as-a-service for arbitrary user-defined tasks. v1 ships a curated task only. The community-task submission surface (`/submit`) lands in v1, but task selection stays with us.
@@ -37,11 +39,11 @@ Three commitments shape every architectural decision. They are non-negotiable.
3739

3840
## 2. System overview
3941

40-
CodeArena's runtime decomposes into five components plus an off-system test corpus. The picture below is the steady-state v1 architecture; see §4 for what's deployed today vs scheduled.
42+
CoderCup's runtime decomposes into five components plus an off-system test corpus. The picture below is the steady-state v1 architecture; see §4 for what's deployed today vs scheduled.
4143

4244
```
4345
┌─────────────────────────────────────────────────┐
44-
CodeArena leaderboard (FE) │
46+
CoderCup leaderboard (FE) │
4547
│ Next.js 14 App Router · static export · CDN │
4648
│ Reads leaderboard.json + agents/<slug>.json │
4749
│ Live JSONL stream rendered as run timeline │
@@ -264,19 +266,19 @@ The two calibration constants (`max_bugs = 20`, `max_usd_imputed = 50`) are plac
264266

265267
## 5. Security model
266268

267-
CodeArena runs in AWS account **<AWS_ACCOUNT_ID>** in **us-east-1**. The account is **shared with TestSprite production workloads**. Every CodeArena resource enforces a four-rule guardrail; if you are adding a resource to this codebase, it must pass all four:
269+
CoderCup runs in AWS account **<AWS_ACCOUNT_ID>** in **us-east-1**. The account is **shared with TestSprite production workloads**. Every CoderCup resource enforces a four-rule guardrail; if you are adding a resource to this codebase, it must pass all four:
268270

269271
1. **Naming**: prefixed `codearena-`. No exceptions.
270-
2. **Tagging**: tagged `Project=CodeArena` and `ManagedBy=ClaudeCode` via `Tags.of(app)` in `infra/cdk/bin/codearena.ts`. CDK applies these uniformly; new top-level constructs inherit.
271-
3. **IAM scoping**: any policy attached to a CodeArena role uses resource ARN patterns matching the `codearena-` prefix. No `Resource: "*"` outside the specific service-level patterns AWS requires (e.g., CloudWatch metric publishing is namespace-conditional, not resource-conditional). The runner role's S3 grants use `Bucket.grantReadWrite(role)` which CDK expands to the bucket ARN and `<bucket-arn>/*` — exactly the codearena buckets, nothing else.
272-
4. **No cross-system reads**: CodeArena drivers / Lambdas never read TestSprite production DynamoDB tables, never write to TestSprite production S3 buckets, never assume the TestSprite production-tier IAM roles. The TestSprite SaaS that's the referee runs in an isolated environment, so a production-tier outage there cannot affect CodeArena.
272+
2. **Tagging**: tagged `Project=CoderCup` and `ManagedBy=ClaudeCode` via `Tags.of(app)` in `infra/cdk/bin/codearena.ts`. CDK applies these uniformly; new top-level constructs inherit.
273+
3. **IAM scoping**: any policy attached to a CoderCup role uses resource ARN patterns matching the `codearena-` prefix. No `Resource: "*"` outside the specific service-level patterns AWS requires (e.g., CloudWatch metric publishing is namespace-conditional, not resource-conditional). The runner role's S3 grants use `Bucket.grantReadWrite(role)` which CDK expands to the bucket ARN and `<bucket-arn>/*` — exactly the codearena buckets, nothing else.
274+
4. **No cross-system reads**: CoderCup drivers / Lambdas never read TestSprite production DynamoDB tables, never write to TestSprite production S3 buckets, never assume the TestSprite production-tier IAM roles. The TestSprite SaaS that's the referee runs in an isolated environment, so a production-tier outage there cannot affect CoderCup.
273275

274-
**Secrets**: there are none managed by CodeArena infrastructure. Each agent CLI uses account-login auth on the runner EC2; those credential files live in `/home/ec2-user/.config/*` on the box, owned by the OS user, never serialized into our IaC or commit history.
276+
**Secrets**: there are none managed by CoderCup infrastructure. Each agent CLI uses account-login auth on the runner EC2; those credential files live in `/home/ec2-user/.config/*` on the box, owned by the OS user, never serialized into our IaC or commit history.
275277

276278
**Public surfaces**:
277279
- `https://main.d2wicurs2ws3dd.amplifyapp.com` (and eventual custom domain) — public read of leaderboard.
278280
- CloudFront distribution fronting `codearena-public-data-*` — public read of leaderboard.json, agents/*.json, runs/*/live.jsonl. Bucket itself is BPA-blocked; only the OAC CloudFront principal can read.
279-
- `https://github.com/TestSprite/CodeArena` — public source, public commit history.
281+
- `https://github.com/TestSprite/CoderCup` — public source, public commit history.
280282

281283
**Private surfaces**:
282284
- `codearena-runs-*` S3 bucket — private, no public principals.
@@ -289,7 +291,7 @@ CodeArena runs in AWS account **<AWS_ACCOUNT_ID>** in **us-east-1**. The account
289291

290292
## 6. Operational model — autonomous build
291293

292-
CodeArena is built end-to-end by an autonomous Claude Code session. The build follows three loops:
294+
CoderCup is built end-to-end by an autonomous Claude Code session. The build follows three loops:
293295

294296
1. **Build loop**: design (in `docs/codearena-v1/`) → implement (in `app/`, `runners/`, `scoring/`, `infra/`) → commit → push → Amplify auto-deploys for FE / `cdk deploy` for infra.
295297
2. **Verify loop**: after a feature lands, invoke the TestSprite verify skill. It runs the relevant TestSprite tests via the CLI, inspects the results, logs friction observations for the TestSprite team. Tests created during the verify loop stay in the project's TestSprite suite — they accumulate into the v1 release suite.
@@ -320,8 +322,8 @@ Two regimes: launch month (May 26 – June 25) and steady state (June 26 onward)
320322
- EC2 stop-and-start, weekdays-only ≈ ~$15/mo
321323
- Other infra ≈ $15-25/mo
322324

323-
**Out of scope from CodeArena's own AWS budget**:
324-
- The agent-built apps' hosting (each ships to its own Amplify app post-launch; counted in the agent's `usd_imputed` indirectly via deploy minutes, not CodeArena infra).
325+
**Out of scope from CoderCup's own AWS budget**:
326+
- The agent-built apps' hosting (each ships to its own Amplify app post-launch; counted in the agent's `usd_imputed` indirectly via deploy minutes, not CoderCup infra).
325327
- Inference token spend if we ever bypass the subscriptions (currently NOT planned — all three CLIs are subscription-billed).
326328

327329
CloudFront egress is the volatile term during launch week. At $0.085/GB outbound, a 1 TB launch-day surge is ~$85. A separate $500 launch-month allowance is carved out for this overshoot scenario; outside launch week, egress is negligible.

infra/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# /infra — AWS CDK stacks for CodeArena
1+
# /infra — AWS CDK stacks for CoderCup
22

33
Deploys to **us-east-1**. The AWS account may be shared with other workloads; every resource must be prefixed `codearena-` and tagged `Project=CodeArena ManagedBy=ClaudeCode`. See `/docs/design.md` for the rules.
44

infra/cdk/bin/codearena.ts

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,13 @@ const env = {
1616
const data = new CodeArenaDataStack(app, 'CodeArenaDataStack', {
1717
env,
1818
description:
19-
'CodeArena W1 data layer - DynamoDB single-table + run/public S3 buckets. SHARED account: all resources prefixed codearena-.',
19+
'CoderCup W1 data layer - DynamoDB single-table + run/public S3 buckets. SHARED account: all resources prefixed codearena-.',
2020
});
2121

2222
new CodeArenaRunnerStack(app, 'CodeArenaRunnerStack', {
2323
env,
2424
description:
25-
'CodeArena W2 runner - persistent EC2 hosting Claude / Codex / Antigravity CLIs with account login. Connect via SSM Session Manager.',
25+
'CoderCup W2 runner - persistent EC2 hosting Claude / Codex / Antigravity CLIs with account login. Connect via SSM Session Manager.',
2626
publicDataBucket: data.publicDataBucket,
2727
runsBucket: data.runsBucket,
2828
mainTable: data.mainTable,
@@ -31,14 +31,14 @@ new CodeArenaRunnerStack(app, 'CodeArenaRunnerStack', {
3131
const cdn = new CodeArenaPublicCdnStack(app, 'CodeArenaPublicCdnStack', {
3232
env,
3333
description:
34-
'CodeArena W2 m2-2 piece-4 - CloudFront fronting codearena-public-data via OAC. Bucket stays BPA-enabled; CloudFront is the only public read path.',
34+
'CoderCup W2 m2-2 piece-4 - CloudFront fronting codearena-public-data via OAC. Bucket stays BPA-enabled; CloudFront is the only public read path.',
3535
publicDataBucket: data.publicDataBucket,
3636
});
3737

3838
new CodeArenaComputeStack(app, 'CodeArenaComputeStack', {
3939
env,
4040
description:
41-
'CodeArena W2 m2-2 piece-2 + piece-3 - score-runner Lambda (S3-trigger) + publisher Lambda + EventBridge cron + Secrets Manager container for the TestSprite API key.',
41+
'CoderCup W2 m2-2 piece-2 + piece-3 - score-runner Lambda (S3-trigger) + publisher Lambda + EventBridge cron + Secrets Manager container for the TestSprite API key.',
4242
mainTable: data.mainTable,
4343
publicDataBucket: data.publicDataBucket,
4444
runsBucket: data.runsBucket,

infra/cdk/lib/codearena-data-stack.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ import {
1616
} from 'aws-cdk-lib/aws-s3';
1717

1818
/**
19-
* W1 data layer for CodeArena. Intentionally minimal:
19+
* W1 data layer for CoderCup. Intentionally minimal:
2020
* - codearena-main DynamoDB table (single-table design with GSI1 for leaderboard sort).
2121
* - codearena-public-data-<account> S3 bucket (private; CloudFront added in W2 when
2222
* leaderboard.json starts flowing).
@@ -27,7 +27,7 @@ import {
2727
*
2828
* Shared-account guardrails (see [[project-codearena-w1-decisions]]):
2929
* - every resource name is prefixed `codearena-`
30-
* - every resource is tagged Project=CodeArena ManagedBy=ClaudeCode (via Tags.of(app) in bin/)
30+
* - every resource is tagged Project=CoderCup ManagedBy=ClaudeCode (via Tags.of(app) in bin/)
3131
* - Block Public Access is fully enforced on every bucket
3232
* - DynamoDB uses point-in-time-recovery and AWS-managed encryption
3333
*/
@@ -92,7 +92,7 @@ export class CodeArenaDataStack extends Stack {
9292

9393
new CfnOutput(this, 'MainTableName', {
9494
value: this.mainTable.tableName,
95-
description: 'DynamoDB single-table for CodeArena entities',
95+
description: 'DynamoDB single-table for CoderCup entities',
9696
});
9797
new CfnOutput(this, 'PublicDataBucketName', {
9898
value: this.publicDataBucket.bucketName,

infra/cdk/lib/codearena-public-cdn-stack.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ import { Bucket } from 'aws-cdk-lib/aws-s3';
2727
* `*.cloudfront.net` default domain.
2828
*
2929
* Per [[feedback-aws-account-boundary]] TIER 3: codearena-* resources only,
30-
* Project=CodeArena ManagedBy=ClaudeCode tags applied via Tags.of(app).
30+
* Project=CoderCup ManagedBy=ClaudeCode tags applied via Tags.of(app).
3131
*/
3232
export interface CodeArenaPublicCdnStackProps extends StackProps {
3333
publicDataBucket: Bucket;

infra/cdk/lib/codearena-runner-stack.ts

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ import { Bucket } from 'aws-cdk-lib/aws-s3';
2626
import { Table } from 'aws-cdk-lib/aws-dynamodb';
2727

2828
/**
29-
* W2 runner host for CodeArena. Provisions ONE persistent EC2 that hosts all
29+
* W2 runner host for CoderCup. Provisions ONE persistent EC2 that hosts all
3030
* three agent CLIs (claude, codex, antigravity). The operator connects via
3131
* SSM Session Manager once per agent to run the account-login flow; subscription
3232
* sessions persist on disk. The driver scripts then invoke each CLI headlessly.
@@ -104,7 +104,7 @@ export class CodeArenaRunnerStack extends Stack {
104104
actions: ['cloudwatch:PutMetricData'],
105105
resources: ['*'],
106106
conditions: {
107-
StringEquals: { 'cloudwatch:namespace': 'CodeArena/Runner' },
107+
StringEquals: { 'cloudwatch:namespace': 'CoderCup/Runner' },
108108
},
109109
}),
110110
);
@@ -128,7 +128,7 @@ export class CodeArenaRunnerStack extends Stack {
128128
// + StartJob) and clean up via DeleteApp. Tag-conditional resource
129129
// scoping is awkward on Amplify because the app id is unknown at
130130
// CreateApp time; v1 narrows by region + account instead, and the
131-
// driver tags every CreateApp with Project=CodeArena +
131+
// driver tags every CreateApp with Project=CoderCup +
132132
// ManagedBy=ClaudeCode + AgentSlug + RunId for accounting.
133133
role.addToPolicy(
134134
new PolicyStatement({
@@ -194,7 +194,7 @@ export class CodeArenaRunnerStack extends Stack {
194194
'mkdir -p /opt/codearena',
195195
'chown ec2-user:ec2-user /opt/codearena',
196196
'echo "export PATH=/usr/local/bin:/usr/bin:/bin:/opt/codearena/bin" >> /home/ec2-user/.bashrc',
197-
'echo "CodeArena runner provisioned $(date -Iseconds)" > /opt/codearena/PROVISIONED',
197+
'echo "CoderCup runner provisioned $(date -Iseconds)" > /opt/codearena/PROVISIONED',
198198
'mkdir -p /home/ec2-user/.codearena',
199199
'chown -R ec2-user:ec2-user /home/ec2-user/.codearena',
200200
);

infra/cdk/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"name": "codearena-infra",
33
"version": "0.1.0",
44
"private": true,
5-
"description": "AWS CDK infrastructure for CodeArena. Deploys to us-east-1. All resources prefixed codearena-, tagged Project=CodeArena ManagedBy=ClaudeCode.",
5+
"description": "AWS CDK infrastructure for CoderCup. Deploys to us-east-1. All resources prefixed codearena-, tagged Project=CodeArena ManagedBy=ClaudeCode.",
66
"scripts": {
77
"build": "tsc",
88
"synth": "cdk synth",

runners/contract/schema.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* Runner contract for CodeArena.
2+
* Runner contract for CoderCup.
33
*
44
* Every agent driver — claude_code_session, codex_cli, vertex_gemini_api, and any
55
* future entrants — consumes a TaskRunInput and produces a RunManifest. The

runners/shared/amplify-deploy.ts

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,15 @@
55
* for the deploy to complete.
66
*
77
* One Amplify app per run, named codearena-run-<run-id>. Tagged
8-
* Project=CodeArena + ManagedBy=ClaudeCode + AgentSlug + RunId so
8+
* Project=CoderCup + ManagedBy=ClaudeCode + AgentSlug + RunId so
99
* AWS Cost Explorer can attribute spend per-run.
1010
*
1111
* Stays in STUB MODE until codearena/github-pat secret exists - the
1212
* Amplify CreateApp call needs the GitHub access token to clone the
1313
* source repo on its end.
1414
*
1515
* Trade-offs vs alternative deploy targets (per the design review
16-
* 2026-05-25): Amplify chosen because the CodeArena leaderboard itself
16+
* 2026-05-25): Amplify chosen because the CoderCup leaderboard itself
1717
* runs on Amplify already, the GitHub App authorization for the
1818
* TestSprite org is already in place, and per-run apps preserve the
1919
* archival value (every historical run has a permanent reachable URL).
@@ -67,13 +67,13 @@ export async function deployRunBranch(
6767
const created = await amplify.send(
6868
new CreateAppCommand({
6969
name: `codearena-run-${opts.runId}`,
70-
description: `CodeArena run artifact for ${opts.agentSlug} on branch ${opts.branchName}`,
70+
description: `CoderCup run artifact for ${opts.agentSlug} on branch ${opts.branchName}`,
7171
repository: SOURCE_REPO,
7272
accessToken: pat,
7373
platform: 'WEB',
7474
enableBranchAutoBuild: true,
7575
tags: {
76-
Project: 'CodeArena',
76+
Project: 'CoderCup',
7777
ManagedBy: 'ClaudeCode',
7878
AgentSlug: opts.agentSlug,
7979
RunId: opts.runId,

0 commit comments

Comments
 (0)