Skip to content

Commit 78d86e2

Browse files
LoCoBench Botclaude
andcommitted
feat: US-010 - Update codereview config with public GitHub repos
- Updated TASK_SG_REPO_NAMES in configs/codereview_2config.sh - Changed from sg-benchmarks clones to public GitHub repos - All three tasks now use: github.com/TryGhost/Ghost, github.com/dotnet/aspnetcore, github.com/calcom/cal.com - Also marked US-009 as passing (was already complete) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent 197a241 commit 78d86e2

3 files changed

Lines changed: 30 additions & 7 deletions

File tree

configs/codereview_2config.sh

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -119,9 +119,9 @@ for t in tasks:
119119
# Sourcegraph repo name mapping for code review tasks
120120
# Each task targets a different public GitHub repo
121121
declare -A TASK_SG_REPO_NAMES=(
122-
["cr-aspnetcore-001"]="github.com/sg-benchmarks/aspnetcore--87525573"
123-
["cr-calcom-001"]="github.com/sg-benchmarks/cal.com--4b99072b"
124-
["cr-ghost-001"]="github.com/sg-benchmarks/Ghost--b43bfc85"
122+
["cr-ghost-001"]="github.com/TryGhost/Ghost"
123+
["cr-aspnetcore-001"]="github.com/dotnet/aspnetcore"
124+
["cr-calcom-001"]="github.com/calcom/cal.com"
125125
)
126126

127127
# Derive short model name for run directory

prd.json

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -131,8 +131,8 @@
131131
"Registered in configs/selected_benchmark_tasks.json with correct metadata"
132132
],
133133
"priority": 9,
134-
"passes": false,
135-
"notes": "Follow same pattern as finalized cr-ghost-001. Choose a cal.com PR from benchmark-pr-mapping with 3+ files."
134+
"passes": true,
135+
"notes": "Used cal.com PR #26801 (feature opt-in scope configuration). 4 defects across 3 TypeScript files: FeatureOptInService.ts, config.ts, _router.ts. Hybrid scoring, inject_defects.sh, all files created and registered."
136136
},
137137
{
138138
"id": "US-010",
@@ -144,8 +144,8 @@
144144
"Verify script syntax: bash -n configs/codereview_3config.sh passes"
145145
],
146146
"priority": 10,
147-
"passes": false,
148-
"notes": "Sourcegraph repo names for public repos: github.com/TryGhost/Ghost, github.com/dotnet/aspnetcore, github.com/calcom/cal.com"
147+
"passes": true,
148+
"notes": "Updated TASK_SG_REPO_NAMES in configs/codereview_2config.sh to use public GitHub repos instead of sg-benchmarks clones. All three tasks mapped. Script syntax validated."
149149
},
150150
{
151151
"id": "US-011",

progress.txt

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,3 +131,26 @@
131131
- MCP tools used: sg_read_file (74%), sg_list_files (19%), sg_keyword_search (6%), sg_find_references (2%). Deep Search: 0%.
132132
- **Recommendation**: Redesign around cross-codebase discovery tasks for MCP to shine
133133
---
134+
## 2026-02-16 - US-009
135+
- Verified cr-calcom-001 task is fully scaffolded (was completed but not marked in PRD)
136+
- Task uses cal.com PR #26801 (feature opt-in scope configuration)
137+
- 4 defects across 3 TypeScript files: FeatureOptInService.ts, config.ts, _router.ts
138+
- All acceptance criteria met: task.toml, instruction.md (no TODOs), Dockerfile, test.sh (executable, hybrid scoring), expected_defects.json, expected_patches/, registered in selected_benchmark_tasks.json
139+
- Files changed: prd.json (marked US-009 as passing)
140+
- **Learnings for future iterations:**
141+
- PRD can get out of sync with actual work if previous iterations didn't update it — always verify actual file state before assuming work needs to be done
142+
- Validation checklist useful: directory exists, files exist, metadata correct, no TODOs, executable permissions, registration in JSON
143+
---
144+
145+
## 2026-02-16 - US-010
146+
- Updated configs/codereview_2config.sh TASK_SG_REPO_NAMES to use public GitHub repos instead of sg-benchmarks clones
147+
- Changed from: github.com/sg-benchmarks/{repo}--{commit} to github.com/{org}/{repo}
148+
- All three tasks now point to: TryGhost/Ghost, dotnet/aspnetcore, calcom/cal.com
149+
- Script syntax validated with bash -n
150+
- Files changed: configs/codereview_2config.sh, prd.json
151+
- **Learnings for future iterations:**
152+
- PRD referred to "codereview_3config.sh" but actual file is "codereview_2config.sh" — the 2config pattern is standard (2 configs: baseline + sourcegraph_full)
153+
- Public GitHub repos are used for Sourcegraph when the actual repos are available (vs sg-benchmarks clones for pinned commits)
154+
- TASK_SG_REPO_NAMES uses format github.com/{org}/{repo} for public repos
155+
---
156+

0 commit comments

Comments
 (0)