Skip to content

Commit a995a89

Browse files
sjarmakclaude
sjarmak
and
claude
committed
feat: backfill reviewers.json for all 410 benchmark tasks
- Add scripts/backfill_reviewers.py: queries GitHub API for PR reviewers and top contributors per task's code areas. Handles sg-evals mirrors, bot filtering, and multiple discovery strategies (source_pr, oracle, Dockerfile, task name inference). - Generate reviewers.json for 410/410 canonical tasks (76% have suggested_reviewers with real contributor data). - Update skills/mine-tasks/SKILL.md: add Step 3e (SDLC reviewer extraction) and Step 4c-extra (org-scale reviewer extraction). - Update skills/scaffold-task/SKILL.md: add Template 5 (reviewers.json) with unified schema for both PR-based and git-log-frequency discovery. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 01b8113 commit a995a89

File tree

409 files changed

+13134
-6
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

409 files changed

+13134
-6
lines changed

‎benchmarks/csb_org_compliance/ccx-compliance-051/reviewers.json‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
{
2+
"task_id": "CCX-compliance-051",
3+
"repos": [
4+
"sourcegraph-testing/prometheus-common",
5+
"prometheus/prometheus"
6+
],
7+
"top_contributors": [
8+
{
9+
"login": "paulojmdias",
10+
"commits": 11
11+
},
12+
{
13+
"login": "mmorel-35",
14+
"commits": 6
15+
},
16+
{
17+
"login": "bwplotka",
18+
"commits": 5
19+
},
20+
{
21+
"login": "matt-gp",
22+
"commits": 3
23+
},
24+
{
25+
"login": "machine424",
26+
"commits": 3
27+
}
28+
],
29+
"code_areas": [
30+
"cmd/prometheus/",
31+
"config/config.go/",
32+
"discovery/openstack/",
33+
"scrape/scrape.go/",
34+
"tracing/tracing.go/",
35+
"config/http_config.go/",
36+
"cmd/promtool/",
37+
"discovery/triton/",
38+
"web/web.go/",
39+
"storage/remote/"
40+
],
41+
"suggested_reviewers": [
42+
"paulojmdias",
43+
"mmorel-35",
44+
"bwplotka"
45+
],
46+
"discovery_method": "git_log_frequency"
47+
}

‎benchmarks/csb_org_compliance/ccx-compliance-052/reviewers.json‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
{
2+
"task_id": "CCX-compliance-052",
3+
"repos": [
4+
"envoyproxy/envoy",
5+
"grpc/grpc"
6+
],
7+
"top_contributors": [
8+
{
9+
"login": "agrawroh",
10+
"commits": 13
11+
},
12+
{
13+
"login": "adisuissa",
14+
"commits": 7
15+
},
16+
{
17+
"login": "wbpcode",
18+
"commits": 5
19+
},
20+
{
21+
"login": "wdauchy",
22+
"commits": 3
23+
},
24+
{
25+
"login": "TAOXUY",
26+
"commits": 3
27+
}
28+
],
29+
"code_areas": [
30+
"source/extensions/",
31+
"source/common/",
32+
"envoy/extensions/"
33+
],
34+
"suggested_reviewers": [
35+
"agrawroh",
36+
"adisuissa",
37+
"wbpcode"
38+
],
39+
"discovery_method": "git_log_frequency"
40+
}

‎benchmarks/csb_org_compliance/ccx-compliance-053/reviewers.json‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
{
2+
"task_id": "CCX-compliance-053",
3+
"repos": [
4+
"apache/kafka"
5+
],
6+
"top_contributors": [
7+
{
8+
"login": "AndrewJSchofield",
9+
"commits": 5
10+
},
11+
{
12+
"login": "mimaison",
13+
"commits": 5
14+
},
15+
{
16+
"login": "mingyen066",
17+
"commits": 4
18+
},
19+
{
20+
"login": "DL1231",
21+
"commits": 4
22+
},
23+
{
24+
"login": "lianetm",
25+
"commits": 4
26+
}
27+
],
28+
"code_areas": [
29+
"clients/src/",
30+
"metadata/src/"
31+
],
32+
"suggested_reviewers": [
33+
"AndrewJSchofield",
34+
"mimaison",
35+
"mingyen066"
36+
],
37+
"discovery_method": "git_log_frequency"
38+
}

‎benchmarks/csb_org_compliance/ccx-compliance-115/reviewers.json‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
{
2+
"task_id": "CCX-compliance-115",
3+
"repos": [
4+
"django/django"
5+
],
6+
"top_contributors": [
7+
{
8+
"login": "jacobtylerwalls",
9+
"commits": 21
10+
},
11+
{
12+
"login": "nessita",
13+
"commits": 15
14+
},
15+
{
16+
"login": "felixxm",
17+
"commits": 8
18+
},
19+
{
20+
"login": "RealOrangeOne",
21+
"commits": 4
22+
},
23+
{
24+
"login": "cliffordgama",
25+
"commits": 3
26+
}
27+
],
28+
"code_areas": [
29+
"docs/faq/",
30+
"docs/releases/",
31+
"docs/internals/",
32+
"tests/model_inheritance_regress/",
33+
"django/contrib/"
34+
],
35+
"suggested_reviewers": [
36+
"jacobtylerwalls",
37+
"nessita",
38+
"felixxm"
39+
],
40+
"discovery_method": "git_log_frequency"
41+
}

‎benchmarks/csb_org_compliance/ccx-compliance-118/reviewers.json‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
{
2+
"task_id": "CCX-compliance-118",
3+
"repos": [
4+
"django/django"
5+
],
6+
"top_contributors": [
7+
{
8+
"login": "felixxm",
9+
"commits": 7
10+
},
11+
{
12+
"login": "Antoliny0919",
13+
"commits": 6
14+
},
15+
{
16+
"login": "jacobtylerwalls",
17+
"commits": 5
18+
},
19+
{
20+
"login": "timgraham",
21+
"commits": 3
22+
},
23+
{
24+
"login": "sarahboyce",
25+
"commits": 3
26+
}
27+
],
28+
"code_areas": [
29+
"tests/admin_filters/",
30+
"django/contrib/"
31+
],
32+
"suggested_reviewers": [
33+
"felixxm",
34+
"Antoliny0919",
35+
"jacobtylerwalls"
36+
],
37+
"discovery_method": "git_log_frequency"
38+
}

‎benchmarks/csb_org_compliance/ccx-compliance-124/reviewers.json‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"task_id": "CCX-compliance-124",
3+
"repos": [
4+
"nicknisi/firefox"
5+
],
6+
"code_areas": [
7+
"dom/security/",
8+
"dom/interfaces/"
9+
],
10+
"discovery_method": "github_api"
11+
}

‎benchmarks/csb_org_compliance/ccx-compliance-182/reviewers.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"task_id": "CCX-compliance-182",
3+
"repos": [
4+
"kubernetes/kubernetes"
5+
],
6+
"code_areas": [
7+
"sg-evals/api--v0.32.0/",
8+
"sg-evals/client-go--v0.32.0/",
9+
"sg-evals/kubernetes--v1.32.0/"
10+
],
11+
"discovery_method": "github_api"
12+
}

‎benchmarks/csb_org_compliance/ccx-compliance-183/reviewers.json‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"task_id": "CCX-compliance-183",
3+
"repos": [
4+
"kubernetes/kubernetes"
5+
],
6+
"code_areas": [
7+
"sg-evals/kubernetes--v1.32.0/",
8+
"sg-evals/etcd-io-etcd/"
9+
],
10+
"discovery_method": "github_api"
11+
}

‎benchmarks/csb_org_compliance/ccx-compliance-184/reviewers.json‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"task_id": "CCX-compliance-184",
3+
"repos": [
4+
"kubernetes/kubernetes"
5+
],
6+
"code_areas": [
7+
"sg-evals/kubernetes--v1.32.0/"
8+
],
9+
"discovery_method": "github_api"
10+
}

‎benchmarks/csb_org_compliance/ccx-compliance-185/reviewers.json‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"task_id": "CCX-compliance-185",
3+
"repos": [
4+
"prometheus/prometheus"
5+
],
6+
"code_areas": [
7+
"prometheus/prometheus/"
8+
],
9+
"discovery_method": "github_api"
10+
}

0 commit comments

Comments
 (0)