You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: benchmarks/README.md
+39-3Lines changed: 39 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,12 @@
1
1
# CodeContextBench Benchmarks
2
2
3
-
170 tasks organized into 8 suites aligned with the software development lifecycle (SDLC). Each suite targets a distinct phase of engineering work. The canonical task selection is in [`selected_benchmark_tasks.json`](../configs/selected_benchmark_tasks.json).
3
+
This directory contains SDLC-aligned suites plus MCP-unique org-scale retrieval suites. The canonical selected task catalog is in [`selected_benchmark_tasks.json`](../configs/selected_benchmark_tasks.json) (currently 251 selected tasks across 19 suites).
4
4
5
5
See [`docs/TASK_SELECTION.md`](../docs/TASK_SELECTION.md) for selection methodology.
6
6
7
7
---
8
8
9
-
## Suite Overview
9
+
## SDLC Suite Overview
10
10
11
11
| Suite | SDLC Phase | Tasks | Description |
12
12
|-------|-----------|------:|-------------|
@@ -22,6 +22,29 @@ See [`docs/TASK_SELECTION.md`](../docs/TASK_SELECTION.md) for selection methodol
22
22
23
23
---
24
24
25
+
## MCP-Unique Suite Overview (Selected Catalog)
26
+
27
+
These suites measure cross-repo discovery, tracing, and org-scale code intelligence use cases. Counts below reflect the current selected catalog in [`selected_benchmark_tasks.json`](../configs/selected_benchmark_tasks.json) (some suite directories may contain additional draft/deferred tasks that are not selected).
28
+
29
+
| Suite | Tasks | Description |
30
+
|-------|------:|-------------|
31
+
|`ccb_mcp_compliance`| 7 | Compliance, audit, and provenance workflows |
32
+
|`ccb_mcp_crossorg`| 5 | Cross-org discovery and authoritative repo identification |
0 commit comments