docs: refresh benchmark catalog docs and add technical report

- README: update MCP-unique section (6 suites/12 tasks -> 11 suites/81
  tasks), total 251, repo structure reflects all 11 ccb_mcp_* suites
- benchmarks/README: replace 3 removed protonmail tasks with new envoy
  and terraform fix tasks
- TASK_CATALOG: same protonmail -> envoy/terraform replacement
- Dockerfiles: clone-as-claude migration for 84 MCP-unique baselines
- Promote fix_haiku_20260226_new3tasks to official

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: sjarmak

README.md

@@ -81,19 +81,24 @@ Eight suites organized by software development lifecycle phase:
 
 ## MCP-Unique Suites (Org-Scale Context Retrieval)
 
-Six additional suites measure cross-repo discovery, symbol resolution, dependency tracing, and deep-search-driven investigation in polyrepo environments.
+Eleven additional suites measure cross-repo discovery, symbol resolution, dependency tracing, and deep-search-driven investigation in polyrepo environments.
 
 | Suite | Category | Tasks | Description |
 |-------|----------|------:|-------------|
-| `ccb_mcp_crossrepo_tracing` | A: Dependency Tracing | 3 | Cross-repo dependency chains, blast radius, symbol resolution |
-| `ccb_mcp_security` | B: Vulnerability Remediation | 2 | CVE mapping, missing auth middleware across repos |
-| `ccb_mcp_incident` | D: Incident Debugging | 1 | Error-to-code-path tracing across microservices |
-| `ccb_mcp_onboarding` | E: Onboarding & Comprehension | 3 | API consumption mapping, end-to-end flow, architecture maps |
-| `ccb_mcp_crossorg` | G: Cross-Org Discovery | 2 | Interface implementations and authoritative repo identification across orgs |
-| `ccb_mcp_platform` | J: Platform Knowledge | 1 | Service template discovery and tribal knowledge |
-| **Total** | | **12** | |
-
-The table above shows the 12 tasks evaluated in official runs. The full MCP-unique catalog has 20 tasks across 8 suites (including compliance and migration, pending first runs). **Combined catalog total: 190 tasks** (170 SDLC + 20 MCP-unique).
+| `ccb_mcp_crossrepo_tracing` | A: Dependency Tracing | 9 | Cross-repo dependency chains, blast radius, symbol resolution |
+| `ccb_mcp_security` | B: Vulnerability Remediation | 10 | CVE mapping, missing auth middleware across repos |
+| `ccb_mcp_migration` | C: Framework Migration | 7 | API migrations, breaking changes across repos |
+| `ccb_mcp_incident` | D: Incident Debugging | 11 | Error-to-code-path tracing across microservices |
+| `ccb_mcp_onboarding` | E: Onboarding & Comprehension | 11 | API consumption mapping, end-to-end flow, architecture maps |
+| `ccb_mcp_compliance` | F: Compliance | 7 | Standards adherence, audit, and provenance workflows |
+| `ccb_mcp_crossorg` | G: Cross-Org Discovery | 5 | Interface implementations and authoritative repo identification across orgs |
+| `ccb_mcp_domain` | H: Domain Lineage | 10 | Config propagation, architecture patterns, domain analysis |
+| `ccb_mcp_org` | I: Organizational Context | 5 | Agentic discovery, org-wide coding correctness |
+| `ccb_mcp_platform` | J: Platform Knowledge | 5 | Service template discovery and tribal knowledge |
+| `ccb_mcp_crossrepo` | Legacy | 1 | Cross-repo discovery (compatibility) |
+| **Total** | | **81** | |
+
+**Combined catalog total: 251 tasks** (170 SDLC + 81 MCP-unique). Of these, 212 are fully paired (baseline + MCP results) in official runs; the remaining 39 MCP-unique tasks have MCP results but are missing baselines.
 
 Both baseline and MCP-Full agents have access to **all repos** in each task's fixture. The only difference is the method: baseline reads code locally, MCP-Full uses Sourcegraph MCP tools (local code is truncated). This ensures we measure whether MCP tools help agents work better — not whether MCP can access repos the baseline can't.
 
@@ -133,12 +138,17 @@ benchmarks/              # Task definitions organized by SDLC phase + MCP-unique
   ccb_secure/            #   Security & Compliance (20 tasks)
   ccb_test/              #   Testing & QA (20 tasks)
   ccb_understand/        #   Requirements & Discovery (20 tasks)
-  ccb_mcp_crossrepo_tracing/  #   MCP-unique: cross-repo dependency tracing (3 tasks)
-  ccb_mcp_security/      #   MCP-unique: vulnerability remediation (2 tasks)
-  ccb_mcp_incident/      #   MCP-unique: incident debugging (1 task)
-  ccb_mcp_onboarding/    #   MCP-unique: onboarding & comprehension (3 tasks)
-  ccb_mcp_crossorg/      #   MCP-unique: cross-org discovery (2 tasks)
-  ccb_mcp_platform/      #   MCP-unique: platform knowledge (1 task)
+  ccb_mcp_compliance/    #   MCP-unique: compliance & audit (7 tasks)
+  ccb_mcp_crossorg/      #   MCP-unique: cross-org discovery (5 tasks)
+  ccb_mcp_crossrepo/     #   MCP-unique: legacy cross-repo (1 task)
+  ccb_mcp_crossrepo_tracing/  #   MCP-unique: dependency tracing (9 tasks)
+  ccb_mcp_domain/        #   MCP-unique: domain lineage (10 tasks)
+  ccb_mcp_incident/      #   MCP-unique: incident debugging (11 tasks)
+  ccb_mcp_migration/     #   MCP-unique: framework migration (7 tasks)
+  ccb_mcp_onboarding/    #   MCP-unique: onboarding (11 tasks)
+  ccb_mcp_org/           #   MCP-unique: org context (5 tasks)
+  ccb_mcp_platform/      #   MCP-unique: platform knowledge (5 tasks)
+  ccb_mcp_security/      #   MCP-unique: vulnerability remediation (10 tasks)
 configs/                 # Run configs and task selection
   _common.sh             #   Shared infra: token refresh, parallel execution, multi-account
   sdlc_suite_2config.sh  #   Generic SDLC runner (used by phase wrappers below)

benchmarks/README.md

@@ -126,9 +126,9 @@ Diagnosing and fixing real bugs across production codebases (SWE-bench Pro, PyTo
 | `openlibrary-fntocli-adapter-fix-001` | FnToCLI adapter list inputs paths |
 | `openlibrary-search-query-fix-001` | Work search query parsing normalization |
 | `openlibrary-solr-boolean-fix-001` | Solr boolean clause limit alignment |
-| `protonmail-conv-testhooks-fix-001` | Conversation message view test hooks |
-| `protonmail-dropdown-sizing-fix-001` | Dropdown unified sizing configuration |
-| `protonmail-holiday-calendar-fix-001` | Public holiday calendar management |
+| `envoy-dfp-host-leak-fix-001` | Dynamic forward proxy host header memory leak |
+| `envoy-udp-proxy-cds-fix-001` | UDP proxy crash on dynamic CDS/EDS cluster update |
+| `terraform-plan-null-unknown-fix-001` | Terraform plan null/unknown value rendering |
 | `pytorch-cudnn-version-fix-001` | Expose cuDNN runtime version |
 | `pytorch-dynamo-keyerror-fix-001` | Fix dynamo keyerror and attribute |
 | `pytorch-release-210-fix-001` | Release 2.10 bug fix changes |

benchmarks/ccb_mcp_compliance/ccx-compliance-051/environment/Dockerfile

@@ -11,22 +11,22 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
      \
     && rm -rf /var/lib/apt/lists/*
 
+# Create claude user BEFORE any work so files are owned correctly from the
+# start.  This avoids a post-clone chown -R layer that doubles image size
+# and takes 15-30 min on overlay2 (copy-on-write duplicates every inode).
+RUN adduser --disabled-password --gecos '' claude 2>/dev/null || true
+RUN mkdir -p /workspace /logs/agent /logs/verifier && \
+    chown -R claude:claude /workspace /logs
+
+USER claude
 WORKDIR /workspace
 
-# Clone local checkout repos (baseline config: agent has local access to these)
-# No local checkout repos specified for this fixture
-
 # Initialize git identity for agent commits
 RUN git config --global user.email "agent@example.com" && \
     git config --global user.name "Agent" && \
     git config --global safe.directory '*'
 
-# Create log directories
-RUN mkdir -p /logs/agent /logs/verifier
-
-# Pre-create claude user and set ownership at build time so Harbor's
-# runtime chown is a no-op (avoids 15-30 min delay on large repos).
-RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \
-    for d in /workspace /app /testbed /logs; do [ -d "$d" ] && chown -R claude:claude "$d"; done || true
+# Switch back to root for Harbor's runtime setup
+USER root
 
 ENTRYPOINT []

benchmarks/ccb_mcp_compliance/ccx-compliance-052/environment/Dockerfile

@@ -11,8 +11,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     g++ make \
     && rm -rf /var/lib/apt/lists/*
 
+# Create claude user BEFORE cloning so files are owned correctly from the
+# start.  This avoids a post-clone chown -R layer that doubles image size
+# and takes 15-30 min on overlay2 (copy-on-write duplicates every inode).
+RUN adduser --disabled-password --gecos '' claude 2>/dev/null || true
+RUN mkdir -p /workspace /logs/agent /logs/verifier && \
+    chown -R claude:claude /workspace /logs
+
+# Clone as claude — files land claude-owned, no separate chown layer needed.
+USER claude
 WORKDIR /workspace
-
 # Clone local checkout repos (baseline config: agent has local access to these)
 RUN git clone --depth 1 https://github.com/sg-evals/envoy--v1.31.2 /workspace/envoy--v1.31.2
 RUN git clone --depth 1 https://github.com/sg-evals/data-plane-api--84e84367 /workspace/data-plane-api--84e84367
@@ -24,12 +32,7 @@ RUN git config --global user.email "agent@example.com" && \
     git config --global user.name "Agent" && \
     git config --global safe.directory '*'
 
-# Create log directories
-RUN mkdir -p /logs/agent /logs/verifier
-
-# Pre-create claude user and set ownership at build time so Harbor's
-# runtime chown is a no-op (avoids 15-30 min delay on large repos).
-RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \
-    for d in /workspace /app /testbed /logs; do [ -d "$d" ] && chown -R claude:claude "$d"; done || true
+# Switch back to root for Harbor's runtime setup
+USER root
 
 ENTRYPOINT []

benchmarks/ccb_mcp_compliance/ccx-compliance-053/environment/Dockerfile

@@ -11,8 +11,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     default-jdk \
     && rm -rf /var/lib/apt/lists/*
 
+# Create claude user BEFORE cloning so files are owned correctly from the
+# start.  This avoids a post-clone chown -R layer that doubles image size
+# and takes 15-30 min on overlay2 (copy-on-write duplicates every inode).
+RUN adduser --disabled-password --gecos '' claude 2>/dev/null || true
+RUN mkdir -p /workspace /logs/agent /logs/verifier && \
+    chown -R claude:claude /workspace /logs
+
+# Clone as claude — files land claude-owned, no separate chown layer needed.
+USER claude
 WORKDIR /workspace
-
 # Clone local checkout repos (baseline config: agent has local access to these)
 RUN git clone --depth 1 https://github.com/sg-evals/kafka--0753c489 /workspace/kafka--0753c489
 RUN git clone --depth 1 https://github.com/sg-evals/flink--0cc95fcc /workspace/flink--0cc95fcc
@@ -23,12 +31,7 @@ RUN git config --global user.email "agent@example.com" && \
     git config --global user.name "Agent" && \
     git config --global safe.directory '*'
 
-# Create log directories
-RUN mkdir -p /logs/agent /logs/verifier
-
-# Pre-create claude user and set ownership at build time so Harbor's
-# runtime chown is a no-op (avoids 15-30 min delay on large repos).
-RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \
-    for d in /workspace /app /testbed /logs; do [ -d "$d" ] && chown -R claude:claude "$d"; done || true
+# Switch back to root for Harbor's runtime setup
+USER root
 
 ENTRYPOINT []

benchmarks/ccb_mcp_compliance/ccx-compliance-057-ds/environment/Dockerfile

@@ -10,8 +10,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     python3 \
     && rm -rf /var/lib/apt/lists/*
 
-WORKDIR /workspace
+# Create claude user BEFORE cloning so files are owned correctly from the
+# start.  This avoids a post-clone chown -R layer that doubles image size
+# and takes 15-30 min on overlay2 (copy-on-write duplicates every inode).
+RUN adduser --disabled-password --gecos '' claude 2>/dev/null || true
+RUN mkdir -p /workspace /logs/agent /logs/verifier && \
+    chown -R claude:claude /workspace /logs
 
+# Clone as claude — files land claude-owned, no separate chown layer needed.
+USER claude
+WORKDIR /workspace
 # Clone all fixture repos (baseline has full local access to every repo)
 RUN git clone --depth 1 https://github.com/sg-evals/grafana--v11.4.0.git /workspace/grafana
 RUN git clone --depth 1 https://github.com/sg-evals/loki--v3.3.4.git /workspace/loki
@@ -21,7 +29,7 @@ RUN git config --global user.email "agent@example.com" && \
     git config --global user.name "Agent" && \
     git config --global safe.directory '*'
 
-# Create log directories
-RUN mkdir -p /logs/agent /logs/verifier
+# Switch back to root for Harbor's runtime setup
+USER root
 
 ENTRYPOINT []

benchmarks/ccb_mcp_compliance/ccx-compliance-057/environment/Dockerfile

@@ -11,8 +11,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     golang-go \
     && rm -rf /var/lib/apt/lists/*
 
+# Create claude user BEFORE cloning so files are owned correctly from the
+# start.  This avoids a post-clone chown -R layer that doubles image size
+# and takes 15-30 min on overlay2 (copy-on-write duplicates every inode).
+RUN adduser --disabled-password --gecos '' claude 2>/dev/null || true
+RUN mkdir -p /workspace /logs/agent /logs/verifier && \
+    chown -R claude:claude /workspace /logs
+
+# Clone as claude — files land claude-owned, no separate chown layer needed.
+USER claude
 WORKDIR /workspace
-
 # Clone local checkout repos (baseline config: agent has local access to these)
 RUN git clone --depth 1 https://github.com/sg-evals/grafana--26d36ec /workspace/grafana--26d36ec
 
@@ -21,12 +29,7 @@ RUN git config --global user.email "agent@example.com" && \
     git config --global user.name "Agent" && \
     git config --global safe.directory '*'
 
-# Create log directories
-RUN mkdir -p /logs/agent /logs/verifier
-
-# Pre-create claude user and set ownership at build time so Harbor's
-# runtime chown is a no-op (avoids 15-30 min delay on large repos).
-RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \
-    for d in /workspace /app /testbed /logs; do [ -d "$d" ] && chown -R claude:claude "$d"; done || true
+# Switch back to root for Harbor's runtime setup
+USER root
 
 ENTRYPOINT []

benchmarks/ccb_mcp_compliance/ccx-compliance-115/environment/Dockerfile

@@ -11,8 +11,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     python3 python3-pip \
     && rm -rf /var/lib/apt/lists/*
 
+# Create claude user BEFORE cloning so files are owned correctly from the
+# start.  This avoids a post-clone chown -R layer that doubles image size
+# and takes 15-30 min on overlay2 (copy-on-write duplicates every inode).
+RUN adduser --disabled-password --gecos '' claude 2>/dev/null || true
+RUN mkdir -p /workspace /logs/agent /logs/verifier && \
+    chown -R claude:claude /workspace /logs
+
+# Clone as claude — files land claude-owned, no separate chown layer needed.
+USER claude
 WORKDIR /workspace
-
 # Clone local checkout repos (baseline config: agent has local access to these)
 RUN git clone --depth 1 https://github.com/sg-evals/django--674eda1c /workspace/django--674eda1c
 
@@ -21,12 +29,7 @@ RUN git config --global user.email "agent@example.com" && \
     git config --global user.name "Agent" && \
     git config --global safe.directory '*'
 
-# Create log directories
-RUN mkdir -p /logs/agent /logs/verifier
-
-# Pre-create claude user and set ownership at build time so Harbor's
-# runtime chown is a no-op (avoids 15-30 min delay on large repos).
-RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \
-    for d in /workspace /app /testbed /logs; do [ -d "$d" ] && chown -R claude:claude "$d"; done || true
+# Switch back to root for Harbor's runtime setup
+USER root
 
 ENTRYPOINT []

benchmarks/ccb_mcp_compliance/ccx-compliance-118/environment/Dockerfile

@@ -11,8 +11,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     python3 python3-pip \
     && rm -rf /var/lib/apt/lists/*
 
+# Create claude user BEFORE cloning so files are owned correctly from the
+# start.  This avoids a post-clone chown -R layer that doubles image size
+# and takes 15-30 min on overlay2 (copy-on-write duplicates every inode).
+RUN adduser --disabled-password --gecos '' claude 2>/dev/null || true
+RUN mkdir -p /workspace /logs/agent /logs/verifier && \
+    chown -R claude:claude /workspace /logs
+
+# Clone as claude — files land claude-owned, no separate chown layer needed.
+USER claude
 WORKDIR /workspace
-
 # Clone local checkout repos (baseline config: agent has local access to these)
 RUN git clone --depth 1 https://github.com/sg-evals/django--674eda1c /workspace/django--674eda1c
 
@@ -21,12 +29,7 @@ RUN git config --global user.email "agent@example.com" && \
     git config --global user.name "Agent" && \
     git config --global safe.directory '*'
 
-# Create log directories
-RUN mkdir -p /logs/agent /logs/verifier
-
-# Pre-create claude user and set ownership at build time so Harbor's
-# runtime chown is a no-op (avoids 15-30 min delay on large repos).
-RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \
-    for d in /workspace /app /testbed /logs; do [ -d "$d" ] && chown -R claude:claude "$d"; done || true
+# Switch back to root for Harbor's runtime setup
+USER root
 
 ENTRYPOINT []

benchmarks/ccb_mcp_compliance/ccx-compliance-124/environment/Dockerfile

@@ -11,8 +11,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     g++ make \
     && rm -rf /var/lib/apt/lists/*
 
+# Create claude user BEFORE cloning so files are owned correctly from the
+# start.  This avoids a post-clone chown -R layer that doubles image size
+# and takes 15-30 min on overlay2 (copy-on-write duplicates every inode).
+RUN adduser --disabled-password --gecos '' claude 2>/dev/null || true
+RUN mkdir -p /workspace /logs/agent /logs/verifier && \
+    chown -R claude:claude /workspace /logs
+
+# Clone as claude — files land claude-owned, no separate chown layer needed.
+USER claude
 WORKDIR /workspace
-
 # Clone local checkout repos (baseline config: agent has local access to these)
 RUN git clone --depth 1 https://github.com/sg-evals/firefox--871325b8 /workspace/firefox--871325b8
 
@@ -21,12 +29,7 @@ RUN git config --global user.email "agent@example.com" && \
     git config --global user.name "Agent" && \
     git config --global safe.directory '*'
 
-# Create log directories
-RUN mkdir -p /logs/agent /logs/verifier
-
-# Pre-create claude user and set ownership at build time so Harbor's
-# runtime chown is a no-op (avoids 15-30 min delay on large repos).
-RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \
-    for d in /workspace /app /testbed /logs; do [ -d "$d" ] && chown -R claude:claude "$d"; done || true
+# Switch back to root for Harbor's runtime setup
+USER root
 
 ENTRYPOINT []