|
34 | 34 | - The investigation task test.sh (inv-debug-001) is the closest template for docgen verifiers — same Python scoring engine |
35 | 35 | - docgen_2config.sh has two arrays to populate: ALL_TASK_IDS and TASK_SG_REPO_NAMES |
36 | 36 | --- |
| 37 | + |
| 38 | +## 2026-02-16 - US-002 |
| 39 | +- Created docgen-arch-002: Istio Pilot Discovery Service architecture doc task |
| 40 | +- Files created: |
| 41 | + - benchmarks/ccb_docgen/docgen-arch-002/task.toml (category=architecture_doc, language=go, difficulty=hard, time_limit_sec=1200) |
| 42 | + - benchmarks/ccb_docgen/docgen-arch-002/instruction.md (covers DiscoveryServer, ADS, PushContext, ServiceRegistry, multi-cluster support) |
| 43 | + - benchmarks/ccb_docgen/docgen-arch-002/tests/ground_truth.json (13 required_topics, 5 file_references, 3 data_flow, 4 extension_points) |
| 44 | + - benchmarks/ccb_docgen/docgen-arch-002/tests/test.sh (copied from docgen-arch-001, generic Python-based verifier) |
| 45 | + - benchmarks/ccb_docgen/docgen-arch-002/environment/Dockerfile (blobless clone of istio/istio at commit f8af3cae) |
| 46 | +- Files modified: |
| 47 | + - configs/docgen_2config.sh (added docgen-arch-002 to ALL_TASK_IDS and TASK_SG_REPO_NAMES) |
| 48 | + - configs/selected_benchmark_tasks.json (registered task with mcp_benefit_score=0.860, total_selected: 190→191) |
| 49 | +- **Learnings for future iterations:** |
| 50 | + - Istio Pilot architecture has 4 key layers: DiscoveryServer (xDS gRPC), ADS (connection model), PushContext (config snapshot), ServiceRegistry (K8s + multi-cluster) |
| 51 | + - Used WebSearch and WebFetch to research Istio architecture before writing ground truth |
| 52 | + - Found specific commit (f8af3caeb1) from GitHub's master branch via WebFetch |
| 53 | + - Ground truth covers pilot/pkg/xds/ (discovery.go, ads.go), pilot/pkg/model/push_context.go, pilot/pkg/serviceregistry/kube/controller/ |
| 54 | + - Same 4-category scoring pattern as docgen-arch-001: required_topics (0.40), file_references (0.25), data_flow (0.20), extension_points (0.15) |
| 55 | + - MCP advantage higher for Istio (0.860) than Envoy (0.855) due to more cross-file dependencies (0.92 vs 0.90) |
| 56 | + - test.sh is reusable across all docgen architecture tasks — just copy from docgen-arch-001 |
| 57 | +--- |
0 commit comments