Skip to content

feat(results): support configured storage branch#1365

Merged
christso merged 2 commits into
mainfrom
feat/results-repo-branch
Jun 12, 2026
Merged

feat(results): support configured storage branch#1365
christso merged 2 commits into
mainfrom
feat/results-repo-branch

Conversation

@christso

@christso christso commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add optional results.branch / ProjectResultsConfig.branch storage-branch support while preserving default-branch behavior when absent.
  • Route git-native status/sync/direct-push/list/materialize flows to origin/<branch> when configured.
  • Preserve project registry round-trips and document that branch_prefix is only for temporary result/PR branches.

Result-portability framing

AgentV's positioning is that teams should build portable eval packs on AgentV instead of bespoke eval frameworks. Result portability is part of that claim: outputs, traces, and result artifacts should sync through boring git-native storage and remain inspectable by Dashboard/remote sync without custom scripts.

This PR makes non-default storage branches (for example agentv-results) a small, explicit AgentV capability rather than an implicit assumption about the results repo default branch. The capability is generic results storage only; it does not absorb Harvey/Irys-specific benchmark shapes or domain schemas into AgentV core.

Separate results repositories remain the preferred/current path for shared remote results because they isolate generated artifacts from source history and keep permissions/retention policies simple. This PR only lets that dedicated results repo use a configured storage branch when teams want the repo default branch reserved for docs, bootstrap material, or governance files.

Design notes

  • This is intentionally minimal: no storage-branch auto-creation. A configured branch must already exist.
  • Missing configured branches fail with actionable orphan-branch initialization guidance.
  • Cache keys now include the storage ref so Dashboard git-native run listing does not mix default-branch and configured-branch results.
  • Schema restraint: results.branch is a storage ref, not a benchmark/result-shape extension point.

Red/green UAT evidence

  • Red (before change): direct push with runtime branch: "agentv-results" still pushed to default main (files_on_main=2, files_on_agentv_results=0).
  • Green (after change): identical direct-push scenario checked out and pushed agentv-results (clone_current_branch=agentv-results, files_on_main=0, files_on_agentv_results=2).
  • Operational proof is intentionally boring: Dashboard git-native listing, remote sync, materialization, and direct push all exercise the configured branch through existing AgentV flows, with no bespoke scripts.

Verification

  • bun test packages/core/test/evaluation/loaders/config-loader.test.ts packages/core/test/projects.test.ts packages/core/test/evaluation/validation/config-validator.test.ts packages/core/test/evaluation/results-repo.test.ts apps/cli/test/commands/results/serve.test.ts
  • bun test packages/core/test/evaluation/results-repo.test.ts apps/cli/test/commands/results/serve.test.ts
  • bun run typecheck
  • bun run lint
  • bun run test
  • bun run validate:examples
  • bun run build
  • Push hook also ran workspace typechecks + biome check . successfully.

Unsupported edge cases / follow-ups

  • Configured storage branch must already exist; initialize once with an orphan branch before syncing/pushing.
  • Repos with no usable default branch may still fail the initial clone before AgentV can check out the configured storage branch.
  • No changes to branch_prefix semantics.

Post-Deploy Monitoring & Validation

  • Watch GitHub Actions for build/typecheck/lint/test regressions.
  • For configured projects, validate Dashboard/API remote status has no last_error and that new runs land on the configured storage branch, not the results repo default branch.
  • Rollback path: remove results.branch from config/registry to return to prior default-branch behavior, or revert this PR.

Review note

Manual review/simplification pass completed; no actionable findings. Subagent review was not run because this harness only permits subagent tools when the user explicitly requests them.

Review fix: missing configured storage branch fallback

  • Addressed reviewer finding that a missing configured results.branch could be masked by filesystem fallback from the clone's checked-out default branch.
  • Fix: configured-branch status failures now report remote results as unavailable, and CLI run count/listing fallback no longer reads materialized files when a storage branch is configured.
  • Red evidence: the regression test failed against the pre-fix PR code with available=true and run_count=1 from the default branch despite branch: agentv-results being missing.
  • Green evidence: the same test now passes with available=false, sync_status=unavailable, run_count=0, no remote run listed, and direct detail lookup returning 404.

@christso christso marked this pull request as ready for review June 12, 2026 10:32
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 12, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 462c9a9
Status: ✅  Deploy successful!
Preview URL: https://a83ce9da.agentv.pages.dev
Branch Preview URL: https://feat-results-repo-branch.agentv.pages.dev

View logs

@christso christso merged commit a46dc26 into main Jun 12, 2026
8 checks passed
@christso christso deleted the feat/results-repo-branch branch June 12, 2026 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant