Skip to content

docs(evals): plan AgentV eval authoring extensibility#1316

Merged
christso merged 6 commits into
mainfrom
docs/agentv-extensibility-plan
Jun 7, 2026
Merged

docs(evals): plan AgentV eval authoring extensibility#1316
christso merged 6 commits into
mainfrom
docs/agentv-extensibility-plan

Conversation

@christso

@christso christso commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds the AgentV eval authoring extensibility plan produced from the public demo and private framework-parity research.
  • Grounds recommendations in comparable framework patterns from promptfoo, Braintrust, Phoenix, and Pi coding-agent skills/extensions.
  • Keeps the plan conservative: improve docs/templates/adapters first, add only proven primitive schema gaps to core.

Key Decisions

  • Keep AgentV core as a lightweight primitive execution layer.
  • Use private parity artifacts for promptfoo/Braintrust/Phoenix comparison instead of public competitor examples.
  • Treat Pi coding-agent style skills/extensions as the authoring-pattern inspiration for progressive-disclosure eval builders.
  • Prioritize benchmark provenance docs, rubric criterion operator support, narrow templates/adapters, and trace/replay follow-up Beads.

Evidence

  • Branch includes the plan at docs/plans/2026-06-06-001-agentv-eval-authoring-extensibility-plan.md.
  • Related Beads record completion of private parity artifacts and follow-up work split under av-r0s.5.

@christso christso merged commit 4e3d685 into main Jun 7, 2026
8 checks passed
@christso christso deleted the docs/agentv-extensibility-plan branch June 7, 2026 05:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant