Skip to content

fix(eval): remove public preprocessor authoring#1656

Merged
christso merged 1 commit into
mainfrom
promptfoo-preprocessor-removal
Jul 5, 2026
Merged

fix(eval): remove public preprocessor authoring#1656
christso merged 1 commit into
mainfrom
promptfoo-preprocessor-removal

Conversation

@christso

@christso christso commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

Public authoring now has one output-shaping contract: use transform. The SDK no longer exports or serializes preprocessor helper options, defineEval() rejects top-level preprocessors, and the YAML loader no longer parses suite-level preprocessor config after rejecting authored usage.

The file-output example and current docs now present this as a transform workflow (examples/features/file-transforms) instead of a preprocessor example. Internal content conversion plumbing remains available only behind non-public runtime types, and retained source traceability uses content_transform_command terminology instead of preprocessor_command.

Validation

  • bun install
  • bun test packages/core/test/evaluation/preprocessors-yaml.test.ts packages/core/test/evaluation/transform-runtime.test.ts packages/core/test/evaluation/validation/eval-validator.test.ts packages/core/test/evaluation/validation/eval-file-schema.test.ts
  • bun test packages/core/test/evaluation/source-traceability.test.ts packages/core/test/evaluation/orchestrator.test.ts packages/core/test/evaluation/llm-grader-multimodal.test.ts
  • bun --filter @agentv/core typecheck
  • bun --filter @agentv/sdk test
  • bun --filter @agentv/sdk typecheck
  • bun run validate:examples
  • bun run lint
  • bun --filter @agentv/sdk build

Live provider/grader dogfood was not rerun here because this PR hardens public authoring rejection and SDK/docs surfaces; the downstream transform docs/example dogfood gate remains tracked by av-kfik.16.

Related

Related: av-kfik.34.4


Compound Engineering
Codex

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: c9a44ac
Status: ✅  Deploy successful!
Preview URL: https://67495f50.agentv.pages.dev
Branch Preview URL: https://promptfoo-preprocessor-remov.agentv.pages.dev

View logs

@christso christso marked this pull request as ready for review July 5, 2026 03:26
@christso christso merged commit 94e91a5 into main Jul 5, 2026
8 checks passed
@christso christso deleted the promptfoo-preprocessor-removal branch July 5, 2026 03:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant