Skip to content

Add OpenAI-compatible embedding API support for evaluation#10

Open
cindehaa wants to merge 2 commits into
fresh-stack:mainfrom
cindehaa:main
Open

Add OpenAI-compatible embedding API support for evaluation#10
cindehaa wants to merge 2 commits into
fresh-stack:mainfrom
cindehaa:main

Conversation

@cindehaa

Copy link
Copy Markdown

Summary

  • Tooling only
  • Adds a way to evaluate API-based embedding models via generic encoder for OpenAI-compatible providers

Validation

  • Ran the full pipeline (embed → retrieve → evaluate → write runfile + results JSON) end-to-end against the live voyage-4-large API on a small langchain slice
  • OpenAI client works against Voyage's endpoint, the response is data[].embedding-shaped, and the run produces a valid TREC runfile and metrics JSON

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant