refactor(providers)!: adopt GenericEmbeddingModel for together/openrouter/mistral embeddings#2123
Open
ojhermann wants to merge 2 commits into
Open
Conversation
…uter/mistral embeddings Collapse the hand-rolled Together, OpenRouter, and Mistral embedding models onto the shared GenericEmbeddingModel<Ext> via a new OpenAIEmbeddingsCompatible path hook, mirroring OpenAICompatibleProvider::completion_path. Together embeddings now honor dimensions (via ndims), encoding_format, and user, which the previous hand-rolled copy silently dropped. EmbeddingResponse.usage becomes optional so providers that omit it (OpenRouter for some models, Together historically) are not regressed now that they share one response type. Removes the now-dead per-provider embedding response/format types and their orphaned error envelopes. Azure (needs the path hook plus deployment-scoped URLs) and Copilot (blocked on async auth headers) are left out of scope per the issue. Fixes 0xPlaygrounds#2078.
3c83d39 to
8d9b3a9
Compare
ojhermann
commented
Jul 12, 2026
ojhermann
commented
Jul 12, 2026
ojhermann
commented
Jul 12, 2026
… conventions Add unit tests covering per-provider embeddings_path resolution and the optional-usage handling: - mistral: asserts POST /v1/embeddings, embedding parse, and usage mapping - openrouter: asserts POST .../api/v1/embeddings and zeroed usage when absent - together: adds a negative test that `dimensions` is omitted when ndims unset Also trim the module/alias docs and inline comments added on the migrated files to match each provider's existing convention (mirroring the completion.rs siblings), per review feedback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2078.
Summary
Collapses the hand-rolled Together, OpenRouter, and Mistral embedding models onto the shared
GenericEmbeddingModel<Ext>— the embeddings analogue of the #2035 → #2038/#2040GenericCompletionModelmigration. Each provider'sembedding.rsshrinks from a ~140–190-line near-verbatim copy of the OpenAIPOST /embeddingsflow to a thin type alias.As called out in the issue, the Together copy had silently drifted: it dropped
dimensions(andencoding_format/user). Routing it through the generic model restores that behavior — covered by a new regression test.Azure (needs the path hook plus deployment-scoped URLs) and Copilot (blocked on async auth headers) are left out of scope per the issue.
What changed
OpenAIEmbeddingsCompatible(openai/embedding.rs) with a singleembeddings_path()method defaulting to/embeddings.GenericEmbeddingModel<Ext>is now bounded on it and resolves the request path throughself.client.ext().embeddings_path()instead of a hardcoded literal.embedding.rsbecomepub type EmbeddingModel<H> = GenericEmbeddingModel<Ext, H>aliases; their extensions implement the hook (Together and Mistral override to/v1/embeddingsbecause their base URLs are bare hosts; OpenRouter uses the default since its base URL already carries/api/v1).EmbeddingResponse.usageis nowOptionin the generic model. OpenAI always populates usage, but OpenRouter returns it only for some models and Together historically didn't surface it at all — making it optional avoids regressing those providers now that they share one response type. A missingusagemaps to a zeroedcompletion::Usage.ApiResponse/ApiErrorResponseerror envelopes.Design note: dedicated trait vs. a method on
ProviderThe path hook could instead have been a method on the base
client::Providertrait — every extension already implementsProvider, soGenericEmbeddingModelwouldn't need a new bound and no newimplblocks would be required. I went with a dedicated trait deliberately:Provideris implemented by every provider regardless of capability, including ones whose embeddings aren't OpenAI-shaped (Cohere, Gemini, Voyage) or that have no embeddings at all. Anembeddings_path()onProviderwould be meaningless or misleading for all of them. The path is a property of "speaks the OpenAI embeddings dialect," not of "is a provider."completion_pathlives on theOpenAICompatibleProvidercapability trait, not onProvider. A dedicated embeddings trait keeps the two symmetric.GenericEmbeddingModel<Ext>on the trait means it only type-checks for an extension that has declared an OpenAI-compatible embeddings path. OnProviderthat guarantee is lost (every extension would satisfy the bound).OpenAICompatibleProvider: OpenAI's own embeddings run onOpenAIResponsesExt, which does not implement that trait (onlyOpenAICompletionsExtdoes), so that bound would exclude OpenAI itself.The cost is one trivial
impl OpenAIEmbeddingsCompatible for XExt {}per embeddings-capable provider — which doubles as an explicit, greppable opt-in, the same way each provider explicitly implementsOpenAICompatibleProvidertoday.Breaking changes
Removes these now-redundant public items (no in-repo references; superseded by the shared OpenAI-compatible types):
together::{EmbeddingResponse, EmbeddingData, Usage}and thetogether_ai_api_typesmoduleopenrouter::{EncodingFormat, EmbeddingResponse, EmbeddingData}mistral::{EmbeddingResponse, EmbeddingData}The
EmbeddingModelhandle and its constructors (new,with_model,with_encoding_format,encoding_format,user) are preserved via the generic model.Points for reviewer input
Two deliberate choices where your steer would help — both are easy to change, and neither is load-bearing for the refactor itself. Flagging them so you don't have to reverse-engineer the intent.
1. Breaking vs. non-breaking
I've made this breaking (
refactor(providers)!) to mirror the completion migration (#2040, also!), deleting the now-redundant per-provider types listed above.It can be made non-breaking instead: keep those old types in place and mark them
#[deprecated]rather than deleting them — they're no longer referenced internally, so leaving them costs nothing and preserves downstream imports. I defaulted to the clean removal for consistency with #2040, but I'm equally happy to switch to the deprecation approach, or to land it breaking now and deprecate-then-remove later — whichever fits your release plans.2. Documentation of the collapsed alias files
Each migrated
embedding.rsmirrors its owncompletion.rssibling:together/embedding.rskeeps a one-line alias doc (its completion sibling has one), whilemistralandopenrouterare left bare (theirs are bare). This keeps each provider directory internally consistent — but means the three embedding files aren't uniform with each other, reflecting a pre-existing inconsistency across those providers rather than one introduced here.The alternative is cross-file uniformity (e.g. make all three aliases bare), at the cost of
together/embedding.rsthen diverging from its own completion sibling. I leaned toward mirroring the siblings; if you'd prefer uniformity I can align them here, or it could be a small follow-up that normalizes alias docs across all providers (completion + embedding) in one consistency pass. Your call.Testing
cargo clippy -p rig-core --all-targetsis clean on the changed files;cargo test -p rig-core --lib embeddingpasses (37 tests, 2#[ignore]).Before this change the generic embedding model had only error-path unit tests, so the behaviors this refactor actually introduces — per-provider
embeddings_path()resolution, the now-optionalusagehandling, and each provider's real response deserializing into the shared OpenAI type — had no coverage. The new tests (each driving a provider through a recording HTTP client) target exactly those:/v1/embeddingswithdimensionswhenndimsis set (the drift the issue flags — the old copy silently dropped it) and omitsdimensionswhen it isn't. → the/v1path override + the regression fix./v1/embeddings, parses a real mistral-shaped body (id+ ausageobject the shared type must tolerate), and checks token usage maps through. → the second path override + cross-provider deserialization + the usage-present branch.…/api/v1/embeddings(confirming the default path is correct over a base URL that already carries/api/v1, i.e. no override needed) and that usage is zeroed when the response omits it. → the default-path decision + the usage-absent branch.Together these lock all three path resolutions, both branches of the new optional-
usagehandling, and cross-provider response parsing. The provider-level embedding smoke tests are#[ignore]live tests requiring API keys and were not run.AI assistance
This change was written with AI assistance. I reviewed every line, understand the design and its trade-offs, and stand behind it per the contributing guidelines.