Skip to content

refactor(providers)!: adopt GenericEmbeddingModel for together/openrouter/mistral embeddings#2123

Open
ojhermann wants to merge 2 commits into
0xPlaygrounds:mainfrom
ojhermann:refactor/embeddings-generic-model
Open

refactor(providers)!: adopt GenericEmbeddingModel for together/openrouter/mistral embeddings#2123
ojhermann wants to merge 2 commits into
0xPlaygrounds:mainfrom
ojhermann:refactor/embeddings-generic-model

Conversation

@ojhermann

@ojhermann ojhermann commented Jul 12, 2026

Copy link
Copy Markdown

Fixes #2078.

Summary

Collapses the hand-rolled Together, OpenRouter, and Mistral embedding models onto the shared GenericEmbeddingModel<Ext> — the embeddings analogue of the #2035#2038/#2040 GenericCompletionModel migration. Each provider's embedding.rs shrinks from a ~140–190-line near-verbatim copy of the OpenAI POST /embeddings flow to a thin type alias.

As called out in the issue, the Together copy had silently drifted: it dropped dimensions (and encoding_format/user). Routing it through the generic model restores that behavior — covered by a new regression test.

Azure (needs the path hook plus deployment-scoped URLs) and Copilot (blocked on async auth headers) are left out of scope per the issue.

What changed

  • New hook OpenAIEmbeddingsCompatible (openai/embedding.rs) with a single embeddings_path() method defaulting to /embeddings. GenericEmbeddingModel<Ext> is now bounded on it and resolves the request path through self.client.ext().embeddings_path() instead of a hardcoded literal.
  • Together / OpenRouter / Mistral embedding.rs become pub type EmbeddingModel<H> = GenericEmbeddingModel<Ext, H> aliases; their extensions implement the hook (Together and Mistral override to /v1/embeddings because their base URLs are bare hosts; OpenRouter uses the default since its base URL already carries /api/v1).
  • EmbeddingResponse.usage is now Option in the generic model. OpenAI always populates usage, but OpenRouter returns it only for some models and Together historically didn't surface it at all — making it optional avoids regressing those providers now that they share one response type. A missing usage maps to a zeroed completion::Usage.
  • Removed the now-dead per-provider embedding response/format types and their orphaned ApiResponse/ApiErrorResponse error envelopes.

Design note: dedicated trait vs. a method on Provider

The path hook could instead have been a method on the base client::Provider trait — every extension already implements Provider, so GenericEmbeddingModel wouldn't need a new bound and no new impl blocks would be required. I went with a dedicated trait deliberately:

  • Altitude. Provider is implemented by every provider regardless of capability, including ones whose embeddings aren't OpenAI-shaped (Cohere, Gemini, Voyage) or that have no embeddings at all. An embeddings_path() on Provider would be meaningless or misleading for all of them. The path is a property of "speaks the OpenAI embeddings dialect," not of "is a provider."
  • Consistency. This mirrors the completion side exactly: completion_path lives on the OpenAICompatibleProvider capability trait, not on Provider. A dedicated embeddings trait keeps the two symmetric.
  • A real invariant. Bounding GenericEmbeddingModel<Ext> on the trait means it only type-checks for an extension that has declared an OpenAI-compatible embeddings path. On Provider that guarantee is lost (every extension would satisfy the bound).
  • It also can't simply reuse OpenAICompatibleProvider: OpenAI's own embeddings run on OpenAIResponsesExt, which does not implement that trait (only OpenAICompletionsExt does), so that bound would exclude OpenAI itself.

The cost is one trivial impl OpenAIEmbeddingsCompatible for XExt {} per embeddings-capable provider — which doubles as an explicit, greppable opt-in, the same way each provider explicitly implements OpenAICompatibleProvider today.

Breaking changes

Removes these now-redundant public items (no in-repo references; superseded by the shared OpenAI-compatible types):

  • together::{EmbeddingResponse, EmbeddingData, Usage} and the together_ai_api_types module
  • openrouter::{EncodingFormat, EmbeddingResponse, EmbeddingData}
  • mistral::{EmbeddingResponse, EmbeddingData}

The EmbeddingModel handle and its constructors (new, with_model, with_encoding_format, encoding_format, user) are preserved via the generic model.

Points for reviewer input

Two deliberate choices where your steer would help — both are easy to change, and neither is load-bearing for the refactor itself. Flagging them so you don't have to reverse-engineer the intent.

1. Breaking vs. non-breaking

I've made this breaking (refactor(providers)!) to mirror the completion migration (#2040, also !), deleting the now-redundant per-provider types listed above.

It can be made non-breaking instead: keep those old types in place and mark them #[deprecated] rather than deleting them — they're no longer referenced internally, so leaving them costs nothing and preserves downstream imports. I defaulted to the clean removal for consistency with #2040, but I'm equally happy to switch to the deprecation approach, or to land it breaking now and deprecate-then-remove later — whichever fits your release plans.

2. Documentation of the collapsed alias files

Each migrated embedding.rs mirrors its own completion.rs sibling: together/embedding.rs keeps a one-line alias doc (its completion sibling has one), while mistral and openrouter are left bare (theirs are bare). This keeps each provider directory internally consistent — but means the three embedding files aren't uniform with each other, reflecting a pre-existing inconsistency across those providers rather than one introduced here.

The alternative is cross-file uniformity (e.g. make all three aliases bare), at the cost of together/embedding.rs then diverging from its own completion sibling. I leaned toward mirroring the siblings; if you'd prefer uniformity I can align them here, or it could be a small follow-up that normalizes alias docs across all providers (completion + embedding) in one consistency pass. Your call.

Testing

cargo clippy -p rig-core --all-targets is clean on the changed files; cargo test -p rig-core --lib embedding passes (37 tests, 2 #[ignore]).

Before this change the generic embedding model had only error-path unit tests, so the behaviors this refactor actually introduces — per-provider embeddings_path() resolution, the now-optional usage handling, and each provider's real response deserializing into the shared OpenAI type — had no coverage. The new tests (each driving a provider through a recording HTTP client) target exactly those:

  • Together — asserts it POSTs to /v1/embeddings with dimensions when ndims is set (the drift the issue flags — the old copy silently dropped it) and omits dimensions when it isn't. → the /v1 path override + the regression fix.
  • Mistral — asserts POST /v1/embeddings, parses a real mistral-shaped body (id + a usage object the shared type must tolerate), and checks token usage maps through. → the second path override + cross-provider deserialization + the usage-present branch.
  • OpenRouter — asserts POST …/api/v1/embeddings (confirming the default path is correct over a base URL that already carries /api/v1, i.e. no override needed) and that usage is zeroed when the response omits it. → the default-path decision + the usage-absent branch.

Together these lock all three path resolutions, both branches of the new optional-usage handling, and cross-provider response parsing. The provider-level embedding smoke tests are #[ignore] live tests requiring API keys and were not run.

AI assistance

This change was written with AI assistance. I reviewed every line, understand the design and its trade-offs, and stand behind it per the contributing guidelines.

…uter/mistral embeddings

Collapse the hand-rolled Together, OpenRouter, and Mistral embedding
models onto the shared GenericEmbeddingModel<Ext> via a new
OpenAIEmbeddingsCompatible path hook, mirroring
OpenAICompatibleProvider::completion_path. Together embeddings now honor
dimensions (via ndims), encoding_format, and user, which the previous
hand-rolled copy silently dropped.

EmbeddingResponse.usage becomes optional so providers that omit it
(OpenRouter for some models, Together historically) are not regressed now
that they share one response type. Removes the now-dead per-provider
embedding response/format types and their orphaned error envelopes.

Azure (needs the path hook plus deployment-scoped URLs) and Copilot
(blocked on async auth headers) are left out of scope per the issue.

Fixes 0xPlaygrounds#2078.
@ojhermann ojhermann force-pushed the refactor/embeddings-generic-model branch from 3c83d39 to 8d9b3a9 Compare July 12, 2026 16:00
Comment thread crates/rig-core/src/providers/mistral/client.rs Outdated
Comment thread crates/rig-core/src/providers/mistral/embedding.rs Outdated
Comment thread crates/rig-core/src/providers/mistral/embedding.rs Outdated
… conventions

Add unit tests covering per-provider embeddings_path resolution and the
optional-usage handling:
- mistral: asserts POST /v1/embeddings, embedding parse, and usage mapping
- openrouter: asserts POST .../api/v1/embeddings and zeroed usage when absent
- together: adds a negative test that `dimensions` is omitted when ndims unset

Also trim the module/alias docs and inline comments added on the migrated
files to match each provider's existing convention (mirroring the
completion.rs siblings), per review feedback.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(providers): adopt GenericEmbeddingModel for together/openrouter/mistral (then azure) embeddings

1 participant