Add 0091 Cohere /v2/embed wire mapping (#216)

* Add 0091 Cohere /v2/embed wire mapping

Add the retrieval-provider 0091 Cohere /v2/embed wire mapping (section
8.4) as a new CohereEmbeddingProvider. input_type is mandatory on the
Cohere wire: query maps to search_query, document to search_document, an
absent input_type to search_document (the bulk-indexing default), and an
unrecognized value is rejected pre-send. embeddings.float parses in input
order, output_dimension maps from dimensions, and truncate NONE is
fail-loud. embedding_types requests float, merging a caller's other
precisions rather than clobbering them. Over the 96-input Cohere cap the
call chunk-and-stitches: one request per chunk, vectors concatenated in
input order with each chunk's count validated, input_tokens summed, and
response_id from the first chunk.

Un-defer conformance fixtures 032-037, add a cohere embed branch to the
harness, and flip conformance.toml. Env-gated live tests cover embed and
output_dimension.

* Validate embedding_types elements before preserving them

The embedding_types merge only checked list-ness and non-emptiness, so a
caller extra containing non-string elements (e.g. [{"x": 1}]) was
preserved and would be sent as an invalid Cohere wire payload,
contradicting the "malformed falls back to [float]" comment. Require a
non-empty list of non-empty strings before preserving it; anything else
falls back to ["float"].

* List all bundled providers in the retrieval docstring

The module docstring named only the OpenAI, Cohere-rerank, and TEI
providers, omitting the Jina providers and the new CohereEmbeddingProvider.
List the full bundled set (OpenAI embed, TEI embed + rerank, Jina and
Cohere embed + rerank) to match what the module exports.

* Correct the Cohere input_type extras-escape-hatch wording

The error message, docstring, and comment claimed Cohere's other input_type
values (classification / clustering / image) ride the extras pass-through
bag. That is unachievable: input_type is a declared config field (so a
caller cannot route it through model_extra), and an unrecognized value is
rejected pre-send. Reword to the achievable behavior -- those values are
outside OA's {query, document} value space and not reachable through this
mapping; the precision escape hatch (embedding_types) is unaffected. The
§8.4 wording it was copied from is flagged to spec separately.

* Correct the 0091 note input_type extras wording

The conformance.toml 0091 note repeated the unachievable claim that
Cohere's classification / clustering / image input_type values ride the
extras pass-through bag. Reword to match the code: those values are
outside OA's {query, document} value space and not reachable through this
mapping (input_type is a declared field; widening it is deferred).

Author: chris-colinsky

conformance.toml

@@ -875,10 +875,11 @@ note = "Retrieval-provider Cohere /v2/rerank wire mapping (§8.4) -- formalizes
 
 # Spec v0.86.0 (proposal 0091).  Cohere embeddings wire mapping
 # (retrieval-provider §8.4, POST /v2/embed) extending §8.4 to both
-# Cohere endpoints.  Not-yet; wire fixtures 032-037 defer with it (no
-# harness wire-capture primitive; the Cohere provider is unshipped).
+# Cohere endpoints.
 [proposals."0091"]
-status = "not-yet"
+status = "implemented"
+since = "0.16.0"
+note = "Retrieval-provider Cohere /v2/embed wire mapping (§8.4) -- the embed half of §8.4, extending the vendor section to both Cohere endpoints (the §8.2 Jina embed+rerank pattern). CohereEmbeddingProvider posts POST {base_url}/v2/embed with {model, input_type, texts (string array), embedding_types: [float], truncate: NONE, output_dimension?}; base_url defaults to https://api.cohere.com (origin only; the provider appends /v2/embed), sent with api_key as Authorization: Bearer. input_type is a MANDATORY wire field (Cohere v2 requires it), so the mapping always sends a value from the closed set: query -> search_query, document -> search_document, and an ABSENT OA input_type -> search_document (the bulk-indexing default -- the case no other embed mapping has, since §8.1 / §8.2 omit the field when absent and §8.3 has none); an unrecognized OA input_type is a pre-send provider_invalid_request (§7, no request issued), NOT silently coerced to the default. Cohere's other input_type values (classification / clustering / image) are outside OA's {query, document} value space and not reachable through this mapping (input_type is a declared field, so it cannot ride the extras bag; widening it is a deferred change). embedding_types: [float] is sent EXPLICITLY (so the type-keyed response is guaranteed to carry the embeddings.float key the mapping reads); other precisions (int8 / uint8 / binary / base64) ride extras. dimensions -> Cohere's output_dimension (Matryoshka) when set, omitted otherwise. truncate: NONE is sent explicitly (fail-loud -- an over-length input errors rather than being silently truncated; the 400 maps to provider_invalid_request, the point where §8.4's embed half diverges from its rerank half). It parses {id, embeddings: {float: [[float, ...], ...]}, texts, meta: {billed_units: {input_tokens}}}: embeddings.float assembles to EmbeddingResponse.vectors IN INPUT ORDER (positional -- Cohere returns them in request order, NOT index-keyed like OpenAI / Jina data[]); meta.billed_units.input_tokens -> EmbeddingUsage.input_tokens (a record when the body surfaces it, else usage = null, never fabricated); top-level id -> response_id; Cohere echoes no model field, so EmbeddingResponse.model is the bound id. The §4 cross-impl invariants (one vector per input, uniform dimensionality) are enforced against the stitched result. It implements the MANDATORY batch chunk-and-stitch (§8.4 96-input cap / the §8 general embed rule): when len(input) exceeds 96 it splits into consecutive <=96 slices, issues one /v2/embed per slice with identical per-call params (only texts differs), concatenates the per-chunk embeddings.float in input order, sums input_tokens across chunks, and takes response_id from the first chunk; raw is the verbatim response object for a single call or the list of per-chunk objects for a chunked call (proposal 0096). One EmbeddingEvent per embed() call (the whole stitched call, not per chunk); input_type flows into EmbeddingEvent.request_params with absence-is-meaningful semantics (the caller's original query / document value, NOT the resolved search_document default the wire carries). Errors map per §8.4: 401/403 -> provider_authentication, 429 -> provider_rate_limit, 404 -> provider_invalid_model, 400/422 (malformed / over-length) -> provider_invalid_request, else -> provider_unavailable. genai_system = 'cohere'. Conformance fixtures 032-037 pass via the general wire-capture harness (cohere_embedding_provider block; request-origin asserted against base_url; the per-chunk expected_wire_request list + count for 037). The embedding observability + usage-nullable contract ride 0059b / 0093."
 
 # Spec v0.87.0 (proposal 0092).  Embedding-mapping batch chunking, the
 # general §8 rule (consecutive <=cap chunks, one request each, vectors

src/openarmature/retrieval/__init__.py

@@ -1,9 +1,10 @@
 """The retrieval-provider capability.
 
 The embedding + rerank provider protocols, their response types, and the
-bundled reference providers (an OpenAI-compatible embedding provider, a
-Cohere-shape reranker, and the TEI embedding + rerank providers). Embedding and
-rerank are sibling surfaces on the same capability.
+bundled reference providers: an OpenAI-compatible embedding provider, the
+TEI embedding + rerank providers, and the Jina and Cohere embedding +
+rerank providers. Embedding and rerank are sibling surfaces on the same
+capability.
 """
 
 from __future__ import annotations
@@ -16,7 +17,7 @@
     validate_rerank_input,
     validate_rerank_response,
 )
-from .providers.cohere import CohereRerankProvider
+from .providers.cohere import CohereEmbeddingProvider, CohereRerankProvider
 from .providers.jina import JinaEmbeddingProvider, JinaRerankProvider
 from .providers.openai import OpenAIEmbeddingProvider
 from .providers.tei import TeiEmbeddingProvider, TeiRerankProvider
@@ -31,6 +32,7 @@
 )
 
 __all__ = [
+    "CohereEmbeddingProvider",
     "CohereRerankProvider",
     "EmbeddingProvider",
     "EmbeddingResponse",

src/openarmature/retrieval/providers/__init__.py

@@ -2,12 +2,13 @@
 
 from __future__ import annotations
 
-from .cohere import CohereRerankProvider
+from .cohere import CohereEmbeddingProvider, CohereRerankProvider
 from .jina import JinaEmbeddingProvider, JinaRerankProvider
 from .openai import OpenAIEmbeddingProvider
 from .tei import TeiEmbeddingProvider, TeiRerankProvider
 
 __all__ = [
+    "CohereEmbeddingProvider",
     "CohereRerankProvider",
     "JinaEmbeddingProvider",
     "JinaRerankProvider",