Add TEI wire mapping (0077) (#208)

Add the first retrieval wire mapping: TeiEmbeddingProvider (POST /embed)
and TeiRerankProvider (POST /rerank) against the TEI self-hosted
endpoints. The rerank provider chunk-and-stitches when the document pool
exceeds chunk_size (TEI's max-client-batch-size): one request per
consecutive slice, indices re-based to absolute positions, a global sort
by score descending, then top_k. TEI returns no usage object, so both
responses carry usage = null (never fabricated).

Add input_type to EmbeddingRuntimeConfig (the cross-vendor query /
document knob, realized here as TEI's prompt_name or a client-side
prefix). Widen EmbeddingResponse.raw / RerankResponse.raw to dict | list,
carrying the provider's verbatim response (proposal 0096, adopted ahead
of the v0.90.0 pin): a bare-array response as its list, a chunked call as
the list of per-request responses.

Introduce the general wire-capture conformance directive family
(mapping / expected_wire_request[+_count] / *_provider blocks) that the
remaining wire mappings reuse, plus a live env-gated TEI integration
test. Un-defer fixtures 013-017; conformance.toml 0077 -> implemented,
0093 -> partial.

Author: chris-colinsky

conformance.toml

@@ -770,11 +770,13 @@ note = "The model's output tool calls get an output-side home on the openarmatur
 status = "not-yet"
 
 # Spec v0.72.0 (proposal 0077).  Retrieval-provider TEI wire mapping
-# (§8 / §8.1) plus the input_type embedding knob (§2 / §3).  Python has
-# not yet shipped the retrieval wire mappings; not-yet.  retrieval-
-# provider fixtures 013-017 defer with it.
+# (§8 / §8.1) plus the input_type embedding knob (§2 / §3).  The first
+# concrete §8 mapping; introduces the conformance wire-capture directive
+# family reused by 0078 / 0079 / 0090 / 0091.
 [proposals."0077"]
-status = "not-yet"
+status = "implemented"
+since = "0.16.0"
+note = "Retrieval-provider TEI (Text Embeddings Inference) wire mapping (§8 / §8.1) -- the first concrete §8 mapping. EmbeddingRuntimeConfig gains the declared input_type field (§2 / §3, the cross-vendor query/document knob; absent = symmetric), flowed into EmbeddingEvent.request_params with absence-is-meaningful semantics (graph-engine §6). Two bundled providers ship: TeiEmbeddingProvider posts POST {base_url}/embed with {inputs[, prompt_name][, dimensions]} -- input_type is realized as TEI's native prompt_name looked up from the construction input_type -> prompt_name map (server-side prompts), or a client-side query_prefix / document_prefix prepend when only prefixes are configured; absent input_type sends neither (byte-identical to the symmetric path). TEI /embed returns a bare vector array in input order and no usage object, so EmbeddingResponse.usage = null (never fabricated). TeiRerankProvider posts POST {base_url}/rerank with {query, texts, truncate: false, return_text} -- texts maps directly onto documents, return_documents -> return_text; it parses the bare [{index, score, text?}] response, reads the text echo only when present, sorts by score descending, and carries response_id + usage null. It implements the MANDATORY rerank chunk-and-stitch (§8.1): when len(documents) exceeds the construction chunk_size (default 32, TEI's max-client-batch-size) it splits into consecutive <= chunk_size slices, issues one /rerank per slice (same query), re-bases each chunk's response index to its absolute position, concatenates, then applies §6 -- a GLOBAL sort by score descending and top_k AFTER the stitch, never per-chunk. truncate: false is sent explicitly on /rerank (fail-loud); the resulting TEI over-length error (HTTP 413 / 422) maps to provider_invalid_request (§7), joined by 400; other statuses map like the sibling providers (401/403 auth, 429 rate_limit, 404 invalid_model, else unavailable). Both providers dispatch the typed EmbeddingEvent / RerankEvent (graph-engine §6) per call, mutually exclusive with the failure variant. genai_system = 'tei'. Conformance fixtures 013-017 pass, driven through a new general wire-capture harness (mapping directive; tei_embedding_provider / tei_rerank_provider construction blocks; expected_wire_request as a single body or a per-chunk list + expected_wire_request_count + expected_wire_request_absent_keys, asserting the exact wire key set and the /embed / /rerank path) reused by the deferred 0078 / 0079 / 0090 / 0091 fixtures. The embed client-side chunk-and-stitch (0092, fixture 038) stays deferred."
 
 # Spec v0.73.0 (proposal 0078).  Retrieval-provider Jina hosted wire
 # mapping (§8.2).  Not-yet; retrieval-provider fixtures 018-022 defer
@@ -885,10 +887,10 @@ status = "not-yet"
 # Spec v0.88.0 (proposal 0093).  Nullable provider usage records:
 # EmbeddingResponse.usage / RerankResponse.usage become record|null and
 # the embedding usage emission (OTel §5.5.8 gen_ai.usage.input_tokens,
-# Langfuse §8.4.5 usageDetails.input) becomes conditional.  Not-yet; the
-# no-usage fixtures 139-143 defer with it.  Follow-up when they un-defer:
-# widen the shipped EmbeddingResponse.usage to nullable + make its
-# emission conditional; 0060 builds RerankResponse.usage nullable from
-# the start.
+# Langfuse §8.4.5 usageDetails.input) is conditional.  Partial: the
+# response-type widening ships; the no-usage observability fixtures
+# 139-143 stay deferred as the 0093-last PR.
 [proposals."0093"]
-status = "not-yet"
+status = "partial"
+since = "0.16.0"
+note = "The response-type nullable widening ships: RerankResponse.usage is record | null from the start (0060a), and EmbeddingResponse.usage is widened to EmbeddingUsage | null here with 0077 (the OpenAIEmbeddingProvider still builds an EmbeddingUsage record; the TeiEmbeddingProvider surfaces usage = null since TEI /embed returns no usage object, never fabricated -- §4). Both bundled observers already emit the embedding usage conditionally (OTel §5.5.8 gen_ai.usage.input_tokens guarded by `event.usage is not None`; Langfuse §8.4.5 usageDetails.input built only when event.usage is not None), landed with the embedding observability in 0059b, so a null-usage EmbeddingEvent renders without a usage attribute rather than crashing. partial because the no-usage OBSERVABILITY fixtures (139 / 140 / 143, asserting the conditional-emission contract end-to-end) stay deferred as the 0093-last PR."

src/openarmature/retrieval/__init__.py

@@ -1,9 +1,9 @@
 """The retrieval-provider capability.
 
 The embedding + rerank provider protocols, their response types, and the
-bundled reference providers (an OpenAI-compatible embedding provider and a
-Cohere-shape reranker). Embedding and rerank are sibling surfaces on the same
-capability.
+bundled reference providers (an OpenAI-compatible embedding provider, a
+Cohere-shape reranker, and the TEI embedding + rerank providers). Embedding and
+rerank are sibling surfaces on the same capability.
 """
 
 from __future__ import annotations
@@ -18,6 +18,7 @@
 )
 from .providers.cohere import CohereRerankProvider
 from .providers.openai import OpenAIEmbeddingProvider
+from .providers.tei import TeiEmbeddingProvider, TeiRerankProvider
 from .response import (
     EmbeddingResponse,
     EmbeddingRuntimeConfig,
@@ -40,6 +41,8 @@
     "RerankRuntimeConfig",
     "RerankUsage",
     "ScoredDocument",
+    "TeiEmbeddingProvider",
+    "TeiRerankProvider",
     "validate_embedding_input",
     "validate_embedding_response",
     "validate_rerank_input",

src/openarmature/retrieval/providers/__init__.py

@@ -2,6 +2,13 @@
 
 from __future__ import annotations
 
+from .cohere import CohereRerankProvider
 from .openai import OpenAIEmbeddingProvider
+from .tei import TeiEmbeddingProvider, TeiRerankProvider
 
-__all__ = ["OpenAIEmbeddingProvider"]
+__all__ = [
+    "CohereRerankProvider",
+    "OpenAIEmbeddingProvider",
+    "TeiEmbeddingProvider",
+    "TeiRerankProvider",
+]

src/openarmature/retrieval/providers/openai.py

@@ -92,8 +92,9 @@ def _classify_embedding_http_error(resp: httpx.Response) -> LlmProviderError:
 # Absence-is-meaningful per observability §5.5.2: only caller-supplied keys
 # appear in the event's request_params -- "the field was not supplied for
 # this call", distinct from a supplied zero. The input_type field joins this
-# set with proposal 0077. For embedding the event-param names coincide with
-# the wire-body keys, so the same dict feeds both the event and the body.
+# set with proposal 0079 (OpenAI-compatible input_type is its wire mapping's
+# job, not 0077's). For embedding the event-param names coincide with the
+# wire-body keys, so the same dict feeds both the event and the body.
 def _request_params_from_config(config: EmbeddingRuntimeConfig | None) -> dict[str, Any]:
     """Extract the supplied embedding request parameters for the event."""
     if config is None: