Skip to content

Commit c959e2d

Browse files
Correct 0092 note to all-or-nothing usage rule
The conformance note described the pre-hardening per-chunk usage semantics (sum when reported, else null). chunk_and_stitch_embed now combines usage all-or-nothing: it sums only when every chunk reports a figure, and a mixed report yields usage = null rather than a partial sum. Bring the note in line with the code.
1 parent 0301843 commit c959e2d

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

conformance.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -887,7 +887,7 @@ note = "Retrieval-provider Cohere /v2/embed wire mapping (§8.4) -- the embed ha
887887
[proposals."0092"]
888888
status = "implemented"
889889
since = "0.16.0"
890-
note = "Retrieval-provider general §8 *Batch chunking* rule for the embedding side + fixture 038 (TEI /embed chunk-and-stitch by the construction chunk_size). The shared chunk_and_stitch_embed helper in _wire.py realizes the rule mapping-agnostically: split the inputs into consecutive <=cap slices preserving order, issue one request per slice via a per-mapping closure (every field other than the chunked input list identical across slices), concatenate the per-chunk vectors in input order (so §4's one-vector-per-input + input-order invariants hold across the whole call, enforced against the stitched result), combine the per-chunk usage record-aware (sum EmbeddingUsage.input_tokens when the provider reports usage, else usage = null -- never fabricated), take response_id from the first chunk, and set raw to the single response for a single-request call or the list of per-chunk responses in request order (proposal 0096). TeiEmbeddingProvider.embed() adopts it with cap = its construction chunk_size (TEI's max-client-batch-size, default 32): an over-cap embed call chunk-and-stitches (fixture 038: chunk_size 2 over 5 inputs -> 3 /embed requests sized 2/2/1 with identical per-call params), and since TEI /embed reports no usage and no id the stitched usage is null and response_id is null; the len(input) <= chunk_size path issues a single /embed request (the pre-0092 behavior preserved). CohereEmbeddingProvider migrates its existing 96-input chunk path (0091, fixture 037) onto the same helper (cap = 96, closure POSTs /v2/embed) with identical behavior. OpenAIEmbeddingProvider adopts it at §8.3's fixed 2048-input cap (closure POSTs /v1/embeddings): an over-2048 embed call now chunk-and-stitches instead of sending one over-cap request the wire rejects. OpenAI's summed-token ceiling is NOT addressed by the count-based rule -- an over-token chunk still fails loud as provider_invalid_request. Jina is the cap-free mapping the rule exempts (server-side batching), so it does not chunk. The per-chunk vector count is validated before stitching so a positional misalignment cannot pass on a compensating total, and the response identity (response_id + response_model) is taken from the FIRST chunk -- the model falls back to the bound identifier for the wires that carry none (TEI's bare array, Cohere /v2/embed), while OpenAI reports the body's model per §4. Conformance fixtures 037 (Cohere, unchanged) + 038 (TEI) pass via the general wire-capture harness; no retrieval fixtures remain deferred. The spec ships NO §8.3 over-cap fixture, so the OpenAI chunking is covered by unit tests only (2049 inputs -> 2 requests sized 2048/1) -- flagged to spec as a fixture-set gap."
890+
note = "Retrieval-provider general §8 *Batch chunking* rule for the embedding side + fixture 038 (TEI /embed chunk-and-stitch by the construction chunk_size). The shared chunk_and_stitch_embed helper in _wire.py realizes the rule mapping-agnostically: split the inputs into consecutive <=cap slices preserving order, issue one request per slice via a per-mapping closure (every field other than the chunked input list identical across slices), concatenate the per-chunk vectors in input order (so §4's one-vector-per-input + input-order invariants hold across the whole call, enforced against the stitched result), combine the per-chunk usage all-or-nothing (sum EmbeddingUsage.input_tokens only when EVERY chunk reports a figure, else usage = null -- a mixed report is an unaccountable total, never a partial sum or a fabricated figure), take response_id from the first chunk, and set raw to the single response for a single-request call or the list of per-chunk responses in request order (proposal 0096). TeiEmbeddingProvider.embed() adopts it with cap = its construction chunk_size (TEI's max-client-batch-size, default 32): an over-cap embed call chunk-and-stitches (fixture 038: chunk_size 2 over 5 inputs -> 3 /embed requests sized 2/2/1 with identical per-call params), and since TEI /embed reports no usage and no id the stitched usage is null and response_id is null; the len(input) <= chunk_size path issues a single /embed request (the pre-0092 behavior preserved). CohereEmbeddingProvider migrates its existing 96-input chunk path (0091, fixture 037) onto the same helper (cap = 96, closure POSTs /v2/embed) with identical behavior. OpenAIEmbeddingProvider adopts it at §8.3's fixed 2048-input cap (closure POSTs /v1/embeddings): an over-2048 embed call now chunk-and-stitches instead of sending one over-cap request the wire rejects. OpenAI's summed-token ceiling is NOT addressed by the count-based rule -- an over-token chunk still fails loud as provider_invalid_request. Jina is the cap-free mapping the rule exempts (server-side batching), so it does not chunk. The per-chunk vector count is validated before stitching so a positional misalignment cannot pass on a compensating total, and the response identity (response_id + response_model) is taken from the FIRST chunk -- the model falls back to the bound identifier for the wires that carry none (TEI's bare array, Cohere /v2/embed), while OpenAI reports the body's model per §4. Conformance fixtures 037 (Cohere, unchanged) + 038 (TEI) pass via the general wire-capture harness; no retrieval fixtures remain deferred. The spec ships NO §8.3 over-cap fixture, so the OpenAI chunking is covered by unit tests only (2049 inputs -> 2 requests sized 2048/1) -- flagged to spec as a fixture-set gap."
891891

892892
# Spec v0.88.0 (proposal 0093). Nullable provider usage records:
893893
# EmbeddingResponse.usage / RerankResponse.usage become record|null and

0 commit comments

Comments
 (0)