Skip to content

[bot] Google GenAI: instrument Batch API (client.batches.create / create_embeddings) #332

@braintrust-bot

Description

@braintrust-bot

What's missing

The Google GenAI (google-genai SDK) exposes a Batch API for asynchronous bulk generation and embedding requests at reduced cost. None of these surfaces are instrumented by the Braintrust integration today:

SDK method Purpose
client.batches.create() Submit a batch of generation requests
client.batches.create_embeddings() Submit a batch of embedding requests
client.batches.get() Retrieve batch job status
client.batches.list() List recent batch jobs
client.batches.cancel() Cancel an in-progress batch
client.batches.delete() Delete a batch job

The primary instrumentation targets are batches.create() and batches.create_embeddings() (the execution surfaces). The CRUD-style methods (get, list, cancel, delete) are secondary but included for completeness.

Asymmetry with other providers

Braintrust docs status

  • The Braintrust tracing docs list Gemini as a supported provider but do not mention batch processing for Google GenAI.
  • No Braintrust documentation page covers Google GenAI batch instrumentation.

Upstream sources

  • Google AI docs: Batch mode — documents the Batch API (currently v1beta), including batches.create(), batches.create_embeddings(), and lifecycle methods.
  • API status: The Batch API uses v1beta endpoints (generativelanguage.googleapis.com/v1beta/...). It is documented and available in the google-genai SDK but has not been promoted to v1 GA.

Local files inspected

  • py/src/braintrust/integrations/google_genai/patchers.py — defines patchers for Models.generate_content, generate_content_stream, embed_content, generate_images, and Interactions CRUD (sync + async). Zero batch-related patchers.
  • py/src/braintrust/integrations/google_genai/tracing.py — wrapper functions for the above surfaces. No batch wrapper functions.
  • py/src/braintrust/integrations/google_genai/integration.py — registers the above patchers. No batch registration.
  • py/src/braintrust/integrations/anthropic/tracing.py (lines 565-619, 1159-1207) — reference implementation of batch instrumentation (Batches / AsyncBatches wrapper classes, _start_batch_create_span, _log_batch_create_to_span, _start_batch_results_span).

Notes

The v1beta status means this API may change before GA. Implementation could follow the Anthropic batch pattern (wrapper classes around the batch resource) and should be gated on the google-genai SDK version that exposes client.batches.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions