Skip to content

Feature - Add support to batching and files API's for Anthropic and OpenAI#951

Open
joao-salomao wants to merge 12 commits into
prism-php:mainfrom
joao-salomao:feature/batch-api-for-anthropic-and-openai
Open

Feature - Add support to batching and files API's for Anthropic and OpenAI#951
joao-salomao wants to merge 12 commits into
prism-php:mainfrom
joao-salomao:feature/batch-api-for-anthropic-and-openai

Conversation

@joao-salomao

@joao-salomao joao-salomao commented Mar 14, 2026

Copy link
Copy Markdown
Contributor

Description

Adds a Files API and Batch Processing API for Anthropic and OpenAI providers, enabling asynchronous processing of large volumes of LLM requests at reduced cost.


Files API (src/Files/)

Shared foundation:

  • FileData / FileListResult / DeleteFileResult — normalized response DTOs
  • UploadFileRequest, DownloadFileRequest, DeleteFileRequest, ListFilesRequest, GetFileMetadataRequest — typed request value objects

Fluent builder (Prism::files()):

  • upload(content, filename, mimeType?) — multipart file upload
  • list(limit?, afterId?, beforeId?) — cursor-paginated listing
  • getMetadata(fileId) — fetch file metadata
  • download(fileId) — download raw file content as a string
  • delete(fileId) — delete a file

Provider implementations:

  • Both OpenAI and Anthropic implement all five operations under Providers/{Provider}/Handlers/Files/
  • Shared response parsing in HandlesFileResponse concerns per provider
  • Pagination parameter support varies by provider: beforeId is Anthropic-only; purpose filter (via withProviderOptions) is OpenAI-only
  • OpenAI requires a purpose field on upload (batch, assistants, fine-tune, etc.)
  • Anthropic restricts downloads to files generated by Claude (e.g. code execution outputs)

Provider contract (Provider.php):

  • uploadFile(), listFiles(), getFileMetadata(), downloadFile(), deleteFile() — default to throwing unsupportedProviderAction

Batch Processing API (src/Batch/)

Shared foundation:

  • BatchRequest / BatchRequestItem — input DTOs; BatchRequestItem wraps a TextRequest with a customId
  • BatchJob / BatchJobRequestCounts — normalized job representation across providers, including inputFileId, outputFileId, errorFileId, timestamps, and status
  • BatchListResult — paginated listing response
  • BatchResultItem — individual result with text, usage, and error info
  • BatchStatus / BatchResultStatus — enums normalizing provider-specific statuses
  • GetBatchResultsRequest, RetrieveBatchRequest, ListBatchesRequest, CancelBatchRequest — typed request value objects

Fluent builder (Prism::batch()):

  • create(items?, inputFileId?) — create a batch
  • retrieve(batchId) — poll for status
  • list(limit?, afterId?, beforeId?) — cursor-paginated listing
  • getResults(batchId) — retrieve all results as BatchResultItem[]
  • cancel(batchId) — cancel an in-progress batch

Provider contract (Provider.php):

  • batch(), retrieveBatch(), listBatches(), getBatchResults(), cancelBatch() — default to throwing unsupportedProviderAction

Anthropic implementation:

  • One handler per operation: Create, Retrieve, ListBatches, Results, Cancel
  • HandlesBatchResponse and MapsBatchResults concerns for shared mapping
  • Pre-create validation enforcing the 100,000 request limit and 256 MB payload size
  • JSONL result parsing with full buffering (returns BatchResultItem[])

OpenAI implementation:

  • Same handler-per-operation structure
  • HandlesBatchResponse and MapsBatchResults concerns
  • Items-first creation: pass BatchRequestItem[] directly — Prism serializes them to JSONL using BuildsRequestBody::buildHttpRequestPayload() and auto-uploads via the Files API; no manual file handling required
  • File-first creation: pass a pre-uploaded inputFileId directly as an escape hatch
  • Providing both items and inputFileId throws a PrismException
  • Results handler uses injected retrieveBatch and downloadFile closures for testable coordination
  • BuildsRequestBody concern extracted from text/stream/batch handlers, eliminating duplicated request body mapping

Fluent Builder — shared traits

Both Prism::files() and Prism::batch() use the same traits as the existing text/structured/embeddings builders:

  • ConfiguresProvidersusing(), whenProvider()
  • ConfiguresClientwithClientOptions(), withClientRetry()
  • HasProviderOptionswithProviderOptions()

TextRequest ergonomics

All constructor parameters except model now have sensible defaults, making it straightforward to construct a TextRequest directly for batch use without needing the full fluent builder chain:

new TextRequest(
    model: 'claude-sonnet-4-20250514',
    messages: [new UserMessage('Summarise: ...')],
)

Documentation

  • docs/core-concepts/files.md — full Files API reference including upload, list (with provider support table), metadata, download, delete, pagination, and provider-conditional options
  • docs/core-concepts/batch.md — full Batch API reference including create-poll-retrieve workflow, provider-specific notes, status table, result item properties, listing, cancellation, and links to the official Anthropic and OpenAI batch docs

Tests

  • Per-provider integration tests for all five batch operations: CreateTest, RetrieveTest, ListTest, ResultsTest, CancelTest
  • Unit tests for Results handlers on both providers: ResultsUnitTest — mocks HTTP/callbacks, covering succeeded/errored/expired/canceled parsing, blank-line skipping, and null output file ID handling
  • Integration tests for all five file operations on both providers

Breaking Changes

None. All new methods on Provider default to throwing unsupportedProviderAction, so existing custom providers are unaffected. The only structural change to an existing class is TextRequest, where constructor parameters now have defaults — all existing call sites using named arguments or the fluent builder are unaffected.

@joao-salomao joao-salomao changed the title feat(batch): add batch processing support for Anthropic and OpenAI Feature - Add support to batching and files API's for Anthropic and OpenAI Mar 15, 2026
@joao-salomao
joao-salomao marked this pull request as ready for review March 15, 2026 23:29
…xtRequest

- Use ?? [] on items to avoid passing null to buildAndUploadFile()
- Cast json_encode() result to string to satisfy non-empty-string return type
- Change clientRetry default from [] to [0] to satisfy array{0: int} type constraint

Made-with: Cursor
@pedro-fernandes-vinta

Copy link
Copy Markdown

Waiting for this feature to be merged to decrease our API usage, thanks for the feature!

@phill-ug

Copy link
Copy Markdown

@joao-salomao great PR! @sixlive - can you please give this priority as it would really help? Many thanks!

@guths

guths commented Mar 31, 2026

Copy link
Copy Markdown

@joao-salomao nice PR. @sixlive would be good to have this feature.

@dair-ug

dair-ug commented Mar 31, 2026

Copy link
Copy Markdown

@joao-salomao such a great feature! @sixlive approve this, please!

@alexandre-da-costa

Copy link
Copy Markdown

Nice stuff, lets merge!

@pentacore

Copy link
Copy Markdown

Would love for this feature to be implemented, its sorely missed in my applications atm

@joao-salomao-ug

Copy link
Copy Markdown

@sixlive any chance you can take a look at this PR?

@JacobHonore

Copy link
Copy Markdown

Unfortunately this has been proposed before and rejected.
I hope it can make it into prism this time, as it is really useful!
Tagging you here, @khalidmaquilang in case you didnt notice this new PR :)

@joao-salomao-ug

joao-salomao-ug commented May 22, 2026

Copy link
Copy Markdown

Unfortunately this has been proposed before and rejected. I hope it can make it into prism this time, as it is really useful! Tagging you here, @khalidmaquilang in case you didnt notice this new PR :)

I understand @JacobHonore. For companies running large workloads, the Batch API is a must-have; we are talking about 50% of discount from providers like Anthropic, OpenAI, and Gemini. This needs to be reconsidered.

@hamannjames

Copy link
Copy Markdown

I want to add to the voices in here. This should be merged!

@sixlive

wishborn added a commit to Particle-Academy/prism that referenced this pull request Jul 6, 2026
… OpenAI

# Conflicts:
#	src/Prism.php
#	src/Providers/OpenAI/Handlers/Stream.php
#	src/Providers/OpenAI/Handlers/Text.php
#	src/Providers/Provider.php
#	src/Text/Request.php
wishborn added a commit to Particle-Academy/prism that referenced this pull request Jul 6, 2026
…table

- Restore the closing brace lost in the Prism facade root during conflict
  resolution.
- The payload-size test from prism-php#951 allocated ten 256MB strings
  and could never run under the default memory limit; the guard now reads an
  overridable config value (prism.anthropic.batch.max_payload_bytes,
  defaulting to the documented 256MB constant) and the test exercises it with
  a 1KB limit.
- Fold the withReasoning() gate (prism-php#1018) into the new shared
  OpenAI BuildsRequestBody concern.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants