Feature - Add support to batching and files API's for Anthropic and OpenAI#951
Feature - Add support to batching and files API's for Anthropic and OpenAI#951joao-salomao wants to merge 12 commits into
Conversation
…ob management and result handling
…h job management, result handling, and error mapping
…-anthropic-and-openai
…lue for data retrieval
… provider methods for batch management
…delete, and metadata retrieval functionalities
…e batch job handling with inputFileId support
…xtRequest
- Use ?? [] on items to avoid passing null to buildAndUploadFile()
- Cast json_encode() result to string to satisfy non-empty-string return type
- Change clientRetry default from [] to [0] to satisfy array{0: int} type constraint
Made-with: Cursor
…s from OpenAI responses
…update tests for empty array responses
|
Waiting for this feature to be merged to decrease our API usage, thanks for the feature! |
|
@joao-salomao great PR! @sixlive - can you please give this priority as it would really help? Many thanks! |
|
@joao-salomao nice PR. @sixlive would be good to have this feature. |
|
@joao-salomao such a great feature! @sixlive approve this, please! |
|
Nice stuff, lets merge! |
|
Would love for this feature to be implemented, its sorely missed in my applications atm |
|
@sixlive any chance you can take a look at this PR? |
|
Unfortunately this has been proposed before and rejected. |
I understand @JacobHonore. For companies running large workloads, the Batch API is a must-have; we are talking about 50% of discount from providers like Anthropic, OpenAI, and Gemini. This needs to be reconsidered. |
|
I want to add to the voices in here. This should be merged! |
… OpenAI # Conflicts: # src/Prism.php # src/Providers/OpenAI/Handlers/Stream.php # src/Providers/OpenAI/Handlers/Text.php # src/Providers/Provider.php # src/Text/Request.php
…table - Restore the closing brace lost in the Prism facade root during conflict resolution. - The payload-size test from prism-php#951 allocated ten 256MB strings and could never run under the default memory limit; the guard now reads an overridable config value (prism.anthropic.batch.max_payload_bytes, defaulting to the documented 256MB constant) and the test exercises it with a 1KB limit. - Fold the withReasoning() gate (prism-php#1018) into the new shared OpenAI BuildsRequestBody concern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Description
Adds a Files API and Batch Processing API for Anthropic and OpenAI providers, enabling asynchronous processing of large volumes of LLM requests at reduced cost.
Files API (
src/Files/)Shared foundation:
FileData/FileListResult/DeleteFileResult— normalized response DTOsUploadFileRequest,DownloadFileRequest,DeleteFileRequest,ListFilesRequest,GetFileMetadataRequest— typed request value objectsFluent builder (
Prism::files()):upload(content, filename, mimeType?)— multipart file uploadlist(limit?, afterId?, beforeId?)— cursor-paginated listinggetMetadata(fileId)— fetch file metadatadownload(fileId)— download raw file content as a stringdelete(fileId)— delete a fileProvider implementations:
Providers/{Provider}/Handlers/Files/HandlesFileResponseconcerns per providerbeforeIdis Anthropic-only;purposefilter (viawithProviderOptions) is OpenAI-onlypurposefield on upload (batch,assistants,fine-tune, etc.)Provider contract (
Provider.php):uploadFile(),listFiles(),getFileMetadata(),downloadFile(),deleteFile()— default to throwingunsupportedProviderActionBatch Processing API (
src/Batch/)Shared foundation:
BatchRequest/BatchRequestItem— input DTOs;BatchRequestItemwraps aTextRequestwith acustomIdBatchJob/BatchJobRequestCounts— normalized job representation across providers, includinginputFileId,outputFileId,errorFileId, timestamps, and statusBatchListResult— paginated listing responseBatchResultItem— individual result with text, usage, and error infoBatchStatus/BatchResultStatus— enums normalizing provider-specific statusesGetBatchResultsRequest,RetrieveBatchRequest,ListBatchesRequest,CancelBatchRequest— typed request value objectsFluent builder (
Prism::batch()):create(items?, inputFileId?)— create a batchretrieve(batchId)— poll for statuslist(limit?, afterId?, beforeId?)— cursor-paginated listinggetResults(batchId)— retrieve all results asBatchResultItem[]cancel(batchId)— cancel an in-progress batchProvider contract (
Provider.php):batch(),retrieveBatch(),listBatches(),getBatchResults(),cancelBatch()— default to throwingunsupportedProviderActionAnthropic implementation:
Create,Retrieve,ListBatches,Results,CancelHandlesBatchResponseandMapsBatchResultsconcerns for shared mappingBatchResultItem[])OpenAI implementation:
HandlesBatchResponseandMapsBatchResultsconcernsBatchRequestItem[]directly — Prism serializes them to JSONL usingBuildsRequestBody::buildHttpRequestPayload()and auto-uploads via the Files API; no manual file handling requiredinputFileIddirectly as an escape hatchitemsandinputFileIdthrows aPrismExceptionResultshandler uses injectedretrieveBatchanddownloadFileclosures for testable coordinationBuildsRequestBodyconcern extracted from text/stream/batch handlers, eliminating duplicated request body mappingFluent Builder — shared traits
Both
Prism::files()andPrism::batch()use the same traits as the existing text/structured/embeddings builders:ConfiguresProviders—using(),whenProvider()ConfiguresClient—withClientOptions(),withClientRetry()HasProviderOptions—withProviderOptions()TextRequestergonomicsAll constructor parameters except
modelnow have sensible defaults, making it straightforward to construct aTextRequestdirectly for batch use without needing the full fluent builder chain:Documentation
docs/core-concepts/files.md— full Files API reference including upload, list (with provider support table), metadata, download, delete, pagination, and provider-conditional optionsdocs/core-concepts/batch.md— full Batch API reference including create-poll-retrieve workflow, provider-specific notes, status table, result item properties, listing, cancellation, and links to the official Anthropic and OpenAI batch docsTests
CreateTest,RetrieveTest,ListTest,ResultsTest,CancelTestResultshandlers on both providers:ResultsUnitTest— mocks HTTP/callbacks, covering succeeded/errored/expired/canceled parsing, blank-line skipping, and null output file ID handlingBreaking Changes
None. All new methods on
Providerdefault to throwingunsupportedProviderAction, so existing custom providers are unaffected. The only structural change to an existing class isTextRequest, where constructor parameters now have defaults — all existing call sites using named arguments or the fluent builder are unaffected.