Skip to content

Commit 8f2003b

Browse files
wishbornwishborn
committed
Merge prism-php#951 — feat: batching and files APIs for Anthropic and OpenAI
# Conflicts: # src/Prism.php # src/Providers/OpenAI/Handlers/Stream.php # src/Providers/OpenAI/Handlers/Text.php # src/Providers/Provider.php # src/Text/Request.php
2 parents fa3a886 + 6d902aa commit 8f2003b

114 files changed

Lines changed: 5446 additions & 71 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/.vitepress/config.mts‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,14 @@ export default defineConfig({
133133
text: "Audio",
134134
link: "/core-concepts/audio",
135135
},
136+
{
137+
text: "Batch Processing",
138+
link: "/core-concepts/batch",
139+
},
140+
{
141+
text: "Files",
142+
link: "/core-concepts/files",
143+
},
136144
{
137145
text: "Schemas",
138146
link: "/core-concepts/schemas",

‎docs/core-concepts/batch.md‎

Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
# Batch Processing
2+
3+
The Batch API lets you submit large numbers of requests for asynchronous processing. Instead of sending each prompt one at a time and waiting for a response, you queue them all at once and collect the results later. This makes batch processing significantly cheaper and better suited for offline workloads like dataset generation, bulk classification, or nightly report jobs.
4+
5+
> [!IMPORTANT]
6+
> For deeper background, see the official provider documentation:
7+
> - [Anthropic](https://platform.claude.com/docs/en/build-with-claude/batch-processing)
8+
> - [OpenAI](https://developers.openai.com/api/docs/guides/batch/)
9+
10+
## How It Works
11+
12+
1. **Create** a batch (submit your requests)
13+
2. **Poll** the batch status until it reaches `Completed`
14+
3. **Retrieve** the results
15+
16+
```php
17+
use Prism\Prism\Facades\Prism;
18+
use Prism\Prism\Enums\Provider;
19+
use Prism\Prism\Batch\BatchStatus;
20+
21+
// 1. Create
22+
$job = Prism::batch()
23+
->using(Provider::Anthropic)
24+
->create(items: $items);
25+
26+
// 2. Poll
27+
while ($job->status !== BatchStatus::Completed) {
28+
sleep(30);
29+
$job = Prism::batch()
30+
->using(Provider::Anthropic)
31+
->retrieve($job->id);
32+
}
33+
34+
// 3. Get results
35+
$results = Prism::batch()
36+
->using(Provider::Anthropic)
37+
->getResults($job->id);
38+
39+
foreach ($results as $result) {
40+
echo "{$result->customId}: {$result->text}\n";
41+
}
42+
```
43+
44+
## Creating a Batch
45+
46+
### Anthropic
47+
48+
Anthropic takes a list of `BatchRequestItem` objects directly. Each item wraps a normal `TextRequest` and a unique `customId` you'll use to match results back to the original input.
49+
50+
```php
51+
use Prism\Prism\Facades\Prism;
52+
use Prism\Prism\Enums\Provider;
53+
use Prism\Prism\Batch\BatchRequestItem;
54+
use Prism\Prism\Text\Request as TextRequest;
55+
use Prism\Prism\ValueObjects\Messages\UserMessage;
56+
57+
$items = collect($yourData)->map(fn ($row) => new BatchRequestItem(
58+
customId: "row-{$row->id}",
59+
request: new TextRequest(
60+
model: 'claude-sonnet-4-20250514',
61+
messages: [new UserMessage("Summarise: {$row->content}")],
62+
),
63+
))->all();
64+
65+
$job = Prism::batch()
66+
->using(Provider::Anthropic)
67+
->create(items: $items);
68+
69+
echo $job->id; // "msgbatch_01..."
70+
echo $job->status; // BatchStatus::Validating
71+
```
72+
73+
### OpenAI
74+
75+
OpenAI accepts the same `items` array as Anthropic. Prism automatically builds the JSONL payload and uploads it via the [Files API](/core-concepts/files) before creating the batch — you don't need to handle the file upload yourself.
76+
77+
```php
78+
use Prism\Prism\Facades\Prism;
79+
use Prism\Prism\Enums\Provider;
80+
use Prism\Prism\Batch\BatchRequestItem;
81+
use Prism\Prism\Text\Request as TextRequest;
82+
use Prism\Prism\ValueObjects\Messages\UserMessage;
83+
84+
$items = collect($yourData)->map(fn ($row) => new BatchRequestItem(
85+
customId: "row-{$row->id}",
86+
request: new TextRequest(
87+
model: 'gpt-4o',
88+
messages: [new UserMessage("Summarise: {$row->content}")],
89+
),
90+
))->all();
91+
92+
$job = Prism::batch()
93+
->using(Provider::OpenAI)
94+
->create(items: $items);
95+
96+
echo $job->id;
97+
```
98+
99+
#### Bringing your own file
100+
101+
If you've already uploaded a JSONL file via the [Files API](/core-concepts/files), pass its ID directly via `inputFileId` instead:
102+
103+
```php
104+
$file = Prism::files()
105+
->using(Provider::OpenAI)
106+
->withProviderOptions(['purpose' => 'batch'])
107+
->upload(content: $jsonlContent, filename: 'batch-input.jsonl');
108+
109+
$job = Prism::batch()
110+
->using(Provider::OpenAI)
111+
->create(inputFileId: $file->id);
112+
```
113+
114+
> [!NOTE]
115+
> Providing both `items` and `inputFileId` at the same time throws a `PrismException`. Use one or the other.
116+
117+
## Polling for Completion
118+
119+
The batch doesn't complete immediately. Poll the status until it reaches a terminal state:
120+
121+
```php
122+
use Prism\Prism\Batch\BatchStatus;
123+
124+
do {
125+
sleep(30);
126+
127+
$job = Prism::batch()
128+
->using(Provider::Anthropic)
129+
->retrieve($job->id);
130+
131+
} while (! in_array($job->status, [
132+
BatchStatus::Completed,
133+
BatchStatus::Failed,
134+
BatchStatus::Expired,
135+
BatchStatus::Cancelled,
136+
]));
137+
```
138+
139+
### BatchStatus values
140+
141+
| Status | Meaning |
142+
|---|---|
143+
| `Validating` | Requests are being validated |
144+
| `InProgress` | Processing is underway |
145+
| `Finalizing` | Results are being prepared |
146+
| `Completed` | All requests have finished |
147+
| `Failed` | The batch failed to process |
148+
| `Cancelling` | Cancellation is in progress |
149+
| `Cancelled` | The batch was cancelled |
150+
| `Expired` | The batch expired before completing |
151+
152+
### Request counts
153+
154+
The `BatchJob` object includes a `requestCounts` property that breaks down progress:
155+
156+
```php
157+
echo "Total: {$job->requestCounts->total}";
158+
echo "Processing: {$job->requestCounts->processing}";
159+
echo "Succeeded: {$job->requestCounts->succeeded}";
160+
echo "Failed: {$job->requestCounts->failed}";
161+
echo "Expired: {$job->requestCounts->expired}";
162+
```
163+
164+
## Retrieving Results
165+
166+
Once the batch is `Completed`, call `getResults()` to get all the result items back as an array:
167+
168+
```php
169+
use Prism\Prism\Batch\BatchResultStatus;
170+
171+
$results = Prism::batch()
172+
->using(Provider::Anthropic)
173+
->getResults($job->id);
174+
175+
foreach ($results as $result) {
176+
if ($result->status === BatchResultStatus::Succeeded) {
177+
echo "{$result->customId}: {$result->text}\n";
178+
echo "Tokens: {$result->usage->promptTokens} in / {$result->usage->completionTokens} out\n";
179+
} else {
180+
echo "{$result->customId} failed: {$result->errorMessage}\n";
181+
}
182+
}
183+
```
184+
185+
### BatchResultItem properties
186+
187+
| Property | Type | Description |
188+
|---|---|---|
189+
| `customId` | `string` | The ID you assigned when creating the item |
190+
| `status` | `BatchResultStatus` | `Succeeded`, `Errored`, `Canceled`, or `Expired` |
191+
| `text` | `?string` | The generated text (null if not succeeded) |
192+
| `usage` | `?Usage` | Token counts |
193+
| `messageId` | `?string` | Provider message ID |
194+
| `model` | `?string` | Model used |
195+
| `errorType` | `?string` | Error code if failed |
196+
| `errorMessage` | `?string` | Human-readable error detail |
197+
198+
## Listing Batches
199+
200+
Browse previously created batches with optional pagination:
201+
202+
```php
203+
$list = Prism::batch()
204+
->using(Provider::OpenAI)
205+
->list(limit: 20);
206+
207+
foreach ($list->batches as $job) {
208+
echo "{$job->id}: {$job->status->name}\n";
209+
}
210+
211+
// Next page
212+
if ($list->hasMore) {
213+
$nextPage = Prism::batch()
214+
->using(Provider::OpenAI)
215+
->list(limit: 20, afterId: $list->lastId);
216+
}
217+
```
218+
219+
## Cancelling a Batch
220+
221+
```php
222+
$job = Prism::batch()
223+
->using(Provider::Anthropic)
224+
->cancel($job->id);
225+
226+
echo $job->status->name; // "Cancelling"
227+
```
228+
229+
## Provider-Conditional Options
230+
231+
Use `whenProvider()` when your batch logic needs to handle both providers from the same code path:
232+
233+
```php
234+
$job = Prism::batch()
235+
->using($providerName)
236+
->whenProvider('openai', fn ($r) => $r->withProviderOptions(['completion_window' => '24h']))
237+
->create(items: $items, inputFileId: $inputFileId);
238+
```
239+
240+
## Client Options
241+
242+
```php
243+
$job = Prism::batch()
244+
->using(Provider::Anthropic)
245+
->withClientOptions(['timeout' => 60])
246+
->withClientRetry(3, 500)
247+
->create(items: $items);
248+
```
249+
250+
## Error Handling
251+
252+
```php
253+
use Prism\Prism\Exceptions\PrismException;
254+
use Throwable;
255+
256+
try {
257+
$job = Prism::batch()
258+
->using(Provider::Anthropic)
259+
->create(items: $items);
260+
} catch (PrismException $e) {
261+
Log::error('Batch creation failed', ['error' => $e->getMessage()]);
262+
} catch (Throwable $e) {
263+
Log::error('Unexpected error', ['error' => $e->getMessage()]);
264+
}
265+
```
266+
267+
> [!TIP]
268+
> Individual result items can have their own failure status independent of the overall batch status. Always check `$result->status` on each item — a `Completed` batch can still contain `Errored` or `Expired` result items.

0 commit comments

Comments
 (0)