Skip to content

Commit ef8cf3d

Browse files
s-sajid-alis-sajid-ali
committed
minor
1 parent 0a88e23 commit ef8cf3d

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

‎docs/genai/04_how_to_guides/04_batch_inference.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Batch Inference
22

3-
When immediate results are not needed, for instance in transforming large datasets of unstructured data with LLMs, batch inference adds convenience while offering lower costs. Typical completion window are 24/48h as LLM inference providers run your workload when the load on the inference server is low. If you are interested in harnessing this feature, reach out to us at [`genai-research-support@nyu.edu`](mailto:genai-research-support@nyu.edu) and we will set up a cloud storage bucket for you.
3+
When immediate results are not needed, for instance in transforming large datasets of unstructured data with LLMs, batch inference adds convenience while offering lower costs. The typical completion window is 24h as LLM inference providers run your workload when the load on the inference server is low. If you are interested in harnessing this feature, reach out to us at [`genai-research-support@nyu.edu`](mailto:genai-research-support@nyu.edu) and we will set up a cloud storage bucket for you.
44

55
:::info
66
Batch processing is only supported for LLMs that can be accessed via the `@vertexai` provider.

0 commit comments

Comments
 (0)