This task has two main parts;
-
update default values of VectorStore methods' batch_size parameter to inherit from the VectorStore's internal batch_size in order to have a single (default) source of truth.
-
investigate performance based on batch_size for a range of hardware setups and use-cases, in order to inform a recommendation for the default batch_size.
Key requirements: cannot break or perform significantly worse on the smallest instances this may run on (2vCPUs), and optimised for a 'typical' cloud deployment with 4/8vCPUs.
Note, these are all assumed to be smaller or equal to a typical 'local' instance with 8+vCPUs.
Workflow:
- create a branch from this issue, check it out locally
- do part 1, open a draft PR
- do part 2, present your findings & recommendations to the rest of the dev team
- from the outcome of that discussion, make the update to the default value and mark the PR ready for review
This task has two main parts;
update default values of
VectorStoremethods'batch_sizeparameter to inherit from theVectorStore's internalbatch_sizein order to have a single (default) source of truth.investigate performance based on
batch_sizefor a range of hardware setups and use-cases, in order to inform a recommendation for the defaultbatch_size.Key requirements: cannot break or perform significantly worse on the smallest instances this may run on (2vCPUs), and optimised for a 'typical' cloud deployment with 4/8vCPUs.
Note, these are all assumed to be smaller or equal to a typical 'local' instance with 8+vCPUs.
Workflow: