Skip to content

Review 'batch_size' behaviour #181

Description

@lukeroantreeONS

This task has two main parts;

  • update default values of VectorStore methods' batch_size parameter to inherit from the VectorStore's internal batch_size in order to have a single (default) source of truth.

  • investigate performance based on batch_size for a range of hardware setups and use-cases, in order to inform a recommendation for the default batch_size.
    Key requirements: cannot break or perform significantly worse on the smallest instances this may run on (2vCPUs), and optimised for a 'typical' cloud deployment with 4/8vCPUs.
    Note, these are all assumed to be smaller or equal to a typical 'local' instance with 8+vCPUs.

Workflow:

  • create a branch from this issue, check it out locally
  • do part 1, open a draft PR
  • do part 2, present your findings & recommendations to the rest of the dev team
  • from the outcome of that discussion, make the update to the default value and mark the PR ready for review

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions