diff --git a/docs-website/reference/integrations-api/sentence_transformers.md b/docs-website/reference/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference/integrations-api/sentence_transformers.md
+++ b/docs-website/reference/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.18/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.18/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.18/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.18/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.19/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.19/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.19/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.19/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.20/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.20/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.20/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.20/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.21/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.21/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.21/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.21/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.22/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.22/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.22/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.22/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.23/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.23/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.23/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.23/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.24/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.24/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.24/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.24/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.25/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.25/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.25/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.25/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.26/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.26/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.26/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.26/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.27/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.27/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.27/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.27/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.28/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.28/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.28/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.28/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.29/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.29/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.29/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.29/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.30/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.30/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.30/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.30/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict
diff --git a/docs-website/reference_versioned_docs/version-2.31-unstable/integrations-api/sentence_transformers.md b/docs-website/reference_versioned_docs/version-2.31-unstable/integrations-api/sentence_transformers.md
index fbe28984f89..b2be5f991a2 100644
--- a/docs-website/reference_versioned_docs/version-2.31-unstable/integrations-api/sentence_transformers.md
+++ b/docs-website/reference_versioned_docs/version-2.31-unstable/integrations-api/sentence_transformers.md
@@ -217,7 +217,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -262,6 +263,12 @@ Creates a SentenceTransformersDocumentEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for small batches and inconsistent across batches. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings, compatible with query embeddings
+ quantized with the same ranges.
#### to_dict
@@ -621,7 +628,8 @@ __init__(
] = "float32",
encode_kwargs: dict[str, Any] | None = None,
backend: Literal["torch", "onnx", "openvino"] = "torch",
- revision: str | None = None
+ revision: str | None = None,
+ quantization_ranges: list[list[float]] | None = None
) -> None
```
@@ -664,6 +672,11 @@ Create a SentenceTransformersTextEmbedder component.
for more information on acceleration and quantization options.
- **revision** (str | None) – The specific model version to use. It can be a branch name, a tag name, or a commit id,
for a stored model on Hugging Face.
+- **quantization_ranges** (list\[list\[float\]\] | None) – Calibration ranges to use when `precision` is "int8" or "uint8", with shape `(2, embedding_dim)`:
+ minimum values in the first row and maximum values in the second.
+ Scalar quantization calibrates the min/max range from the batch being encoded, which is degenerate
+ for a single text and produces meaningless embeddings. Pass ranges computed from a representative
+ sample of embeddings to get consistent quantized embeddings.
#### to_dict