You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[Multimodal Embeddings with `gemini-embedding-2-preview`](#multimodal-embeddings-with-gemini-embedding-2-preview)
36
+
-[License](#license)
36
37
37
38
## Overview
38
39
39
40
[Google Gen AI](https://ai.google.dev/) provides access to Google's Gemini models through the new Google Gen AI SDK. This integration enables the usage of Google's latest generative models via the updated API interface.
40
41
Google Gen AI is compatible with both the Gemini Developer API and the Vertex AI API.
41
42
42
-
Haystack supports the latest [Gemini models](https://ai.google.dev/models/gemini) like `gemini-2.0-flash` and `text-embedding-004` for tasks such as **chat completion**, **function calling**, **streaming responses** and **embedding generation**.
43
+
Haystack supports the latest [Gemini models](https://ai.google.dev/models/gemini) for tasks such as **chat completion**, **function calling**, **streaming responses** and **embedding generation**.
44
+
45
+
**Generative models:**`gemini-3.1-flash-lite-preview`, `gemini-3.1-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-pro`, `gemini-2.5-flash-lite`, and the Gemini 2.0 series (e.g. `gemini-2.0-flash`).
46
+
47
+
**Embedding models:**`gemini-embedding-2-preview` (multimodal, multilingual) and `gemini-embedding-001` (multilingual).
48
+
49
+
> 🖼️ Learn how to build multimodal search systems using Gemini Embedding 2 to embed text, images, video, audio, and PDFs in [Multimodal Search with Gemini Embedding 2 in Haystack](https://haystack.deepset.ai/blog/multimodal-embeddings-gemini-haystack).
Once installed, you will have access to the Haystack Chat components:
55
62
56
-
-[`GoogleGenAIChatGenerator`](https://docs.haystack.deepset.ai/docs/googlegenaichatgenerator): Use this component with [Gemini models](https://ai.google.dev/gemini-api/docs/models/gemini#model-variations), such as '**gemini-2.0-flash**' for chat completion and function calling.
57
-
-`GoogleGenAIDocumentEmbedder`: Use this component with [Google GenAI models](https://ai.google.dev/gemini-api/docs/embeddings#embeddings-models), such as '**text-embedding-004**' for generating embeddings for documents.
58
-
-`GoogleGenAITextEmbedder`: Use this component with [Google GenAI models](https://ai.google.dev/gemini-api/docs/embeddings#embeddings-models), such as '**text-embedding-004**' for generating embeddings for text.
63
+
-[`GoogleGenAIChatGenerator`](https://docs.haystack.deepset.ai/docs/googlegenaichatgenerator): Use this component with [Gemini models](https://ai.google.dev/gemini-api/docs/models/gemini#model-variations), such as **gemini-3.1-flash-lite-preview** or **gemini-2.5-pro** for chat completion and function calling.
64
+
-[`GoogleGenAIDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/googlegenaidocumentembedder): Use this component with [Google GenAI embedding models](https://ai.google.dev/gemini-api/docs/embeddings#embeddings-models), such as **gemini-embedding-001** for generating embeddings for documents.
65
+
-[`GoogleGenAITextEmbedder`](https://docs.haystack.deepset.ai/docs/googlegenaitextembedder): Use this component with [Google GenAI embedding models](https://ai.google.dev/gemini-api/docs/embeddings#embeddings-models), such as **gemini-embedding-001** for generating embeddings for text.
66
+
-[`GoogleGenAIMultimodalDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/googlegenaimultimodaldocumentembedder): Use this component with [Google GenAI multimodal embedding models](https://ai.google.dev/gemini-api/docs/embeddings#embeddings-models), such as **gemini-embedding-2-preview** for generating embeddings for text, image, PDF, video and audio.
59
67
60
68
To use this component with the Gemini Developer API and get an API key, visit [Google AI Studio](https://aistudio.google.com/).
61
69
To use this component with the Vertex AI API, visit [Google Cloud > Vertex AI](https://cloud.google.com/vertex-ai).
@@ -93,9 +101,9 @@ from haystack_integrations.components.generators.google_genai import GoogleGenAI
### Chat Generation with `gemini-3.1-flash-lite-preview`
97
105
98
-
To use Gemini model for chat generation, set the `GOOGLE_API_KEY` or `GEMINI_API_KEY` environment variable and then initialize a `GoogleGenAIChatGenerator` with `"gemini-2.0-flash"`:
106
+
To use Gemini model for chat generation, set the `GOOGLE_API_KEY` or `GEMINI_API_KEY` environment variable and then initialize a `GoogleGenAIChatGenerator` with `"gemini-3.1-flash-lite-preview"`:
99
107
100
108
```python
101
109
import os
@@ -105,7 +113,7 @@ from haystack_integrations.components.generators.google_genai import GoogleGenAI
Embeddings are vector representations of text that capture semantic meaning. They power use cases like **semantic search**, **retrieval-augmented generation (RAG)**, and **similarity comparison** e.g. finding documents or passages that are closest in meaning to a query.
209
+
210
+
This integration provides three embedder components:
211
+
212
+
-**`GoogleGenAIDocumentEmbedder`** — Embeds Haystack `Document` objects (text content). Use it when indexing documents into a vector store or when you need to embed multiple documents in one call. The resulting embeddings are stored on the documents and can be used for document retrieval.
213
+
-**`GoogleGenAITextEmbedder`** — Embeds a single string. Use it when you need to embed a search query or any standalone text to compare against document embeddings (e.g. in a RAG pipeline for the query side).
214
+
-**`GoogleGenAIMultimodalDocumentEmbedder`** — Embeds documents that can contain **images**, **videos**, or **PDFs** (via `meta["file_path"]`). Use the `gemini-embedding-2-preview` model for multimodal retrieval, e.g. searching over mixed media with text or image queries.
215
+
216
+
For text-only pipelines, use the same model (e.g. `gemini-embedding-001`) for documents and queries so that their embeddings live in the same vector space. For multimodal content, use `gemini-embedding-2-preview` with the appropriate `task_type` in the config.
217
+
218
+
#### Document Embedding
219
+
220
+
Use `GoogleGenAIDocumentEmbedder` to create embeddings for documents before storing them in a document store or using them in retrieval:
199
221
200
222
```python
201
223
import os
@@ -204,15 +226,17 @@ from haystack_integrations.components.embedders.google_genai import GoogleGenAID
The returned embedding is a list of floats (e.g. 3072 dimensions for `gemini-embedding-001`). In a typical RAG pipeline you would pass this query embedding to a retriever to find the most similar document embeddings from your index.
263
+
264
+
#### Multimodal Embeddings with `gemini-embedding-2-preview`
265
+
266
+
`GoogleGenAIMultimodalDocumentEmbedder` embeds documents that reference **text**, **images**, **audios**, **videos**, or **PDFs** via `meta["file_path"]`. It uses the `gemini-embedding-2-preview` model, which produces a unified embedding space for text, images, and other modalities—so you can index mixed media and retrieve with text or image queries.
267
+
268
+
Set `config={"task_type": "RETRIEVAL_DOCUMENT"}` when embedding documents for indexing, and use `RETRIEVAL_QUERY` when embedding queries. Use the same model and task type pairing so document and query embeddings are comparable.
269
+
270
+
Example: embedding multiple files (text, audio, video, images, PDF) and writing them to a document store:
271
+
272
+
```python
273
+
import os
274
+
from haystack.document_stores.in_memory import InMemoryDocumentStore
275
+
from haystack import Document
276
+
from haystack_integrations.components.embedders.google_genai import GoogleGenAIMultimodalDocumentEmbedder
0 commit comments