diff --git a/docs-website/reference/integrations-api/google_vertex.md b/docs-website/reference/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference/integrations-api/google_vertex.md +++ b/docs-website/reference/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.18/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.18/integrations-api/google_vertex.md index fc702ca62a..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.18/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.18/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1032 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAITextGenerator – Deserialized component. - +#### run -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder - - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. +Prompts the model to generate text. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **prompt** (str) – The prompt to use for text generation. -**Returns**: +**Returns:** -Deserialized component. +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.19/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.19/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.19/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.19/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.20/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.20/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.20/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.20/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.21/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.21/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.21/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.21/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.22/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.22/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.22/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.22/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.23/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.23/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.23/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.23/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.24/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.24/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.24/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.24/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.25/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.25/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.25/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.25/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.26/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.26/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.26/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.26/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.27/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.27/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.27/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.27/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.28/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.28/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.28/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.28/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.29/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.29/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.29/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.29/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.30/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.30/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.30/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.30/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer. diff --git a/docs-website/reference_versioned_docs/version-2.31/integrations-api/google_vertex.md b/docs-website/reference_versioned_docs/version-2.31/integrations-api/google_vertex.md index 897f7e5fa6..78cac5ef8f 100644 --- a/docs-website/reference_versioned_docs/version-2.31/integrations-api/google_vertex.md +++ b/docs-website/reference_versioned_docs/version-2.31/integrations-api/google_vertex.md @@ -5,144 +5,296 @@ description: "Google Vertex integration for Haystack" slug: "/integrations-google-vertex" --- - -## Module haystack\_integrations.components.generators.google\_vertex.gemini +## haystack_integrations.components.embedders.google_vertex.document_embedder - +### VertexAIDocumentEmbedder -### VertexAIGeminiGenerator +Embed text using Vertex AI Embeddings API. -`VertexAIGeminiGenerator` enables text generation using Google Gemini models. +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). Usage example: + ```python -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator +from haystack import Document +from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder +doc = Document(content="I love pizza!") -gemini = VertexAIGeminiGenerator() -result = gemini.run(parts = ["What is the most interesting thing you know?"]) -for answer in result["replies"]: - print(answer) +document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") ->>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... ->>> 2. **The Unseen Universe:** The vast majority of the universe is ... ->>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... ->>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... ->>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... ->>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... ->>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... ->>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... ->>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... ->>> 10. **Consciousness:** The nature of human consciousness and how it arises ... +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) +# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, ``` - - -#### VertexAIGeminiGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-2.0-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - system_instruction: Optional[Union[str, ByteStream, Part]] = None, - streaming_callback: Optional[Callable[[StreamingChunk], - None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_DOCUMENT", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + batch_size: int = 32, + max_tokens_total: int = 20000, + time_sleep: int = 30, + retries: int = 3, + progress_bar: bool = True, + truncate_dim: Optional[int] = None, + meta_fields_to_embed: Optional[list[str]] = None, + embedding_separator: str = "\n", +) -> None ``` -Multi-modal generator using Gemini model via Google Vertex AI. +Generate Document Embedder using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `generation_config`: The generation config to use. -Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) -object or a dictionary of parameters. -Accepted fields are: - - temperature - - top_p - - top_k - - candidate_count - - max_output_tokens - - stop_sequences -- `safety_settings`: The safety settings to use. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `system_instruction`: Default system instruction to use for generating content. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiGenerator.to\_dict +**Parameters:** + +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **batch_size** (int) – The number of documents to process in a single batch. +- **max_tokens_total** (int) – The maximum number of tokens to process in total. +- **time_sleep** (int) – The time to sleep between retries in seconds. +- **retries** (int) – The number of retries in case of failure. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. +- **meta_fields_to_embed** (Optional\[list\[str\]\]) – A list of metadata fields to include in the embeddings. +- **embedding_separator** (str) – The separator to use between different embeddings. + +**Raises:** + +- ValueError – If the provided model is not in the list of supported models. + +#### get_text_embedding_input ```python -def to_dict() -> dict[str, Any] +get_text_embedding_input(batch: list[Document]) -> list[TextEmbeddingInput] ``` -Serializes the component to a dictionary. +Converts a batch of Document objects into a list of TextEmbeddingInput objects. + +Args: +batch (List[Document]): A list of Document objects to be converted. + +Returns: +List\[TextEmbeddingInput\]: A list of TextEmbeddingInput objects created from the input documents. + +#### embed_batch_by_smaller_batches + +```python +embed_batch_by_smaller_batches( + batch: list[str], subbatch: list[str] = 1 +) -> list[list[float]] +``` + +Embeds a batch of text strings by dividing them into smaller sub-batches. +Args: +batch (List[str]): A list of text strings to be embedded. +subbatch (int, optional): The size of the smaller sub-batches. Defaults to 1. +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. +Raises: +Exception: If embedding fails at the item level, an exception is raised with the error details. + +#### embed_batch + +```python +embed_batch(batch: list[str]) -> list[list[float]] +``` + +Generate embeddings for a batch of text strings. + +Args: +batch (List[str]): A list of text strings to be embedded. + +Returns: +List\[List[float]\]: A list of embeddings, where each embedding is a list of floats. + +#### run + +```python +run(documents: list[Document]) +``` + +Processes all documents in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` -**Returns**: +Serializes the component to a dictionary. -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiGenerator" +from_dict(data: dict[str, Any]) -> VertexAIDocumentEmbedder ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- VertexAIDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.google_vertex.text_embedder + +### VertexAITextEmbedder + +Embed text using VertexAI Text Embeddings API. + +See available models in the official +[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). + +Usage example: -- `data`: Dictionary to deserialize from. +```python +from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder -**Returns**: +text_to_embed = "I love pizza!" -Deserialized component. +text_embedder = VertexAITextEmbedder(model="text-embedding-005") - +print(text_embedder.run(text_to_embed)) +# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +``` -#### VertexAIGeminiGenerator.run +#### __init__ ```python -@component.output_types(replies=list[str]) -def run(parts: Variadic[Union[str, ByteStream, Part]], - streaming_callback: Optional[Callable[[StreamingChunk], None]] = None) +__init__( + model: Literal[ + "text-embedding-004", + "text-embedding-005", + "textembedding-gecko-multilingual@001", + "text-multilingual-embedding-002", + "text-embedding-large-exp-03-07", + ], + task_type: Literal[ + "RETRIEVAL_DOCUMENT", + "RETRIEVAL_QUERY", + "SEMANTIC_SIMILARITY", + "CLASSIFICATION", + "CLUSTERING", + "QUESTION_ANSWERING", + "FACT_VERIFICATION", + "CODE_RETRIEVAL_QUERY", + ] = "RETRIEVAL_QUERY", + gcp_region_name: Optional[Secret] = Secret.from_env_var( + "GCP_DEFAULT_REGION", strict=False + ), + gcp_project_id: Optional[Secret] = Secret.from_env_var( + "GCP_PROJECT_ID", strict=False + ), + progress_bar: bool = True, + truncate_dim: Optional[int] = None, +) -> None ``` -Generates content using the Gemini model. +Initializes the TextEmbedder with the specified model, task type, and GCP configuration. -**Arguments**: +**Parameters:** -- `parts`: Prompt for the model. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. +- **model** (Literal['text-embedding-004', 'text-embedding-005', 'textembedding-gecko-multilingual@001', 'text-multilingual-embedding-002', 'text-embedding-large-exp-03-07']) – Name of the model to use. +- **task_type** (Literal['RETRIEVAL_DOCUMENT', 'RETRIEVAL_QUERY', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING', 'QUESTION_ANSWERING', 'FACT_VERIFICATION', 'CODE_RETRIEVAL_QUERY']) – The type of task for which the embeddings are being generated. + For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). +- **gcp_region_name** (Optional\[Secret\]) – The default location to use when making API calls, if not set uses us-central-1. +- **gcp_project_id** (Optional\[Secret\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **progress_bar** (bool) – Whether to display a progress bar during processing. +- **truncate_dim** (Optional\[int\]) – The dimension to truncate the embeddings to, if specified. -**Returns**: +#### run -A dictionary with the following keys: -- `replies`: A list of generated content. +```python +run(text: Union[list[Document], list[str], str]) +``` + +Processes text in batches while adhering to the API's token limit per request. + +**Parameters:** + +- **text** (Union\[list\[Document\], list\[str\], str\]) – The text to embed. + +**Returns:** + +- – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> VertexAITextEmbedder +``` + +Deserializes the component from a dictionary. - +**Parameters:** -## Module haystack\_integrations.components.generators.google\_vertex.captioner +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. - +**Returns:** + +- VertexAITextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.google_vertex.captioner ### VertexAIImageCaptioner @@ -152,6 +304,7 @@ Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: + ```python import requests @@ -173,16 +326,16 @@ for caption in result["captions"]: >>> two gold robots are standing next to each other in the desert ``` - - -#### VertexAIImageCaptioner.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` Generate image captions using a Google Vertex AI model. @@ -190,1033 +343,760 @@ Generate image captions using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.get_captions()` documentation. -#### VertexAIImageCaptioner.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageCaptioner.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageCaptioner" +from_dict(data: dict[str, Any]) -> VertexAIImageCaptioner ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIImageCaptioner – Deserialized component. -#### VertexAIImageCaptioner.run +#### run ```python -@component.output_types(captions=list[str]) -def run(image: ByteStream) +run(image: ByteStream) ``` Prompts the model to generate captions for the given image. -**Arguments**: +**Parameters:** -- `image`: The image to generate captions for. +- **image** (ByteStream) – The image to generate captions for. -**Returns**: +**Returns:** -A dictionary with the following keys: +- – A dictionary with the following keys: - `captions`: A list of captions generated by the model. - +## haystack_integrations.components.generators.google_vertex.chat.gemini -## Module haystack\_integrations.components.generators.google\_vertex.code\_generator +### VertexAIGeminiChatGenerator - +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. -### VertexAICodeGenerator +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -This component enables code generation using Google Vertex AI generative model. +### Usage example -`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. +````python +from haystack.dataclasses import ChatMessage +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator + +gemini_chat = VertexAIGeminiChatGenerator() + +messages = [ChatMessage.from_user("Tell me the name of a movie")] +res = gemini_chat.run(messages) + +print(res["replies"][0].text) +>>> The Shawshank Redemption + +#### With Tool calling: -Usage example: ```python - from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator +from typing import Annotated +from haystack.utils import Secret +from haystack.dataclasses.chat_message import ChatMessage +from haystack.components.tools import ToolInvoker +from haystack.tools import create_tool_from_function - generator = VertexAICodeGenerator() +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - result = generator.run(prefix="def to_json(data):") +# example function to get the current weather +def get_current_weather( + location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", + unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", +) -> str: + return f"The weather in {location} is sunny. The temperature is 20 {unit}." - for answer in result["replies"]: - print(answer) +tool = create_tool_from_function(get_current_weather) +tool_invoker = ToolInvoker(tools=[tool]) - >>> ```python - >>> import json - >>> - >>> def to_json(data): - >>> """Converts a Python object to a JSON string. - >>> - >>> Args: - >>> data: The Python object to convert. - >>> - >>> Returns: - >>> A JSON string representing the Python object. - >>> """ - >>> - >>> return json.dumps(data) - >>> ``` -``` +gemini_chat = VertexAIGeminiChatGenerator( + model="gemini-2.0-flash-exp", + tools=[tool], +) +user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] +replies = gemini_chat.run(messages=user_message)["replies"] +print(replies[0].tool_calls) + +# actually invoke the tool +tool_messages = tool_invoker.run(messages=replies)["tool_messages"] +messages = user_message + replies + tool_messages - +# transform the tool call result into a human readable message +final_replies = gemini_chat.run(messages=messages)["replies"] +print(final_replies[0].text) +```` -#### VertexAICodeGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "code-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-1.5-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + tools: Optional[list[Tool]] = None, + tool_config: Optional[ToolConfig] = None, + streaming_callback: Optional[StreamingCallbackT] = None +) ``` -Generate code using a Google Vertex AI model. +`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. - - - -#### VertexAICodeGenerator.to\_dict +**Parameters:** + +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. + Defaults to None. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – Configuration for the generation process. + See the \[GenerationConfig documentation\](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig + for a list of supported arguments. +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – Safety settings to use when generating content. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. +- **tool_config** (Optional\[ToolConfig\]) – The tool config to use. See the documentation for [ToolConfig] + (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from + the stream. The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAICodeGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAICodeGenerator" +from_dict(data: dict[str, Any]) -> VertexAIGeminiChatGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIGeminiChatGenerator – Deserialized component. - - -#### VertexAICodeGenerator.run +#### run ```python -@component.output_types(replies=list[str]) -def run(prefix: str, suffix: Optional[str] = None) +run( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) ``` -Generate code using a Google Vertex AI model. +**Parameters:** -**Arguments**: +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -- `prefix`: Code before the current point. -- `suffix`: Code after the current point. +**Returns:** -**Returns**: +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. -A dictionary with the following keys: -- `replies`: A list of generated code snippets. +#### run_async - +```python +run_async( + messages: list[ChatMessage], + streaming_callback: Optional[StreamingCallbackT] = None, + *, + tools: Optional[list[Tool]] = None +) +``` -## Module haystack\_integrations.components.generators.google\_vertex.image\_generator +Async version of the run method. Generates text based on the provided messages. - +**Parameters:** -### VertexAIImageGenerator +- **messages** (list\[ChatMessage\]) – A list of `ChatMessage` instances, representing the input messages. +- **streaming_callback** (Optional\[StreamingCallbackT\]) – A callback function that is called when a new token is received from the stream. +- **tools** (Optional\[list\[Tool\]\]) – A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set + during component initialization. -This component enables image generation using Google Vertex AI generative model. +**Returns:** -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +- – A dictionary containing the following key: +- `replies`: A list containing the generated responses as `ChatMessage` instances. + +## haystack_integrations.components.generators.google_vertex.code_generator + +### VertexAICodeGenerator + +This component enables code generation using Google Vertex AI generative model. + +`VertexAICodeGenerator` supports `code-bison`, `code-bison-32k`, and `code-gecko`. Usage example: -```python -from pathlib import Path -from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator +````python + from haystack_integrations.components.generators.google_vertex import VertexAICodeGenerator -generator = VertexAIImageGenerator() -result = generator.run(prompt="Generate an image of a cute cat") -result["images"][0].to_file(Path("my_image.png")) -``` + generator = VertexAICodeGenerator() + + result = generator.run(prefix="def to_json(data):") - + for answer in result["replies"]: + print(answer) + + >>> ```python + >>> import json + >>> + >>> def to_json(data): + >>> """Converts a Python object to a JSON string. + >>> + >>> Args: + >>> data: The Python object to convert. + >>> + >>> Returns: + >>> A JSON string representing the Python object. + >>> """ + >>> + >>> return json.dumps(data) + >>> ``` +```` -#### VertexAIImageGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagegeneration", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "code-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generates images using a Google Vertex AI model. +Generate code using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. +**Parameters:** - +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -#### VertexAIImageGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIImageGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageGenerator" +from_dict(data: dict[str, Any]) -> VertexAICodeGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAICodeGenerator – Deserialized component. -#### VertexAIImageGenerator.run +#### run ```python -@component.output_types(images=list[ByteStream]) -def run(prompt: str, negative_prompt: Optional[str] = None) +run(prefix: str, suffix: Optional[str] = None) ``` -Produces images based on the given prompt. +Generate code using a Google Vertex AI model. -**Arguments**: +**Parameters:** -- `prompt`: The prompt to generate images from. -- `negative_prompt`: A description of what you want to omit in -the generated images. +- **prefix** (str) – Code before the current point. +- **suffix** (Optional\[str\]) – Code after the current point. -**Returns**: +**Returns:** -A dictionary with the following keys: -- `images`: A list of ByteStream objects, each containing an image. +- – A dictionary with the following keys: +- `replies`: A list of generated code snippets. - +## haystack_integrations.components.generators.google_vertex.gemini -## Module haystack\_integrations.components.generators.google\_vertex.question\_answering - - - -### VertexAIImageQA - -This component enables text generation (image captioning) using Google Vertex AI generative models. +### VertexAIGeminiGenerator -Authenticates using Google Cloud Application Default Credentials (ADCs). -For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). +`VertexAIGeminiGenerator` enables text generation using Google Gemini models. Usage example: -```python -from haystack.dataclasses.byte_stream import ByteStream -from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -qa = VertexAIImageQA() - -image = ByteStream.from_file_path("dog.jpg") +```python +from haystack_integrations.components.generators.google_vertex import VertexAIGeminiGenerator -res = qa.run(image=image, question="What color is this dog") -print(res["replies"][0]) +gemini = VertexAIGeminiGenerator() +result = gemini.run(parts = ["What is the most interesting thing you know?"]) +for answer in result["replies"]: + print(answer) ->>> white +>>> 1. **The Origin of Life:** How and where did life begin? The answers to this ... +>>> 2. **The Unseen Universe:** The vast majority of the universe is ... +>>> 3. **Quantum Entanglement:** This eerie phenomenon in quantum mechanics allows ... +>>> 4. **Time Dilation:** Einstein's theory of relativity revealed that time can ... +>>> 5. **The Fermi Paradox:** Despite the vastness of the universe and the ... +>>> 6. **Biological Evolution:** The idea that life evolves over time through natural ... +>>> 7. **Neuroplasticity:** The brain's ability to adapt and change throughout life, ... +>>> 8. **The Goldilocks Zone:** The concept of the habitable zone, or the Goldilocks zone, ... +>>> 9. **String Theory:** This theoretical framework in physics aims to unify all ... +>>> 10. **Consciousness:** The nature of human consciousness and how it arises ... ``` - - -#### VertexAIImageQA.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "imagetext", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "gemini-2.0-flash", + project_id: Optional[str] = None, + location: Optional[str] = None, + generation_config: Optional[Union[GenerationConfig, dict[str, Any]]] = None, + safety_settings: Optional[dict[HarmCategory, HarmBlockThreshold]] = None, + system_instruction: Optional[Union[str, ByteStream, Part]] = None, + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None +) ``` -Answers questions about an image using a Google Vertex AI model. +Multi-modal generator using Gemini model via Google Vertex AI. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - - -#### VertexAIImageQA.to\_dict +**Parameters:** + +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **generation_config** (Optional\[Union\[GenerationConfig, dict\[str, Any\]\]\]) – The generation config to use. + Can either be a [`GenerationConfig`](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig) + object or a dictionary of parameters. + Accepted fields are: + - temperature + - top_p + - top_k + - candidate_count + - max_output_tokens + - stop_sequences +- **safety_settings** (Optional\[dict\[HarmCategory, HarmBlockThreshold\]\]) – The safety settings to use. See the documentation + for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) + and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) + for more details. +- **system_instruction** (Optional\[Union\[str, ByteStream, Part\]\]) – Default system instruction to use for generating content. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. + The callback function accepts StreamingChunk as an argument. + +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIImageQA.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIImageQA" +from_dict(data: dict[str, Any]) -> VertexAIGeminiGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAIGeminiGenerator – Deserialized component. -#### VertexAIImageQA.run +#### run ```python -@component.output_types(replies=list[str]) -def run(image: ByteStream, question: str) +run( + parts: Variadic[Union[str, ByteStream, Part]], + streaming_callback: Optional[Callable[[StreamingChunk], None]] = None, +) ``` -Prompts model to answer a question about an image. - -**Arguments**: - -- `image`: The image to ask the question about. -- `question`: The question to ask. - -**Returns**: +Generates content using the Gemini model. -A dictionary with the following keys: -- `replies`: A list of answers to the question. +**Parameters:** - +- **parts** (Variadic\[Union\[str, ByteStream, Part\]\]) – Prompt for the model. +- **streaming_callback** (Optional\[Callable\\[[StreamingChunk\], None\]\]) – A callback function that is called when a new token is received from the stream. -## Module haystack\_integrations.components.generators.google\_vertex.text\_generator +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of generated content. -### VertexAITextGenerator +## haystack_integrations.components.generators.google_vertex.image_generator -This component enables text generation using Google Vertex AI generative models. +### VertexAIImageGenerator -`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. +This component enables image generation using Google Vertex AI generative model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python - from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator - generator = VertexAITextGenerator() - res = generator.run("Tell me a good interview question for a software engineer.") +```python +from pathlib import Path - print(res["replies"][0]) +from haystack_integrations.components.generators.google_vertex import VertexAIImageGenerator - >>> **Question:** - >>> You are given a list of integers and a target sum. - >>> Find all unique combinations of numbers in the list that add up to the target sum. - >>> - >>> **Example:** - >>> - >>> ``` - >>> Input: [1, 2, 3, 4, 5], target = 7 - >>> Output: [[1, 2, 4], [3, 4]] - >>> ``` - >>> - >>> **Follow-up:** What if the list contains duplicate numbers? +generator = VertexAIImageGenerator() +result = generator.run(prompt="Generate an image of a cute cat") +result["images"][0].to_file(Path("my_image.png")) ``` - - -#### VertexAITextGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "text-bison", - project_id: Optional[str] = None, - location: Optional[str] = None, - **kwargs) +__init__( + *, + model: str = "imagegeneration", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate text using a Google Vertex AI model. +Generates images using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: +**Parameters:** -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `model`: Name of the model to use. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -- `kwargs`: Additional keyword arguments to pass to the model. -For a list of supported arguments see the `TextGenerationModel.predict()` documentation. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageGenerationModel.generate_images()` documentation. - - -#### VertexAITextGenerator.to\_dict +#### to_dict ```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: - -Dictionary with serialized data. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAITextGenerator.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextGenerator" +from_dict(data: dict[str, Any]) -> VertexAIImageGenerator ``` Deserializes the component from a dictionary. -**Arguments**: +**Parameters:** -- `data`: Dictionary to deserialize from. +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -**Returns**: +**Returns:** -Deserialized component. +- VertexAIImageGenerator – Deserialized component. - - -#### VertexAITextGenerator.run +#### run ```python -@component.output_types(replies=list[str], - safety_attributes=dict[str, float], - citations=list[dict[str, Any]]) -def run(prompt: str) +run(prompt: str, negative_prompt: Optional[str] = None) ``` -Prompts the model to generate text. - -**Arguments**: - -- `prompt`: The prompt to use for text generation. +Produces images based on the given prompt. -**Returns**: +**Parameters:** -A dictionary with the following keys: -- `replies`: A list of generated replies. -- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) - of each answer. -- `citations`: A list of citations for each answer. +- **prompt** (str) – The prompt to generate images from. +- **negative_prompt** (Optional\[str\]) – A description of what you want to omit in + the generated images. - +**Returns:** -## Module haystack\_integrations.components.generators.google\_vertex.chat.gemini +- – A dictionary with the following keys: +- `images`: A list of ByteStream objects, each containing an image. - +## haystack_integrations.components.generators.google_vertex.question_answering -### VertexAIGeminiChatGenerator +### VertexAIImageQA -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +This component enables text generation (image captioning) using Google Vertex AI generative models. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -### Usage example -```python -from haystack.dataclasses import ChatMessage -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -gemini_chat = VertexAIGeminiChatGenerator() - -messages = [ChatMessage.from_user("Tell me the name of a movie")] -res = gemini_chat.run(messages) - -print(res["replies"][0].text) ->>> The Shawshank Redemption - -#### With Tool calling: +Usage example: ```python -from typing import Annotated -from haystack.utils import Secret -from haystack.dataclasses.chat_message import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.tools import create_tool_from_function - -from haystack_integrations.components.generators.google_vertex import VertexAIGeminiChatGenerator - -__example function to get the current weather__ - -def get_current_weather( - location: Annotated[str, "The city for which to get the weather, e.g. 'San Francisco'"] = "Munich", - unit: Annotated[str, "The unit for the temperature, e.g. 'celsius'"] = "celsius", -) -> str: - return f"The weather in {location} is sunny. The temperature is 20 {unit}." - -tool = create_tool_from_function(get_current_weather) -tool_invoker = ToolInvoker(tools=[tool]) +from haystack.dataclasses.byte_stream import ByteStream +from haystack_integrations.components.generators.google_vertex import VertexAIImageQA -gemini_chat = VertexAIGeminiChatGenerator( - model="gemini-2.0-flash-exp", - tools=[tool], -) -user_message = [ChatMessage.from_user("What is the temperature in celsius in Berlin?")] -replies = gemini_chat.run(messages=user_message)["replies"] -print(replies[0].tool_calls) +qa = VertexAIImageQA() -__actually invoke the tool__ +image = ByteStream.from_file_path("dog.jpg") -tool_messages = tool_invoker.run(messages=replies)["tool_messages"] -messages = user_message + replies + tool_messages +res = qa.run(image=image, question="What color is this dog") -__transform the tool call result into a human readable message__ +print(res["replies"][0]) -final_replies = gemini_chat.run(messages=messages)["replies"] -print(final_replies[0].text) +>>> white ``` - - -#### VertexAIGeminiChatGenerator.\_\_init\_\_ +#### __init__ ```python -def __init__(*, - model: str = "gemini-1.5-flash", - project_id: Optional[str] = None, - location: Optional[str] = None, - generation_config: Optional[Union[GenerationConfig, - dict[str, Any]]] = None, - safety_settings: Optional[dict[HarmCategory, - HarmBlockThreshold]] = None, - tools: Optional[list[Tool]] = None, - tool_config: Optional[ToolConfig] = None, - streaming_callback: Optional[StreamingCallbackT] = None) +__init__( + *, + model: str = "imagetext", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -`VertexAIGeminiChatGenerator` enables chat completion using Google Gemini models. +Answers questions about an image using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. For available models, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models. -- `project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `location`: The default location to use when making API calls, if not set uses us-central-1. -Defaults to None. -- `generation_config`: Configuration for the generation process. -See the [GenerationConfig documentation](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.GenerationConfig -for a list of supported arguments. -- `safety_settings`: Safety settings to use when generating content. See the documentation -for [HarmBlockThreshold](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmBlockThreshold) -and [HarmCategory](https://cloud.google.com/python/docs/reference/aiplatform/latest/vertexai.generative_models.HarmCategory) -for more details. -- `tools`: A list of tools for which the model can prepare calls. -- `tool_config`: The tool config to use. See the documentation for [ToolConfig] -(https://cloud.google.com/vertex-ai/generative-ai/docs/reference/python/latest/vertexai.generative_models.ToolConfig) -- `streaming_callback`: A callback function that is called when a new token is received from -the stream. The callback function accepts StreamingChunk as an argument. - - - -#### VertexAIGeminiChatGenerator.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: +**Parameters:** -Dictionary with serialized data. +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `ImageTextModel.ask_question()` documentation. - - -#### VertexAIGeminiChatGenerator.from\_dict +#### to_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIGeminiChatGenerator" +to_dict() -> dict[str, Any] ``` -Deserializes the component from a dictionary. - -**Arguments**: - -- `data`: Dictionary to deserialize from. - -**Returns**: +Serializes the component to a dictionary. -Deserialized component. +**Returns:** - +- dict\[str, Any\] – Dictionary with serialized data. -#### VertexAIGeminiChatGenerator.run +#### from_dict ```python -@component.output_types(replies=list[ChatMessage]) -def run(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +from_dict(data: dict[str, Any]) -> VertexAIImageQA ``` -**Arguments**: +Deserializes the component from a dictionary. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- VertexAIImageQA – Deserialized component. -#### VertexAIGeminiChatGenerator.run\_async +#### run ```python -@component.output_types(replies=list[ChatMessage]) -async def run_async(messages: list[ChatMessage], - streaming_callback: Optional[StreamingCallbackT] = None, - *, - tools: Optional[list[Tool]] = None) +run(image: ByteStream, question: str) ``` -Async version of the run method. Generates text based on the provided messages. - -**Arguments**: +Prompts model to answer a question about an image. -- `messages`: A list of `ChatMessage` instances, representing the input messages. -- `streaming_callback`: A callback function that is called when a new token is received from the stream. -- `tools`: A list of tools for which the model can prepare calls. If set, it will override the `tools` parameter set -during component initialization. +**Parameters:** -**Returns**: +- **image** (ByteStream) – The image to ask the question about. +- **question** (str) – The question to ask. -A dictionary containing the following key: -- `replies`: A list containing the generated responses as `ChatMessage` instances. +**Returns:** - +- – A dictionary with the following keys: +- `replies`: A list of answers to the question. -## Module haystack\_integrations.components.embedders.google\_vertex.document\_embedder +## haystack_integrations.components.generators.google_vertex.text_generator - +### VertexAITextGenerator -### VertexAIDocumentEmbedder +This component enables text generation using Google Vertex AI generative models. -Embed text using Vertex AI Embeddings API. +`VertexAITextGenerator` supports `text-bison`, `text-unicorn` and `text-bison-32k` models. -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). +Authenticates using Google Cloud Application Default Credentials (ADCs). +For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). Usage example: -```python -from haystack import Document -from haystack_integrations.components.embedders.google_vertex import VertexAIDocumentEmbedder -doc = Document(content="I love pizza!") +````python + from haystack_integrations.components.generators.google_vertex import VertexAITextGenerator -document_embedder = VertexAIDocumentEmbedder(model="text-embedding-005") + generator = VertexAITextGenerator() + res = generator.run("Tell me a good interview question for a software engineer.") -result = document_embedder.run([doc]) -print(result['documents'][0].embedding) -# [-0.044606007635593414, 0.02857724390923977, -0.03549133986234665, -``` + print(res["replies"][0]) - + >>> **Question:** + >>> You are given a list of integers and a target sum. + >>> Find all unique combinations of numbers in the list that add up to the target sum. + >>> + >>> **Example:** + >>> + >>> ``` + >>> Input: [1, 2, 3, 4, 5], target = 7 + >>> Output: [[1, 2, 4], [3, 4]] + >>> ``` + >>> + >>> **Follow-up:** What if the list contains duplicate numbers? +```` -#### VertexAIDocumentEmbedder.\_\_init\_\_ +#### __init__ ```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_DOCUMENT", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - batch_size: int = 32, - max_tokens_total: int = 20000, - time_sleep: int = 30, - retries: int = 3, - progress_bar: bool = True, - truncate_dim: Optional[int] = None, - meta_fields_to_embed: Optional[list[str]] = None, - embedding_separator: str = "\n") -> None +__init__( + *, + model: str = "text-bison", + project_id: Optional[str] = None, + location: Optional[str] = None, + **kwargs: Optional[str] +) ``` -Generate Document Embedder using a Google Vertex AI model. +Generate text using a Google Vertex AI model. Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official [Google documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc). -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `batch_size`: The number of documents to process in a single batch. -- `max_tokens_total`: The maximum number of tokens to process in total. -- `time_sleep`: The time to sleep between retries in seconds. -- `retries`: The number of retries in case of failure. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. -- `meta_fields_to_embed`: A list of metadata fields to include in the embeddings. -- `embedding_separator`: The separator to use between different embeddings. - -**Raises**: - -- `ValueError`: If the provided model is not in the list of supported models. - - - -#### VertexAIDocumentEmbedder.get\_text\_embedding\_input - -```python -def get_text_embedding_input( - batch: list[Document]) -> list[TextEmbeddingInput] -``` - -Converts a batch of Document objects into a list of TextEmbeddingInput objects. - -**Arguments**: - -- `batch` _List[Document]_ - A list of Document objects to be converted. - +**Parameters:** -**Returns**: +- **project_id** (Optional\[str\]) – ID of the GCP project to use. By default, it is set during Google Cloud authentication. +- **model** (str) – Name of the model to use. +- **location** (Optional\[str\]) – The default location to use when making API calls, if not set uses us-central-1. +- **kwargs** – Additional keyword arguments to pass to the model. + For a list of supported arguments see the `TextGenerationModel.predict()` documentation. -- `List[TextEmbeddingInput]` - A list of TextEmbeddingInput objects created from the input documents. - - - -#### VertexAIDocumentEmbedder.embed\_batch\_by\_smaller\_batches - -```python -def embed_batch_by_smaller_batches(batch: list[str], - subbatch=1) -> list[list[float]] -``` - -Embeds a batch of text strings by dividing them into smaller sub-batches. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. -- `subbatch` _int, optional_ - The size of the smaller sub-batches. Defaults to 1. - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - -**Raises**: - -- `Exception` - If embedding fails at the item level, an exception is raised with the error details. - - - -#### VertexAIDocumentEmbedder.embed\_batch +#### to_dict ```python -def embed_batch(batch: list[str]) -> list[list[float]] -``` - -Generate embeddings for a batch of text strings. - -**Arguments**: - -- `batch` _List[str]_ - A list of text strings to be embedded. - - -**Returns**: - -- `List[List[float]]` - A list of embeddings, where each embedding is a list of floats. - - - -#### VertexAIDocumentEmbedder.run - -```python -@component.output_types(documents=list[Document]) -def run(documents: list[Document]) -``` - -Processes all documents in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `documents`: A list of documents to embed. - -**Returns**: - -A dictionary with the following keys: -- `documents`: A list of documents with embeddings. - - - -#### VertexAIDocumentEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] +to_dict() -> dict[str, Any] ``` Serializes the component to a dictionary. -**Returns**: +**Returns:** -Dictionary with serialized data. +- dict\[str, Any\] – Dictionary with serialized data. - - -#### VertexAIDocumentEmbedder.from\_dict +#### from_dict ```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAIDocumentEmbedder" +from_dict(data: dict[str, Any]) -> VertexAITextGenerator ``` Deserializes the component from a dictionary. -**Arguments**: - -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. -Deserialized component. +**Returns:** - +- VertexAITextGenerator – Deserialized component. -## Module haystack\_integrations.components.embedders.google\_vertex.text\_embedder +#### run - - -### VertexAITextEmbedder - -Embed text using VertexAI Text Embeddings API. - -See available models in the official -[Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#syntax). - -Usage example: ```python -from haystack_integrations.components.embedders.google_vertex import VertexAITextEmbedder - -text_to_embed = "I love pizza!" - -text_embedder = VertexAITextEmbedder(model="text-embedding-005") - -print(text_embedder.run(text_to_embed)) -# {'embedding': [-0.08127457648515701, 0.03399784862995148, -0.05116401985287666, ...] +run(prompt: str) ``` - - -#### VertexAITextEmbedder.\_\_init\_\_ - -```python -def __init__(model: Literal[ - "text-embedding-004", - "text-embedding-005", - "textembedding-gecko-multilingual@001", - "text-multilingual-embedding-002", - "text-embedding-large-exp-03-07", -], - task_type: Literal[ - "RETRIEVAL_DOCUMENT", - "RETRIEVAL_QUERY", - "SEMANTIC_SIMILARITY", - "CLASSIFICATION", - "CLUSTERING", - "QUESTION_ANSWERING", - "FACT_VERIFICATION", - "CODE_RETRIEVAL_QUERY", - ] = "RETRIEVAL_QUERY", - gcp_region_name: Optional[Secret] = Secret.from_env_var( - "GCP_DEFAULT_REGION", strict=False), - gcp_project_id: Optional[Secret] = Secret.from_env_var( - "GCP_PROJECT_ID", strict=False), - progress_bar: bool = True, - truncate_dim: Optional[int] = None) -> None -``` - -Initializes the TextEmbedder with the specified model, task type, and GCP configuration. - -**Arguments**: - -- `model`: Name of the model to use. -- `task_type`: The type of task for which the embeddings are being generated. -For more information see the official [Google documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api#tasktype). -- `gcp_region_name`: The default location to use when making API calls, if not set uses us-central-1. -- `gcp_project_id`: ID of the GCP project to use. By default, it is set during Google Cloud authentication. -- `progress_bar`: Whether to display a progress bar during processing. -- `truncate_dim`: The dimension to truncate the embeddings to, if specified. - - - -#### VertexAITextEmbedder.run - -```python -@component.output_types(embedding=list[float]) -def run(text: Union[list[Document], list[str], str]) -``` - -Processes text in batches while adhering to the API's token limit per request. - -**Arguments**: - -- `text`: The text to embed. - -**Returns**: - -A dictionary with the following keys: -- `embedding`: The embedding of the input text. - - - -#### VertexAITextEmbedder.to\_dict - -```python -def to_dict() -> dict[str, Any] -``` - -Serializes the component to a dictionary. - -**Returns**: - -Dictionary with serialized data. - - - -#### VertexAITextEmbedder.from\_dict - -```python -@classmethod -def from_dict(cls, data: dict[str, Any]) -> "VertexAITextEmbedder" -``` - -Deserializes the component from a dictionary. - -**Arguments**: +Prompts the model to generate text. -- `data`: Dictionary to deserialize from. +**Parameters:** -**Returns**: +- **prompt** (str) – The prompt to use for text generation. -Deserialized component. +**Returns:** +- – A dictionary with the following keys: +- `replies`: A list of generated replies. +- `safety_attributes`: A dictionary with the [safety scores](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/responsible-ai#safety_attribute_descriptions) + of each answer. +- `citations`: A list of citations for each answer.