diff --git a/docs-website/docs/concepts/pipelines/asyncpipeline.mdx b/docs-website/docs/concepts/pipelines/asyncpipeline.mdx index 75d8ad20e57..e113b331bac 100644 --- a/docs-website/docs/concepts/pipelines/asyncpipeline.mdx +++ b/docs-website/docs/concepts/pipelines/asyncpipeline.mdx @@ -46,7 +46,7 @@ In an `AsyncPipeline`, components such as A and B will run in parallel _only if You can control the maximum number of components that run simultaneously using the `concurrency_limit` parameter to ensure controlled resource usage. -You can find more details in our [API Reference](/reference/pipeline-api#asyncpipeline), or directly in the pipeline's [GitHub code](https://github.com/deepset-ai/haystack/blob/main/haystack/core/pipeline/async_pipeline.py). +You can find more details in our [API Reference](/reference/pipeline-api#pipeline), or directly in the pipeline's [GitHub code](https://github.com/deepset-ai/haystack/blob/main/haystack/core/pipeline/pipeline.py). ## Example diff --git a/docs-website/docs/overview/migration.mdx b/docs-website/docs/overview/migration.mdx index b3984016ab1..92fdffc6476 100644 --- a/docs-website/docs/overview/migration.mdx +++ b/docs-website/docs/overview/migration.mdx @@ -169,7 +169,7 @@ If you need help migrating a 1.x node without a 2.x counterpart, open an [issue] | Haystack 1.x | Description | Haystack 2.x | | --- | --- | --- | | Ranker | Orders documents based on how relevant they are to the query. **Example usage:** In a query pipeline, after a keyword-based Retriever to rank the documents it returns. | [Rankers](../pipeline-components/rankers.mdx) | -| Reader | Finds an answer by selecting a text span in documents. **Example usage:** In a query pipeline when you want to know the location of the answer. | [ExtractiveReader](../pipeline-components/readers/extractivereader.mdx) | +| Reader | Finds an answer by selecting a text span in documents. **Example usage:** In a query pipeline when you want to know the location of the answer. | [TransformersExtractiveReader](../pipeline-components/readers/transformersextractivereader.mdx) | | Retriever | Fetches relevant documents from the Document Store. **Example usage:** Coupling Retriever with a Reader in a query pipeline to speed up the search (the Reader only goes through the documents it gets from the Retriever). | [Retrievers](../pipeline-components/retrievers.mdx) | | QuestionGenerator | When given a document, it generates questions this document can answer. **Example usage:** Auto-suggested questions in your search app. | Prompt [Builders](../pipeline-components/builders.mdx) with dedicated prompt, [Generators](../pipeline-components/generators.mdx) | diff --git a/docs-website/docs/overview/platform-components.mdx b/docs-website/docs/overview/platform-components.mdx index bbefdfcd0d8..9f25bc5fb3f 100644 --- a/docs-website/docs/overview/platform-components.mdx +++ b/docs-website/docs/overview/platform-components.mdx @@ -40,7 +40,6 @@ The Haystack Enterprise Platform currently supports **224 components** and **55 | [DocumentWriter](https://docs.haystack.deepset.ai/docs/documentwriter) | Writer | ✅ Available | | [DOCXToDocument](https://docs.haystack.deepset.ai/docs/docxtodocument) | Converter | ✅ Available | | [EmbeddingBasedDocumentSplitter](https://docs.haystack.deepset.ai/docs/embeddingbaseddocumentsplitter) | Preprocessor | ✅ Available | -| [ExtractiveReader](https://docs.haystack.deepset.ai/docs/extractivereader) | Reader | ✅ Available | | [FallbackChatGenerator](https://docs.haystack.deepset.ai/docs/fallbackchatgenerator) | Generator | ✅ Available | | [FileToFileContent](https://docs.haystack.deepset.ai/docs/filetofilecontent) | Converter | ✅ Available | | [FileTypeRouter](https://docs.haystack.deepset.ai/docs/filetyperouter) | Router | ✅ Available | @@ -110,6 +109,7 @@ The Haystack Enterprise Platform currently supports **224 components** and **55 | [TikaDocumentConverter](https://docs.haystack.deepset.ai/docs/tikadocumentconverter) | Converter | ✅ Available | | [ToolInvoker](https://docs.haystack.deepset.ai/docs/toolinvoker) | Component | ✅ Available | | [TopPSampler](https://docs.haystack.deepset.ai/docs/toppsampler) | Sampler | ✅ Available | +| [TransformersExtractiveReader](https://docs.haystack.deepset.ai/docs/transformersextractivereader) | Reader | ✅ Available | | [TransformersSimilarityRanker](https://docs.haystack.deepset.ai/docs/transformerssimilarityranker) | Ranker | ✅ Available | | [TransformersTextRouter](https://docs.haystack.deepset.ai/docs/transformerstextrouter) | Router | ✅ Available | | [TransformersZeroShotDocumentClassifier](https://docs.haystack.deepset.ai/docs/transformerszeroshotdocumentclassifier) | Classifier | ✅ Available | diff --git a/docs-website/docs/pipeline-components/generators.mdx b/docs-website/docs/pipeline-components/generators.mdx index d742f44d490..fa7d5edb50c 100644 --- a/docs-website/docs/pipeline-components/generators.mdx +++ b/docs-website/docs/pipeline-components/generators.mdx @@ -24,7 +24,6 @@ Generators are responsible for generating text after you give them a prompt. The | [CohereChatGenerator](generators/coherechatgenerator.mdx) | Enables chat completion using Cohere's LLMs. | ✅ | | [CohereGenerator](generators/coheregenerator.mdx) | Queries the LLM using Cohere API. | ✅ | | [CometAPIChatGenerator](generators/cometapichatgenerator.mdx) | Enables chat completion using AI models through the Comet API. | ✅ | -| [DALLEImageGenerator](generators/dalleimagegenerator.mdx) | Generate images using OpenAI's image generation models such as `gpt-image-2`. | ❌ | | [FallbackChatGenerator](generators/fallbackchatgenerator.mdx) | A ChatGenerator wrapper that tries multiple Chat Generators sequentially until one succeeds. | ✅ | | [GoogleAIGeminiChatGenerator](generators/googleaigeminichatgenerator.mdx) | Enables chat completion using Google Gemini models. **_This integration will be deprecated soon. We recommend using [GoogleGenAIChatGenerator](generators/googlegenaichatgenerator.mdx) integration instead._** | ✅ | | [GoogleAIGeminiGenerator](generators/googleaigeminigenerator.mdx) | Enables text generation using Google Gemini models. **_This integration will be deprecated soon. We recommend using [GoogleGenAIChatGenerator](generators/googlegenaichatgenerator.mdx) integration instead._** | ✅ | @@ -45,6 +44,7 @@ Generators are responsible for generating text after you give them a prompt. The | [OllamaGenerator](generators/ollamagenerator.mdx) | Provides an interface to generate text using an LLM running on Ollama. | ✅ | | [OpenAIChatGenerator](generators/openaichatgenerator.mdx) | Enables chat completion using OpenAI's large language models (LLMs). | ✅ | | [OpenAIGenerator](generators/openaigenerator.mdx) | Enables text generation using OpenAI's large language models (LLMs). | ✅ | +| [OpenAIImageGenerator](generators/openaiimagegenerator.mdx) | Generate images using OpenAI's image generation models such as `gpt-image-2`. | ❌ | | [OpenAIResponsesChatGenerator](generators/openairesponseschatgenerator.mdx) | Enables chat completion using OpenAI's Responses API with support for reasoning models. | ✅ | | [OpenRouterChatGenerator](generators/openrouterchatgenerator.mdx) | Enables chat completion with any model hosted on OpenRouter. | ✅ | | [PerplexityChatGenerator](generators/perplexitychatgenerator.mdx) | Enables chat completion using models via the Perplexity Agent API. | ✅ | diff --git a/docs-website/docs/pipeline-components/generators/dalleimagegenerator.mdx b/docs-website/docs/pipeline-components/generators/openaiimagegenerator.mdx similarity index 73% rename from docs-website/docs/pipeline-components/generators/dalleimagegenerator.mdx rename to docs-website/docs/pipeline-components/generators/openaiimagegenerator.mdx index 862c814a7c5..6a405beada1 100644 --- a/docs-website/docs/pipeline-components/generators/dalleimagegenerator.mdx +++ b/docs-website/docs/pipeline-components/generators/openaiimagegenerator.mdx @@ -1,11 +1,11 @@ --- -title: "DALLEImageGenerator" -id: dalleimagegenerator -slug: "/dalleimagegenerator" +title: "OpenAIImageGenerator" +id: openaiimagegenerator +slug: "/openaiimagegenerator" description: "Generate images using OpenAI's image generation models such as `gpt-image-2`." --- -# DALLEImageGenerator +# OpenAIImageGenerator Generate images using OpenAI's image generation models such as `gpt-image-2`. @@ -18,33 +18,33 @@ Generate images using OpenAI's image generation models such as `gpt-image-2`. | **Mandatory run variables** | `prompt`: A string containing the prompt for the model | | **Output variables** | `images`: A list of generated images

`revised_prompt`: A string containing the prompt that was used to generate the image, if there was any revision to the prompt made by OpenAI | | **API reference** | [Generators](/reference/generators-api) | -| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/openai_dalle.py | +| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/openai_image_generator.py | | **Package name** | `haystack-ai` | ## Overview -The `DALLEImageGenerator` component generates images using OpenAI's image generation models (such as `gpt-image-2`). +The `OpenAIImageGenerator` component generates images using OpenAI's image generation models (such as `gpt-image-2`). By default, the component uses the `gpt-image-2` model, `"auto"` quality, and 1024x1024 resolution. You can change these parameters using `model` (during component initialization), `quality`, and `size` (during component initialization or run) parameters. -`DALLEImageGenerator` needs an OpenAI key to work. It uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`: +`OpenAIImageGenerator` needs an OpenAI key to work. It uses an `OPENAI_API_KEY` environment variable by default. Otherwise, you can pass an API key at initialization with `api_key`: ``` -image_generator = DALLEImageGenerator(api_key=Secret.from_token("")) +image_generator = OpenAIImageGenerator(api_key=Secret.from_token("")) ``` -Check our [API reference](/reference/generators-api#dalleimagegenerator) for the detailed component parameters description, or the [OpenAI documentation](https://developers.openai.com/api/reference/resources/images/methods/generate) for the details on OpenAI API parameters. +Check our [API reference](/reference/generators-api#openaiimagegenerator) for the detailed component parameters description, or the [OpenAI documentation](https://developers.openai.com/api/reference/resources/images/methods/generate) for the details on OpenAI API parameters. ## Usage ### On its own ```python -from haystack.components.generators import DALLEImageGenerator +from haystack.components.generators import OpenAIImageGenerator -image_generator = DALLEImageGenerator() +image_generator = OpenAIImageGenerator() response = image_generator.run("Show me a picture of a black cat.") print(response) @@ -52,11 +52,11 @@ print(response) ### In a pipeline -In the following pipeline, we first set up a `PromptBuilder` that will structure the image description with a detailed template describing various artistic elements. The pipeline then passes this structured prompt into a `DALLEImageGenerator` to generate the image based on this detailed description. +In the following pipeline, we first set up a `PromptBuilder` that will structure the image description with a detailed template describing various artistic elements. The pipeline then passes this structured prompt into an `OpenAIImageGenerator` to generate the image based on this detailed description. ```python from haystack import Pipeline -from haystack.components.generators import DALLEImageGenerator +from haystack.components.generators import OpenAIImageGenerator from haystack.components.builders import PromptBuilder prompt_builder = PromptBuilder( @@ -70,7 +70,7 @@ prompt_builder = PromptBuilder( Additional details: {details}""", ) -image_generator = DALLEImageGenerator() +image_generator = OpenAIImageGenerator() pipeline = Pipeline() pipeline.add_component("prompt_builder", prompt_builder) diff --git a/docs-website/docs/pipeline-components/readers.mdx b/docs-website/docs/pipeline-components/readers.mdx index df8bcae994b..d50c2776e4b 100644 --- a/docs-website/docs/pipeline-components/readers.mdx +++ b/docs-website/docs/pipeline-components/readers.mdx @@ -9,4 +9,4 @@ description: "Readers are pipeline components that pinpoint answers in documents Readers are pipeline components that pinpoint answers in documents. They’re used in extractive question answering systems. -Currently, there's one Reader available in Haystack: [ExtractiveReader](readers/extractivereader.mdx). \ No newline at end of file +Currently, there's one Reader available in Haystack: [TransformersExtractiveReader](readers/transformersextractivereader.mdx). diff --git a/docs-website/docs/pipeline-components/readers/extractivereader.mdx b/docs-website/docs/pipeline-components/readers/extractivereader.mdx deleted file mode 100644 index 60691f6a8b6..00000000000 --- a/docs-website/docs/pipeline-components/readers/extractivereader.mdx +++ /dev/null @@ -1,113 +0,0 @@ ---- -title: "ExtractiveReader" -id: extractivereader -slug: "/extractivereader" -description: "Use this component in extractive question answering pipelines based on a query and a list of documents." ---- - -# ExtractiveReader - -Use this component in extractive question answering pipelines based on a query and a list of documents. - -:::warning[Deprecated] - -`ExtractiveReader` is deprecated and will be removed in Haystack 3.0. It has moved to the `transformers-haystack` package and was renamed to `TransformersExtractiveReader`. See [TransformersExtractiveReader](transformersextractivereader.mdx) for the updated documentation. - -::: - -
- -| | | -| --- | --- | -| **Most common position in a pipeline** | In query pipelines, after a component that returns a list of documents, such as a [Retriever](../retrievers.mdx) | -| **Mandatory init variables** | `token`: The Hugging Face API token. Can be set with `HF_API_TOKEN` or `HF_TOKEN` env var. | -| **Mandatory run variables** | `documents`: A list of documents

`query`: A query string | -| **Output variables** | `answers`: A list of [`ExtractedAnswer`](../../concepts/data-classes.mdx#extractedanswer) objects | -| **API reference** | [Readers](/reference/readers-api) | -| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/readers/extractive.py | -| **Package name** | `haystack-ai` | - -
- -## Overview - -`ExtractiveReader` locates and extracts answers to a given query from the document text. It's used in extractive QA systems where you want to know exactly where the answer is located within the document. It's usually coupled with a Retriever that precedes it, but you can also use it with other components that fetch documents. - -Readers assign a _probability_ to answers. This score ranges from 0 to 1, indicating how well the results the Reader returned match the query. Probability closest to 1 means the model has high confidence in the answer's relevance. The Reader sorts the answers based on their probability scores, with higher probability listed first. You can limit the number of answers the Reader returns in the optional `top_k` parameter. - -You can use the probability to set the quality expectations for your system. To do that, use the `confidence_score` parameter of the Reader to set a minimum probability threshold for answers. For example, setting `confidence_threshold` to `0.7` means only answers with a probability higher than 0.7 will be returned. - -By default, the Reader includes a scenario where no answer to the query is found in the document text (`no_answer=True`). In this case, it returns an additional `ExtractedAnswer` with no text and the probability that none of the `top_k` answers are correct. For example, if `top_k=4` the system will return four answers and an additional empty one. Each answer has a probability assigned. If the empty answer has a probability of 0.5, it means that's the probability that none of the returned answers is correct. To receive only the actual top_k answers, set the `no_answer` parameter to `False` when initializing the component. - -### Models - -Here are the models that we recommend for using with `ExtractiveReader`: - -| | | | -| --- | --- | --- | -| Model URL | Description | Language | -| [deepset/roberta-base-squad2-distilled](https://huggingface.co/deepset/roberta-base-squad2-distilled) (default) | A distilled model, relatively fast and with good performance. | English | -| [deepset/roberta-large-squad2](https://huggingface.co/deepset/roberta-large-squad2) | A large model with good performance. Slower than the distilled one. | English | -| [deepset/tinyroberta-squad2](https://huggingface.co/deepset/tinyroberta-squad2) | A distilled version of roberta-large-squad2 model, very fast. | English | -| [deepset/xlm-roberta-base-squad2](https://huggingface.co/deepset/xlm-roberta-base-squad2) | A base multilingual model with good speed and performance. | Multilingual | - -You can also view other question answering models on [Hugging Face](https://huggingface.co/models?pipeline_tag=question-answering). - -## Usage - -### On its own - -Below is an example that uses the `ExtractiveReader` outside of a pipeline. The Reader gets the query and the documents at runtime. It should return two answers and an additional third answer with no text and the probability that the `top_k` answers are incorrect. - -```python -from haystack import Document -from haystack.components.readers import ExtractiveReader - -docs = [ - Document(content="Paris is the capital of France."), - Document(content="Berlin is the capital of Germany."), -] - -reader = ExtractiveReader() - -reader.run(query="What is the capital of France?", documents=docs, top_k=2) -``` - -### In a pipeline - -Below is an example of a pipeline that retrieves a document from an `InMemoryDocumentStore` based on keyword search (using `InMemoryBM25Retriever`). It then uses the `ExtractiveReader` to extract the answer to our query from the top retrieved documents. - -With the ExtractiveReader’s `top_k` set to 2, an additional, third answer with no text and the probability that the other `top_k` answers are incorrect is also returned. - -```python -from haystack import Document, Pipeline -from haystack.document_stores.in_memory import InMemoryDocumentStore -from haystack.components.retrievers.in_memory import InMemoryBM25Retriever -from haystack.components.readers import ExtractiveReader - -docs = [ - Document(content="Paris is the capital of France."), - Document(content="Berlin is the capital of Germany."), - Document(content="Rome is the capital of Italy."), - Document(content="Madrid is the capital of Spain."), -] -document_store = InMemoryDocumentStore() -document_store.write_documents(docs) - -retriever = InMemoryBM25Retriever(document_store=document_store) -reader = ExtractiveReader() - -extractive_qa_pipeline = Pipeline() -extractive_qa_pipeline.add_component(instance=retriever, name="retriever") -extractive_qa_pipeline.add_component(instance=reader, name="reader") - -extractive_qa_pipeline.connect("retriever.documents", "reader.documents") - -query = "What is the capital of France?" -extractive_qa_pipeline.run( - data={ - "retriever": {"query": query, "top_k": 3}, - "reader": {"query": query, "top_k": 2}, - }, -) -``` diff --git a/docs-website/docs/pipeline-components/retrievers/alloydbkeywordretriever.mdx b/docs-website/docs/pipeline-components/retrievers/alloydbkeywordretriever.mdx index 749474e75ec..a5e963842fa 100644 --- a/docs-website/docs/pipeline-components/retrievers/alloydbkeywordretriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/alloydbkeywordretriever.mdx @@ -13,7 +13,7 @@ A keyword-based Retriever that fetches documents matching a query from the Alloy | | | | --- | --- | -| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of an [AlloyDBDocumentStore](../../document-stores/alloydbdocumentstore.mdx) | | **Mandatory run variables** | `query`: A string | | **Output variables** | `documents`: A list of documents (matching the query) | diff --git a/docs-website/docs/pipeline-components/retrievers/azureaisearchbm25retriever.mdx b/docs-website/docs/pipeline-components/retrievers/azureaisearchbm25retriever.mdx index 9cdf5103663..a04347b2180 100644 --- a/docs-website/docs/pipeline-components/retrievers/azureaisearchbm25retriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/azureaisearchbm25retriever.mdx @@ -15,7 +15,7 @@ A keyword-based Retriever that fetches documents matching a query from the Azure | | | | --- | --- | -| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before an [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of [`AzureAISearchDocumentStore`](../../document-stores/azureaisearchdocumentstore.mdx) | | **Mandatory run variables** | `query`: A string | | **Output variables** | `documents`: A list of documents (matching the query) | diff --git a/docs-website/docs/pipeline-components/retrievers/chromaqueryretriever.mdx b/docs-website/docs/pipeline-components/retrievers/chromaqueryretriever.mdx index bfd11fa8057..6c760753923 100644 --- a/docs-website/docs/pipeline-components/retrievers/chromaqueryretriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/chromaqueryretriever.mdx @@ -13,7 +13,7 @@ This is a a Retriever compatible with the Chroma Document Store. | | | | --- | --- | -| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [ChromaDocumentStore](../../document-stores/chromadocumentstore.mdx) | | **Mandatory run variables** | `query`: A single query in plain-text format to be processed by the [Retriever](../retrievers.mdx) | | **Output variables** | `documents`: A list of documents | diff --git a/docs-website/docs/pipeline-components/retrievers/elasticsearchbm25retriever.mdx b/docs-website/docs/pipeline-components/retrievers/elasticsearchbm25retriever.mdx index cfbd1d357b5..a02c6cbdfea 100644 --- a/docs-website/docs/pipeline-components/retrievers/elasticsearchbm25retriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/elasticsearchbm25retriever.mdx @@ -13,7 +13,7 @@ A keyword-based Retriever that fetches Documents matching a query from the Elast | | | | --- | --- | -| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of [ElasticsearchDocumentStore](../../document-stores/elasticsearch-document-store.mdx) | | **Mandatory run variables** | `query`: A string | | **Output variables** | `documents`: A list of documents (matching the query) | diff --git a/docs-website/docs/pipeline-components/retrievers/inmemorybm25retriever.mdx b/docs-website/docs/pipeline-components/retrievers/inmemorybm25retriever.mdx index 12e230917b1..3483253d6ff 100644 --- a/docs-website/docs/pipeline-components/retrievers/inmemorybm25retriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/inmemorybm25retriever.mdx @@ -13,7 +13,7 @@ A keyword-based Retriever compatible with InMemoryDocumentStore. | | | | --- | --- | -| **Most common position in a pipeline** | In query pipelines:
In a RAG pipeline, before a [`PromptBuilder`](../builders/promptbuilder.mdx)
In a semantic search pipeline, as the last component
In an extractive QA pipeline, before an [`ExtractiveReader`](../readers/extractivereader.mdx) | +| **Most common position in a pipeline** | In query pipelines:
In a RAG pipeline, before a [`PromptBuilder`](../builders/promptbuilder.mdx)
In a semantic search pipeline, as the last component
In an extractive QA pipeline, before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) | | **Mandatory init variables** | `document_store`: An instance of [InMemoryDocumentStore](../../document-stores/inmemorydocumentstore.mdx) | | **Mandatory run variables** | `query`: A query string | | **Output variables** | `documents`: A list of documents (matching the query) | diff --git a/docs-website/docs/pipeline-components/retrievers/opensearchbm25retriever.mdx b/docs-website/docs/pipeline-components/retrievers/opensearchbm25retriever.mdx index ea2a80b1af3..fe2338d02b6 100644 --- a/docs-website/docs/pipeline-components/retrievers/opensearchbm25retriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/opensearchbm25retriever.mdx @@ -13,7 +13,7 @@ This is a keyword-based Retriever that fetches Documents matching a query from a | | | | --- | --- | -| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of an [OpenSearchDocumentStore](../../document-stores/opensearch-document-store.mdx) | | **Mandatory run variables** | `query`: A query string | | **Output variables** | `documents`: A list of documents matching the query | diff --git a/docs-website/docs/pipeline-components/retrievers/oraclekeywordretriever.mdx b/docs-website/docs/pipeline-components/retrievers/oraclekeywordretriever.mdx index 56059cb9345..09f1c794cea 100644 --- a/docs-website/docs/pipeline-components/retrievers/oraclekeywordretriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/oraclekeywordretriever.mdx @@ -13,7 +13,7 @@ A keyword-based Retriever that fetches documents matching a query from the Oracl | | | | --- | --- | -| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in a keyword search pipeline 3. Before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in a keyword search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of an [OracleDocumentStore](../../document-stores/oracledocumentstore.mdx) | | **Mandatory run variables** | `query`: A string | | **Output variables** | `documents`: A list of documents matching the query | diff --git a/docs-website/docs/pipeline-components/retrievers/pgvectorkeywordretriever.mdx b/docs-website/docs/pipeline-components/retrievers/pgvectorkeywordretriever.mdx index e0a4ffc709b..314004e32b4 100644 --- a/docs-website/docs/pipeline-components/retrievers/pgvectorkeywordretriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/pgvectorkeywordretriever.mdx @@ -13,7 +13,7 @@ This is a keyword-based Retriever that fetches documents matching a query from t | | | | --- | --- | -| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [PgvectorDocumentStore](../../document-stores/pgvectordocumentstore.mdx) | | **Mandatory run variables** | `query`: A string | | **Output variables** | `document`: A list of documents (matching the query) | diff --git a/docs-website/docs/pipeline-components/retrievers/qdranthybridretriever.mdx b/docs-website/docs/pipeline-components/retrievers/qdranthybridretriever.mdx index 9b6debbe61c..ece3ddd2dc6 100644 --- a/docs-website/docs/pipeline-components/retrievers/qdranthybridretriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/qdranthybridretriever.mdx @@ -13,7 +13,7 @@ A Retriever based both on dense and sparse embeddings, compatible with the Qdran | | | | --- | --- | -| **Most common position in a pipeline** | 1\. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline

2. The last component in a hybrid search pipeline
3. After a Text Embedder and before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1\. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline

2. The last component in a hybrid search pipeline
3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx) | | **Mandatory run variables** | `query_embedding`: A dense vector representing the query (a list of floats)

`query_sparse_embedding`: A [`SparseEmbedding`](../../concepts/data-classes.mdx#sparseembedding) object containing a vectorial representation of the query | | **Output variables** | `documents`: A list of documents | diff --git a/docs-website/docs/pipeline-components/retrievers/qdrantsparseembeddingretriever.mdx b/docs-website/docs/pipeline-components/retrievers/qdrantsparseembeddingretriever.mdx index 3f0b0588556..0b748182247 100644 --- a/docs-website/docs/pipeline-components/retrievers/qdrantsparseembeddingretriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/qdrantsparseembeddingretriever.mdx @@ -13,7 +13,7 @@ A Retriever based on sparse embeddings, compatible with the Qdrant Document Stor | | | | --- | --- | -| **Most common position in a pipeline** | 1\. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline

2. The last component in the semantic search pipeline
3. After a Text Embedder and before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1\. After a Text Embedder and before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline

2. The last component in the semantic search pipeline
3. After a Text Embedder and before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [QdrantDocumentStore](../../document-stores/qdrant-document-store.mdx) | | **Mandatory run variables** | `query_sparse_embedding`: A [`SparseEmbedding`](../../concepts/data-classes.mdx#sparseembedding) object containing a vectorial representation of the query | | **Output variables** | `documents`: A list of documents | diff --git a/docs-website/docs/pipeline-components/retrievers/supabasepgvectorkeywordretriever.mdx b/docs-website/docs/pipeline-components/retrievers/supabasepgvectorkeywordretriever.mdx index c29e0066ba9..514c0f081bf 100644 --- a/docs-website/docs/pipeline-components/retrievers/supabasepgvectorkeywordretriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/supabasepgvectorkeywordretriever.mdx @@ -13,7 +13,7 @@ A keyword-based Retriever that fetches documents matching a query from the Supab | | | | --- | --- | -| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [SupabasePgvectorDocumentStore](../../document-stores/supabasedocumentstore.mdx) | | **Mandatory run variables** | `query`: A string | | **Output variables** | `documents`: A list of documents (matching the query) | diff --git a/docs-website/docs/pipeline-components/retrievers/vespakeywordretriever.mdx b/docs-website/docs/pipeline-components/retrievers/vespakeywordretriever.mdx index f67de15a5d3..efbcd232676 100644 --- a/docs-website/docs/pipeline-components/retrievers/vespakeywordretriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/vespakeywordretriever.mdx @@ -13,7 +13,7 @@ A keyword-based Retriever that fetches documents matching a query from the Vespa | | | | --- | --- | -| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the keyword search pipeline 3. Before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the keyword search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [VespaDocumentStore](../../document-stores/vespadocumentstore.mdx) | | **Mandatory run variables** | `query`: A string | | **Output variables** | `documents`: A list of documents (matching the query) | diff --git a/docs-website/docs/pipeline-components/retrievers/weaviatebm25retriever.mdx b/docs-website/docs/pipeline-components/retrievers/weaviatebm25retriever.mdx index a0a6834b6e9..7653af01399 100644 --- a/docs-website/docs/pipeline-components/retrievers/weaviatebm25retriever.mdx +++ b/docs-website/docs/pipeline-components/retrievers/weaviatebm25retriever.mdx @@ -13,7 +13,7 @@ This is a keyword-based Retriever that fetches Documents matching a query from t | | | | --- | --- | -| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before an [`ExtractiveReader`](../readers/extractivereader.mdx) in an extractive QA pipeline | +| **Most common position in a pipeline** | 1. Before a [`PromptBuilder`](../builders/promptbuilder.mdx) in a RAG pipeline 2. The last component in the semantic search pipeline 3. Before a [`TransformersExtractiveReader`](../readers/transformersextractivereader.mdx) in an extractive QA pipeline | | **Mandatory init variables** | `document_store`: An instance of a [WeaviateDocumentStore](../../document-stores/weaviatedocumentstore.mdx) | | **Mandatory run variables** | `query`: A string | | **Output variables** | `documents`: A list of documents (matching the query) | diff --git a/docs-website/sidebars.js b/docs-website/sidebars.js index a309ad10c62..0014b605e19 100644 --- a/docs-website/sidebars.js +++ b/docs-website/sidebars.js @@ -423,7 +423,6 @@ export default { 'pipeline-components/generators/coherechatgenerator', 'pipeline-components/generators/coheregenerator', 'pipeline-components/generators/cometapichatgenerator', - 'pipeline-components/generators/dalleimagegenerator', 'pipeline-components/generators/fallbackchatgenerator', 'pipeline-components/generators/googleaigeminichatgenerator', 'pipeline-components/generators/googleaigeminigenerator', @@ -445,6 +444,7 @@ export default { 'pipeline-components/generators/openaichatgenerator', 'pipeline-components/generators/openairesponseschatgenerator', 'pipeline-components/generators/openaigenerator', + 'pipeline-components/generators/openaiimagegenerator', 'pipeline-components/generators/openrouterchatgenerator', 'pipeline-components/generators/perplexitychatgenerator', 'pipeline-components/generators/sagemakergenerator', @@ -551,7 +551,6 @@ export default { id: 'pipeline-components/readers' }, items: [ - 'pipeline-components/readers/extractivereader', 'pipeline-components/readers/transformersextractivereader', ], },