Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs-website/docs/concepts/agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,17 @@ export OPENAI_API_KEY=<your-openai-key>
export SERPERDEV_API_KEY=<your-serperdev-key>
```

The examples on this page use SerperDev web search component that have moved to the `serperdev-haystack` package. Install it to run the examples:

```shell
pip install serperdev-haystack
```

```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.components.websearch import SerperDevWebSearch
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool

Expand Down
12 changes: 9 additions & 3 deletions docs-website/docs/concepts/agents/multi-agent-systems.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,20 @@ Wrapping an agent inside a `@tool` function gives you full control over what the
This approach works better with smaller LLMs because the tool has a clean, minimal signature.
The coordinator only needs to provide a query string - all the `ChatMessage` construction and result unpacking is hidden inside the function.

The examples on this page use SerperDev web search component that have moved to the `serperdev-haystack` package. Install it to run the examples:

```shell
pip install serperdev-haystack
```

```python
from typing import Annotated
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool, tool
from haystack.components.websearch import SerperDevWebSearch
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.utils import Secret


Expand Down Expand Up @@ -198,7 +204,7 @@ components:
exclude_subdomains: false
search_params: {}
top_k: 3
type: haystack.components.websearch.serper_dev.SerperDevWebSearch
type: haystack_integrations.components.websearch.serperdev.websearch.SerperDevWebSearch
description: Search the web for current information on any topic
inputs_from_state: null
name: web_search
Expand Down Expand Up @@ -254,7 +260,7 @@ from haystack.components.converters import HTMLToDocument
from haystack.components.fetchers.link_content import LinkContentFetcher
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.components.websearch import SerperDevWebSearch
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool, tool
from haystack.utils import Secret
Expand Down
10 changes: 9 additions & 1 deletion docs-website/docs/concepts/components.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,17 @@ Returns a list of Documents ranked by their similarity to the given query.

Components that use heavy resources, like LLMs or embedding models, have a `warm_up()` method that loads the necessary resources (such as models) into memory. This method is automatically called the first time the component runs, so you can use components directly without explicitly calling `warm_up()`:

The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:

```shell
pip install sentence-transformers-haystack
```

```python
from haystack import Document
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
)

doc = Document(content="I love pizza!")
doc_embedder = SentenceTransformersDocumentEmbedder()
Expand Down
12 changes: 10 additions & 2 deletions docs-website/docs/concepts/components/supercomponents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,20 @@ With this decorator, the `to_dict` and `from_dict` serialization is optional, as

The custom HybridRetriever example SuperComponent below turns your query into embeddings, then runs both a BM25 search and an embedding-based search at the same time. It finally merges those two result sets and returns the combined documents.

The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:

```shell
pip install sentence-transformers-haystack
```

```python
# pip install haystack-ai datasets "sentence-transformers>=3.0.0"
# pip install haystack-ai datasets sentence-transformers-haystack

from haystack import Document, Pipeline, super_component
from haystack.components.joiners import DocumentJoiner
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
)
from haystack.components.retrievers import (
InMemoryBM25Retriever,
InMemoryEmbeddingRetriever,
Expand Down
9 changes: 7 additions & 2 deletions docs-website/docs/concepts/pipelines/asyncpipeline.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,18 @@ You can find more details in our [API Reference](/reference/pipeline-api#asyncpi

## Example

The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:

```shell
pip install sentence-transformers-haystack
```

```python
import asyncio

from haystack import AsyncPipeline, Document
from haystack.components.builders import ChatPromptBuilder
from haystack.components.embedders import (
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
Expand All @@ -75,7 +81,6 @@ documents = [
]

docs_embedder = SentenceTransformersDocumentEmbedder()
docs_embedder.warm_up()

document_store = InMemoryDocumentStore()
document_store.write_documents(docs_embedder.run(documents=documents)["documents"])
Expand Down
10 changes: 9 additions & 1 deletion docs-website/docs/concepts/pipelines/creating-pipelines.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,18 @@ For each component you want to use in your pipeline, you must know the names of
Import all the dependencies, like pipeline, documents, Document Store, and all the components you want to use in your pipeline.
For example, to create a semantic document search pipelines, you need the `Document` object, the pipeline, the Document Store, Embedders, and a Retriever:

The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:

```shell
pip install sentence-transformers-haystack
```

```python
from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
)
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
```

Expand Down
9 changes: 7 additions & 2 deletions docs-website/docs/document-stores/oracledocumentstore.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@ It stores documents alongside dense vector embeddings in a native `VECTOR` colum
pip install oracle-haystack
```

The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:

```shell
pip install sentence-transformers-haystack
```

## Connection

`OracleDocumentStore` connects to Oracle using the `OracleConnectionConfig` dataclass, which supports two connection modes:
Expand Down Expand Up @@ -108,7 +114,7 @@ document_store = OracleDocumentStore(
```python
from haystack import Document, Pipeline
from haystack.document_stores.types import DuplicatePolicy
from haystack.components.embedders import (
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
SentenceTransformersTextEmbedder,
)
Expand Down Expand Up @@ -146,7 +152,6 @@ documents = [
doc_embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
doc_embedder.warm_up()
embedded_docs = doc_embedder.run(documents)["documents"]
document_store.write_documents(embedded_docs, policy=DuplicatePolicy.OVERWRITE)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@ description: "Use Supabase as a document store in Haystack, with vector search (
pip install supabase-haystack
```

The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:

```shell
pip install sentence-transformers-haystack
```

## SupabasePgvectorDocumentStore

`SupabasePgvectorDocumentStore` is a thin wrapper around [`PgvectorDocumentStore`](./pgvectordocumentstore.mdx) with Supabase-specific defaults:
Expand Down Expand Up @@ -70,7 +76,7 @@ To learn more about the initialization parameters, see the [API docs](/reference
```python
from haystack import Document, Pipeline
from haystack.document_stores.types.policy import DuplicatePolicy
from haystack.components.embedders import (
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
SentenceTransformersDocumentEmbedder,
)
Expand Down
14 changes: 12 additions & 2 deletions docs-website/docs/document-stores/valkeydocumentstore.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ You can install the Valkey Haystack integration with:
pip install valkey-haystack
```

The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:

```shell
pip install sentence-transformers-haystack
```

## Initialization

To use Valkey as your data storage for Haystack pipelines, you need a Valkey server with the search module running. Initialize a `ValkeyDocumentStore` like this:
Expand Down Expand Up @@ -65,7 +71,9 @@ To write documents to your `ValkeyDocumentStore`, create an indexing pipeline or
from haystack import Pipeline
from haystack.components.converters import MarkdownToDocument
from haystack.components.writers import DocumentWriter
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
)
from haystack.components.preprocessors import DocumentSplitter
from haystack_integrations.document_stores.valkey import ValkeyDocumentStore

Expand Down Expand Up @@ -99,7 +107,9 @@ Once documents are in your `ValkeyDocumentStore`, you can use [`ValkeyEmbeddingR
from haystack import Pipeline
from haystack.utils import Secret
from haystack.dataclasses import ChatMessage
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersTextEmbedder,
)
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.document_stores.valkey import ValkeyDocumentStore
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ Many embedding retrievers generalize poorly to new, unseen domains. This approac

First, prepare all the components that you would need:

The examples on this page use Sentence Transformers embedders that have moved to the `sentence-transformers-haystack` package. Install it to run the examples:

```shell
pip install sentence-transformers-haystack
```

```python
import os
from numpy import array, mean
Expand All @@ -37,7 +43,9 @@ from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders import ChatPromptBuilder
from haystack import component, Document
from haystack.components.converters import OutputAdapter
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack_integrations.components.embedders.sentence_transformers import (
SentenceTransformersDocumentEmbedder,
)
from haystack.dataclasses import ChatMessage

# We need to ensure we have the OpenAI API key in our environment variables
Expand Down Expand Up @@ -69,7 +77,6 @@ adapter = OutputAdapter(
embedder = SentenceTransformersDocumentEmbedder(
model="sentence-transformers/all-MiniLM-L6-v2",
)
embedder.warm_up()


# Adding one custom component that returns one, "average" embedding from multiple (hypothetical) document embeddings
Expand Down
28 changes: 19 additions & 9 deletions docs-website/docs/overview/get-started.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -88,11 +88,19 @@ print(results["llm"]["replies"])
</TabItem>
<TabItem value="huggingface" label="Hugging Face">

[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `haystack-ai` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API.
[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `huggingface-api-haystack` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API.

The examples on this page use the Hugging Face API components and the SerperDev web search component, which have moved to the `huggingface-api-haystack` and `serperdev-haystack` packages. Install them to run the examples:

```shell
pip install huggingface-api-haystack serperdev-haystack
```

```python
from haystack import Pipeline, Document
from haystack.components.generators.chat import HuggingFaceAPIChatGenerator
from haystack_integrations.components.generators.huggingface_api import (
HuggingFaceAPIChatGenerator,
)
from haystack.components.retrievers import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.builders import ChatPromptBuilder
Expand Down Expand Up @@ -389,7 +397,7 @@ from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool
from haystack.components.websearch import SerperDevWebSearch
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.utils import Secret

search_tool = ComponentTool(component=SerperDevWebSearch())
Expand All @@ -411,14 +419,16 @@ print(result["last_message"].text)
</TabItem>
<TabItem value="huggingface" label="Hugging Face">

[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `haystack-ai` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API.
[HuggingFaceAPIChatGenerator](../pipeline-components/generators/huggingfaceapichatgenerator.mdx) is included in the `huggingface-api-haystack` package. You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API.

```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import HuggingFaceAPIChatGenerator
from haystack_integrations.components.generators.huggingface_api import (
HuggingFaceAPIChatGenerator,
)
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool
from haystack.components.websearch import SerperDevWebSearch
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.utils import Secret

search_tool = ComponentTool(component=SerperDevWebSearch())
Expand Down Expand Up @@ -454,7 +464,7 @@ from haystack.components.agents import Agent
from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool
from haystack.components.websearch import SerperDevWebSearch
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.utils import Secret

search_tool = ComponentTool(component=SerperDevWebSearch())
Expand Down Expand Up @@ -492,7 +502,7 @@ from haystack_integrations.components.generators.amazon_bedrock import (
)
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool
from haystack.components.websearch import SerperDevWebSearch
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch

os.environ["AWS_ACCESS_KEY_ID"] = "YOUR_AWS_ACCESS_KEY_ID"
os.environ["AWS_SECRET_ACCESS_KEY"] = "YOUR_AWS_SECRET_ACCESS_KEY"
Expand Down Expand Up @@ -531,7 +541,7 @@ from haystack_integrations.components.generators.google_genai import (
)
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool
from haystack.components.websearch import SerperDevWebSearch
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.utils import Secret

search_tool = ComponentTool(component=SerperDevWebSearch())
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -459,11 +459,11 @@ Both frameworks offer in-memory stores for prototyping and a wide range of produ

<div className="code-comparison">
<div className="code-comparison__column">
<CodeBlock language="python" title="Haystack">{`# pip install haystack-ai sentence-transformers
<CodeBlock language="python" title="Haystack">{`# pip install haystack-ai sentence-transformers-haystack

from haystack import Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersDocumentEmbedder

# Embed and write documents to the document store
document_store = InMemoryDocumentStore()
Expand Down Expand Up @@ -505,7 +505,7 @@ vectorstore.add_documents([
<div className="code-comparison">
<div className="code-comparison__column">
<CodeBlock language="python" title="Haystack">{`from haystack import Pipeline
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
Expand Down
2 changes: 1 addition & 1 deletion docs-website/docs/pipeline-components/agents-1/agent.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ components:
exclude_subdomains: false
search_params: {}
top_k: 3
type: haystack.components.websearch.serper_dev.SerperDevWebSearch
type: haystack_integrations.components.websearch.serperdev.websearch.SerperDevWebSearch
description: Search the web for current information on any topic
inputs_from_state: null
name: web_search
Expand Down
Loading
Loading