Skip to content

Commit 98d73ef

Browse files
bilgeyucelbilgeyucel
authored
add HF code to the get started page (#10446)
1 parent ae6d56a commit 98d73ef

2 files changed

Lines changed: 174 additions & 10 deletions

File tree

‎docs-website/docs/overview/get-started.mdx‎

Lines changed: 87 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,9 @@ slug: "/get-started"
55
description: "Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing, running your first RAG pipeline, adding data and further resources."
66
---
77

8+
import Tabs from '@theme/Tabs';
9+
import TabItem from '@theme/TabItem';
10+
811
# Get Started
912

1013
Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing, running your first RAG pipeline, adding data and further resources.
@@ -39,15 +42,15 @@ If you have any questions, please reach out to us on the [GitHub Discussion](htt
3942

4043
</details>
4144

42-
In the example below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). However, for easier use, you can also set an OpenAI key as an `OPENAI_API_KEY` environment variable.
45+
In the examples below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). You can choose between OpenAI or Hugging Face as your LLM provider. For easier use, you can also set the API key as an environment variable (`OPENAI_API_KEY` or `HF_API_TOKEN`).
46+
47+
<Tabs>
48+
<TabItem value="openai" label="OpenAI" default>
4349

4450
:::note
4551
**Using OpenAIChatGenerator requires an OpenAI API key with sufficient quota.**
4652
New users on the free tier may immediately encounter a `429` ("insufficient_quota") error when running
47-
the example below.
48-
49-
If you do not have enough OpenAI credits, you may skip this example or use an alternative Generator such as
50-
`HuggingFaceAPIChatGenerator`.
53+
this example. If you do not have enough OpenAI credits, try the Hugging Face tab instead.
5154
:::
5255

5356
```python
@@ -112,9 +115,88 @@ results = rag_pipeline.run(
112115
)
113116

114117
print(results["llm"]["replies"])
118+
```
119+
120+
</TabItem>
121+
<TabItem value="huggingface" label="Hugging Face">
122+
123+
:::note
124+
**Using HuggingFaceAPIChatGenerator requires a Hugging Face API token.**
125+
You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API.
126+
This API is rate-limited but perfect for experimentation.
127+
:::
128+
129+
```python
130+
# import necessary dependencies
131+
from haystack import Pipeline, Document
132+
from haystack.components.generators.chat import HuggingFaceAPIChatGenerator
133+
from haystack.components.retrievers import InMemoryBM25Retriever
134+
from haystack.document_stores.in_memory import InMemoryDocumentStore
135+
from haystack.components.builders import ChatPromptBuilder
136+
from haystack.utils import Secret
137+
from haystack.dataclasses import ChatMessage
138+
139+
# create a document store and write documents to it
140+
document_store = InMemoryDocumentStore()
141+
document_store.write_documents([
142+
Document(content="My name is Jean and I live in Paris."),
143+
Document(content="My name is Mark and I live in Berlin."),
144+
Document(content="My name is Giorgio and I live in Rome.")
145+
])
146+
147+
# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer.
148+
prompt_template = [
149+
ChatMessage.from_system(
150+
"""
151+
Given these documents, answer the question.
152+
Documents:
153+
{% for doc in documents %}
154+
{{ doc.content }}
155+
{% endfor %}
156+
Question:
157+
"""
158+
),
159+
ChatMessage.from_user(
160+
"{{question}}"
161+
),
162+
ChatMessage.from_system("Answer:")
163+
]
164+
165+
# create the components adding the necessary parameters
166+
retriever = InMemoryBM25Retriever(document_store=document_store)
167+
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
168+
llm = HuggingFaceAPIChatGenerator(
169+
api_type="serverless_inference_api",
170+
api_params={"model": "Qwen/Qwen2.5-7B-Instruct", "provider": "together"},
171+
token=Secret.from_env_var("HF_API_TOKEN")
172+
)
115173

174+
# Create the pipeline and add the components to it. The order doesn't matter.
175+
# At this stage, the Pipeline validates the components without running them yet.
176+
rag_pipeline = Pipeline()
177+
rag_pipeline.add_component("retriever", retriever)
178+
rag_pipeline.add_component("prompt_builder", prompt_builder)
179+
rag_pipeline.add_component("llm", llm)
180+
181+
# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name").
182+
rag_pipeline.connect("retriever", "prompt_builder.documents")
183+
rag_pipeline.connect("prompt_builder", "llm")
184+
185+
# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components.
186+
question = "Who lives in Paris?"
187+
results = rag_pipeline.run(
188+
{
189+
"retriever": {"query": question},
190+
"prompt_builder": {"question": question},
191+
}
192+
)
193+
194+
print(results["llm"]["replies"])
116195
```
117196

197+
</TabItem>
198+
</Tabs>
199+
118200
### Adding Your Data
119201

120202
Instead of running the RAG pipeline on example data, learn how you can add your own custom data using [Document Stores](../concepts/document-store.mdx).

‎docs-website/versioned_docs/version-2.22/overview/get-started.mdx‎

Lines changed: 87 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,9 @@ slug: "/get-started"
55
description: "Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing, running your first RAG pipeline, adding data and further resources."
66
---
77

8+
import Tabs from '@theme/Tabs';
9+
import TabItem from '@theme/TabItem';
10+
811
# Get Started
912

1013
Have a look at this page to learn how to quickly get up and running with Haystack. It contains instructions for installing, running your first RAG pipeline, adding data and further resources.
@@ -39,15 +42,15 @@ If you have any questions, please reach out to us on the [GitHub Discussion](htt
3942

4043
</details>
4144

42-
In the example below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). However, for easier use, you can also set an OpenAI key as an `OPENAI_API_KEY` environment variable.
45+
In the examples below, we show how to set an API key using a Haystack [Secret](../concepts/secret-management.mdx). You can choose between OpenAI or Hugging Face as your LLM provider. For easier use, you can also set the API key as an environment variable (`OPENAI_API_KEY` or `HF_API_TOKEN`).
46+
47+
<Tabs>
48+
<TabItem value="openai" label="OpenAI" default>
4349

4450
:::note
4551
**Using OpenAIChatGenerator requires an OpenAI API key with sufficient quota.**
4652
New users on the free tier may immediately encounter a `429` ("insufficient_quota") error when running
47-
the example below.
48-
49-
If you do not have enough OpenAI credits, you may skip this example or use an alternative Generator such as
50-
`HuggingFaceAPIChatGenerator`.
53+
this example. If you do not have enough OpenAI credits, try the Hugging Face tab instead.
5154
:::
5255

5356
```python
@@ -112,9 +115,88 @@ results = rag_pipeline.run(
112115
)
113116

114117
print(results["llm"]["replies"])
118+
```
119+
120+
</TabItem>
121+
<TabItem value="huggingface" label="Hugging Face">
122+
123+
:::note
124+
**Using HuggingFaceAPIChatGenerator requires a Hugging Face API token.**
125+
You can get a [free Hugging Face token](https://huggingface.co/settings/tokens) to use the Serverless Inference API.
126+
This API is rate-limited but perfect for experimentation.
127+
:::
128+
129+
```python
130+
# import necessary dependencies
131+
from haystack import Pipeline, Document
132+
from haystack.components.generators.chat import HuggingFaceAPIChatGenerator
133+
from haystack.components.retrievers import InMemoryBM25Retriever
134+
from haystack.document_stores.in_memory import InMemoryDocumentStore
135+
from haystack.components.builders import ChatPromptBuilder
136+
from haystack.utils import Secret
137+
from haystack.dataclasses import ChatMessage
138+
139+
# create a document store and write documents to it
140+
document_store = InMemoryDocumentStore()
141+
document_store.write_documents([
142+
Document(content="My name is Jean and I live in Paris."),
143+
Document(content="My name is Mark and I live in Berlin."),
144+
Document(content="My name is Giorgio and I live in Rome.")
145+
])
146+
147+
# A prompt corresponds to an NLP task and contains instructions for the model. Here, the pipeline will go through each Document to figure out the answer.
148+
prompt_template = [
149+
ChatMessage.from_system(
150+
"""
151+
Given these documents, answer the question.
152+
Documents:
153+
{% for doc in documents %}
154+
{{ doc.content }}
155+
{% endfor %}
156+
Question:
157+
"""
158+
),
159+
ChatMessage.from_user(
160+
"{{question}}"
161+
),
162+
ChatMessage.from_system("Answer:")
163+
]
164+
165+
# create the components adding the necessary parameters
166+
retriever = InMemoryBM25Retriever(document_store=document_store)
167+
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")
168+
llm = HuggingFaceAPIChatGenerator(
169+
api_type="serverless_inference_api",
170+
api_params={"model": "Qwen/Qwen2.5-7B-Instruct", "provider": "together"},
171+
token=Secret.from_env_var("HF_API_TOKEN")
172+
)
115173

174+
# Create the pipeline and add the components to it. The order doesn't matter.
175+
# At this stage, the Pipeline validates the components without running them yet.
176+
rag_pipeline = Pipeline()
177+
rag_pipeline.add_component("retriever", retriever)
178+
rag_pipeline.add_component("prompt_builder", prompt_builder)
179+
rag_pipeline.add_component("llm", llm)
180+
181+
# Arrange pipeline components in the order you need them. If a component has more than one inputs or outputs, indicate which input you want to connect to which output using the format ("component_name.output_name", "component_name, input_name").
182+
rag_pipeline.connect("retriever", "prompt_builder.documents")
183+
rag_pipeline.connect("prompt_builder", "llm")
184+
185+
# Run the pipeline by specifying the first component in the pipeline and passing its mandatory inputs. Optionally, you can pass inputs to other components.
186+
question = "Who lives in Paris?"
187+
results = rag_pipeline.run(
188+
{
189+
"retriever": {"query": question},
190+
"prompt_builder": {"question": question},
191+
}
192+
)
193+
194+
print(results["llm"]["replies"])
116195
```
117196

197+
</TabItem>
198+
</Tabs>
199+
118200
### Adding Your Data
119201

120202
Instead of running the RAG pipeline on example data, learn how you can add your own custom data using [Document Stores](../concepts/document-store.mdx).

0 commit comments

Comments
 (0)