Skip to content

Commit 1d2bcee

Browse files
HaystackBotsjrl
HaystackBot
and
sjrl
authored
docs: sync Core Integrations API reference (tavily) on Docusaurus (#12000)
Co-authored-by: sjrl <10526848+sjrl@users.noreply.github.com>
1 parent fad8ef3 commit 1d2bcee

15 files changed

Lines changed: 1680 additions & 0 deletions

File tree

‎docs-website/reference/integrations-api/tavily.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,118 @@ slug: "/integrations-tavily"
66
---
77

88

9+
## haystack_integrations.components.fetchers.tavily.tavily_fetcher
10+
11+
### TavilyFetcher
12+
13+
A component that uses the Tavily Extract API to fetch and extract content from URLs as Haystack Documents.
14+
15+
This component wraps the Tavily Extract API, which retrieves and parses web page content from
16+
one or more specified URLs. Unlike web search, it fetches content directly from the given URLs
17+
rather than discovering them via a query. PDF URLs are also supported for extraction.
18+
19+
Tavily is an AI-powered search and extraction API optimized for LLM applications. You need a Tavily
20+
API key from [tavily.com](https://tavily.com).
21+
22+
### Usage example
23+
24+
```python
25+
from haystack_integrations.components.fetchers.tavily import TavilyFetcher
26+
from haystack.utils import Secret
27+
28+
fetcher = TavilyFetcher(
29+
api_key=Secret.from_env_var("TAVILY_API_KEY"),
30+
extract_depth="basic",
31+
)
32+
result = fetcher.run(urls=["https://haystack.deepset.ai"])
33+
documents = result["documents"]
34+
meta = result["meta"]
35+
```
36+
37+
#### __init__
38+
39+
```python
40+
__init__(
41+
api_key: Secret = Secret.from_env_var("TAVILY_API_KEY"),
42+
*,
43+
extract_depth: Literal["basic", "advanced"] = "basic",
44+
include_images: bool = False,
45+
extract_params: dict[str, Any] | None = None
46+
) -> None
47+
```
48+
49+
Initialize the TavilyFetcher component.
50+
51+
**Parameters:**
52+
53+
- **api_key** (<code>Secret</code>) – API key for Tavily. Defaults to the `TAVILY_API_KEY` environment variable.
54+
- **extract_depth** (<code>Literal['basic', 'advanced']</code>) – Extraction depth: `"basic"` (fast, lower cost) or `"advanced"` (more data including
55+
tables, higher latency and cost). Defaults to `"basic"`.
56+
- **include_images** (<code>bool</code>) – If `True`, extracted image URLs are included in each Document's metadata under
57+
the `"images"` key. Defaults to `False`.
58+
- **extract_params** (<code>dict\[str, Any\] | None</code>) – Additional parameters passed to the Tavily Extract API, such as `format`,
59+
`include_favicon`, `query`, or `chunks_per_source`.
60+
See the [Tavily Extract API reference](https://docs.tavily.com/documentation/api-reference/endpoint/extract)
61+
for available options.
62+
63+
#### warm_up
64+
65+
```python
66+
warm_up() -> None
67+
```
68+
69+
Initialize the Tavily sync and async clients.
70+
71+
Called automatically on first use. Can be called explicitly to avoid cold-start latency.
72+
73+
#### run
74+
75+
```python
76+
run(
77+
urls: list[str], extract_params: dict[str, Any] | None = None
78+
) -> dict[str, Any]
79+
```
80+
81+
Fetch and extract content from the given URLs using the Tavily Extract API.
82+
83+
**Parameters:**
84+
85+
- **urls** (<code>list\[str\]</code>) – List of URLs to extract content from. Maximum 20 URLs per request.
86+
- **extract_params** (<code>dict\[str, Any\] | None</code>) – Optional per-run override of extract parameters.
87+
If provided, fully replaces the init-time `extract_params`.
88+
89+
**Returns:**
90+
91+
- <code>dict\[str, Any\]</code> – A dictionary with:
92+
- `documents`: List of Documents containing extracted page content.
93+
Each Document's `meta` includes `"url"` and, if `include_images` is True, `"images"`.
94+
- `meta`: Request-level metadata containing `"response_time"`, `"usage"`,
95+
`"request_id"`, and `"failed_results"` for URLs that could not be processed.
96+
97+
#### run_async
98+
99+
```python
100+
run_async(
101+
urls: list[str], extract_params: dict[str, Any] | None = None
102+
) -> dict[str, Any]
103+
```
104+
105+
Asynchronously fetch and extract content from the given URLs using the Tavily Extract API.
106+
107+
**Parameters:**
108+
109+
- **urls** (<code>list\[str\]</code>) – List of URLs to extract content from. Maximum 20 URLs per request.
110+
- **extract_params** (<code>dict\[str, Any\] | None</code>) – Optional per-run override of extract parameters.
111+
If provided, fully replaces the init-time `extract_params`.
112+
113+
**Returns:**
114+
115+
- <code>dict\[str, Any\]</code> – A dictionary with:
116+
- `documents`: List of Documents containing extracted page content.
117+
Each Document's `meta` includes `"url"` and, if `include_images` is True, `"images"`.
118+
- `meta`: Request-level metadata containing `"response_time"`, `"usage"`,
119+
`"request_id"`, and `"failed_results"` for URLs that could not be processed.
120+
9121
## haystack_integrations.components.websearch.tavily.tavily_websearch
10122

11123
### TavilyWebSearch

‎docs-website/reference_versioned_docs/version-2.18/integrations-api/tavily.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,118 @@ slug: "/integrations-tavily"
66
---
77

88

9+
## haystack_integrations.components.fetchers.tavily.tavily_fetcher
10+
11+
### TavilyFetcher
12+
13+
A component that uses the Tavily Extract API to fetch and extract content from URLs as Haystack Documents.
14+
15+
This component wraps the Tavily Extract API, which retrieves and parses web page content from
16+
one or more specified URLs. Unlike web search, it fetches content directly from the given URLs
17+
rather than discovering them via a query. PDF URLs are also supported for extraction.
18+
19+
Tavily is an AI-powered search and extraction API optimized for LLM applications. You need a Tavily
20+
API key from [tavily.com](https://tavily.com).
21+
22+
### Usage example
23+
24+
```python
25+
from haystack_integrations.components.fetchers.tavily import TavilyFetcher
26+
from haystack.utils import Secret
27+
28+
fetcher = TavilyFetcher(
29+
api_key=Secret.from_env_var("TAVILY_API_KEY"),
30+
extract_depth="basic",
31+
)
32+
result = fetcher.run(urls=["https://haystack.deepset.ai"])
33+
documents = result["documents"]
34+
meta = result["meta"]
35+
```
36+
37+
#### __init__
38+
39+
```python
40+
__init__(
41+
api_key: Secret = Secret.from_env_var("TAVILY_API_KEY"),
42+
*,
43+
extract_depth: Literal["basic", "advanced"] = "basic",
44+
include_images: bool = False,
45+
extract_params: dict[str, Any] | None = None
46+
) -> None
47+
```
48+
49+
Initialize the TavilyFetcher component.
50+
51+
**Parameters:**
52+
53+
- **api_key** (<code>Secret</code>) – API key for Tavily. Defaults to the `TAVILY_API_KEY` environment variable.
54+
- **extract_depth** (<code>Literal['basic', 'advanced']</code>) – Extraction depth: `"basic"` (fast, lower cost) or `"advanced"` (more data including
55+
tables, higher latency and cost). Defaults to `"basic"`.
56+
- **include_images** (<code>bool</code>) – If `True`, extracted image URLs are included in each Document's metadata under
57+
the `"images"` key. Defaults to `False`.
58+
- **extract_params** (<code>dict\[str, Any\] | None</code>) – Additional parameters passed to the Tavily Extract API, such as `format`,
59+
`include_favicon`, `query`, or `chunks_per_source`.
60+
See the [Tavily Extract API reference](https://docs.tavily.com/documentation/api-reference/endpoint/extract)
61+
for available options.
62+
63+
#### warm_up
64+
65+
```python
66+
warm_up() -> None
67+
```
68+
69+
Initialize the Tavily sync and async clients.
70+
71+
Called automatically on first use. Can be called explicitly to avoid cold-start latency.
72+
73+
#### run
74+
75+
```python
76+
run(
77+
urls: list[str], extract_params: dict[str, Any] | None = None
78+
) -> dict[str, Any]
79+
```
80+
81+
Fetch and extract content from the given URLs using the Tavily Extract API.
82+
83+
**Parameters:**
84+
85+
- **urls** (<code>list\[str\]</code>) – List of URLs to extract content from. Maximum 20 URLs per request.
86+
- **extract_params** (<code>dict\[str, Any\] | None</code>) – Optional per-run override of extract parameters.
87+
If provided, fully replaces the init-time `extract_params`.
88+
89+
**Returns:**
90+
91+
- <code>dict\[str, Any\]</code> – A dictionary with:
92+
- `documents`: List of Documents containing extracted page content.
93+
Each Document's `meta` includes `"url"` and, if `include_images` is True, `"images"`.
94+
- `meta`: Request-level metadata containing `"response_time"`, `"usage"`,
95+
`"request_id"`, and `"failed_results"` for URLs that could not be processed.
96+
97+
#### run_async
98+
99+
```python
100+
run_async(
101+
urls: list[str], extract_params: dict[str, Any] | None = None
102+
) -> dict[str, Any]
103+
```
104+
105+
Asynchronously fetch and extract content from the given URLs using the Tavily Extract API.
106+
107+
**Parameters:**
108+
109+
- **urls** (<code>list\[str\]</code>) – List of URLs to extract content from. Maximum 20 URLs per request.
110+
- **extract_params** (<code>dict\[str, Any\] | None</code>) – Optional per-run override of extract parameters.
111+
If provided, fully replaces the init-time `extract_params`.
112+
113+
**Returns:**
114+
115+
- <code>dict\[str, Any\]</code> – A dictionary with:
116+
- `documents`: List of Documents containing extracted page content.
117+
Each Document's `meta` includes `"url"` and, if `include_images` is True, `"images"`.
118+
- `meta`: Request-level metadata containing `"response_time"`, `"usage"`,
119+
`"request_id"`, and `"failed_results"` for URLs that could not be processed.
120+
9121
## haystack_integrations.components.websearch.tavily.tavily_websearch
10122

11123
### TavilyWebSearch

‎docs-website/reference_versioned_docs/version-2.19/integrations-api/tavily.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,118 @@ slug: "/integrations-tavily"
66
---
77

88

9+
## haystack_integrations.components.fetchers.tavily.tavily_fetcher
10+
11+
### TavilyFetcher
12+
13+
A component that uses the Tavily Extract API to fetch and extract content from URLs as Haystack Documents.
14+
15+
This component wraps the Tavily Extract API, which retrieves and parses web page content from
16+
one or more specified URLs. Unlike web search, it fetches content directly from the given URLs
17+
rather than discovering them via a query. PDF URLs are also supported for extraction.
18+
19+
Tavily is an AI-powered search and extraction API optimized for LLM applications. You need a Tavily
20+
API key from [tavily.com](https://tavily.com).
21+
22+
### Usage example
23+
24+
```python
25+
from haystack_integrations.components.fetchers.tavily import TavilyFetcher
26+
from haystack.utils import Secret
27+
28+
fetcher = TavilyFetcher(
29+
api_key=Secret.from_env_var("TAVILY_API_KEY"),
30+
extract_depth="basic",
31+
)
32+
result = fetcher.run(urls=["https://haystack.deepset.ai"])
33+
documents = result["documents"]
34+
meta = result["meta"]
35+
```
36+
37+
#### __init__
38+
39+
```python
40+
__init__(
41+
api_key: Secret = Secret.from_env_var("TAVILY_API_KEY"),
42+
*,
43+
extract_depth: Literal["basic", "advanced"] = "basic",
44+
include_images: bool = False,
45+
extract_params: dict[str, Any] | None = None
46+
) -> None
47+
```
48+
49+
Initialize the TavilyFetcher component.
50+
51+
**Parameters:**
52+
53+
- **api_key** (<code>Secret</code>) – API key for Tavily. Defaults to the `TAVILY_API_KEY` environment variable.
54+
- **extract_depth** (<code>Literal['basic', 'advanced']</code>) – Extraction depth: `"basic"` (fast, lower cost) or `"advanced"` (more data including
55+
tables, higher latency and cost). Defaults to `"basic"`.
56+
- **include_images** (<code>bool</code>) – If `True`, extracted image URLs are included in each Document's metadata under
57+
the `"images"` key. Defaults to `False`.
58+
- **extract_params** (<code>dict\[str, Any\] | None</code>) – Additional parameters passed to the Tavily Extract API, such as `format`,
59+
`include_favicon`, `query`, or `chunks_per_source`.
60+
See the [Tavily Extract API reference](https://docs.tavily.com/documentation/api-reference/endpoint/extract)
61+
for available options.
62+
63+
#### warm_up
64+
65+
```python
66+
warm_up() -> None
67+
```
68+
69+
Initialize the Tavily sync and async clients.
70+
71+
Called automatically on first use. Can be called explicitly to avoid cold-start latency.
72+
73+
#### run
74+
75+
```python
76+
run(
77+
urls: list[str], extract_params: dict[str, Any] | None = None
78+
) -> dict[str, Any]
79+
```
80+
81+
Fetch and extract content from the given URLs using the Tavily Extract API.
82+
83+
**Parameters:**
84+
85+
- **urls** (<code>list\[str\]</code>) – List of URLs to extract content from. Maximum 20 URLs per request.
86+
- **extract_params** (<code>dict\[str, Any\] | None</code>) – Optional per-run override of extract parameters.
87+
If provided, fully replaces the init-time `extract_params`.
88+
89+
**Returns:**
90+
91+
- <code>dict\[str, Any\]</code> – A dictionary with:
92+
- `documents`: List of Documents containing extracted page content.
93+
Each Document's `meta` includes `"url"` and, if `include_images` is True, `"images"`.
94+
- `meta`: Request-level metadata containing `"response_time"`, `"usage"`,
95+
`"request_id"`, and `"failed_results"` for URLs that could not be processed.
96+
97+
#### run_async
98+
99+
```python
100+
run_async(
101+
urls: list[str], extract_params: dict[str, Any] | None = None
102+
) -> dict[str, Any]
103+
```
104+
105+
Asynchronously fetch and extract content from the given URLs using the Tavily Extract API.
106+
107+
**Parameters:**
108+
109+
- **urls** (<code>list\[str\]</code>) – List of URLs to extract content from. Maximum 20 URLs per request.
110+
- **extract_params** (<code>dict\[str, Any\] | None</code>) – Optional per-run override of extract parameters.
111+
If provided, fully replaces the init-time `extract_params`.
112+
113+
**Returns:**
114+
115+
- <code>dict\[str, Any\]</code> – A dictionary with:
116+
- `documents`: List of Documents containing extracted page content.
117+
Each Document's `meta` includes `"url"` and, if `include_images` is True, `"images"`.
118+
- `meta`: Request-level metadata containing `"response_time"`, `"usage"`,
119+
`"request_id"`, and `"failed_results"` for URLs that could not be processed.
120+
9121
## haystack_integrations.components.websearch.tavily.tavily_websearch
10122

11123
### TavilyWebSearch

0 commit comments

Comments
 (0)