Skip to content

Commit 79a37a8

Browse files
HaystackBotbogdankostic
HaystackBot
and
bogdankostic
authored
docs: sync Core Integrations API reference (google_drive) on Docusaurus (#11727)
Co-authored-by: bogdankostic <48713846+bogdankostic@users.noreply.github.com>
1 parent c36bd97 commit 79a37a8

14 files changed

Lines changed: 2562 additions & 0 deletions

File tree

‎docs-website/reference/integrations-api/google_drive.md‎

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
---
2+
title: "Google Drive"
3+
id: integrations-google-drive
4+
description: "Google Drive integration for Haystack"
5+
slug: "/integrations-google-drive"
6+
---
7+
8+
9+
## haystack_integrations.components.retrievers.google_drive.retriever
10+
11+
### GoogleDriveRetriever
12+
13+
Retrieves files from Google Drive via the Drive API v3 search (`files.list`) endpoint.
14+
15+
Given a query, the retriever runs a full-text search over the user's Drive (and optionally shared
16+
drives) and maps each matching file to a Haystack `Document`. By default, each `Document` carries
17+
resource metadata (`file_name`, `file_id`, `web_url`, `mime_type`, `file_extension`, author, and
18+
timestamps) and uses the file `description` or `name` as `content`, because the Drive search API does
19+
not return a text snippet. Set `include_content=True` to additionally export native Google
20+
Docs/Sheets/Slides to text and use that as the `Document` content. Binary files (PDF, DOCX, ...) are
21+
never downloaded. Compose a downstream fetcher/converter on the returned `web_url`/`file_id` when full
22+
file content is needed.
23+
24+
The retriever takes a per-user `access_token` as a run input, typically wired from an upstream
25+
`OAuthResolver`. The token must carry a delegated Google OAuth scope that allows search
26+
(for example `https://www.googleapis.com/auth/drive.readonly`). The metadata-only
27+
`drive.metadata.readonly` scope cannot search file content or export documents.
28+
29+
### Usage example
30+
31+
```python
32+
from haystack import Pipeline
33+
from haystack.utils import Secret
34+
from haystack_integrations.components.connectors.oauth import OAuthResolver
35+
from haystack_integrations.utils.oauth import RefreshTokenSource
36+
from haystack_integrations.components.retrievers.google_drive import GoogleDriveRetriever
37+
38+
pipeline = Pipeline()
39+
pipeline.add_component(
40+
"resolver",
41+
OAuthResolver(
42+
token_source=RefreshTokenSource(
43+
token_url="https://oauth2.googleapis.com/token",
44+
client_id="aaa-bbb-ccc",
45+
refresh_token=Secret.from_env_var("GOOGLE_REFRESH_TOKEN"),
46+
scopes=["https://www.googleapis.com/auth/drive.readonly"],
47+
),
48+
),
49+
)
50+
pipeline.add_component("retriever", GoogleDriveRetriever(top_k=5))
51+
pipeline.connect("resolver.access_token", "retriever.access_token")
52+
53+
result = pipeline.run({"retriever": {"query": "quarterly roadmap"}})
54+
documents = result["retriever"]["documents"]
55+
```
56+
57+
#### __init__
58+
59+
```python
60+
__init__(
61+
*,
62+
include_content: bool = False,
63+
top_k: int = 10,
64+
query_filter: str | None = None,
65+
include_shared_drives: bool = False,
66+
order_by: str | None = None,
67+
fields: list[str] | None = None,
68+
api_base_url: str = _DEFAULT_API_BASE_URL,
69+
timeout: float = 30.0,
70+
max_retries: int = 3
71+
) -> None
72+
```
73+
74+
Initialize the retriever.
75+
76+
**Parameters:**
77+
78+
- **include_content** (<code>bool</code>) – When `True`, native Google Docs/Sheets/Slides are exported to text and the
79+
result becomes the `Document` content. Binary files are never downloaded. When `False` (the
80+
default), `content` is the file `description` or `name` and no export request is made.
81+
- **top_k** (<code>int</code>) – The maximum number of documents to return. Maps to the Drive `pageSize` and is
82+
paginated when it exceeds a single page.
83+
- **query_filter** (<code>str | None</code>) – Optional Drive query clause AND-ed with the full-text search term, for example
84+
`"mimeType != 'application/vnd.google-apps.folder'"` or `"'<folderId>' in parents"`.
85+
- **include_shared_drives** (<code>bool</code>) – When `True`, the search spans shared drives as well as the user's My
86+
Drive (sets `includeItemsFromAllDrives`, `supportsAllDrives`, and `corpora=allDrives`).
87+
- **order_by** (<code>str | None</code>) – Optional Drive `orderBy` expression, for example `"modifiedTime desc"`.
88+
- **fields** (<code>list\[str\] | None</code>) – Optional list of file properties to request via the Drive `fields` selection.
89+
Defaults to a standard set covering the returned metadata.
90+
- **api_base_url** (<code>str</code>) – The Drive API base URL. Defaults to `https://www.googleapis.com/drive/v3`.
91+
- **timeout** (<code>float</code>) – The HTTP timeout in seconds for each request to the Drive API.
92+
- **max_retries** (<code>int</code>) – The maximum number of retries on HTTP 429 (rate limit), 500, 502, 503,
93+
or 504 responses. Set to 0 to disable retries.
94+
95+
**Raises:**
96+
97+
- <code>GoogleDriveConfigError</code> – If `top_k` is not positive or `max_retries` is negative.
98+
99+
#### run
100+
101+
```python
102+
run(
103+
query: str, access_token: str | Secret, top_k: int | None = None
104+
) -> dict[str, list[Document]]
105+
```
106+
107+
Search Google Drive and return the matching documents.
108+
109+
**Parameters:**
110+
111+
- **query** (<code>str</code>) – The search query string, matched against the full text of files via
112+
`fullText contains`.
113+
- **access_token** (<code>str | Secret</code>) – A delegated Google OAuth bearer token for the user whose Drive is searched,
114+
typically wired from an upstream `OAuthResolver` (which emits a plain `str`). A `Secret` is also
115+
accepted and resolved internally.
116+
- **top_k** (<code>int | None</code>) – Overrides the `top_k` configured at initialization for this run.
117+
118+
**Returns:**
119+
120+
- <code>dict\[str, list\[Document\]\]</code> – A dictionary with a `documents` key holding the list of retrieved `Document` objects.
121+
122+
**Raises:**
123+
124+
- <code>GoogleDriveConfigError</code> – If `access_token` is a `Secret` that does not resolve to a string.
125+
- <code>GoogleDriveRequestError</code> – If the Drive API returns an error response.
126+
- <code>httpx.HTTPError</code> – If a network-level error occurs (for example a timeout or connection failure).
127+
128+
#### run_async
129+
130+
```python
131+
run_async(
132+
query: str, access_token: str | Secret, top_k: int | None = None
133+
) -> dict[str, list[Document]]
134+
```
135+
136+
Asynchronously search Google Drive and return the matching documents.
137+
138+
**Parameters:**
139+
140+
- **query** (<code>str</code>) – The search query string, matched against the full text of files via
141+
`fullText contains`.
142+
- **access_token** (<code>str | Secret</code>) – A delegated Google OAuth bearer token for the user whose Drive is searched,
143+
typically wired from an upstream `OAuthResolver` (which emits a plain `str`). A `Secret` is also
144+
accepted and resolved internally.
145+
- **top_k** (<code>int | None</code>) – Overrides the `top_k` configured at initialization for this run.
146+
147+
**Returns:**
148+
149+
- <code>dict\[str, list\[Document\]\]</code> – A dictionary with a `documents` key holding the list of retrieved `Document` objects.
150+
151+
**Raises:**
152+
153+
- <code>GoogleDriveConfigError</code> – If `access_token` is a `Secret` that does not resolve to a string.
154+
- <code>GoogleDriveRequestError</code> – If the Drive API returns an error response.
155+
- <code>httpx.HTTPError</code> – If a network-level error occurs (for example a timeout or connection failure).
156+
157+
#### to_dict
158+
159+
```python
160+
to_dict() -> dict[str, Any]
161+
```
162+
163+
Serialize this component to a dictionary.
164+
165+
**Returns:**
166+
167+
- <code>dict\[str, Any\]</code> – The serialized component as a dictionary.
168+
169+
#### from_dict
170+
171+
```python
172+
from_dict(data: dict[str, Any]) -> GoogleDriveRetriever
173+
```
174+
175+
Deserialize this component from a dictionary.
176+
177+
**Parameters:**
178+
179+
- **data** (<code>dict\[str, Any\]</code>) – The dictionary representation of this component.
180+
181+
**Returns:**
182+
183+
- <code>GoogleDriveRetriever</code> – The deserialized component instance.

‎docs-website/reference_versioned_docs/version-2.18/integrations-api/google_drive.md‎

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
---
2+
title: "Google Drive"
3+
id: integrations-google-drive
4+
description: "Google Drive integration for Haystack"
5+
slug: "/integrations-google-drive"
6+
---
7+
8+
9+
## haystack_integrations.components.retrievers.google_drive.retriever
10+
11+
### GoogleDriveRetriever
12+
13+
Retrieves files from Google Drive via the Drive API v3 search (`files.list`) endpoint.
14+
15+
Given a query, the retriever runs a full-text search over the user's Drive (and optionally shared
16+
drives) and maps each matching file to a Haystack `Document`. By default, each `Document` carries
17+
resource metadata (`file_name`, `file_id`, `web_url`, `mime_type`, `file_extension`, author, and
18+
timestamps) and uses the file `description` or `name` as `content`, because the Drive search API does
19+
not return a text snippet. Set `include_content=True` to additionally export native Google
20+
Docs/Sheets/Slides to text and use that as the `Document` content. Binary files (PDF, DOCX, ...) are
21+
never downloaded. Compose a downstream fetcher/converter on the returned `web_url`/`file_id` when full
22+
file content is needed.
23+
24+
The retriever takes a per-user `access_token` as a run input, typically wired from an upstream
25+
`OAuthResolver`. The token must carry a delegated Google OAuth scope that allows search
26+
(for example `https://www.googleapis.com/auth/drive.readonly`). The metadata-only
27+
`drive.metadata.readonly` scope cannot search file content or export documents.
28+
29+
### Usage example
30+
31+
```python
32+
from haystack import Pipeline
33+
from haystack.utils import Secret
34+
from haystack_integrations.components.connectors.oauth import OAuthResolver
35+
from haystack_integrations.utils.oauth import RefreshTokenSource
36+
from haystack_integrations.components.retrievers.google_drive import GoogleDriveRetriever
37+
38+
pipeline = Pipeline()
39+
pipeline.add_component(
40+
"resolver",
41+
OAuthResolver(
42+
token_source=RefreshTokenSource(
43+
token_url="https://oauth2.googleapis.com/token",
44+
client_id="aaa-bbb-ccc",
45+
refresh_token=Secret.from_env_var("GOOGLE_REFRESH_TOKEN"),
46+
scopes=["https://www.googleapis.com/auth/drive.readonly"],
47+
),
48+
),
49+
)
50+
pipeline.add_component("retriever", GoogleDriveRetriever(top_k=5))
51+
pipeline.connect("resolver.access_token", "retriever.access_token")
52+
53+
result = pipeline.run({"retriever": {"query": "quarterly roadmap"}})
54+
documents = result["retriever"]["documents"]
55+
```
56+
57+
#### __init__
58+
59+
```python
60+
__init__(
61+
*,
62+
include_content: bool = False,
63+
top_k: int = 10,
64+
query_filter: str | None = None,
65+
include_shared_drives: bool = False,
66+
order_by: str | None = None,
67+
fields: list[str] | None = None,
68+
api_base_url: str = _DEFAULT_API_BASE_URL,
69+
timeout: float = 30.0,
70+
max_retries: int = 3
71+
) -> None
72+
```
73+
74+
Initialize the retriever.
75+
76+
**Parameters:**
77+
78+
- **include_content** (<code>bool</code>) – When `True`, native Google Docs/Sheets/Slides are exported to text and the
79+
result becomes the `Document` content. Binary files are never downloaded. When `False` (the
80+
default), `content` is the file `description` or `name` and no export request is made.
81+
- **top_k** (<code>int</code>) – The maximum number of documents to return. Maps to the Drive `pageSize` and is
82+
paginated when it exceeds a single page.
83+
- **query_filter** (<code>str | None</code>) – Optional Drive query clause AND-ed with the full-text search term, for example
84+
`"mimeType != 'application/vnd.google-apps.folder'"` or `"'<folderId>' in parents"`.
85+
- **include_shared_drives** (<code>bool</code>) – When `True`, the search spans shared drives as well as the user's My
86+
Drive (sets `includeItemsFromAllDrives`, `supportsAllDrives`, and `corpora=allDrives`).
87+
- **order_by** (<code>str | None</code>) – Optional Drive `orderBy` expression, for example `"modifiedTime desc"`.
88+
- **fields** (<code>list\[str\] | None</code>) – Optional list of file properties to request via the Drive `fields` selection.
89+
Defaults to a standard set covering the returned metadata.
90+
- **api_base_url** (<code>str</code>) – The Drive API base URL. Defaults to `https://www.googleapis.com/drive/v3`.
91+
- **timeout** (<code>float</code>) – The HTTP timeout in seconds for each request to the Drive API.
92+
- **max_retries** (<code>int</code>) – The maximum number of retries on HTTP 429 (rate limit), 500, 502, 503,
93+
or 504 responses. Set to 0 to disable retries.
94+
95+
**Raises:**
96+
97+
- <code>GoogleDriveConfigError</code> – If `top_k` is not positive or `max_retries` is negative.
98+
99+
#### run
100+
101+
```python
102+
run(
103+
query: str, access_token: str | Secret, top_k: int | None = None
104+
) -> dict[str, list[Document]]
105+
```
106+
107+
Search Google Drive and return the matching documents.
108+
109+
**Parameters:**
110+
111+
- **query** (<code>str</code>) – The search query string, matched against the full text of files via
112+
`fullText contains`.
113+
- **access_token** (<code>str | Secret</code>) – A delegated Google OAuth bearer token for the user whose Drive is searched,
114+
typically wired from an upstream `OAuthResolver` (which emits a plain `str`). A `Secret` is also
115+
accepted and resolved internally.
116+
- **top_k** (<code>int | None</code>) – Overrides the `top_k` configured at initialization for this run.
117+
118+
**Returns:**
119+
120+
- <code>dict\[str, list\[Document\]\]</code> – A dictionary with a `documents` key holding the list of retrieved `Document` objects.
121+
122+
**Raises:**
123+
124+
- <code>GoogleDriveConfigError</code> – If `access_token` is a `Secret` that does not resolve to a string.
125+
- <code>GoogleDriveRequestError</code> – If the Drive API returns an error response.
126+
- <code>httpx.HTTPError</code> – If a network-level error occurs (for example a timeout or connection failure).
127+
128+
#### run_async
129+
130+
```python
131+
run_async(
132+
query: str, access_token: str | Secret, top_k: int | None = None
133+
) -> dict[str, list[Document]]
134+
```
135+
136+
Asynchronously search Google Drive and return the matching documents.
137+
138+
**Parameters:**
139+
140+
- **query** (<code>str</code>) – The search query string, matched against the full text of files via
141+
`fullText contains`.
142+
- **access_token** (<code>str | Secret</code>) – A delegated Google OAuth bearer token for the user whose Drive is searched,
143+
typically wired from an upstream `OAuthResolver` (which emits a plain `str`). A `Secret` is also
144+
accepted and resolved internally.
145+
- **top_k** (<code>int | None</code>) – Overrides the `top_k` configured at initialization for this run.
146+
147+
**Returns:**
148+
149+
- <code>dict\[str, list\[Document\]\]</code> – A dictionary with a `documents` key holding the list of retrieved `Document` objects.
150+
151+
**Raises:**
152+
153+
- <code>GoogleDriveConfigError</code> – If `access_token` is a `Secret` that does not resolve to a string.
154+
- <code>GoogleDriveRequestError</code> – If the Drive API returns an error response.
155+
- <code>httpx.HTTPError</code> – If a network-level error occurs (for example a timeout or connection failure).
156+
157+
#### to_dict
158+
159+
```python
160+
to_dict() -> dict[str, Any]
161+
```
162+
163+
Serialize this component to a dictionary.
164+
165+
**Returns:**
166+
167+
- <code>dict\[str, Any\]</code> – The serialized component as a dictionary.
168+
169+
#### from_dict
170+
171+
```python
172+
from_dict(data: dict[str, Any]) -> GoogleDriveRetriever
173+
```
174+
175+
Deserialize this component from a dictionary.
176+
177+
**Parameters:**
178+
179+
- **data** (<code>dict\[str, Any\]</code>) – The dictionary representation of this component.
180+
181+
**Returns:**
182+
183+
- <code>GoogleDriveRetriever</code> – The deserialized component instance.

0 commit comments

Comments
 (0)