Skip to content

Commit 51084ed

Browse files
docs: add federated passthrough queries section
Document the `TABLE(<connector>.system.query(query => '...'))` syntax for pushing native SQL directly to federated data sources. PyAthena supports this out of the box since the SQL is passed through to Athena unchanged. Refs #712
1 parent 3304e03 commit 51084ed

1 file changed

Lines changed: 28 additions & 0 deletions

File tree

docs/usage.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -202,6 +202,34 @@ and the query was a DML statement (the assumption being that you always want to
202202

203203
The S3 staging directory is not checked, so it's possible that the location of the results is not in your provided `s3_staging_dir`.
204204

205+
## Federated passthrough queries
206+
207+
Athena's [federated query](https://docs.aws.amazon.com/athena/latest/ug/connect-to-a-data-source.html) feature lets you query data in external sources (PostgreSQL, MySQL, Snowflake, etc.) through Lambda-based connectors. By default Athena's planner translates your SQL into the source dialect, but for complex queries — or when you need a feature only the source supports — you can use **passthrough queries** to push native SQL directly to the source.
208+
209+
Passthrough queries use the `TABLE(<connector>.system.query(query => '...'))` syntax. Because PyAthena passes the SQL string through to Athena unchanged, **no client-side changes are required** — these queries work out of the box.
210+
211+
```python
212+
from pyathena import connect
213+
214+
cursor = connect(s3_staging_dir="s3://YOUR_S3_BUCKET/path/to/",
215+
region_name="us-west-2").cursor()
216+
217+
cursor.execute("""
218+
SELECT * FROM TABLE(
219+
my_postgres_connector.system.query(
220+
query => 'SELECT id, name FROM public.users WHERE created_at > NOW() - INTERVAL ''7 days'''
221+
)
222+
)
223+
""")
224+
print(cursor.fetchall())
225+
```
226+
227+
The connector (`my_postgres_connector` above) must be registered in Athena as a data source connector first; see the [Athena documentation](https://docs.aws.amazon.com/athena/latest/ug/connectors-passthrough-queries.html) for the connector setup steps and the list of connectors that support passthrough.
228+
229+
```{note}
230+
Result columns from a passthrough query come back with the source system's types, which Athena maps to its own type system. When using `PandasCursor` or `ArrowCursor`, verify that the resulting dtypes match your expectations — connector-specific types (e.g., PostgreSQL `json`, MySQL `geometry`) may be returned as strings.
231+
```
232+
205233
(query-execution-callback)=
206234

207235
## Query execution callback

0 commit comments

Comments
 (0)