|
| 1 | +--- |
| 2 | +title: "SQLAlchemyTableRetriever" |
| 3 | +id: sqlalchemytableretriever |
| 4 | +slug: "/sqlalchemytableretriever" |
| 5 | +description: "Connects to any SQLAlchemy-supported database and executes an SQL query." |
| 6 | +--- |
| 7 | + |
| 8 | +# SQLAlchemyTableRetriever |
| 9 | + |
| 10 | +Connects to any SQLAlchemy-supported database and executes an SQL query. |
| 11 | + |
| 12 | +<div className="key-value-table"> |
| 13 | + |
| 14 | +| | | |
| 15 | +| --- | --- | |
| 16 | +| **Most common position in a pipeline** | Before a [`PromptBuilder`](../builders/promptbuilder.mdx) | |
| 17 | +| **Mandatory init variables** | `drivername`: SQLAlchemy driver name, for example `sqlite`, `postgresql+psycopg2`, `mysql+pymysql`, or `mssql+pyodbc`. For real database backends you will also need `host`, `port`, `database`, `username`, and `password`. | |
| 18 | +| **Mandatory run variables** | `query`: An SQL query to execute | |
| 19 | +| **Output variables** | `dataframe`: The query result as a Pandas DataFrame <br /> <br />`table`: The same result rendered as a Markdown table <br /> <br />`error`: Error message if the query failed, empty string otherwise | |
| 20 | +| **API reference** | [SQLAlchemy](/reference/integrations-sqlalchemy) | |
| 21 | +| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/sqlalchemy | |
| 22 | +| **Package name** | `sqlalchemy-haystack` | |
| 23 | + |
| 24 | +</div> |
| 25 | + |
| 26 | +## Overview |
| 27 | + |
| 28 | +`SQLAlchemyTableRetriever` is a backend-agnostic table retriever: it speaks to anything SQLAlchemy speaks to — PostgreSQL, MySQL, SQLite, MSSQL, and the long tail of dialects covered by third-party drivers. Give it a SQL query and it hands you back the result as both a Pandas DataFrame (under `dataframe`) and a ready-to-render Markdown table (under `table`), which is convenient for piping into a prompt. Results are capped at 10,000 rows. |
| 29 | + |
| 30 | +If the query fails, the component does not raise — it returns an empty DataFrame and puts the SQLAlchemy error string in the `error` output. That makes it safe to drop into a pipeline without wrapping the whole thing in a try/except. |
| 31 | + |
| 32 | +### Connection parameters |
| 33 | + |
| 34 | +The init arguments map directly to SQLAlchemy's URL parts: |
| 35 | + |
| 36 | +- `drivername` — the only strictly required one. Pick the driver that matches your backend, e.g. `postgresql+psycopg2`, `mysql+pymysql`, `sqlite`, `mssql+pyodbc`. |
| 37 | +- `host`, `port`, `database`, `username` — standard connection bits. Pass whatever your backend needs. |
| 38 | +- `password` — a Haystack [Secret](../../concepts/secret-management.mdx). Resolve it from an environment variable with `Secret.from_env_var("MY_DB_PASSWORD")`, or inline with `Secret.from_token("...")` (not recommended for anything other than local tinkering). |
| 39 | + |
| 40 | +For SQLite, `drivername="sqlite"` with `database=":memory:"` is enough — no host/user/password needed. |
| 41 | + |
| 42 | +### `init_script` |
| 43 | + |
| 44 | +Pass `init_script` to run one or more SQL statements once, in a single transaction, the first time the component is warmed up. Typical uses: |
| 45 | + |
| 46 | +- Seeding an in-memory SQLite database for demos or tests. |
| 47 | +- Creating temporary views or session-level settings before queries run. |
| 48 | + |
| 49 | +Each entry in the list is a single statement. |
| 50 | + |
| 51 | +## Usage |
| 52 | + |
| 53 | +Install the `sqlalchemy-haystack` package, plus the driver for your database: |
| 54 | + |
| 55 | +```shell |
| 56 | +pip install sqlalchemy-haystack |
| 57 | +# For PostgreSQL, also install a driver: |
| 58 | +pip install psycopg2-binary |
| 59 | +``` |
| 60 | + |
| 61 | +### On its own |
| 62 | + |
| 63 | +A self-contained example using an in-memory SQLite database seeded via `init_script`: |
| 64 | + |
| 65 | +```python |
| 66 | +from haystack_integrations.components.retrievers.sqlalchemy import SQLAlchemyTableRetriever |
| 67 | + |
| 68 | +retriever = SQLAlchemyTableRetriever( |
| 69 | + drivername="sqlite", |
| 70 | + database=":memory:", |
| 71 | + init_script=[ |
| 72 | + "CREATE TABLE employees (name TEXT, salary INTEGER)", |
| 73 | + "INSERT INTO employees VALUES ('Ada', 90000), ('Linus', 85000), ('Grace', 95000)", |
| 74 | + ], |
| 75 | +) |
| 76 | + |
| 77 | +result = retriever.run(query="SELECT name, salary FROM employees ORDER BY salary DESC") |
| 78 | +print(result["dataframe"]) |
| 79 | +print(result["table"]) |
| 80 | +``` |
| 81 | + |
| 82 | +Connecting to a real backend looks the same — swap the driver and pass connection details: |
| 83 | + |
| 84 | +```python |
| 85 | +from haystack.utils import Secret |
| 86 | +from haystack_integrations.components.retrievers.sqlalchemy import SQLAlchemyTableRetriever |
| 87 | + |
| 88 | +retriever = SQLAlchemyTableRetriever( |
| 89 | + drivername="postgresql+psycopg2", |
| 90 | + host="db.example.com", |
| 91 | + port=5432, |
| 92 | + database="analytics", |
| 93 | + username="readonly", |
| 94 | + password=Secret.from_env_var("ANALYTICS_DB_PASSWORD"), |
| 95 | +) |
| 96 | +``` |
| 97 | + |
| 98 | +### In a pipeline |
| 99 | + |
| 100 | +Use the retriever's Markdown `table` output as context for an LLM — for example, asking an LLM to summarize a query result: |
| 101 | + |
| 102 | +```python |
| 103 | +from haystack import Pipeline |
| 104 | +from haystack.utils import Secret |
| 105 | +from haystack.components.builders import ChatPromptBuilder |
| 106 | +from haystack.components.generators.chat import OpenAIChatGenerator |
| 107 | +from haystack.dataclasses import ChatMessage |
| 108 | +from haystack_integrations.components.retrievers.sqlalchemy import SQLAlchemyTableRetriever |
| 109 | + |
| 110 | +retriever = SQLAlchemyTableRetriever( |
| 111 | + drivername="postgresql+psycopg2", |
| 112 | + host="db.example.com", |
| 113 | + port=5432, |
| 114 | + database="analytics", |
| 115 | + username="readonly", |
| 116 | + password=Secret.from_env_var("ANALYTICS_DB_PASSWORD"), |
| 117 | +) |
| 118 | + |
| 119 | +pipeline = Pipeline() |
| 120 | +pipeline.add_component( |
| 121 | + "builder", |
| 122 | + ChatPromptBuilder( |
| 123 | + template=[ChatMessage.from_user("Describe this table: {{ table }}")], |
| 124 | + required_variables="*", |
| 125 | + ), |
| 126 | +) |
| 127 | +pipeline.add_component("db", retriever) |
| 128 | +pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o")) |
| 129 | + |
| 130 | +pipeline.connect("db.table", "builder.table") |
| 131 | +pipeline.connect("builder.prompt", "llm.messages") |
| 132 | + |
| 133 | +pipeline.run(data={"query": "SELECT employee, salary FROM employees LIMIT 10"}) |
| 134 | +``` |
0 commit comments