An enterprise-grade Data Agent workbench — it reads business definitions through unified semantics, runs complex multi-table, multi-step analysis inside read-only boundaries,
and keeps every step auditable and replayable, turning one question into a trustworthy analysis.
28 datasource types out of the box · Enterprise semantics & context · Self-hosted · Multi-model · Fully auditable
English · 简体中文
Quick Start · Docs · Supported Data Sources · Roadmap · Contributing
When teams let AI query enterprise databases, the real worry is never "can the model write SQL." It is: does it understand business definitions? Could it mutate production data? Could credentials leak into context? Can a conclusion be verified after the fact?
Most tools reduce the problem to prompt → SQL → answer — impressive in a demo, dead on arrival in the enterprise. DataFoundry takes a different path: it puts the agent inside a semantic, policy-aware, evidence-preserving data task system, upgrading natural-language analytics into controllable, trustworthy, verifiable data work.
- 🗄️ 28 datasource types, ready out of the box — From PostgreSQL, MySQL, Snowflake, BigQuery, and ClickHouse to MongoDB, Redis, and Elasticsearch: connect your existing data stack quickly, cut integration cost, and get the agent into real business analysis faster.
- 🧠 Enterprise semantics and context organization — Manage schema, metric definitions, and field relationships in one place, so terms like "GMV" and "retention" resolve to enterprise-approved tables, fields, and definitions — fewer guessed fields, wrong joins, and definition drift, and fundamentally better accuracy.
- 🏠 Self-hosted deployment and multi-model support — Run it inside your own boundary so data never leaves; on the model side, any OpenAI-compatible provider works (Qwen, DeepSeek, GPT, ...), letting you balance security, cost, latency, and quality per scenario.
- 🔒 Safe by default, auditable throughout — Read-only queries, credential isolation, field masking, row limits, and timeouts by default; SQL, tool calls, and event streams are fully persisted and replayable, so every conclusion is backed by evidence.
- 🧩 Deep optimization for complex data tasks — Built for multi-table, multi-field, long-horizon analysis and multi-step reasoning: complex questions get decomposed, verified, and converged into trustworthy conclusions, materialized as tables, charts, and reports the team can reuse.
DataFoundry 0.2 turns the first usable workbench into a more complete, stateful data-agent workflow:
- Branchable, concurrent analysis — Keep multiple sessions running, queue follow-up prompts, restore completed work, and branch from an earlier question or checkpoint without overwriting the original path.
- Evidence-first follow-ups — Reference a complete output or a selected table/text region in the next question; resolved evidence is carried into the governed run context with diagnostics.
- Semantic trace and first-party Data Link — Inspect checkpoint-backed run structure in a semantic Trace DAG. With our newly open-sourced Data Link, connect tables and columns to business concepts, entities, joinable paths, and confidence-scored relationships for stronger agent grounding.
- Reusable outputs and workspace assets — Preview and export tables, charts, reports, SQL, and files; upload files into an active session, then promote supported files for reuse across sessions.
- Production-facing Web foundation — Built-in password authentication, same-origin API proxying, bilingual UI, model connection tests, onboarding, and an auto-provisioned DTC growth analysis case.
See the v0.2.0 release notes for the complete capability and documentation audit.
Default path is the formal stack: password auth + build / start (do not run npm run dev). No business database required — built-in demo data sources, including the DTC Growth Review case, work out of the box.
Two formal environments share the same startup commands:
| Environment | Use for | Public URL | |
|---|---|---|---|
| Formal test | Local / private acceptance | AUTH_EMAIL_DELIVERY=test (links in API console) |
http://127.0.0.1:3000 |
| Real production | Public service | AUTH_EMAIL_DELIVERY=smtp |
Public HTTPS + reverse proxy |
git clone https://github.com/datagallery-lab/datafoundry.git
cd datafoundry
npm install
cp .env.example .env
cp apps/web/.env.example apps/web/.env.localConfigure any OpenAI-compatible model in the root .env, plus auth for formal test or real production (sample defaults lean formal test):
LLM_PROVIDER=openai-compatible
LLM_MODEL=qwen-plus # or deepseek-chat, gpt-4o, ...
LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
LLM_API_KEY=replace-with-your-key
DATAFOUNDRY_AUTH_MODE=password
AUTH_SESSION_SECRET=replace-with-at-least-32-random-characters
AUTH_PUBLIC_BASE_URL=http://127.0.0.1:3000 # real production: https://your.domain
AUTH_EMAIL_DELIVERY=test # real production: smtp + AUTH_SMTP_*apps/web/.env.local:
NEXT_PUBLIC_DATAFOUNDRY_AUTH_MODE=password
NEXT_PUBLIC_AGENT_RUNTIME_URL=
NEXT_PUBLIC_CONFIG_API_URL=
API_PROXY_TARGET=http://127.0.0.1:8787Build and start:
npm run build
npm run build:web
npm run start:api # :8787 — /healthz liveness, /ready readiness
npm run start:web # :3000 — password mode via same-origin BFFOpen http://127.0.0.1:3000/login, register or sign in, go to /data-tasks, and ask:
Show me the tables in this datasource and explain the main fields of each.
You will see the full chain: schema inspection → read-only SQL → SQL audit → table output → replayable run history.
For real production, also configure SMTP and a reverse proxy: deploy/nginx.datafoundry.conf.example (gzip/brotli static assets; keep /api/copilotkit uncompressed and unbuffered for SSE).
Full steps and the two-environment matrix: Quick Start. Contributor hot-reload (
npm run dev) is appendix-only. Connect your own PostgreSQL / MySQL / CSV and more: Data Sources guide.
Coding agents change code, SQL chatbots answer questions, DataFoundry runs data tasks — three different operating objects, risk boundaries, and outputs:
| Works on | Main risk | Output | |
|---|---|---|---|
| Coding agent | Repos, tests, PRs | Breaking code | Patch, commit, PR |
| SQL chatbot | Prompt, SQL, answer | Wrong tables, unsafe access, leaked credentials, no replay | A SQL snippet or an answer |
| DataFoundry | Datasources, files, knowledge, tools, task state | Production data boundaries, business semantics, audit evidence | Replayable data tasks + SQL audit + tables / charts / reports |
On data tasks specifically, DataFoundry's core advantages over a general-purpose coding agent are:
- Accuracy, from data constraints — Pointed at a database, a coding agent tends to guess tables, fields, and definitions; DataFoundry enforces schema-first analysis and constrains query paths through Data Gateway, cutting guessed fields and wrong joins.
- Safety, from controlled execution — Coding agents run commands and write files: powerful, but high-risk against enterprise data. DataFoundry defaults to read-only SQL, credential isolation, field masking, row limits, timeouts, and audit — built for real data environments.
- Speed, from a converging task path — Not because the model reasons faster, but because datasource selection, schema caching, context budgeting, tool policy, and the artifact pipeline eliminate wasted attempts, so analysis converges on results sooner.
- Complex tasks, from data-workflow design — Coding agents excel at code engineering; DataFoundry is built for analysis across many tables, fields, and metrics plus knowledge bases, files, and report outputs, chaining "query, verify, explain, materialize" into one complete flow.
- Adoption, from an enterprise runtime — This is not a demo: the Web workbench, TUI, REST API, CopilotKit / AG-UI, Data Gateway, Skills, MCP, Files, Artifacts, and Metadata combine into an operating foundation for data agents.
Ask → Align semantics → Execute under control → Materialize output → Replay & review
- Define the task — Pick datasources, files, knowledge, and tools, then describe the business question in natural language.
- Align meaning with structure — The agent inspects schema and available context first, grounding terms like "GMV" or "retention" in real tables and fields.
- Execute under control — Data Gateway runs queries inside a read-only boundary with SQL guardrails, row limits, timeouts, and masking; every SQL statement leaves an audit record.
- Materialize output — Results become tables, charts, reports, or files — assets the team can cite.
- Replay and review — Web, TUI, and API share one run history, so every step's evidence is always one click away.
The Web workbench fits day-to-day analysis and demos, the TUI fits terminals and remote servers, and the API / CopilotKit / AG-UI path lets you embed the same trusted runtime into your own product.
Connect through Data Gateway adapters: the built-in DTC Growth Review case works out of the box; DuckDB, SQLite, CSV, Excel, PostgreSQL, and MySQL fit local trials; cloud warehouses, search engines, and NoSQL systems plug in with their own services and credentials.
See the full list in Supported Data Sources.
- The model only receives governed context; datasource credentials, model API keys, and MCP tokens never enter
messages,context, orforwardedProps. - All datasource access goes through Data Gateway — read-only by default, with SQL guardrails, row limits, timeouts, and field masking.
- SQL audit logs, tool-call records, and event streams are fully persisted for after-the-fact review.
- Production-grade multi-tenant auth, centralized secret management, monitoring, and deployment operations require a dedicated design for your environment — see Security.
- Governed data-task workbench — Web and TUI share one TypeScript agent runtime, CopilotKit / AG-UI event stream, replayable run history, SQL audit trail, and unified asset layer.
- Safe data access foundation — Datasource registration, connection testing, schema introspection, table preview, read-only SQL, masking, knowledge imports, MCP resources, skill packages, and model configuration.
- Unified semantic layer — Durable metrics, entities, relationships, lineage, and policies, moving agents from "guessing fields" to "understanding business definitions" and from one-off SQL to a governable data operating layer.
- Autonomous analyst loops — Agents that plan investigations, run controlled experiments, critique findings, and converge on evidence-backed conclusions.
- Evaluation and reliability lab — NL2SQL, retrieval, tool-use, and end-to-end task benchmarks with regression gates and failure forensics.
- Enterprise control plane — Identity, RBAC, approvals, audit export, policy-as-code, and cost limits.
Roadmap discussions are welcome in issues and discussions.
| Goal | Read |
|---|---|
| Run the local demo | Quick Start |
| Understand positioning and scope | Overview · Capabilities |
| Use the Web / TUI | Web workbench guide · TUI guide |
| Connect data sources | Data sources guide · Supported data sources |
| Integrate via API and runtime | REST API · Agent Runtime & AG-UI |
| Understand architecture and security | Architecture overview · Security |
DataFoundry moves quickly, so small, well-scoped PRs are the easiest to merge:
- Open an issue or discussion first for behavioral, protocol, datasource-adapter, or agent-policy changes.
- Keep each PR focused on one runtime boundary or feature area.
- Run
npm run buildplus the targeted smoke checks for what you touched (e.g.npm run smoke:data-gateway). - Update docs when a change affects setup, APIs, datasource configuration, or user-visible output.
- Do not commit credentials, local databases, generated storage, or private benchmark data.
See CONTRIBUTING.md for details.
Join the DataFoundry community to discuss the product, roadmap, and real-world adoption with us. Issues and discussions are always welcome too.
| Slack Community | QQ Group |
 |
 |
| Join Slack | Group ID1048076064 |
DataFoundry is inspired by and built with ideas from excellent open-source projects and communities:
- Thanks to the LINUX DO community for their support and discussions.
- Mastra for agent runtime patterns.
- AG-UI for event stream protocol design.
- CopilotKit for agent-native UX patterns.
- Ink for terminal UI foundations.
- MCP for the tool ecosystem and integration model.
Apache License 2.0. See LICENSE.
DataFoundry is under active development. Current code, public docs, and passing smoke checks are the source of truth.