Skip to content

Commit fc88514

Browse files
iancaooiancaoo
committed
docs: add ecosystem badges to readme
1 parent f9bfe5e commit fc88514

2 files changed

Lines changed: 261 additions & 216 deletions

File tree

‎README.md‎

Lines changed: 133 additions & 113 deletions
Original file line numberDiff line numberDiff line change
@@ -1,178 +1,198 @@
1-
<h1 align="center">DataFoundry 🚀</h1>
1+
<h1 align="center">DataFoundry</h1>
2+
3+
<p align="center"><strong>An AI data analyst your enterprise can actually trust</strong></p>
24

35
<p align="center">
4-
DataFoundry is an out-of-the-box, self-hostable enterprise data agent workbench that upgrades natural-language data analysis into semantically aware, secure, and verifiable data workflows, helping teams reach trusted answers faster while preserving audit-grade analytical evidence.
6+
An open-source, enterprise-grade Data Agent workbench that turns one question into a trustworthy analysis — semantics understood, boundaries enforced, evidence preserved.
57
</p>
68

79
<p align="center">
8-
English · <a href="README_zh.md">简体中文</a>
10+
<strong>English</strong> · <a href="README_zh.md">简体中文</a>
911
</p>
1012

1113
<p align="center">
12-
<a href="docs/zh/quick-start.md"><strong>Quick Start</strong></a>
14+
<a href="#-run-it-in-5-minutes"><strong>Quick Start</strong></a>
1315
·
14-
<a href="docs/README.md"><strong>Docs</strong></a>
16+
<a href="https://datagallery-lab.github.io/datafoundry/"><strong>Docs</strong></a>
1517
·
16-
<a href="docs/zh/reference/supported-datasources.md"><strong>Supported Data Sources</strong></a>
18+
<a href="docs/en/reference/supported-datasources.md"><strong>Supported Data Sources</strong></a>
1719
·
18-
<a href="#-contributing"><strong>Contributing</strong></a>
20+
<a href="#️-roadmap"><strong>Roadmap</strong></a>
1921
·
20-
<a href="#-license"><strong>License</strong></a>
22+
<a href="#-contributing"><strong>Contributing</strong></a>
2123
</p>
2224

23-
## ✨ Why DataFoundry
24-
25-
Modern data agents need more than a chat model. They need selected context, datasource boundaries, SQL policy,
26-
auditable events, durable outputs, and a frontend protocol that can replay the whole run.
27-
28-
DataFoundry puts those pieces behind one runtime:
29-
30-
- 🔎 **Schema-first analysis** — the agent inspects datasource structure before it can run read-only SQL.
31-
- 🧠 **Governed context** — conversation history, memory, tool results, files, and knowledge sources are compiled under one budget.
32-
- 🧾 **Auditable execution** — AG-UI events, SQL audit logs, artifacts, and session history are persisted as replayable records.
33-
- 📦 **Unified assets** — uploads, workspace files, generated outputs, and KB imports share the same deduplicated asset layer.
34-
- 🧩 **Protocol-ready runtime** — CopilotKit / AG-UI clients consume the same events, run state, artifacts, and replay data.
35-
36-
## 🗄️ Bring Your Data Stack
37-
38-
DataFoundry is built around a Data Gateway adapter boundary. The current runtime already recognizes local files,
39-
embedded databases, cloud warehouses, lakehouse engines, operational databases, and search / NoSQL systems.
40-
4125
<p align="center">
42-
<img src="docs/assets/readme/database-wall.png" alt="Supported DataFoundry datasource adapters" width="100%">
26+
<img src="https://img.shields.io/badge/license-Apache--2.0-blue" alt="Apache-2.0" />
27+
<img src="https://img.shields.io/badge/TypeScript-5.x-3178c6?logo=typescript&logoColor=white" alt="TypeScript" />
28+
<img src="https://img.shields.io/badge/self--hostable-local%20first-2ea44f" alt="Self-hostable" />
29+
<img src="https://img.shields.io/badge/PRs-welcome-ff69b4" alt="PRs welcome" />
30+
<img src="https://img.shields.io/badge/status-early%20but%20usable-orange" alt="Status" />
31+
<br />
32+
<a href="https://github.com/mastra-ai/mastra"><img src="https://img.shields.io/badge/Mastra-agent%20runtime-111827" alt="Mastra agent runtime" /></a>
33+
<a href="https://github.com/ag-ui-protocol/ag-ui"><img src="https://img.shields.io/badge/AG--UI-event%20stream-6f42c1" alt="AG-UI event stream" /></a>
34+
<a href="https://github.com/vadimdemedes/ink"><img src="https://img.shields.io/badge/Ink-terminal%20UI-0f766e" alt="Ink terminal UI" /></a>
4335
</p>
4436

45-
## 🧭 How It Works
46-
4737
<p align="center">
48-
<img src="docs/assets/readme/runtime-flow.png" alt="DataFoundry runtime flow" width="100%">
38+
<img src="docs/assets/readme/gui-demo.gif" alt="DataFoundry Web workbench demo" width="100%">
4939
</p>
5040

51-
The frontend talks to a single backend runtime. The backend owns identity, run replay, context assembly, memory,
52-
tool policy, SQL guardrails, file references, and artifact creation. The model sees a governed prompt; it never sees raw
53-
datasource credentials. 🛡️
41+
---
5442

55-
## 🎬 GUI And TUI Preview
43+
## 🤔 What Is DataFoundry
5644

57-
### GUI Preview
45+
When teams let AI query enterprise databases, the real worry is never "can the model write SQL." It is: **does it understand business definitions? Could it mutate production data? Could credentials leak into context? Can a conclusion be verified after the fact?**
5846

59-
The recording below shows the current web workbench experience.
60-
61-
<p align="center">
62-
<img src="docs/assets/readme/gui-demo.gif" alt="DataFoundry GUI demo" width="100%">
63-
</p>
47+
Most tools reduce the problem to `prompt → SQL → answer` — impressive in a demo, dead on arrival in the enterprise. DataFoundry takes a different path: **it puts the agent inside a semantic, policy-aware, evidence-preserving data task system**, upgrading natural-language analytics into controllable, trustworthy, verifiable data work.
6448

65-
### TUI Preview
49+
## ✨ Core Capabilities
6650

67-
The recording below shows the current terminal workflow backed by the same runtime.
51+
- 🧠 **Unified semantics, no guessing** — Metrics, entities, relationships, and business definitions are consolidated into a semantic layer the agent can reuse; before any query it inspects schema and obtains a run-scoped token, so terms like "GMV" and "retention" resolve to enterprise-approved tables, fields, and definitions instead of being re-guessed from table names.
52+
- 🔒 **Read-only security boundary** — Database access never goes through the model or the browser. Data Gateway owns connections, SQL guardrails, row limits, timeouts, and field masking; credentials and API keys never enter model context.
53+
- 🧾 **End-to-end evidence chain** — SQL, tool calls, event streams, generated files, and session history are persisted as traceable records. Teams see the full reasoning path from question to conclusion, not just "an answer."
54+
- 🔁 **Replayable tasks** — Every analysis is a resumable run history that can be reviewed, audited, continued, or handed off to a teammate.
55+
- 🖥️ **Web / TUI / API, three surfaces** — The graphical workbench, terminal UI, and HTTP API share one runtime and event stream — from local analysis to embedding in your own product, one set of logic end to end.
56+
- 🔌 **Open ecosystem** — Plug in MCP servers, skill packages, and workspace tools; on the model side, any OpenAI-compatible provider works (Qwen, DeepSeek, GPT, ...).
57+
- 📊 **Outputs become assets** — Tables, charts, reports, and generated files land in a unified asset layer instead of scattering across chat logs and screenshots.
6858

69-
<p align="center">
70-
<a href="docs/assets/readme/tui-demo.mp4">
71-
<img src="docs/assets/readme/tui-demo.gif" alt="DataFoundry TUI demo" width="100%">
72-
</a>
73-
</p>
59+
## 🚀 Run It In 5 Minutes
7460

75-
## ⚡ Quick Start
61+
No database required — a built-in DuckDB demo datasource works out of the box.
7662

7763
```bash
64+
git clone https://github.com/datagallery-lab/datafoundry.git
65+
cd datafoundry
7866
npm install
7967
cp .env.example .env
8068
cp apps/web/.env.example apps/web/.env.local
8169
npm run dev
8270
```
8371

84-
Open the workbench:
85-
86-
```text
87-
http://127.0.0.1:3000/data-tasks
88-
```
72+
Configure any OpenAI-compatible model in `.env`:
8973

90-
The local workbench includes a demo DuckDB datasource. Live agent runs require a real LLM key in `.env`.
91-
92-
```text
74+
```bash
9375
LLM_PROVIDER=openai-compatible
94-
LLM_MODEL=qwen-plus
76+
LLM_MODEL=qwen-plus # or deepseek-chat, gpt-4o, ...
9577
LLM_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
9678
LLM_API_KEY=replace-with-your-key
9779
```
9880

99-
DeepSeek and other OpenAI-compatible providers use the same provider mode:
81+
Open `http://127.0.0.1:3000/data-tasks` and ask:
10082

10183
```text
102-
LLM_PROVIDER=openai-compatible
103-
LLM_MODEL=deepseek-chat
104-
LLM_BASE_URL=https://api.deepseek.com
105-
LLM_API_KEY=replace-with-your-key
84+
Show me the tables in this datasource and explain the main fields of each.
10685
```
10786

108-
## 🧩 What You Can Build With It
87+
You will see the full chain: schema inspection → read-only SQL → SQL audit → table output → replayable run history.
10988

110-
| Use case | Runtime support |
111-
| --- | --- |
112-
| Natural-language database analysis | Datasource selection, schema inspection, SQL guard, query limit, timeout, audit log, table artifact. |
113-
| File-backed agent work | Session workspace, cross-session workspace assets, file refs, downloads, generated deliverables. |
114-
| Knowledge-assisted analysis | KB imports, document chunks, local search, optional embedding-backed retrieval, governed context injection. |
115-
| Frontend agent UX | CopilotKit / AG-UI streaming, run replay, task state, token usage, artifacts, interaction suspension. |
116-
| Controlled tool extension | Mastra tools, MCP middleware, workspace tools, skill packages, tool observation adapters. |
89+
> See the [Quick Start](docs/en/quick-start.md) for details, and the [Data Sources guide](docs/en/guides/data-sources.md) to connect your own PostgreSQL / MySQL / CSV and more.
11790
118-
## 🛠️ Developer Loop
91+
## 🆚 How It Differs From A SQL Chatbot
11992

120-
```bash
121-
npm run build
122-
npm run smoke:config-api
123-
npm run smoke:data-gateway
124-
npm run smoke:copilotkit
125-
npm run smoke:docs
93+
Coding agents change code, SQL chatbots answer questions, DataFoundry runs data tasks — three different operating objects, risk boundaries, and outputs:
94+
95+
| | Works on | Main risk | Output |
96+
| --- | --- | --- | --- |
97+
| Coding agent | Repos, tests, PRs | Breaking code | Patch, commit, PR |
98+
| SQL chatbot | Prompt, SQL, answer | Wrong tables, unsafe access, leaked credentials, no replay | A SQL snippet or an answer |
99+
| **DataFoundry** | Datasources, files, knowledge, tools, task state | Production data boundaries, business semantics, audit evidence | **Replayable data tasks** + SQL audit + tables / charts / reports |
100+
101+
In one sentence: a SQL chatbot answers a question and is done; DataFoundry compounds into a reusable, governable, auditable layer of data operations.
102+
103+
## ⚙️ How A Data Task Runs
104+
105+
```text
106+
Ask → Align semantics → Execute under control → Materialize output → Replay & review
126107
```
127108

128-
Use targeted smoke checks for the package you touch. `package.json` lists the full verification set.
109+
1. **Define the task** — Pick datasources, files, knowledge, and tools, then describe the business question in natural language.
110+
2. **Align meaning with structure** — The agent inspects schema and available context first, grounding terms like "GMV" or "retention" in real tables and fields.
111+
3. **Execute under control** — Data Gateway runs queries inside a read-only boundary with SQL guardrails, row limits, timeouts, and masking; every SQL statement leaves an audit record.
112+
4. **Materialize output** — Results become tables, charts, reports, or files — assets the team can cite.
113+
5. **Replay and review** — Web, TUI, and API share one run history, so every step's evidence is always one click away.
129114

130-
## 🤝 Contributing
115+
<p align="center">
116+
<img src="docs/assets/readme/runtime-flow.png" alt="DataFoundry runtime flow" width="100%">
117+
</p>
131118

132-
DataFoundry is moving quickly, so small, well-scoped contributions are easiest to review.
119+
## 🖥️ More Than A Chat Box
133120

134-
See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, verification, and hygiene rules.
121+
The **Web workbench** fits day-to-day analysis and demos, the **TUI** fits terminals and remote servers, and the **API / CopilotKit / AG-UI** path lets you embed the same trusted runtime into your own product.
135122

136-
1. Open an issue or discussion for behavioral changes, protocol changes, datasource adapters, and agent-policy changes.
137-
2. Keep pull requests focused on one runtime boundary or feature area.
138-
3. Run `npm run build` and the targeted smoke checks for the packages you touched.
139-
4. Update docs when a change affects setup, APIs, datasource configuration, event behavior, or user-visible output.
140-
5. Do not commit credentials, local databases, generated storage, or private benchmark data.
123+
<p align="center">
124+
<a href="docs/assets/readme/tui-demo.mp4">
125+
<img src="docs/assets/readme/tui-demo.gif" alt="DataFoundry TUI demo" width="100%">
126+
</a>
127+
</p>
128+
129+
## 🗄️ Bring Your Data Stack, No Rebuild
130+
131+
Connect through Data Gateway adapters: the built-in DuckDB demo works out of the box; SQLite, CSV, Excel, PostgreSQL, and MySQL fit local trials; cloud warehouses, search engines, and NoSQL systems plug in with their own services and credentials.
132+
133+
<p align="center">
134+
<img src="docs/assets/readme/database-wall.png" alt="Supported DataFoundry data sources" width="100%">
135+
</p>
136+
137+
See the full list in [Supported Data Sources](docs/en/reference/supported-datasources.md).
138+
139+
## 🛡️ Security Boundary
141140

142-
## 🛣️ Progress And Roadmap
141+
- The model only receives governed context; datasource credentials, model API keys, and MCP tokens never enter `messages`, `context`, or `forwardedProps`.
142+
- All datasource access goes through Data Gateway — read-only by default, with SQL guardrails, row limits, timeouts, and field masking.
143+
- SQL audit logs, tool-call records, and event streams are fully persisted for after-the-fact review.
144+
- Production-grade multi-tenant auth, centralized secret management, monitoring, and deployment operations require a dedicated design for your environment — see [Security](docs/en/security.md).
143145

144-
- [ ] **Semantic data operating layer** - Build a durable business semantic layer for metrics, entities, joins, lineage, policies, and reusable analytical concepts.
145-
- [ ] **Autonomous analyst loops** - Let agents plan investigations, run controlled experiments, critique findings, and converge on evidence-backed conclusions.
146-
- [ ] **Evaluation and reliability lab** - Create repeatable NL2SQL, retrieval, tool-use, and end-to-end task benchmarks with regression gates and failure forensics.
147-
- [ ] **Multimodal knowledge fabric** - Unify tables, documents, notebooks, charts, images, logs, and generated files into one governed context and retrieval fabric.
148-
- [ ] **Agent app platform** - Expose DataFoundry as a platform for domain-specific analytical agents, reusable workflows, custom tools, and shareable agent apps.
149-
- [ ] **Enterprise control plane** - Add multi-tenant governance for identity, RBAC, approvals, audit export, policy-as-code, cost limits, and deployment operations.
146+
## 🗺️ Roadmap
150147

151-
### Recent Progress
148+
- [x] **Governed data-task workbench** — Web and TUI share one TypeScript agent runtime, CopilotKit / AG-UI event stream, replayable run history, SQL audit trail, and unified asset layer.
149+
- [x] **Safe data access foundation** — Datasource registration, connection testing, schema introspection, table preview, read-only SQL, masking, knowledge imports, MCP resources, skill packages, and model configuration.
150+
- [ ] **Unified semantic layer** — Durable metrics, entities, relationships, lineage, and policies, moving agents from "guessing fields" to "understanding business definitions" and from one-off SQL to a governable data operating layer.
151+
- [ ] **Autonomous analyst loops** — Agents that plan investigations, run controlled experiments, critique findings, and converge on evidence-backed conclusions.
152+
- [ ] **Evaluation and reliability lab** — NL2SQL, retrieval, tool-use, and end-to-end task benchmarks with regression gates and failure forensics.
153+
- [ ] **Enterprise control plane** — Identity, RBAC, approvals, audit export, policy-as-code, and cost limits.
152154

153-
- **2026-07-01: DataFoundry first complete release** - DataFoundry now ships as a full-stack governed AI data workbench: a Web workbench and TUI share one TypeScript agent runtime, CopilotKit / AG-UI event stream, replayable run history, task state, SQL audit trail, artifact output, and unified file asset layer. The first release connects natural-language analysis to real datasource registration, connection testing, schema introspection, table preview, read-only SQL execution, row limits, masking, knowledge imports, workspace files, MCP resources, packaged skills, a built-in data-analysis skill, and model configuration. In short, DataFoundry has moved from an agent demo into an extensible data-agent operating foundation for safe, traceable, and reusable analytical work.
155+
Roadmap discussions are welcome in issues and discussions.
154156

155157
## 📚 Documentation
156158

157-
<table>
158-
<tr>
159-
<td><a href="docs/zh/quick-start.md"><strong>Quick Start</strong></a><br/>Install, configure a model key, and run the workbench.</td>
160-
<td><a href="docs/zh/overview.md"><strong>Product Overview</strong></a><br/>Understand the product positioning and analysis workflow.</td>
161-
</tr>
162-
<tr>
163-
<td><a href="docs/zh/reference/supported-datasources.md"><strong>Supported Data Sources</strong></a><br/>Datasource types, fields, and connection boundaries.</td>
164-
<td><a href="docs/zh/reference/agent-runtime.md"><strong>Agent Runtime</strong></a><br/>CopilotKit / AG-UI run input, events, and safety boundaries.</td>
165-
</tr>
166-
<tr>
167-
<td><a href="docs/zh/reference/rest-api.md"><strong>REST API</strong></a><br/>HTTP endpoints for local development and integration.</td>
168-
<td><a href="docs/zh/architecture/overview.md"><strong>Architecture</strong></a><br/>High-level runtime, Data Gateway, files, knowledge, and artifacts.</td>
169-
</tr>
170-
</table>
171-
172-
## 🧪 Status
173-
174-
DataFoundry is under active development. Current code, public docs, and passing smoke checks are the source of truth.
159+
| Goal | Read |
160+
| --- | --- |
161+
| Run the local demo | [Quick Start](docs/en/quick-start.md) |
162+
| Understand positioning and scope | [Overview](docs/en/overview.md) · [Capabilities](docs/en/capabilities.md) |
163+
| Use the Web / TUI | [Web workbench guide](docs/en/guides/web-workbench.md) · [TUI guide](docs/en/guides/tui.md) |
164+
| Connect data sources | [Data sources guide](docs/en/guides/data-sources.md) · [Supported data sources](docs/en/reference/supported-datasources.md) |
165+
| Integrate via API and runtime | [REST API](docs/en/reference/rest-api.md) · [Agent Runtime & AG-UI](docs/en/reference/agent-runtime.md) |
166+
| Understand architecture and security | [Architecture overview](docs/en/architecture/overview.md) · [Security](docs/en/security.md) |
167+
168+
## 🤝 Contributing
169+
170+
DataFoundry moves quickly, so small, well-scoped PRs are the easiest to merge:
171+
172+
1. Open an issue or discussion first for behavioral, protocol, datasource-adapter, or agent-policy changes.
173+
2. Keep each PR focused on one runtime boundary or feature area.
174+
3. Run `npm run build` plus the targeted smoke checks for what you touched (e.g. `npm run smoke:data-gateway`).
175+
4. Update docs when a change affects setup, APIs, datasource configuration, or user-visible output.
176+
5. Do not commit credentials, local databases, generated storage, or private benchmark data.
177+
178+
See [CONTRIBUTING.md](CONTRIBUTING.md) for details.
179+
180+
## 🙏 Built With
181+
182+
DataFoundry stands on the shoulders of these excellent open-source projects: [Mastra](https://github.com/mastra-ai/mastra) (agent runtime), [AG-UI](https://github.com/ag-ui-protocol/ag-ui) (event stream protocol), [CopilotKit](https://github.com/CopilotKit/CopilotKit) (agent UX), [Ink](https://github.com/vadimdemedes/ink) (terminal UI), and [MCP](https://modelcontextprotocol.io) (tool ecosystem).
183+
184+
## ⭐ Star History
185+
186+
If this project helps you, a star is the most direct way to support us.
187+
188+
<p align="center">
189+
<a href="https://star-history.com/#datagallery-lab/datafoundry&Date">
190+
<img src="https://api.star-history.com/svg?repos=datagallery-lab/datafoundry&type=Date" alt="Star History Chart" width="70%">
191+
</a>
192+
</p>
175193

176194
## 📄 License
177195

178196
Apache License 2.0. See [LICENSE](LICENSE).
197+
198+
> DataFoundry is under active development. Current code, public docs, and passing smoke checks are the source of truth.

0 commit comments

Comments
 (0)