Skip to content

Commit af061f9

Browse files
authored
feat(db): add LibreDB embedded provider (#95)
* docs(features): update headings and fix markdown indentation in FEATURES.md * docs(providers): add design specification for LibreDB provider * docs(providers): add approved design spec for LibreDB embedded provider * docs(plans): add LibreDB embedded provider implementation plan * feat(db): register libredb database type and @libredb/libredb dependency * feat(db): add LibreDBProvider lifecycle, metadata, and factory wiring * fix(db): validate libredb path on connect to prevent silent in-memory open * feat(db): implement libredb getSchema prefix grouping * feat(db): implement libredb query command grammar * fix(db): reject unmatched quotes in libredb command parsing * test(db): lock libredb monitoring surface * feat(ui): make libredb selectable with a file-path connection form * docs(providers): add libredb provider reference (tri-sync) * docs(db): clarify disconnect close-guard comment in libredb provider * feat(libredb): catalog-aware schema and reserved-namespace filtering (@libredb/libredb 0.0.3) Upgrade the embedded LibreDB provider for @libredb/libredb 0.0.3, which adds a persisted catalog and a public reserved-namespace contract. - Catalog-aware getSchema: read catalog(db) to render faithful per-kind views. Relational tables show their declared columns and types (primary key marked); document collections show a generic id/document pair; uncataloged keys keep the raw key/value view. Cataloged-but-empty namespaces are surfaced too. The query grammar (get/put/delete/prefix/range) is unchanged. - Reserved-namespace filtering: internal keys (the catalog, and any future reserved sub-namespace) are excluded from getSchema and from range/prefix query results via the package's pinned isReservedKey predicate, accessed through the lazily-loaded module rather than a hardcoded prefix. isReservedKey is marker-based (U+0000), so it hides the entire reserved namespace, not just catalog entries; the database forbids user namespace names under the marker (assertUserName), so no user data can be hidden. - Require @libredb/libredb ^0.0.3. Tri-sync: provider code, integration tests (catalog-aware, reserved-namespace, and a widening case; 22 pass), and docs/providers/libredb.md updated together. Gate green: typecheck, lint (0 errors), test, build, build:lib. * docs(archived): add superseded design spec for LibreDB provider * fix(libredb): address PR review notes - ConnectionModal: correct font-mediumr -> font-medium on the new file-path label - docs/providers/libredb.md: use 'bun run test' (not bare 'bun test') per CLAUDE.md - docs/archived design sketch: connection example uses the 'database' field, not 'path' - provider: drop redundant prepareQuery override (base default suffices; not called by query()) * fix(libredb): address follow-up PR review notes - archived design doc: fix relative link to ../providers/libredb.md (was broken from docs/archived/) - ConnectionModal: use a provider-neutral file-path placeholder (the form is shared by all file-based providers, not just libredb) * chore(.gitignore): add data/ directory to ignore list * docs(providers): add libredb to DATABASE_PROVIDERS overview * feat(libredb): wire schema-explorer Scan/Generate Command for libredb The shared query generators treated every queryLanguage:'json' provider as MongoDB, so libredb's menu actions emitted unrunnable Mongo find JSON. Add an optional ProviderCapabilities.queryDialect discriminator (undefined for all existing providers -> zero behavior change) and route generation to LibreDB commands when queryDialect==='libredb': - Scan Keys -> 'prefix <group>:' (or 'get <name>' for a bare key) - Generate Command -> a runnable, comment-free get/put/delete cheatsheet Open-closed: no other provider touched. Covered by query-generators unit tests and the libredb capabilities test; docs/providers/libredb.md section 5.3 added. * feat(libredb): explanatory, runnable Generate Command cheatsheet Make the schema-explorer 'Generate Command' output self-explanatory and directly runnable: - Parser skips blank and '#' comment lines and runs the first real command, so a commented multi-line cheatsheet runs directly (select a line, or run-all runs the first command). Comments only count when a line starts with '#'. - Generator emits a use-case comment above each command, and every command line is a concrete, directly-runnable example (no <placeholder> tokens). The put example value is column-aware: JSON object from a relational table's columns, a small object for a document collection, or a plain string for raw kv. Covered by query-generators unit tests and new provider comment-handling tests; docs/providers/libredb.md 5.1/5.3 updated. * feat(libredb): dedicated Monaco syntax highlighting for libredb command tabs libredb tabs reused the JSON Monaco language, so commands rendered flat (and '#' comments / verbs were not coloured). Add a first-class 'libredb' tab type (mirroring the existing 'redis' precedent), driven off capabilities.queryDialect, and a minimal Monaco language (verbs as keywords, '#' line comments, strings, numbers) that maps onto the existing db-dark theme. The editor language is derived from the tab type at both render sites (standalone Studio + embedded StudioWorkspace), so highlighting works in both. No other provider affected. Extended the QueryEditor test's Monaco mock with the languages API now used in beforeMount. * fix(libredb): drop unused LIBREDB_LANGUAGE_ID export (knip) * fix(libredb): address Copilot review — path hardening, catalog reconciliation, editor polish - connect() now resolves + validates the file path (path.resolve + reject traversal/null-byte), mirroring the SQLite provider; the resolved path is reused for open()/statSync. Closes the CodeQL 'uncontrolled data in path' alert and the sibling-parity gap. (test: null-byte path rejected) - catalogEntryFor() only reconciles ':*' prefix groups; a bare (no-colon) key is never catalog-upgraded even if its name matches a namespace. (test: bare key stays key/value) - Monaco tokenizer: a key like 'users:1' is one identifier token, not 'users' + number. - Hide the Format button for languages without a formatter (libredb), instead of showing an affordance that does nothing. All local gates green: lint, typecheck, knip, test (18 groups), build, build:lib, helm. * fix(docker): bundle @libredb/libredb into the standalone runner image The libredb provider lazy-imports @libredb/libredb, so Next.js output-file tracing leaves it out of the standalone server bundle and a libredb connection would fail at runtime with 'package not available' inside the container. Copy it explicitly (same pattern as better-sqlite3) so the provider works in Docker. Verified: package present + file create/WAL/fsync succeed as the non-root nextjs user, persisting to /app/data.
1 parent ed42647 commit af061f9

29 files changed

Lines changed: 3135 additions & 39 deletions

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,3 +104,4 @@ npmjs-token
104104

105105
.token
106106
.sonar-token
107+
data/

Dockerfile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,12 @@ COPY --from=builder /usr/src/app/node_modules/bindings ./node_modules/bindings
6060
COPY --from=builder /usr/src/app/node_modules/file-uri-to-path ./node_modules/file-uri-to-path
6161
# prebuild-install is only needed at build time, not runtime
6262

63+
# Copy the embedded LibreDB database package. The libredb provider lazy-imports
64+
# it (await import('@libredb/libredb')) so it stays out of client bundles, but
65+
# that also means Next.js output-file-tracing does not include it in the
66+
# standalone server bundle — copy it explicitly so the provider works at runtime.
67+
COPY --from=builder /usr/src/app/node_modules/@libredb/libredb ./node_modules/@libredb/libredb
68+
6369
# Create non-root user for security
6470
RUN addgroup --system --gid 1001 nodejs && \
6571
adduser --system --uid 1001 nextjs && \

bun.lock

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/DATABASE_PROVIDERS.md

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,10 @@ src/lib/db/
3030
│ │ └── mssql.ts # SQL Server Strategy
3131
│ ├── document/ # Document Database Providers
3232
│ │ └── mongodb.ts # MongoDB Strategy
33-
│ └── keyvalue/ # Key-Value Providers
34-
│ └── redis.ts # Redis Strategy
33+
│ ├── keyvalue/ # Key-Value Providers
34+
│ │ └── redis.ts # Redis Strategy
35+
│ └── embedded/ # Embedded (in-process) Providers
36+
│ └── libredb.ts # LibreDB Strategy
3537
└── utils/
3638
├── pool-manager.ts # Connection pool utilities
3739
└── query-limiter.ts # SELECT auto-LIMIT (analyzeQuery/applyQueryLimit)
@@ -48,10 +50,11 @@ BaseDatabaseProvider (abstract)
4850
│ ├── OracleProvider │
4951
│ └── MSSQLProvider │
5052
├── MongoDBProvider ────────────────────────┤ Document Database
51-
└── RedisProvider ──────────────────────────┘ Key-Value Store
53+
├── RedisProvider ──────────────────────────┤ Key-Value Store
54+
└── LibreDBProvider ────────────────────────┘ Embedded (key-value)
5255
```
5356

54-
`SQLBaseProvider` provides SQL-specific helpers (LIMIT injection, identifier escaping, placeholder generation). Non-SQL databases like MongoDB and Redis extend `BaseDatabaseProvider` directly.
57+
`SQLBaseProvider` provides SQL-specific helpers (LIMIT injection, identifier escaping, placeholder generation). Non-SQL databases like MongoDB, Redis, and LibreDB extend `BaseDatabaseProvider` directly. LibreDB is embedded (opened in-process from a file, like SQLite) but, having no SQL, it is a key-value-style provider rather than a SQL one.
5558

5659
**Key files:**
5760

@@ -90,7 +93,7 @@ QueryEditor /api/db/query
9093

9194
## Supported Databases
9295

93-
Seven providers are supported. For the per-provider reference (driver, pooling, query format,
96+
Eight providers are supported. For the per-provider reference (driver, pooling, query format,
9497
monitoring, limitations, …) see the prime docs in **[`docs/providers/`](./providers/README.md)**:
9598

9699
| Provider | type-id | Family | Reference |
@@ -102,6 +105,7 @@ monitoring, limitations, …) see the prime docs in **[`docs/providers/`](./prov
102105
| SQLite | `sqlite` | SQL (embedded) | [providers/sqlite.md](./providers/sqlite.md) |
103106
| Redis | `redis` | Key-Value | [providers/redis.md](./providers/redis.md) |
104107
| MongoDB | `mongodb` | Document | [providers/mongodb.md](./providers/mongodb.md) |
108+
| LibreDB | `libredb` | Embedded (key-value) | [providers/libredb.md](./providers/libredb.md) |
105109

106110
## Core Interface
107111

@@ -351,6 +355,7 @@ is kept in sync with its per-provider doc). Don't copy a skeleton from this guid
351355
| Embedded / file SQL | `SQLBaseProvider` | `sqlite.ts` | [sqlite.md](./providers/sqlite.md) |
352356
| Document store | `BaseDatabaseProvider` | `mongodb.ts` | [mongodb.md](./providers/mongodb.md) |
353357
| Key-value store | `BaseDatabaseProvider` | `redis.ts` | [redis.md](./providers/redis.md) |
358+
| Embedded (in-process, no wire protocol) | `BaseDatabaseProvider` | `embedded/libredb.ts` | [libredb.md](./providers/libredb.md) |
354359

355360
**Implement the abstract methods** from the `DatabaseProvider` interface: `connect`, `disconnect`,
356361
`query`, `getSchema`, `getHealth`, `runMaintenance`, plus the monitoring set (`getOverview`,

docs/FEATURES.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# LibreDB Studio Expert Features
22

3-
## 🚀 Implemented Features
3+
## Implemented Features
44

55
### 1. Monaco SQL IDE Experience
66
* **VS Code Engine:** Integrated Monaco Editor for a professional coding environment.
@@ -122,6 +122,6 @@
122122
* **Search & Filter:** Locate tables by name, with compact and detailed view modes.
123123
* **Export:** Save diagrams as SVG or PNG for documentation.
124124

125-
## 🛠 Roadmap
125+
## Roadmap
126126

127127
Upcoming "expert" features are tracked under [`docs/backlogs/`](backlogs/).
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# LibreDB Studio Provider — Design Spec (Option A: embedded raw-KV browser)
2+
3+
> **SUPERSEDED.** This precursor sketch has been superseded by:
4+
> - [`docs/superpowers/specs/2026-06-24-libredb-embedded-provider-design.md`](../superpowers/specs/2026-06-24-libredb-embedded-provider-design.md) — the approved design spec
5+
> - [`docs/providers/libredb.md`](../providers/libredb.md) — the provider reference documentation (implementation is complete)
6+
>
7+
> Original status note: design spec for a future implementation session. NOT built yet.
8+
> Lets LibreDB Studio connect to a local LibreDB database file (the embedded `libredb` package —
9+
> the open-source database in the sibling `libredb-database` repo) by importing the package and
10+
> opening the file IN the Studio server process — the same embedded pattern Studio uses for SQLite.
11+
> `libredb-database` reference document path: /home/cevheri/projects/libredb/libredb-database/DESIGN.md, /home/cevheri/projects/libredb/libredb-database/README.md
12+
## 1. Why embedded, not a connection
13+
14+
LibreDB has no server and no wire protocol — that is a deliberate, locked design decision (embedded-first). So Studio connects the SQLite way: import the `libredb` package and open the file in-process. The file must live on the Studio server's filesystem. There is no "remote LibreDB" to connect to. (If a networked LibreDB is ever wanted, it would be a separate optional server product — not this provider.)
15+
16+
## 2. The key constraint: what this provider can and cannot show
17+
18+
A LibreDB file is raw ordered key-value bytes. The lens (kv / document / relational) and any relational table schema live in application code, NOT on disk. So **Option A shows the file as a raw ordered-KV store** — keys grouped by prefix, values shown as-is. It does NOT reconstruct relational tables/schemas, because that information is not persisted.
19+
20+
Richer per-kind views (real collections, tables, schemas) are "Option B" and are **blocked on a database-side catalog** (see `libredb-database` DESIGN.md section 6.3). When that ships, this provider can read `catalog(db)` and present faithful views; until then, raw KV is the honest representation.
21+
22+
## 3. Shape: like the Redis/Mongo providers, not the SQL editor
23+
24+
LibreDB has no SQL, so this is a NoSQL-style provider:
25+
26+
- **Base class:** extends `BaseDatabaseProvider` (NOT `SQLBaseProvider`). Drive behaviour through capabilities/labels; no `=== 'libredb'` type-checks outside the provider class (per CLAUDE.md).
27+
- **Connection:** `{ type: 'libredb', database: '/path/to/data.libredb' }` — file-based, reusing the `database` field like SQLite. Server-side only (touches the filesystem).
28+
- **`getSchema()`:** open the file (`open({ path })`), scan the ordered KV via the package's range/prefix API, and group keys by their `:`-prefix into "tables" — the same convention the Redis provider uses for key prefixes. Each group lists its keys; values are rendered as-is and JSON pretty-printed when parseable.
29+
- **Query model:** a structured / command form (NOT SQL) — `get`, `put`, `delete`, `range(start, end)`, `prefix(p)` — mapping onto the package's kv-level access. Reads return `{ key, value }` rows; writes go through the package (file-backed writes are durable / fsync'd).
30+
- **Capabilities/labels:** declare no-SQL; "tables" = key prefixes; expose read + write. The kernel's multi-key `transact` may be surfaced as an advanced affordance later (see open forks).
31+
32+
## 4. Tri-sync deliverables (per CLAUDE.md provider invariant)
33+
34+
All three in the same PR, 1:1 for type-id `libredb`:
35+
36+
- **Code:** `src/lib/db/providers/<family>/libredb.ts` (a new family, or grouped with the embedded/file providers).
37+
- **Docs:** `docs/providers/libredb.md` (mirrors the code; this design spec is the precursor, not that doc).
38+
- **Tests:** `tests/integration/db/libredb-provider.test.ts` — open a temp file via the package, write a few keys across prefixes, assert `getSchema()` groups them and queries return them; test JSON-value rendering and a write round-trip.
39+
40+
## 5. Dependency & build notes
41+
42+
- Add `libredb` as a dependency. It is a TypeScript/ESM package; verify it works in Studio's Bun/Next server runtime **and** survives `build:lib` (tsup) so the embedded `@libredb/studio` npm package still builds.
43+
- The provider is server-side only, like the SQLite and storage providers — it must not be imported into client bundles.
44+
- LibreDB's API is synchronous; wrap its calls to satisfy the provider interface's async contract (they resolve immediately).
45+
46+
## 6. Open forks (ratify at build time)
47+
48+
- Provider "family" name: a new `embedded`/`file` family vs grouping under an existing one.
49+
- Whether to expose the kernel `transact` (multi-key atomic writes) in the query UI now or defer.
50+
- Read-only vs read-write in v1 — recommend read-write (the package supports it; matches other providers).
51+
- In-memory connections (`open()` with no path) are ephemeral and empty per Studio session, so likely omit; the provider targets a persisted file.
52+
53+
## 7. Later (depends on libredb-database)
54+
55+
When the database-side catalog (libredb-database DESIGN.md 6.3) ships, extend this provider to read `catalog(db)` and present faithful per-kind views: document collections, relational tables with their columns/types, and kv namespaces — replacing the raw-prefix grouping with real schema-aware browsing.

0 commit comments

Comments
 (0)