docs(providers): add comprehensive MongoDB reference (final provider)#82
Conversation
Add docs/providers/mongodb.md documenting the MongoDB document provider (official mongodb driver). Covers the JSON/MQL query format and all operations, BSON serialization, sampling-based flat schema inference, the find-100-cap vs unbounded aggregate, MongoClient pooling, validate/compact/dbCheck/killOp maintenance, serverStatus/currentOp/$indexStats monitoring with real index scans, overridden collection labels, and known gaps (sampled schema, unbounded aggregate, no EXPLAIN/ transactions/cancel, collStats deprecation, privilege-gated monitoring). Completes the docs/providers/ set: postgres, mysql, oracle, mssql, sqlite, redis, mongodb.
There was a problem hiding this comment.
Pull request overview
Adds a comprehensive MongoDB provider reference under docs/providers/, documenting the provider’s JSON/MQL query format, BSON serialization, schema inference, monitoring, maintenance, and test strategy to match the current implementation.
Changes:
- Added
docs/providers/mongodb.mdas the provider’s single, end-to-end reference. - Documented MongoDB-specific behaviors (sampling-based schema inference,
finddefault limit, monitoring/maintenance mappings, connection-string handling).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| Supported operations: `find`, `findOne`, `aggregate`, `count`, `distinct`, `insertOne`, `insertMany`, | ||
| `updateOne`, `updateMany`, `deleteOne`, `deleteMany`. See the | ||
| [`API_DOCS.md` MongoDB Query Format](../API_DOCS.md) section (under `POST /api/db/query`) and | ||
| [`CLAUDE.md`](../../CLAUDE.md) for the request shape. |
There was a problem hiding this comment.
Correct — real code↔doc drift. MongoDBProvider expects operation: "count" (it calls countDocuments internally) and supports distinct (field from the first key of options.projection). Fixed docs/API_DOCS.md in this PR: replaced countDocuments with count and added distinct. (mongodb.md itself already listed both correctly.)
The MongoDBProvider expects operation: "count" (not "countDocuments") and also supports "distinct" (field taken from options.projection). Correct the API_DOCS.md operations list accordingly. Surfaced during the mongodb.md review (#82).
Verified each review claim against mongodb.ts / route.ts before documenting: - prepareQuery is NOT a true no-op — it returns limit: options.limit||100 which the /api/db/query route uses for hasMore/pagination; clarified wording - fixed off-by-one citation (find 100-cap is line 252, not 251) - per-operation options handling: findOne honours only projection (sort/skip/limit silently ignored); aggregate ignores options entirely; distinct takes its field from the first key of options.projection (no dedicated field param) — documented the required shape and flagged as limitations - getPerformanceMetrics "queries/sec" is actually total ops/sec (q+i+u+d ÷ uptime) - active-sessions "user" column shows op.client (host:port), not a real user - getIndexStats indexType only distinguishes text vs btree (hashed/geo/wildcard mislabelled); serializeDocument normalises only a subset of BSON types - unlimited option ignored (hasMore may be wrong); getSchema serial round-trips - softened the "never use bun run test" note to match CLAUDE.md's pre-commit gate; tidied the connection-string template
|
…& schema-timeout fix (0.9.27) (#89) * fix(sql): quote schema-qualified table names per-segment The dialect-aware quoteIdentifier treated a schema-qualified table name as a single identifier, so 'employees.department' became '"employees.department"' — which Postgres reads as one relation literally named with a dot, failing with a query error. Sidebar 'select top 100' on any schema-qualified table was broken. Add quoteQualifiedName(): split on '.', quote each segment only-when-needed, and rejoin. 'employees.department' stays unquoted; 'public.Order' becomes 'public."Order"'. Use it for table names in the query generators and the data profiler (columns still use single-identifier quoting). Verified end-to-end against a Neon Postgres database: the generated 'SELECT * FROM employees.department LIMIT 50;' runs, whereas the old quoted form failed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(package): bump version to 0.9.23 * chore(package): bump version to 0.9.24 * fix(db/postgres): prevent schema introspection timeout on large schemas getSchema() ran a single query that joins five information_schema-based CTEs (columns, PKs, foreign keys, indexes + table sizes). PostgreSQL 12+ inlines single-reference CTEs, and because the planner estimates rows=1 for information_schema views it chose nested-loop joins that re-execute the expensive fk/index CTEs many times. On schemas with 100+ tables this exploded to ~295s and always hit the connection's 60s statement_timeout, so the schema tree never loaded. Two complementary fixes: 1. Mark all CTEs AS MATERIALIZED so each is computed once instead of being re-executed inside nested loops (~295s -> ~2.6s on a 122-table schema). 2. Split schema loading into two phases so a slow/failing stats query can never block the table list: - getSchemaList() -> tables + columns + PKs (fast, ~50ms) [/api/db/schema/list] - getSchemaRelations() -> foreign keys + indexes (heavy, ~2.3s) [/api/db/schema/relations] The client renders the tree from the fast list immediately, then merges relations asynchronously. If relations fail/time out they are logged and skipped — the table list stays intact. getSchema() is kept (MATERIALIZED) for consumers that need the full schema (diff, AI context). New optional provider methods getSchemaList/getSchemaRelations fall back to getSchema()/[] for providers that don't implement them. * test(db/schema): cover two-phase schema introspection (list + relations) Adds regression coverage for the schema-introspection timeout fix (0d60c47), which split schema loading into a fast structural list and a heavy, best-effort relations phase. None of the new code paths were previously tested. Provider (tests/integration/db/postgres-provider.test.ts, +11): - getSchemaList(): returns columns + PKs but intentionally empty indexes/foreignKeys, isPrimary detection, non-public schema prefixing, negative reltuples row_count clamped to 0, and null/empty columns. - getSchemaRelations(): FK/index data keyed by display name, non-public prefixing on both the table and the referenced table, public references kept bare, empty/null fk-index tolerance, and null index columns coerced to []. API routes (new files, 15 tests): - /api/db/schema/list: uses getSchemaList() when present, falls back to getSchema() otherwise, plus auth (401), empty-body (400), missing-type (400), ConnectionError (503) and DatabaseError/generic (500). - /api/db/schema/relations: uses getSchemaRelations() when present, returns [] fallback otherwise, plus the same auth/validation/error matrix. Hook (tests/hooks/use-connection-manager.test.ts, +4): - phase 1 renders the table list, phase 2 merges FKs/indexes by table name; - relations failure does NOT wipe the table list and shows no error toast (the core resilience contract of the fix); - phase 1 failure short-circuits so the heavy relations endpoint is never hit; - tables absent from the relations payload are left unchanged. Full suite green: lint (0 errors), typecheck, 2050 unit/api/integration + 251 hooks + component groups, 0 failures. * fix(db/postgres): hoist schema SQL to module scope + dedupe CTEs for Sonar gate The SonarCloud PR quality gate failed on #71 with 53.3% coverage and 6.6% duplication on new code. Both stem from how the schema introspection SQL was written, not from missing tests. Coverage: the getSchema/getSchemaList/getSchemaRelations SQL lived in multi-line template literals *inside the method bodies*. bun's coverage instruments the interior lines of such literals as 0-hit in any test process that imports the module without exercising the method; the merged lcov (per-file isolated runs) then reports those SQL lines as uncovered even though the methods are tested. Verified empirically: a module-level `const` template is evaluated once at load and reported covered everywhere, while an in-body one is not. Hoisting the three queries to module scope takes postgres.ts new-code line coverage 28% -> 100% (overall new-code coverage 53.9% -> ~99%). Duplication: the hoisted queries shared the same CTEs (tables/columns/pk across full+list, fk/index across full+relations). Extracted each CTE to a single reusable fragment const and compose the three queries from them, removing the duplicated blocks. The SQL text produced is unchanged. Tests: the two route specs intentionally keep their mock.module setup inline (a shared helper breaks bun's per-file mock scoping under the non-isolated `bun run test`, clobbering @/lib/db mocks across files). Added an empty-object ({}) body case to each for the last uncovered branch. Test mock/setup boilerplate is expected harness scaffolding, so tests are excluded from Sonar copy-paste detection (sonar.cpd.exclusions). No behavioural change: same SQL, same provider/route/hook logic. Verified: lint (0 errors), typecheck, 2052 unit/api/integration + hooks + component groups (0 failures), production build, full coverage regenerated. * chore(package): bump version to 0.9.25 * refactor(api/schema): extract shared handler to remove route duplication The Sonar PR gate still flagged 6.2% duplication on new code: the /api/db/schema/list and /api/db/schema/relations routes shared ~39 identical lines of body parsing, auth, connection resolution and error handling (the only difference is which provider method they call). Extract that boilerplate into handleSchemaRequest() in src/lib/api/schema-route.ts; each route is now a thin wrapper that passes a `load` callback selecting the provider method (with the getSchemaList->getSchema and getSchemaRelations->[] fallbacks preserved). Removes both 39-line duplicated blocks. Coverage holds at 100% on new code (the shared handler is fully exercised by the existing route specs). Verified: typecheck, 2052 core tests + 0 failures, coverage regenerated (schema-route.ts 32/32 lines). * fix(db/postgres): join constraint_column_usage on constraint_schema for cross-schema FKs Copilot review on #71 flagged a pre-existing bug in the FK introspection (from 0d60c47, relocated verbatim into CTE_FK_INFO during the dedupe): JOIN information_schema.constraint_column_usage ccu ON ccu.constraint_name = tc.constraint_name AND ccu.table_schema = tc.table_schema -- wrong For a foreign key, constraint_column_usage reports the *referenced* (parent) table's columns, so ccu.table_schema is the referenced schema while tc.table_schema is the referencing schema. Joining them requires the two to be equal, which silently drops every cross-schema foreign key and makes referencedSchema always equal the referencing schema — defeating the non-public referenced-table prefixing (e.g. billing.accounts). Join on the constraint's own schema instead (ccu.constraint_schema = tc.constraint_schema). Because the FK CTE is now single-sourced, this fixes both getSchema() and getSchemaRelations(). Added a SQL-level regression guard: the existing specs mock the query result, so they can't catch a bad join; the new test asserts the emitted SQL joins on constraint_schema and not table_schema. Real cross-schema FK behaviour should be confirmed against a live Postgres via E2E. * fix(ci): restore missing test:ci script used by the npm publish gate The npm-publish workflow's Validate job runs `bun run test:ci`, but the script was dropped from package.json somewhere after v0.9.19 (it was added in #66). The v0.9.25 tag push therefore failed with "Script not found test:ci", blocking the npm publish (the GitHub release, tag and Docker images published fine). Restore it to the per-file-isolated form the publish gate expects: test:ci = bash tests/run-core.sh && bun run test:components This mirrors ci.yml's reliable path and avoids bun's process-wide mock.module load-order flakiness that the single-process `bun run test` is prone to. * fix(schema-explorer): update select action label from 'Select Top 100' to 'Select Top 50' * bump: upgrade version to 0.9.26 * fix(e2e): make admin-dashboard tests seed-state-independent The 'default tab is overview' and 'can switch to operations tab' E2E tests asserted on empty-state-only copy ('Command Center', the 'Select connection' placeholder), which disappears once seed connections load the populated dashboard. They passed in CI (no seed file) but failed locally with SEED_CONFIG_PATH set. Add stable data-testid hooks (admin-content-overview/-operations) to the TabsContent wrappers and assert on testid visibility + tab aria-selected, so the tests hold in both empty and populated states. Verified: 32/32 full E2E pass with seeds loaded, 8/8 admin pass with seeds disabled. * feat(redis): add Redis provider support with query format and error handling - Introduced Redis connection configuration and query formats in CLAUDE.md and API_DOCS.md. - Added support for both plain Redis commands and JSON command objects. - Implemented schema introspection and health metrics retrieval for Redis. - Enhanced Redis provider tests to cover error handling for malformed commands and Redis-side errors. * bump: upgrade app version to 0.9.27 * docs(providers): add comprehensive Redis provider reference (#75) * docs(providers): add comprehensive Redis provider reference Add docs/providers/redis.md — the single reference point for the Redis provider covering design rationale, architecture (Strategy Pattern + base class), connection, query interface, schema mapping, monitoring, capabilities, error handling, testing, usage, and known limitations. Also serves as the template for the other provider docs under docs/providers/. * docs(providers): address review — TYPE sampling, analyze line count, TLS wording * docs(providers): add comprehensive PostgreSQL provider reference (#76) * docs(providers): add comprehensive PostgreSQL provider reference Add docs/providers/postgresql.md documenting the PostgreSQL provider — the reference implementation for the SQL provider family. Covers the SQLBaseProvider layer, connection pooling/SSL, automatic LIMIT injection, transactions and query cancellation, the MATERIALIZED-CTE two-phase schema introspection (and its performance/coverage rationale), pg_stat_* monitoring with graceful degradation, maintenance, capabilities, error mapping, testing, and known limitations. * docs(providers): rename to postgres.md to match the canonical type-id Convention: doc filename mirrors the code type-id / source filename (providers/sql/postgres.ts -> docs/providers/postgres.md) for a 1:1 code<->doc mapping. The official name "PostgreSQL" is retained in the document title and prose. * docs(providers): address review — queryTimeout source, SSL precedence, getHealth fallback, tx pooling, config validation - queryTimeout is ProviderOptions.queryTimeout (not pool), applied as statement_timeout - SSL step 2 also triggers on options.ssl === true, not just cloud hosts - getHealth returns a placeholder row (no pg_stat_activity fallback — that's getSlowQueries) - transaction client is checked out from the pool, not held "outside" it - the two connection forms are not mutually exclusive; connection string wins when both given * docs(providers): address review — test:ci runner & statement_timeout error class - Testing: `bun run test` runs the core group in one process (mock.module flaky); CI uses `test:ci` (run-core.sh per-file isolation) / `test:coverage`, not `bun run test` - Error mapping: PG statement_timeout emits "canceling statement due to statement timeout" → classified as QueryCancelledError (cancellation check precedes timeout), not TimeoutError; TimeoutError is for generic/acquire timeouts - getHealth: clarified as placeholder path (not pg_stat_activity fallback) in coverage notes * docs: add provider tri-sync invariant to CLAUDE.md (#77) Providers are the lifeblood of the project: for each provider, code, docs, and tests form a 1:1 triad keyed by the canonical type-id, and a change to any one side must sync the other two in the same PR. Documented as a callout under Architecture → Key Patterns → Database Abstraction. * docs(providers): add comprehensive MySQL provider reference (#78) * docs(providers): add comprehensive MySQL provider reference Add docs/providers/mysql.md documenting the MySQL provider, framed as a diff against the PostgreSQL SQL reference. Covers the mysql2 pool config (only `max` honored; no statement_timeout wiring), backtick quoting, binary→hex row sanitization, N+1 single-database schema introspection (no two-phase split), KILL QUERY cancellation, transactions, performance_schema monitoring with its three distinct degradation modes, analyze/optimize/check/kill maintenance, error mapping, testing, and known limitations. * docs(providers): address review — connectionString bypass & TimeoutError nuance - buildPoolConfig connectionString path returns {...baseConfig, uri} and skips the discrete branch, so ssl/connection.ssl/auto-SSL and timezone are ignored (must be encoded in the URI) — now documented in §4.2/§4.3 - error handling: mapDatabaseError DOES emit TimeoutError for any "timeout"/ "timed out" driver message (e.g. lock-wait timeout); the MySQL-specific gap is only the lack of a server-side timeout derived from queryTimeout * docs(providers): address review — rowCount for non-SELECT & MAX_EXECUTION_TIME - rowCount is rows.length only for array (SELECT) results; non-SELECT returns a ResultSetHeader and the provider reports rowCount: 0 (affected-rows not surfaced) - future-work: MAX_EXECUTION_TIME is the per-statement limit; wait_timeout is idle-connection only and wouldn't bound query execution — dropped it * docs(providers): add comprehensive Oracle provider reference (#79) * docs(providers): add comprehensive Oracle provider reference Add docs/providers/oracle.md documenting the Oracle provider (oracledb Thin mode), framed as a diff vs the PostgreSQL SQL reference. Covers EZConnect service-name connect string, FETCH FIRST pagination, owner-scoped 5-query schema introspection, connection.break() cancellation, transactions without an auto-rollback timeout, the absence of an SSL config path (TLS via wallet/connect string), privilege-guarded V$-view monitoring, DBMS_STATS/index-rebuild/kill maintenance, overridden Oracle UI labels, ORA-* error mapping, testing, and known limitations. * docs(providers): address review — queryTimeout is a separate ProviderOptions field, not part of DEFAULT_POOL_CONFIG * docs(providers): document Oracle data-type & feature gaps (LOB, NUMBER, EXPLAIN, callTimeout, privileges) Add a §5.3 on CLOB/BLOB (returned as Lob stream objects, not configured via fetchAsString/fetchAsBuffer) and NUMBER precision loss, and expand Known Limitations with: LOB rendering gap, large-NUMBER precision, EXPLAIN advertised but not implemented for Oracle (UI builder is pg/mysql only), connection.callTimeout as the fix for no server-side timeout, ALTER SYSTEM / V$ privilege requirements, and the module-global oracledb settings side effect. * docs(providers): clarify Oracle connectionString is EZConnect/TNS, not oracle:// connectionString is passed straight to oracledb.createPool({ connectString }), which expects EZConnect/TNS. The oracle:// scheme is only a UI paste-parser convenience that decomposes into discrete fields and never reaches the driver. * docs(providers): add comprehensive Microsoft SQL Server (mssql) reference (#80) * docs(providers): add comprehensive Microsoft SQL Server (mssql) reference Add docs/providers/mssql.md documenting the SQL Server provider (node-mssql), framed as a diff vs the PostgreSQL SQL reference, and remove the stray sqlserver.md placeholder (type-id naming: file = mssql, prose = "SQL Server"). Covers encrypt-by-default/Azure-aware TLS, TOP / OFFSET-FETCH pagination, five-query cross-schema introspection, the fully-wired pool+requestTimeout (server-side query timeout), rowsAffected-based rowCount, real blocked-session detection and real index scan counts, mssql.Transaction (no auto-rollback timeout), request.cancel() cancellation, DMV monitoring with privilege guards, analyze/check/optimize/kill maintenance, overridden labels, SQL-Server error mapping, and known gaps (connectionString ignored by buildConfig, EXPLAIN advertised-not-implemented, non-Azure cert trust, unsanitized binary). * docs(providers): address review — requestTimeout is driver-enforced, soften connectionString wording - requestTimeout is enforced client-side by the mssql/Tedious driver, not a server-enforced statement timeout (Postgres statement_timeout is the latter); reworded §3.5 and §11 accordingly (+ fixed the anchor link) - connectionString-only config: reworded from "silently connect to localhost" to targeting an unintended server and likely failing on missing fields * docs(providers): document SQL Server data-type & enterprise gaps Add §5.3 (parameter type inference, BIGINT/DECIMAL/MONEY precision loss, binary Buffer, single-recordset) and expand Known Limitations with numeric precision, untyped parameter binding, missing Always On options (MultiSubnetFailover / ApplicationIntent ReadOnly routing), and Azure SQL DMV/DBCC caveats. * docs(providers): address review — grammar, cancel wording, index-scan claim, cancellation error class - "does bound" -> "does impose a request timeout" - cancelQuery: returns false if no tracked Request, else true if cancel() didn't throw (doesn't confirm cancellation took effect) - index scans: Postgres ALSO reports real counts (pg_stat_user_indexes.idx_scan); SQL Server is unique only for blocked-session detection - error table: cancellation-pattern messages map to QueryCancelledError before the timeout check * docs(providers): add comprehensive SQLite reference (#81) * docs(providers): add comprehensive SQLite reference (with deployment-constraint framing) Add docs/providers/sqlite.md. Leads with the strategic framing the product needs: SQLite-as-target is an embedded, server-local, bun:sqlite-only engine — fit for self-hosted / local-dev / edge and zero-config trials, NOT a multi-tenant SaaS target. Disentangles the two SQLites (storage better-sqlite3 vs target bun:sqlite). Covers file-path resolution + traversal guard, connect PRAGMAs (WAL/FK), read/write dispatch, the absence of transactions/cancellation/pooling, single-schema PRAGMA introspection, minimal/estimated monitoring, vacuum/analyze/reindex/check maintenance, real-engine (:memory:) testing, and known limitations. * docs(providers): address review — stable anchor (drop emoji from heading) & accurate path-traversal note - Removed the⚠️ from the "Deployment constraint" heading (variation-selector made the GitHub anchor brittle); moved it into the body and fixed the §0 link - Corrected the path-traversal claim: resolved !== normalize(resolved) is a no-op (path.resolve already normalises), so only NUL bytes are rejected; documented the ineffective guard as a known limitation * docs(providers): add comprehensive MongoDB reference (final provider) (#82) * docs(providers): add comprehensive MongoDB reference (final provider) Add docs/providers/mongodb.md documenting the MongoDB document provider (official mongodb driver). Covers the JSON/MQL query format and all operations, BSON serialization, sampling-based flat schema inference, the find-100-cap vs unbounded aggregate, MongoClient pooling, validate/compact/dbCheck/killOp maintenance, serverStatus/currentOp/$indexStats monitoring with real index scans, overridden collection labels, and known gaps (sampled schema, unbounded aggregate, no EXPLAIN/ transactions/cancel, collStats deprecation, privilege-gated monitoring). Completes the docs/providers/ set: postgres, mysql, oracle, mssql, sqlite, redis, mongodb. * docs(api): fix MongoDB operations list to match the provider The MongoDBProvider expects operation: "count" (not "countDocuments") and also supports "distinct" (field taken from options.projection). Correct the API_DOCS.md operations list accordingly. Surfaced during the mongodb.md review (#82). * docs(providers): incorporate deep-review findings into mongodb.md Verified each review claim against mongodb.ts / route.ts before documenting: - prepareQuery is NOT a true no-op — it returns limit: options.limit||100 which the /api/db/query route uses for hasMore/pagination; clarified wording - fixed off-by-one citation (find 100-cap is line 252, not 251) - per-operation options handling: findOne honours only projection (sort/skip/limit silently ignored); aggregate ignores options entirely; distinct takes its field from the first key of options.projection (no dedicated field param) — documented the required shape and flagged as limitations - getPerformanceMetrics "queries/sec" is actually total ops/sec (q+i+u+d ÷ uptime) - active-sessions "user" column shows op.client (host:port), not a real user - getIndexStats indexType only distinguishes text vs btree (hashed/geo/wildcard mislabelled); serializeDocument normalises only a subset of BSON types - unlimited option ignored (hasMore may be wrong); getSchema serial round-trips - softened the "never use bun run test" note to match CLAUDE.md's pre-commit gate; tidied the connection-string template * docs: refresh SQL_ALIAS_COMPLETION.md to current code locations (#84) The alias-aware completion feature is live; this doc is its sole reference, so it's kept (not archived) — but the completion provider was refactored out of QueryEditor.tsx into src/lib/editor/sql-completions.ts. Update the module structure, Key Components, and Related Files accordingly; QueryEditor.tsx now hosts the editor and registers the provider. Core API (extractAliases/ resolveAlias) and types are unchanged. * docs: make docs/providers/ the prime provider reference; slim DATABASE_PROVIDERS.md (#83) * docs: make docs/providers/ the prime provider reference; slim DATABASE_PROVIDERS.md The per-provider docs under docs/providers/ are now the canonical reference for each database. This wires them up and removes the now-duplicated provider detail from DATABASE_PROVIDERS.md while preserving its unique value. - Add docs/providers/README.md: index of all 7 prime docs (provider/type-id/ family/driver/query-language table) + conventions + links to the architecture/ authoring guide and the API contract. - DATABASE_PROVIDERS.md is repurposed as the cross-cutting "architecture + how-to-add-a-provider" guide: the duplicated per-provider sections (Supported Databases table, MongoDB Query Format, Provider-Specific Features, Reference: Existing Providers) now point to the prime docs; fixed the stale SQLite driver (better-sqlite3 -> bun:sqlite for the target provider) and completed the architecture file-tree + provider hierarchy (oracle/mssql/redis). - README.md: add a Provider reference docs pointer and correct the SQLite driver. No content lost — the unique architecture overview, interface contracts, and the full "Adding a New Database Provider" guide remain in DATABASE_PROVIDERS.md. * docs: trim Step 2's inlined provider skeletons in DATABASE_PROVIDERS.md The "Adding a New Provider" guide inlined ~265 lines of synthetic SQL/non-SQL provider class skeletons (CockroachDB/Redis-like). Now that each provider has a detailed doc and the real source files are the best template, replace the skeletons with concise guidance: which base to extend, which real file to copy as a template (linked), the abstract methods to implement, and the metadata hooks to override. Kept the "what the base class gives you for free" / "what SQLBaseProvider adds" tables and the interface-contracts reference. DATABASE_PROVIDERS.md: 895 -> 583 lines; no longer carries code that can drift from the providers it describes. * docs: address review — type-id filename + mapDatabaseError query arg - Step 2: provider files are named by the canonical type-id (<type-id>.ts), aligning with docs/providers/README.md and the code/docs/tests convention - mapDatabaseError's 3rd arg is the raw query string (SQL or JSON per errors.ts), not "sql" — reworded so it's accurate for non-SQL providers too * docs: consolidate editor/UI docs + root cleanup (drift fixes, archive stale plan) (#87) * fix: correct schema-explorer select-limit label + make admin E2E seed-state-independent (#73) * fix(sql): quote schema-qualified table names per-segment The dialect-aware quoteIdentifier treated a schema-qualified table name as a single identifier, so 'employees.department' became '"employees.department"' — which Postgres reads as one relation literally named with a dot, failing with a query error. Sidebar 'select top 100' on any schema-qualified table was broken. Add quoteQualifiedName(): split on '.', quote each segment only-when-needed, and rejoin. 'employees.department' stays unquoted; 'public.Order' becomes 'public."Order"'. Use it for table names in the query generators and the data profiler (columns still use single-identifier quoting). Verified end-to-end against a Neon Postgres database: the generated 'SELECT * FROM employees.department LIMIT 50;' runs, whereas the old quoted form failed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(package): bump version to 0.9.23 * chore(package): bump version to 0.9.24 * fix(db/postgres): prevent schema introspection timeout on large schemas getSchema() ran a single query that joins five information_schema-based CTEs (columns, PKs, foreign keys, indexes + table sizes). PostgreSQL 12+ inlines single-reference CTEs, and because the planner estimates rows=1 for information_schema views it chose nested-loop joins that re-execute the expensive fk/index CTEs many times. On schemas with 100+ tables this exploded to ~295s and always hit the connection's 60s statement_timeout, so the schema tree never loaded. Two complementary fixes: 1. Mark all CTEs AS MATERIALIZED so each is computed once instead of being re-executed inside nested loops (~295s -> ~2.6s on a 122-table schema). 2. Split schema loading into two phases so a slow/failing stats query can never block the table list: - getSchemaList() -> tables + columns + PKs (fast, ~50ms) [/api/db/schema/list] - getSchemaRelations() -> foreign keys + indexes (heavy, ~2.3s) [/api/db/schema/relations] The client renders the tree from the fast list immediately, then merges relations asynchronously. If relations fail/time out they are logged and skipped — the table list stays intact. getSchema() is kept (MATERIALIZED) for consumers that need the full schema (diff, AI context). New optional provider methods getSchemaList/getSchemaRelations fall back to getSchema()/[] for providers that don't implement them. * test(db/schema): cover two-phase schema introspection (list + relations) Adds regression coverage for the schema-introspection timeout fix (0d60c47), which split schema loading into a fast structural list and a heavy, best-effort relations phase. None of the new code paths were previously tested. Provider (tests/integration/db/postgres-provider.test.ts, +11): - getSchemaList(): returns columns + PKs but intentionally empty indexes/foreignKeys, isPrimary detection, non-public schema prefixing, negative reltuples row_count clamped to 0, and null/empty columns. - getSchemaRelations(): FK/index data keyed by display name, non-public prefixing on both the table and the referenced table, public references kept bare, empty/null fk-index tolerance, and null index columns coerced to []. API routes (new files, 15 tests): - /api/db/schema/list: uses getSchemaList() when present, falls back to getSchema() otherwise, plus auth (401), empty-body (400), missing-type (400), ConnectionError (503) and DatabaseError/generic (500). - /api/db/schema/relations: uses getSchemaRelations() when present, returns [] fallback otherwise, plus the same auth/validation/error matrix. Hook (tests/hooks/use-connection-manager.test.ts, +4): - phase 1 renders the table list, phase 2 merges FKs/indexes by table name; - relations failure does NOT wipe the table list and shows no error toast (the core resilience contract of the fix); - phase 1 failure short-circuits so the heavy relations endpoint is never hit; - tables absent from the relations payload are left unchanged. Full suite green: lint (0 errors), typecheck, 2050 unit/api/integration + 251 hooks + component groups, 0 failures. * fix(db/postgres): hoist schema SQL to module scope + dedupe CTEs for Sonar gate The SonarCloud PR quality gate failed on #71 with 53.3% coverage and 6.6% duplication on new code. Both stem from how the schema introspection SQL was written, not from missing tests. Coverage: the getSchema/getSchemaList/getSchemaRelations SQL lived in multi-line template literals *inside the method bodies*. bun's coverage instruments the interior lines of such literals as 0-hit in any test process that imports the module without exercising the method; the merged lcov (per-file isolated runs) then reports those SQL lines as uncovered even though the methods are tested. Verified empirically: a module-level `const` template is evaluated once at load and reported covered everywhere, while an in-body one is not. Hoisting the three queries to module scope takes postgres.ts new-code line coverage 28% -> 100% (overall new-code coverage 53.9% -> ~99%). Duplication: the hoisted queries shared the same CTEs (tables/columns/pk across full+list, fk/index across full+relations). Extracted each CTE to a single reusable fragment const and compose the three queries from them, removing the duplicated blocks. The SQL text produced is unchanged. Tests: the two route specs intentionally keep their mock.module setup inline (a shared helper breaks bun's per-file mock scoping under the non-isolated `bun run test`, clobbering @/lib/db mocks across files). Added an empty-object ({}) body case to each for the last uncovered branch. Test mock/setup boilerplate is expected harness scaffolding, so tests are excluded from Sonar copy-paste detection (sonar.cpd.exclusions). No behavioural change: same SQL, same provider/route/hook logic. Verified: lint (0 errors), typecheck, 2052 unit/api/integration + hooks + component groups (0 failures), production build, full coverage regenerated. * chore(package): bump version to 0.9.25 * refactor(api/schema): extract shared handler to remove route duplication The Sonar PR gate still flagged 6.2% duplication on new code: the /api/db/schema/list and /api/db/schema/relations routes shared ~39 identical lines of body parsing, auth, connection resolution and error handling (the only difference is which provider method they call). Extract that boilerplate into handleSchemaRequest() in src/lib/api/schema-route.ts; each route is now a thin wrapper that passes a `load` callback selecting the provider method (with the getSchemaList->getSchema and getSchemaRelations->[] fallbacks preserved). Removes both 39-line duplicated blocks. Coverage holds at 100% on new code (the shared handler is fully exercised by the existing route specs). Verified: typecheck, 2052 core tests + 0 failures, coverage regenerated (schema-route.ts 32/32 lines). * fix(db/postgres): join constraint_column_usage on constraint_schema for cross-schema FKs Copilot review on #71 flagged a pre-existing bug in the FK introspection (from 0d60c47, relocated verbatim into CTE_FK_INFO during the dedupe): JOIN information_schema.constraint_column_usage ccu ON ccu.constraint_name = tc.constraint_name AND ccu.table_schema = tc.table_schema -- wrong For a foreign key, constraint_column_usage reports the *referenced* (parent) table's columns, so ccu.table_schema is the referenced schema while tc.table_schema is the referencing schema. Joining them requires the two to be equal, which silently drops every cross-schema foreign key and makes referencedSchema always equal the referencing schema — defeating the non-public referenced-table prefixing (e.g. billing.accounts). Join on the constraint's own schema instead (ccu.constraint_schema = tc.constraint_schema). Because the FK CTE is now single-sourced, this fixes both getSchema() and getSchemaRelations(). Added a SQL-level regression guard: the existing specs mock the query result, so they can't catch a bad join; the new test asserts the emitted SQL joins on constraint_schema and not table_schema. Real cross-schema FK behaviour should be confirmed against a live Postgres via E2E. * fix(ci): restore missing test:ci script used by the npm publish gate The npm-publish workflow's Validate job runs `bun run test:ci`, but the script was dropped from package.json somewhere after v0.9.19 (it was added in #66). The v0.9.25 tag push therefore failed with "Script not found test:ci", blocking the npm publish (the GitHub release, tag and Docker images published fine). Restore it to the per-file-isolated form the publish gate expects: test:ci = bash tests/run-core.sh && bun run test:components This mirrors ci.yml's reliable path and avoids bun's process-wide mock.module load-order flakiness that the single-process `bun run test` is prone to. * fix(schema-explorer): update select action label from 'Select Top 100' to 'Select Top 50' * bump: upgrade version to 0.9.26 * fix(e2e): make admin-dashboard tests seed-state-independent The 'default tab is overview' and 'can switch to operations tab' E2E tests asserted on empty-state-only copy ('Command Center', the 'Select connection' placeholder), which disappears once seed connections load the populated dashboard. They passed in CI (no seed file) but failed locally with SEED_CONFIG_PATH set. Add stable data-testid hooks (admin-content-overview/-operations) to the TabsContent wrappers and assert on testid visibility + tab aria-selected, so the tests hold in both empty and populated states. Verified: 32/32 full E2E pass with seeds loaded, 8/8 admin pass with seeds disabled. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: group editor/UI docs under docs/editor and docs/ui; fix drift (#85) Five feature docs were loose in docs/ root. Group them by concern and fix drift found while auditing each against current code (tri-sync: docs now mirror the code they describe). Moves (git mv, history preserved; renamed to lowercase-kebab): - docs/MONACO_EDITOR_PERFORMANCE.md -> docs/editor/monaco-performance.md (LIVE, no change) - docs/SQL_ALIAS_COMPLETION.md -> docs/editor/sql-alias-completion.md - docs/QUERY_OPTIMIZATION.md -> docs/editor/query-optimization.md (LIVE, no change) - docs/THEMING.md -> docs/ui/theming.md - docs/LOGIN_PAGE.md -> docs/ui/login-page.md Drift fixes: - sql-alias-completion: completion provider is src/lib/editor/sql-completions.ts (registerSQLCompletionProvider), not QueryEditor.tsx; QueryEditor only registers it. - theming: Studio is dark-mode only — the dark class is hardcoded on <body> in layout.tsx; next-themes is installed but not wired (no ThemeProvider). Rewrote the "Switching Themes" + troubleshooting sections, added chart-color hex values for both modes, added missing light-mode muted-foreground. - login-page: removed the "Quick Access" test-button section (documented, never implemented). No docs were archived — all five describe live features. Indexes + links: - Added docs/editor/README.md and docs/ui/README.md (with source maps). - Updated root README.md theming links and added editor/login entries. - Updated .devin/wiki.json theming-doc reference. - Left docs/releases/* untouched (frozen historical snapshots). * docs: archive stale TECHNICAL_PLAN; fix drift in FEATURES + ARCHITECTURE Continues the docs/ root cleanup. Archive: - docs/TECHNICAL_PLAN.md -> docs/archived/ (git mv). Dec-2025 roadmap whose every phase has shipped (Monaco, autocomplete, multi-tab, virtualized grid, inline editing, EXPLAIN, ERD); no inbound links. Joins AI_PLAN.md et al. FEATURES.md (drift fixes, all verified against code): - AI section: "Gemini 2.5 Flash" -> "Multi-Provider LLM"; documented the flexible provider support and the shipped AI suite (index-advisor, impact, query-safety, describe-schema). - Added the Redis key-value provider (was entirely absent despite shipping). - Added OIDC SSO to the Authentication section. - Moved Visual Schema Explorer (ERD) from "Planned" to Implemented (#18) — it ships via src/components/SchemaDiagram.tsx; "Planned" section now points to docs/backlogs/. - Fixed duplicate section numbers (two #8, two #10) -> sequential 1..18. ARCHITECTURE.md (drift fixes, all verified): - "8 database backends" -> "7" (it lists exactly 7). - Added §4.6 Workspace Abstraction documenting the npm-package embedding layer (src/workspace/, src/exports/, tsup build:lib) — central per CLAUDE.md, previously undocumented. - Directory tree: added workspace/, exports/, lib/editor/, lib/seed/, lib/api/, components/monitoring/, app/monitoring/, results-grid/, SchemaDiagram.tsx; expanded the api/ai/ route list. Verified before editing: provider count (7), SchemaDiagram.tsx exists, all 8 ai/ routes exist, every src/ dir referenced exists. Avoided DATABASE_PROVIDERS.md (in flight on the providers branch) and left docs/releases/* frozen. * docs(helm): fix HELM_CHART drift; defer values to chart README Audited docs/HELM_CHART.md against charts/libredb-studio/. Fix-in-place (keeps the architecture/rationale focus; defers the values reference to the chart's own README). Fixes (all verified against the chart + CI): - Version Management: corrected the appVersion claim. CI does NOT enforce appVersion == package.json. The real guard (ci.yml "Verify appVersion is valid") fails only if appVersion is *ahead* of package.json; being behind is tolerated with an info notice. Removed the false "helm-lint job enforces equality" wording (in Version Management and Known Limitations). - Env vars: added NODE_OPTIONS (--max-old-space-size=384, hardcoded in configmap.yaml) and SEED_CONFIG_PATH/SEED_CACHE_TTL_MS. - Chart structure: added seed-configmap.yaml (was omitted). - Added §8 Seed Connections architecture note (config mount, env wiring, runtime credential resolution). - Added a pointer to charts/libredb-studio/README.md as the authoritative values reference. Left untouched (pending user confirmation): the Helm repo URL (libredb.org vs github.io) — CLAUDE.md/this doc say libredb.org while the chart README + CI use github.io; canonical domain not determinable from main. * docs(features): fix markdown indentation (Copilot review) Several section headings/bullets were indented 4 spaces, which GitHub/CommonMark renders as code blocks instead of headings/lists (Copilot flagged lines 40, 70, 117). Normalized the whole file flush-left; sub-bullets under SQL/Document/ Key-Value providers keep their single 4-space nesting level. * docs(helm): align chart README repo URL to canonical libredb.org The chart README told users 'helm repo add libredb https://libredb.github.io/ libredb-studio'. Verified that github.io 301-redirects to the canonical custom domain https://libredb.org/libredb-studio/ (the live Helm index.yaml is served there), which is what HELM_CHART.md and CLAUDE.md already use. Aligned the README to the canonical URL so all docs agree. * docs: fix drift in OIDC + STORAGE (SEED_CONNECTIONS verified clean) Audited SEED_CONNECTIONS.md, OIDC.md, STORAGE.md against current code. SEED_CONNECTIONS.md — LIVE, no changes. Every claim (config schema, env vars, credential resolution, RBAC, Helm seed-configmap wiring) matches src/lib/seed/. OIDC.md (fix-then-keep): - buildLogoutUrl snippet corrected to match src/lib/oidc.ts: uses searchParams.set() (not "+ params"), precise Auth0 host check, and the Zitadel RP-Initiated Logout special-case (/oidc/v1/end_session, detected via the role claim). Added Zitadel to the provider logout-endpoints table. - OIDCClaims interface: replaced the non-existent `name?` field with the actual `preferred_username?` field. - Username resolution note: corrected to the real fallback chain (claims.email || claims.preferred_username || claims.sub || role). STORAGE.md (fix-then-keep): - SQLite upsert: "INSERT OR REPLACE" -> actual "INSERT ... ON CONFLICT ... DO UPDATE" (sqlite.ts:105). - StorageSyncState: added the missing `isReady` field (use-storage-sync.ts:13). All claims verified against source (oidc.ts, callback route, sqlite provider, use-storage-sync). main and dev are identical for these three files, so no divergence risk. * docs(helm): clarify seed-connection env source + configurable mount (Copilot) Addresses two Copilot review notes on HELM_CHART.md: - SEED_CONFIG_PATH / SEED_CACHE_TTL_MS are set directly on the Deployment (deployment.yaml:118-120), not via the app ConfigMap — table Source column + section intro clarified. - Seed §8: the mount filename is configurable via seedConnections.configMapKey (default seed-connections.yaml), and enabling the feature without seedConnections.config OR existingConfigMap provisions nothing. The third Copilot note (FEATURES.md:56, Redis 'CLIENT LIST') is a false positive — getActiveSessions() uses this.client.client('LIST') at redis.ts:455 (ioredis exposes CLIENT LIST as .client('LIST')), so the doc and CLAUDE.md are correct. No change. * docs(editor): fix duplicated paragraph from dev merge resolution The 'dev' merge into this branch left two redundant descriptions of the completion provider in Section 2 (one from #84, one from #85). Collapsed into a single accurate paragraph: registerSQLCompletionProvider -> registerCompletion ItemProvider('sql', …) with trigger characters '.' and space, registered from QueryEditor.tsx. Verified triggerCharacters ['.', ' '] at sql-completions.ts:121. * docs(claude): add branching and PR guidelines for development workflow Introduced a new section outlining that feature/work branches should target `dev` instead of `main`. This clarifies the integration process and prevents divergence by ensuring all PRs are opened against the `dev` branch, which is the active integration branch. The `main` branch is reserved for deliberate promotions from `dev`. * docs(claude): streamline development commands and clarify coverage isolation Updated the development commands section for clarity and conciseness, removing redundant lines and ensuring consistent formatting. Added emphasis on the importance of running `bun run build:lib` after component changes. Clarified the coverage isolation explanation to highlight the impact of `bun`'s `mock.module()` on test execution across processes, preventing nondeterministic failures in CI. * docs: fix provider/driver + OIDC logout drift (Copilot review #87) All four Copilot notes were valid (verified against code): - FEATURES.md: SQLite DB provider uses `bun:sqlite` (db/providers/sql/sqlite.ts:63), not better-sqlite3 (that's the storage layer). Oracle/SQL Server don't use performance_schema — Oracle uses ALL_*/DBA_* views, SQL Server uses sys.dm_* DMVs; reworded those bullets. MySQL's performance_schema claim kept (genuine, 11 refs). - OIDC.md: buildLogoutUrl() only special-cases Auth0 + Zitadel; Keycloak/Okta/Azure AD fall back to /protocol/openid-connect/logout. Added a note + "Wired?" column so the endpoint table no longer implies Okta/Azure are specially handled. * docs(claude): update SQLite database driver reference to bun:sqlite Replaced the SQLite driver mention from `better-sqlite3` to `bun:sqlite` in the databases section to reflect the correct usage in the project. This change ensures accurate documentation for developers regarding the database drivers supported by LibreDB Studio. * docs(claude): slim CLAUDE.md to rules/conventions + add pre-commit hook Per current Anthropic guidance (code.claude.com/docs/en/memory + the Claude Code best-practices post): CLAUDE.md should hold rules/conventions/gotchas, target <200 lines, and push path-specific or procedural content into .claude/rules/ or hooks. Applied: - CLAUDE.md 366 → 82 lines. Kept the non-derivable layer (branching rule, provider tri-sync, coverage-isolation + twMerge/Lucide gotchas, build:lib rule, pre-commit rule). Trimmed code-derivable detail (full dir tree, env-var dump, connection structs, tech-stack prose) to pointers into docs/ and .env.example. - Extracted the Platform Integration Rules into .claude/rules/platform-integration.md with `paths:` frontmatter (src/components, **/*.tsx, globals.css, workspace) so they load only when touching UI — out of every-session context. - Added a PreToolUse hook on `git commit` running lint + typecheck + test + build (blocks on failure) as enforcement; the documented rule remains the reference. Note: the new .claude/settings.json hook may need `/hooks` (or a restart) to activate this session, since the settings watcher only tracks files present at startup. * docs: fix GitHub capitalization + seed-connection failure mode (Copilot #87) - CLAUDE.md: "## Github" → "## GitHub" (brand capitalization). - HELM_CHART.md §8: corrected the seed-connections misconfiguration note. Verified against the chart: seed-configmap.yaml only renders when `config` is set, but the Deployment unconditionally mounts the `seed-config` volume (referencing <release>-seed-connections) when enabled, and that volume is NOT `optional`. So enabling with neither config nor existingConfigMap makes the pods fail to start — not a benign "provisions nothing". --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(api): sync API_DOCS to current routes + correct auth model (#88) * docs(api): sync API_DOCS to current routes + correct auth model Audited API_DOCS.md against all 36 route handlers + the proxy middleware. Corrected drift in documented endpoints: - Overview/Key Features: DB list now includes Oracle, SQL Server, Redis. - Response Format: removed the fictional `{data, status}` global envelope — there is none; documented the real per-endpoint shapes. - Auth model: rewritten around `src/proxy.ts` — auth is middleware-enforced; public routes are `/api/auth/*`, `GET /api/db/health`, `GET /api/storage/config`. This is why `ai/*` handlers don't call getSession() (NOT a vuln — verified the middleware gates them). - login: requires `{email, password}` (was `{password}`); error "Invalid email or password". - auth/me: user payload is `{role, username}` (was `{role, iat, exp}`). - logout: documented the OIDC `redirectUrl` field. - query: documented the `pagination` object (limit/offset/hasMore/totalReturned/wasLimited). - ai/chat: added `queryLanguage` + `conversationHistory`; clarified 401/429/400 are LLM-provider errors, not session auth; model default 2.0→2.5-flash. Added user-facing endpoints that were undocumented: - AI suite (nl2sql, explain, query-safety, impact, index-advisor, autopilot, describe-schema), Storage API, Connections API, Admin API. - Listed internal frontend-only /api/db/* routes under a single "internal" note rather than documenting each (they track the UI and change often). No code changed. The "unauthenticated ai/*/disconnect" flagged in review was a false positive — proxy.ts gates all non-public routes. * docs(api): tighten auth/health, storage-config shape, AI validation (Copilot #88) All Copilot notes verified against code: - /api/db/health is excluded from the middleware for BOTH methods; GET is public, POST self-checks (JSON 401). Public-endpoints note corrected (was "GET only" + blanket "everything else redirected"). - /api/storage/config returns { provider, serverMode } (factory.ts:38), not just provider — fixed example + description. - "Other AI endpoints" table: renamed "Required input" → "Key input"; marked index-advisor/autopilot inputs as all-optional; narrowed the "validates …" sentence to the endpoints that actually validate (nl2sql/explain/query-safety/ impact/describe-schema) — index-advisor/autopilot accept partial payloads. * docs(features): correct Schema Explorer context-menu labels The bullet listed stale labels ("Select Top 100", "Generate Template Query"). The actual defaults (TableItem.tsx) are "Select Top 50" and "Generate Query", and labels adapt per provider via getLabels(). Surfaced in the dev->main review (#89). --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>



Summary
Adds
docs/providers/mongodb.md— the seventh and final provider reference underdocs/providers/. Per the tri-sync invariant it mirrors the currentmongodb.tsexactly (everyfile:lineverified). MongoDB is the document family: it extendsBaseDatabaseProviderdirectly (like Redis) and speaks JSON/MQL, not SQL.MongoDB-specific points captured
{collection, operation, …}; all 11 operations (find/findOne/aggregate/count/distinct + the writes)<Binary: N bytes>placeholder, recursivemixed(a|b)typesfindcapped at 100,aggregateunbounded — flagged as a real result-set riskMongoClientbuilt-in pool (max/min/idle/connect/serverSelection all mapped)connectionStringsupport (used directly — unlike SQL Server)validate/compact/dbCheck/killOp(reindexremoved in 6.0+)serverStatus/currentOp/$indexStats(real index scans) — privilege-gated, degrades gracefullyKnown limitations (current vs ought-to-be)
Sampled/flat schema, unbounded aggregate, no EXPLAIN/transactions/cancel,
collStatsdeprecation (6.2+), privilege-gated monitoring, Binary placeholder.Quality notes
file:lineverified againstdev(mongodb.ts,factory.ts).test:ci/test:coveragerunner guidance; comparative claims (real index scans) checked against sibling code.🎉 This completes the provider documentation initiative — all 7 providers (
postgres,mysql,oracle,mssql,sqlite,redis,mongodb) now have a comprehensivedocs/providers/<type-id>.mdreference.