Skip to content

feat: add endpoint to retrieve Mermaid diagram of SQL connection database structure#1750

Merged
Artuomka merged 1 commit into
mainfrom
backend_table_db_structure
Apr 30, 2026
Merged

feat: add endpoint to retrieve Mermaid diagram of SQL connection database structure#1750
Artuomka merged 1 commit into
mainfrom
backend_table_db_structure

Conversation

@Artuomka
Copy link
Copy Markdown
Collaborator

@Artuomka Artuomka commented Apr 30, 2026

Summary by CodeRabbit

  • New Features

    • Added database connection diagram generation functionality, providing visual schema representations with table structures, primary/foreign key relationships, and human-readable database summaries. Diagrams are generated in Mermaid ER format and are supported for SQL connections only.
  • Tests

    • Added end-to-end tests validating diagram generation, error handling for unsupported connection types, and authentication enforcement.

Copilot AI review requested due to automatic review settings April 30, 2026 14:30
@Artuomka Artuomka enabled auto-merge April 30, 2026 14:31
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 30, 2026

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

Introduces a new "Get Connection Diagram" feature enabling SQL database connections to generate Mermaid ER diagrams and descriptions from their schema metadata. The implementation includes a new controller endpoint, use case, utilities for diagram generation, data structures, DTOs, and comprehensive E2E tests.

Changes

Cohort / File(s) Summary
Enum & Dependency Injection
backend/src/common/data-injection.tokens.ts
Adds GET_CONNECTION_DIAGRAM enum member to UseCaseType for use-case registration.
Data Structures & DTOs
backend/src/entities/connection/application/data-structures/get-connection-diagram.ds.ts, backend/src/entities/connection/application/dto/connection-diagram-response.dto.ts
Introduces request data structure with connectionId, masterPwd, userId; introduces response DTO with connection metadata, Mermaid diagram string, description, and generation timestamp.
Controller & Module Wiring
backend/src/entities/connection/connection.controller.ts, backend/src/entities/connection/connection.module.ts
Adds protected GET /connection/diagram/:connectionId endpoint with input validation; registers GetConnectionDiagramUseCase in module and adds authenticated route.
Use Case & Interface
backend/src/entities/connection/use-cases/get-connection-diagram.use.case.ts, backend/src/entities/connection/use-cases/use-cases.interfaces.ts
Implements use case that decrypts connection, validates SQL type, queries database schema, collects table metadata, and generates Mermaid diagram; defines IGetConnectionDiagram interface.
Utilities
backend/src/entities/connection/utils/build-mermaid-er-diagram.util.ts, backend/src/entities/connection/utils/is-sql-connection-type.util.ts
Adds diagram builder generating Mermaid ER syntax with table aliases, columns annotated with PK/FK markers, relationships, and human-readable descriptions; adds SQL connection type validator.
Messages & Exceptions
backend/src/exceptions/text/messages.ts
Adds DIAGRAM_NOT_SUPPORTED_FOR_CONNECTION_TYPE message constant for non-SQL connections.
E2E Tests
backend/test/ava-tests/non-saas-tests/non-saas-connection-diagram-e2e.test.ts
Comprehensive test suite validating successful diagram generation for SQL connections, error handling for unsupported types and unauthenticated requests, and Mermaid diagram output structure.

Sequence Diagram

sequenceDiagram
    participant Client
    participant Controller
    participant UseCase
    participant DB
    participant Diagram as Diagram Builder

    Client->>Controller: GET /connection/diagram/:connectionId
    Controller->>Controller: Validate connectionId & extract masterPwd, userId
    Controller->>UseCase: execute(GetConnectionDiagramDs)
    UseCase->>DB: Decrypt & load connection by ID
    UseCase->>UseCase: Validate SQL connection type
    UseCase->>DB: Resolve userEmail for agent-based connections
    UseCase->>DB: Validate schema cache & retrieve tables
    UseCase->>DB: Fetch table metadata (structure, PKs, FKs)
    UseCase->>Diagram: buildMermaidErDiagram(databaseName, tables)
    Diagram->>Diagram: Generate ER diagram with relationships
    Diagram-->>UseCase: Return diagram + description
    UseCase-->>Controller: Return ConnectionDiagramResponseDTO
    Controller-->>Client: HTTP 200 with diagram & metadata
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A diagram blooms from seeds of data small,
Tables dancing, relationships enthrall,
Mermaid's magic draws the ER way,
SQL secrets now have light of day!

🚥 Pre-merge checks | ✅ 4 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Security Check ⚠️ Warning Pull request introduces critical security vulnerabilities: broken access control (OWASP A01), incomplete Mermaid diagram escaping (OWASP A03), and unbounded concurrent database queries (OWASP A04). Add authorization check to verify user owns connection, update escapeQuotes() to handle control characters, implement bounded concurrency for metadata queries, and add cross-user access test.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main feature: adding an endpoint to retrieve a Mermaid diagram representing SQL connection database structure.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch backend_table_db_structure

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 0/1 reviews remaining, refill in 60 minutes.

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new authenticated backend endpoint to generate and return a Mermaid erDiagram representation (plus a textual summary) of a SQL connection’s database schema, along with end-to-end coverage for the new behavior.

Changes:

  • Added GET /connection/diagram/:connectionId controller route, DI wiring, and a new use case to build the diagram from DB metadata.
  • Introduced utilities to detect SQL connection types and to render Mermaid ER diagrams + descriptions.
  • Added non-SaaS AVA E2E tests for the new endpoint (success, unsupported DB type, unauthenticated).

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
backend/test/ava-tests/non-saas-tests/non-saas-connection-diagram-e2e.test.ts Adds E2E coverage for the new connection diagram endpoint.
backend/src/exceptions/text/messages.ts Adds a new message for rejecting non-SQL connection diagram requests; minor formatting adjustment.
backend/src/entities/connection/utils/is-sql-connection-type.util.ts Adds helper to classify which connection types are treated as SQL for diagram support.
backend/src/entities/connection/utils/build-mermaid-er-diagram.util.ts Implements Mermaid erDiagram + description string builder from table metadata.
backend/src/entities/connection/use-cases/use-cases.interfaces.ts Adds IGetConnectionDiagram interface.
backend/src/entities/connection/use-cases/get-connection-diagram.use.case.ts Implements schema introspection and diagram generation use case.
backend/src/entities/connection/connection.module.ts Wires the new use case and routes; adds middleware route entry.
backend/src/entities/connection/connection.controller.ts Exposes GET /connection/diagram/:connectionId endpoint with guards/swagger metadata.
backend/src/entities/connection/application/dto/connection-diagram-response.dto.ts Adds response DTO for diagram payload.
backend/src/entities/connection/application/data-structures/get-connection-diagram.ds.ts Adds input DS for the new use case.
backend/src/common/data-injection.tokens.ts Adds DI token GET_CONNECTION_DIAGRAM.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +35 to +41
const connection = await this._dbContext.connectionRepository.findAndDecryptConnection(connectionId, masterPwd);
if (!connection) {
throw new HttpException({ message: Messages.CONNECTION_NOT_FOUND }, HttpStatus.BAD_REQUEST);
}
if (!isSqlConnectionType(connection.type)) {
throw new BadRequestException(Messages.DIAGRAM_NOT_SUPPORTED_FOR_CONNECTION_TYPE);
}
Copy link

Copilot AI Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

findAndDecryptConnection() can throw Messages.MASTER_PASSWORD_MISSING / Messages.MASTER_PASSWORD_INCORRECT (see custom-connection-repository-extension.ts), but this use case doesn't catch/translate those errors. That will surface as a 500 for a client-side error when the master password header is missing/incorrect. Catch these specific errors and rethrow an HttpException with HttpStatus.BAD_REQUEST (and the same type values used elsewhere, e.g. no_master_key / invalid_master_key).

Copilot uses AI. Check for mistakes.
Comment on lines +57 to +60
const realTables = tables.filter((t) => !t.isView);
const tableInputs: Array<MermaidTableInput> = await Promise.all(
realTables.map((t) => this.collectTableInfo(dao, t.tableName, userEmail)),
);
Copy link

Copilot AI Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Promise.all(realTables.map(...)) will kick off 3 queries per table concurrently (structure/PK/FK). On connections with many tables this can overwhelm the DB / exceed pool limits and cause avoidable timeouts. Consider adding a concurrency limit / batching (e.g. process tables in chunks) so the endpoint remains reliable for larger schemas.

Copilot uses AI. Check for mistakes.
Comment on lines +85 to +90
private async safe<T>(fn: () => Promise<T>, fallback: T): Promise<T> {
try {
return await fn();
} catch {
return fallback;
}
Copy link

Copilot AI Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The safe() helper swallows all errors and returns a fallback without logging or surfacing partial-failure information. This can lead to silently incomplete diagrams and makes production failures hard to diagnose. At minimum, log (or capture to Sentry) the table name + operation when a DAO call fails, or propagate an error so the client can see the failure reason.

Copilot uses AI. Check for mistakes.
t.is(typeof body.diagram, 'string');
t.is(typeof body.description, 'string');
t.is(typeof body.generatedAt, 'string');
t.notThrows(() => new Date(body.generatedAt));
Copy link

Copilot AI Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

t.notThrows(() => new Date(body.generatedAt)) doesn't actually validate the timestamp because new Date(...) never throws (it returns an Invalid Date object on bad input). Use an assertion that checks parsability instead (e.g. !Number.isNaN(Date.parse(body.generatedAt)) or new Date(...).getTime() is finite).

Suggested change
t.notThrows(() => new Date(body.generatedAt));
t.false(Number.isNaN(Date.parse(body.generatedAt)));

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants