Skip to content

Grafana postgresql support#8870

Open
fabioluciano wants to merge 12 commits into
apache:mainfrom
fabioluciano:grafana-postgresql-support
Open

Grafana postgresql support#8870
fabioluciano wants to merge 12 commits into
apache:mainfrom
fabioluciano:grafana-postgresql-support

Conversation

@fabioluciano
Copy link
Copy Markdown
Contributor

@fabioluciano fabioluciano commented May 8, 2026

Summary

This PR adds comprehensive PostgreSQL support for Grafana dashboards, enabling DevLake to work with PostgreSQL databases in addition to MySQL.

Key Changes:

  1. PostgreSQL Dashboard Support

    • Created automated conversion script (convert-mysql-to-postgresql.py) to translate MySQL-specific SQL syntax to PostgreSQL
    • Generated complete set of PostgreSQL-compatible dashboards (40+ dashboards) covering all data sources (DORA, GitHub, GitLab, Jira, Jenkins, etc.)
    • Fixed SQL dialect differences: FIELD()CASE WHEN, column alias handling, computed field naming
  2. Development Environment Improvements

    • Split Docker Compose configurations into separate MySQL and PostgreSQL environments
    • docker-compose-dev-mysql.yml: MySQL stack (ports 3001/8083/4001/4181)
    • docker-compose-dev-postgresql.yml: PostgreSQL stack (ports 3002/8084/4002/4182)
    • Different project names to allow parallel development/testing
  3. Dashboard Organization

    • Restructured dashboards into mysql/ and postgresql/ directories
    • Standardized filenames to kebab-case for consistency
    • Removed outdated archived dashboards
  4. Configuration Updates

    • Disabled foldersFromFilesStructure in dashboard provisioning to prevent redundant folder creation
    • Optimized Grafana Dockerfile by combining RUN commands
    • Updated entrypoint script to support both database types

Does this close any open issues?

Closes #[issue-number-if-applicable]

Screenshots

N/A - Backend database compatibility changes

Other Information

The conversion script handles the following MySQL → PostgreSQL transformations:

  • FIELD() function → CASE WHEN statements
  • Backtick identifiers → double quotes
  • DATE_ADD/DATE_SUB → interval arithmetic
  • String concatenation (CONCAT) → || operator
  • Column alias deduplication (fixes _computed suffix issues)

Both MySQL and PostgreSQL dashboard sets are now maintained in parallel

@fabioluciano fabioluciano changed the title Draft: Grafana postgresql support Grafana postgresql support May 8, 2026
- Disable ambiguous alias pattern that caused _computed suffix duplication
- Add MySQL FIELD() function to PostgreSQL CASE WHEN conversion
- Fix adoption_pct and deploy_count column name issues
Remove outdated dashboard files from _archive directory
Standardize dashboard filenames from CamelCase to kebab-case for consistency
- Fix _computed suffix duplication in column aliases
- Convert MySQL FIELD() to PostgreSQL CASE WHEN
- Apply all SQL dialect fixes from updated conversion script
- Rename dashboard files to kebab-case for consistency
- Rename docker-compose-dev.yml to docker-compose-dev-postgresql.yml
- Create separate docker-compose-dev-mysql.yml
- Use different project names to avoid conflicts
- Assign unique ports for each stack (MySQL: 3001/8083/4001/4181, PostgreSQL: 3002/8084/4000/4180)
Create dedicated compose file for MySQL development environment with unique ports and project name
Set foldersFromFilesStructure to false to prevent creating postgresql/mysql subdirectories in Grafana UI
Merge plugin installation and permission setup into single RUN statement
@fabioluciano fabioluciano marked this pull request as ready for review May 11, 2026 18:57
@dosubot dosubot Bot added size:XS This PR changes 0-9 lines, ignoring generated files. component/ext This issue or PR relates to external components, such as Grafana pr-type/feature-development This PR is to develop a new feature labels May 11, 2026
@fabioluciano
Copy link
Copy Markdown
Contributor Author

This PR seens big, but isnt. The number of lines is big bcs of the grafana dashboard that i converted to postgresql using the python script.

I tested evry single query :)

@klesh
Copy link
Copy Markdown
Contributor

klesh commented May 17, 2026

Great job! I believe this is a significant improvement to the project.

@klesh
Copy link
Copy Markdown
Contributor

klesh commented May 17, 2026

Could you please fix the failed check?

@fabioluciano
Copy link
Copy Markdown
Contributor Author

Sure! I will poke u when I finish it

@klesh
Copy link
Copy Markdown
Contributor

klesh commented May 21, 2026

down stream Pr: apache/incubator-devlake-helm-chart#371

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/ext This issue or PR relates to external components, such as Grafana pr-type/feature-development This PR is to develop a new feature size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants