Skip to content

cleanup: remove root-level layout_config and components from Dashboar…#1360

Open
NaveenCode wants to merge 1 commit into
mainfrom
cleanup/remove-dashboard-root-layout-fields
Open

cleanup: remove root-level layout_config and components from Dashboar…#1360
NaveenCode wants to merge 1 commit into
mainfrom
cleanup/remove-dashboard-root-layout-fields

Conversation

@NaveenCode
Copy link
Copy Markdown
Contributor

@NaveenCode NaveenCode commented May 19, 2026

Summary

  • Permanently removes deprecated root-level layout_config and components fields from the Dashboard system
  • All dashboard data now lives exclusively in dashboard.tabs[].layout_config and dashboard.tabs[].components
  • Adds cleanup_frozen_dashboard_root_fields management command to clean up existing ReportSnapshot JSON blobs

Migration steps (on deploy)

  1. Run data migration: python manage.py cleanup_frozen_dashboard_root_fields
  2. Run schema migration: python manage.py migrate ddpui 0161

Test plan

  • All 1592 backend tests pass (uv run pytest ddpui/tests/ --ignore=ddpui/tests/integration_tests)
  • Create a dashboard — verify no layout_config/components in API response
  • Open an existing report — verify it renders correctly
  • Duplicate a dashboard — verify tabs are copied correctly

Summary by CodeRabbit

  • Refactor
    • Dashboard structure reorganized to use tabs-based content organization
    • Database schema updated to reflect dashboard tabs structure
    • Services and APIs updated to support the new dashboard organization
    • Added management command for cleanup of legacy frozen dashboard data

Review Change Stack

…d (DALGO-1296)

- Drop layout_config and components columns from Dashboard model (migration 0161)
- Remove fields from DashboardResponse and FrozenDashboardConfig schemas
- Update report_service, dashboard_service, chart_service, and duplicate API to iterate tabs
- Add cleanup_frozen_dashboard_root_fields management command for ReportSnapshot JSON cleanup
- Delete obsolete migrate_dashboards_to_tabs and migrate_report_snapshots_to_tabs commands
- Update all test fixtures and assertions to use tabs structure
@NaveenCode NaveenCode self-assigned this May 19, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 19, 2026

Walkthrough

This pull request completes the migration from a flat layout_config/components dashboard structure to a tab-based hierarchy. The model, database schema, snapshots, and service layer are refactored to store and query UI configuration exclusively within tabs, with stale root fields removed and cleanup tooling provided for existing snapshots.

Changes

Tab-Based Dashboard Structure Refactor

Layer / File(s) Summary
Data model and schema definition
ddpui/models/dashboard.py, ddpui/schemas/dashboard_schema.py, ddpui/schemas/report_schema.py
Removes layout_config and components JSONField declarations from Dashboard model; Dashboard.to_json() stops serializing those fields; DashboardResponse Pydantic schema removes both fields; FrozenDashboardConfig schema updates from root fields to tabs, filter_layout, and filters to describe frozen dashboard state.
Database schema migration
ddpui/migrations/0161_remove_dashboard_root_layout_components.py
Adds migration that removes layout_config and components columns from the dashboard table, depending on prior 0160_add_dashboard_tabs migration.
Report snapshot freezing and cleanup
ddpui/core/reports/report_service.py, ddpui/management/commands/cleanup_frozen_dashboard_root_fields.py
ReportService._freeze_dashboard now stores only tab-based metadata, filters, and filter layout (excluding root fields); _extract_chart_ids derives chart IDs exclusively from components nested in dashboard.tabs with deduplication; new management command iterates ReportSnapshot records, identifies those with stale root-level fields in frozen_dashboard, and bulk-updates to remove them via --dry-run-aware batching.
Dashboard and chart service refactoring
ddpui/services/dashboard_service.py, ddpui/services/chart_service.py
DashboardService.get_dashboard_response removes legacy filter-component migration code; apply_filters and get_dashboard_charts now traverse tabs[*].components instead of root fields; validate_dashboard_config performs per-tab layout/component consistency checks; ChartService.get_chart_dashboards scans each tab's components for matching chart entries using nested-loop breaking.
Dashboard duplication simplification
ddpui/api/dashboard_native_api.py
Removes import copy; in duplicate_dashboard, eliminates deep-copy and field remapping of root layout_config/components; dashboard creation no longer explicitly sets those fields, relying on defaults; duplication only remaps tabs via copy_tabs_with_filter_remapping.
Comprehensive test fixture and assertion updates
ddpui/tests/{models,services,api_tests,core/reports,schema_tests}/*.py
Test fixtures (sample_dashboard, public_dashboard, tab_dashboard) updated to define dashboards via tabs containing per-tab layout_config and components; assertions in 30+ ranges shifted to verify chart/text components within tabs[*].components instead of root fields; schema test fixture cleaned of removed fields; removed assertions expecting empty root fields in tab-based dashboards; added test validating chart ID deduplication across multiple tabs.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • DalgoT4D/DDP_backend#1263: Updates ReportService snapshot freezing and chart-ID extraction logic that this PR refactors for tab-based structure.
  • DalgoT4D/DDP_backend#1286: Modifies FrozenDashboardConfig schema to add dashboard_id field alongside the tabs/filter_layout/filters changes here.
  • DalgoT4D/DDP_backend#1284: Updates snapshot freezing serialization in ReportService to use Pydantic model_dump() instead of dict() alongside schema changes here.

Suggested reviewers

  • siddhant3030
  • Ishankoradia
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'cleanup: remove root-level layout_config and components from Dashboard' accurately summarizes the main change in the PR—it removes deprecated root-level Dashboard fields and consolidates data structure into tabs.
Docstring Coverage ✅ Passed Docstring coverage is 83.87% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch cleanup/remove-dashboard-root-layout-fields

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@sentry
Copy link
Copy Markdown

sentry Bot commented May 19, 2026

Codecov Report

❌ Patch coverage is 19.44444% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.72%. Comparing base (75cb6fd) to head (a9b17c6).

Files with missing lines Patch % Lines
ddpui/services/dashboard_service.py 0.00% 29 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1360      +/-   ##
==========================================
+ Coverage   58.66%   58.72%   +0.05%     
==========================================
  Files         132      132              
  Lines       15690    15653      -37     
==========================================
- Hits         9205     9192      -13     
+ Misses       6485     6461      -24     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ddpui/services/dashboard_service.py`:
- Around line 1053-1056: The component-validation loop is currently outside the
"for tab in dashboard.tabs or []" loop causing only the last tab's components to
be validated; move the loop that iterates over components (the "for
component_id, component_config in components.items():" block) inside the tab
loop so each tab's components are validated, ensure "components =
tab.get('components') or {}" is set per tab before that inner loop, and keep
using the same identifiers (components, component_id, component_config,
component_type) to preserve existing logic.

In `@ddpui/tests/models/test_dashboard_models.py`:
- Line 145: The fixture uses a non-production chart component shape ("chart-1":
{"type": "chart", "chart_id": 1}); update the component to match the tab
component contract by replacing the root chart_id with a config object using
config.chartId (e.g., under the "chart-1" component keep "type": "chart" and add
"config": {"chartId": 1}) so tests reflect the real production schema used by
services.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3504812d-f6fe-47e2-9dff-76ce875ea57f

📥 Commits

Reviewing files that changed from the base of the PR and between 75cb6fd and a9b17c6.

📒 Files selected for processing (20)
  • ddpui/api/dashboard_native_api.py
  • ddpui/core/reports/report_service.py
  • ddpui/management/commands/cleanup_frozen_dashboard_root_fields.py
  • ddpui/management/commands/migrate_dashboards_to_tabs.py
  • ddpui/management/commands/migrate_report_snapshots_to_tabs.py
  • ddpui/migrations/0161_remove_dashboard_root_layout_components.py
  • ddpui/models/dashboard.py
  • ddpui/schemas/dashboard_schema.py
  • ddpui/schemas/report_schema.py
  • ddpui/services/chart_service.py
  • ddpui/services/dashboard_service.py
  • ddpui/tests/api_tests/test_charts_api.py
  • ddpui/tests/api_tests/test_dashboard_native_api.py
  • ddpui/tests/api_tests/test_public_report_api.py
  • ddpui/tests/api_tests/test_report_api.py
  • ddpui/tests/core/reports/test_report_service.py
  • ddpui/tests/models/test_dashboard_models.py
  • ddpui/tests/schema_tests/test_dashboard_schema.py
  • ddpui/tests/services/test_chart_service.py
  • ddpui/tests/services/test_dashboard_service.py
💤 Files with no reviewable changes (9)
  • ddpui/management/commands/migrate_report_snapshots_to_tabs.py
  • ddpui/models/dashboard.py
  • ddpui/schemas/report_schema.py
  • ddpui/management/commands/migrate_dashboards_to_tabs.py
  • ddpui/schemas/dashboard_schema.py
  • ddpui/api/dashboard_native_api.py
  • ddpui/tests/api_tests/test_dashboard_native_api.py
  • ddpui/tests/schema_tests/test_dashboard_schema.py
  • ddpui/tests/services/test_dashboard_service.py

Comment on lines +1053 to 1056
for tab in dashboard.tabs or []:
components = tab.get("components") or {}
for component_id, component_config in components.items():
component_type = component_config.get("type")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Validate components inside each tab loop, not only the last tab.

On Line 1055, the component-validation loop is outside the for tab in dashboard.tabs block, so only the final tab’s components are checked.

Proposed fix
-        for tab in dashboard.tabs or []:
-            components = tab.get("components") or {}
-        for component_id, component_config in components.items():
-            component_type = component_config.get("type")
-
-            if not component_type:
-                errors.append(f"Component {component_id} has no type specified")
-                continue
-
-            if component_type not in [ct.value for ct in DashboardComponentType]:
-                errors.append(f"Component {component_id} has invalid type: {component_type}")
-
-            # Validate chart components
-            if component_type == DashboardComponentType.CHART.value:
-                config = component_config.get("config", {})
-                if not config.get("chartId"):
-                    errors.append(f"Chart component {component_id} has no chartId")
+        for tab in dashboard.tabs or []:
+            components = tab.get("components") or {}
+            for component_id, component_config in components.items():
+                component_type = component_config.get("type")
+
+                if not component_type:
+                    errors.append(f"Component {component_id} has no type specified")
+                    continue
+
+                if component_type not in [ct.value for ct in DashboardComponentType]:
+                    errors.append(f"Component {component_id} has invalid type: {component_type}")
+
+                # Validate chart components
+                if component_type == DashboardComponentType.CHART.value:
+                    config = component_config.get("config", {})
+                    if not config.get("chartId"):
+                        errors.append(f"Chart component {component_id} has no chartId")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for tab in dashboard.tabs or []:
components = tab.get("components") or {}
for component_id, component_config in components.items():
component_type = component_config.get("type")
for tab in dashboard.tabs or []:
components = tab.get("components") or {}
for component_id, component_config in components.items():
component_type = component_config.get("type")
if not component_type:
errors.append(f"Component {component_id} has no type specified")
continue
if component_type not in [ct.value for ct in DashboardComponentType]:
errors.append(f"Component {component_id} has invalid type: {component_type}")
# Validate chart components
if component_type == DashboardComponentType.CHART.value:
config = component_config.get("config", {})
if not config.get("chartId"):
errors.append(f"Chart component {component_id} has no chartId")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ddpui/services/dashboard_service.py` around lines 1053 - 1056, The
component-validation loop is currently outside the "for tab in dashboard.tabs or
[]" loop causing only the last tab's components to be validated; move the loop
that iterates over components (the "for component_id, component_config in
components.items():" block) inside the tab loop so each tab's components are
validated, ensure "components = tab.get('components') or {}" is set per tab
before that inner loop, and keep using the same identifiers (components,
component_id, component_config, component_type) to preserve existing logic.

"id": "tab-1",
"title": "Tab 1",
"layout_config": [{"i": "chart-1", "x": 0, "y": 0, "w": 6, "h": 4}],
"components": {"chart-1": {"type": "chart", "chart_id": 1}},
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Use the same chart component schema as production (config.chartId).

This fixture uses chart_id at the component root, which does not match the tab component contract used by services.

Proposed fix
-                "components": {"chart-1": {"type": "chart", "chart_id": 1}},
+                "components": {"chart-1": {"type": "chart", "config": {"chartId": 1}}},
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"components": {"chart-1": {"type": "chart", "chart_id": 1}},
"components": {"chart-1": {"type": "chart", "config": {"chartId": 1}}},
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ddpui/tests/models/test_dashboard_models.py` at line 145, The fixture uses a
non-production chart component shape ("chart-1": {"type": "chart", "chart_id":
1}); update the component to match the tab component contract by replacing the
root chart_id with a config object using config.chartId (e.g., under the
"chart-1" component keep "type": "chart" and add "config": {"chartId": 1}) so
tests reflect the real production schema used by services.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant