Problem
Custom Data Insights charts over data quality (testCaseResolutionStatus, testCaseResult) report fewer resolved incidents than the DQ incident page. Widening the chart's time range or changing its metric does not reconcile the counts, and a Data Insights backfill does not recover the missing history.
Root cause
Data Insights never had a separate store for data quality. DataInsightsApp.createDataQualityDataIndex() creates the live test_case_resolution_status_search_index / test_case_result_search_index — the same indices the incident page and search APIs keep complete — and attaches a di-data-assets-* alias to each. The custom DI charts read those physical indices through that alias.
DataQualityWorkflow then, on each run, deleted a time window from those live indices and re-inserted the exact documents live indexing had already written. It added no data, but it did:
- cap history at 30 days (
retentionDays = 30), silently clamping any longer backfill request;
- destroy live data on
recreateDataAssetsIndex, by calling deleteDataQualityDataIndex() on the live index;
- write through the canonical index name, ignoring rebuild aliases during a reindex.
The di-data-assets-* alias was also attached imperatively at runtime, so a SearchIndexApp reindex dropped it until the next Data Insights run — leaving the charts resolving nothing.
Expected
Custom DI charts over data quality should reflect the full incident history that the incident page already shows, and should survive a reindex, a recreate, and a restart.
Problem
Custom Data Insights charts over data quality (
testCaseResolutionStatus,testCaseResult) report fewer resolved incidents than the DQ incident page. Widening the chart's time range or changing its metric does not reconcile the counts, and a Data Insights backfill does not recover the missing history.Root cause
Data Insights never had a separate store for data quality.
DataInsightsApp.createDataQualityDataIndex()creates the livetest_case_resolution_status_search_index/test_case_result_search_index— the same indices the incident page and search APIs keep complete — and attaches adi-data-assets-*alias to each. The custom DI charts read those physical indices through that alias.DataQualityWorkflowthen, on each run, deleted a time window from those live indices and re-inserted the exact documents live indexing had already written. It added no data, but it did:retentionDays = 30), silently clamping any longer backfill request;recreateDataAssetsIndex, by callingdeleteDataQualityDataIndex()on the live index;The
di-data-assets-*alias was also attached imperatively at runtime, so aSearchIndexAppreindex dropped it until the next Data Insights run — leaving the charts resolving nothing.Expected
Custom DI charts over data quality should reflect the full incident history that the incident page already shows, and should survive a reindex, a recreate, and a restart.