Skip to content

[TECH DEBT]: Data Compare Consolidation #745

@vijaypavann-db

Description

@vijaypavann-db

Is there an existing issue for this?

  • I have searched the existing issues

Category of feature request

Reconcile

Problem statement

In the current codebase, the reconcile_data and _get_mismatch_data functions are tightly integrated with HASH_COLUMN_NAME to identify the mismatch, missing_in_src and missing_in_tgt records; due to this, it can't be reused for any other use cases like Aggregate Data Reconciliation .

Proposed Solution

Need to update these as generic common functions, so that it can be reused for other use cases like Aggregate Data Reconciliation along with Hash Data Reconciliation.

Additional Context

For Aggregate Data Reconciliation, two new functions, join_aggregate_data and _get_mismatch_agg_data, are introduced. Once generic functions are developed, these two functions should be removed and integrated with the common functions.

Metadata

Metadata

Assignees

Labels

feat/reconmaking sure that remorphed query produces the same results as originalrecon redesigntech debtdesign flaws and other cascading effects

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions