HDDS-13664. Persist pendingDeleteBytes only when feature is finalized

[priyeshkaratha]

What changes were proposed in this pull request?

This PR introduces refactors the pending deletion metadata handling logic and comprehensive integration testing for the DataNode data distribution feature upgrade scenario.
The changes include:

1. Code Refactoring: Refactored to improve the pending deletion metadata handling: KeyValueContainerUtil.java
   • Extracted populatePendingDeletionMetadata() method for better separation of concerns
   • Added separate handler methods for different upgrade states
   • Enhanced the conditional logic to properly handle DATA_DISTRIBUTION feature finalization
   • Improved code readability and maintainability
2. Integration Test Addition: Created with comprehensive test coverage for the DN data distribution upgrade scenario, specifically validating: TestDNDataDistributionFinalization.java
   • Pre-finalization behavior (handlePreDataDistributionFeature path)
   • Post-finalization behavior (handlePostDataDistributionFeature path)
   • Missing metadata recalculation scenario (getAggregatePendingDelete path)

What is the link to the Apache JIRA

HDDS-13664

How was this patch tested?

Tested using integration added integration tests.

[priyeshkaratha]

@ChenSammi @sumitagrawl Can you please review the changes?

[errose28]

Why does the title of this PR refer to datanode upgrade finalization but the code changes do not add a datanode layout feature?

[priyeshkaratha]

Why does the title of this PR refer to datanode upgrade finalization but the code changes do not add a datanode layout feature?

updated the title. There is no upgrade actions.

[errose28]

I see the layout feature was added earlier in the feature branch. Please update the design doc with information why a layout feature is required for this. From what I can tell we are only adding new information, which does not usually require layout features.

[priyeshkaratha]

Thank you @ChenSammi for the review and I have addressed all your comments in latest commit.

[ChenSammi]

Can you refactor these two lines of comments to make it correct?

[ChenSammi]

@priyeshkaratha , there is one more place that pendingDeleteBytes is persisted without check the feature finalized or not, KeyValueContainerMetadataInspector#checkAndRepair. We can skip the pendingDeleteBytes check and repair if feature is not finalized.

Please also check and run the fail test locally to see if it's related.

[priyeshkaratha]

Handled checkAndRepair. Validated failed testcase locally and it is passing.

[ChenSammi]

Thanks @priyeshkaratha.