Skip to content

HBASE-29696 Fix incorrect TableInputFormat split boundaries#8491

Open
Sigma-Ma wants to merge 1 commit into
apache:masterfrom
Sigma-Ma:HBASE-29696-fix-table-input-split-boundaries
Open

HBASE-29696 Fix incorrect TableInputFormat split boundaries#8491
Sigma-Ma wants to merge 1 commit into
apache:masterfrom
Sigma-Ma:HBASE-29696-fix-table-input-split-boundaries

Conversation

@Sigma-Ma

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Restore the original start and end rows after Bytes.split calculates the intermediate split keys in TableInputFormatBase#createNInputSplitsUniform.

Synthetic finite boundaries are still used internally for calculating the split keys, but they are no longer exposed as the boundaries of the first and last generated TableSplit.

A regression test was added to cover both empty and non-empty start rows with an open-ended range. The test also verifies continuity between adjacent splits.

Why are the changes needed?

An empty end row represents an unbounded scan in HBase. createNInputSplitsUniform temporarily replaces it with a finite all-0xFF key so that Bytes.split can calculate intermediate keys.

Previously, this temporary key was also used as the end row of the final TableSplit. Since scan stop rows are exclusive, rows equal to or lexicographically greater than the synthetic key, such as 0xFFFF, could be skipped.

Restoring the original empty end row keeps the final split open-ended and preserves the complete input range.

How was this patch tested?

Added TestTableInputFormatBase#testCreateNInputSplitsUniformPreservesOriginalBoundaries, covering:

  • empty start and empty end rows;
  • non-empty start and empty end rows;
  • continuity between adjacent splits;
  • preservation of the original first and last boundaries.

The regression test was run with:

mvn -pl hbase-mapreduce -am
-Dtest=TestTableInputFormatBase#testCreateNInputSplitsUniformPreservesOriginalBoundaries
-Dsurefire.failIfNoSpecifiedTests=false test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants