Skip to content

Untie decimal conversion from arrow metadata#853

Closed
vikrantpuppala wants to merge 3 commits into
mainfrom
metadata-fix
Closed

Untie decimal conversion from arrow metadata#853
vikrantpuppala wants to merge 3 commits into
mainfrom
metadata-fix

Conversation

@vikrantpuppala

Copy link
Copy Markdown
Collaborator

Description

Untie decimal conversion from arrow metadata

Testing

Unit tests

Additional Notes to the Reviewer

@github-actions

Copy link
Copy Markdown

Please ensure that the NEXT_CHANGELOG.md file is updated with any relevant changes.
If this is not necessary for your PR, please include the following in your PR description:
NO_CHANGELOG=true
and rerun the job.

@vikrantpuppala vikrantpuppala requested review from Copilot, jayantsing-db and shivam2680 and removed request for Copilot and shivam2680 June 18, 2025 09:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR decouples decimal conversion logic from arrow metadata by refactoring relevant methods and tests to use a dedicated ColumnInfo object. Key changes include updating conversion functions to accept ColumnInfo instead of string metadata, adjusting related helper methods in utility classes, and updating the tests accordingly.

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/test/java/com/databricks/jdbc/common/util/DatabricksThriftUtilTest.java Updated type retrieval test to use ColumnInfo.
src/test/java/com/databricks/jdbc/api/impl/converters/ArrowToJavaObjectConverterTest.java Revised DECIMAL conversion and other tests to pass ColumnInfo.
src/test/java/com/databricks/jdbc/api/impl/arrow/InlineChunkProviderTest.java, ArrowResultChunkTest.java, ArrowStreamResult.java, DatabricksResultSetMetaDataTest.java Adjusted tests for ColumnInfo-based type extraction and conversion.
src/main/java/com/databricks/jdbc/common/util/DatabricksTypeUtil.java, DatabricksThriftUtil.java Refactored helper methods to support ColumnInfo usage.
src/main/java/com/databricks/jdbc/api/impl/converters/ArrowToJavaObjectConverter.java Modified DECIMAL conversion to use ColumnInfo.
src/main/java/com/databricks/jdbc/api/impl/arrow/InlineChunkProvider.java, ArrowResultChunk.java, DatabricksResultSetMetaData.java Updated field conversion and metadata extraction with ColumnInfo.
Comments suppressed due to low confidence (1)

src/main/java/com/databricks/jdbc/api/impl/converters/ArrowToJavaObjectConverter.java:270

  • The conversion method uses 'typePrecision' from ColumnInfo to set the BigDecimal scale, which can be counterintuitive. Consider clarifying the intended use of typePrecision versus typeScale, or renaming the property to better reflect its purpose.
    int bigDecimalScale = columnInfo.getTypePrecision().intValue();

Comment thread src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSetMetaData.java Outdated
yv.allocateNewSafe();
yv.setSafe(0, 1200);
yv.setValueCount(1);
ColumnInfo intervalColumnInfo = new ColumnInfo();

Copilot AI Jun 18, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Multiple test cases instantiate a new ColumnInfo with default settings repeatedly. Consider extracting a helper method or a constant to generate a default ColumnInfo instance to reduce duplication and improve test readability.

Suggested change
ColumnInfo intervalColumnInfo = new ColumnInfo();
ColumnInfo intervalColumnInfo = createDefaultColumnInfo();

Copilot uses AI. Check for mistakes.
return new BigDecimal(object.toString());
bigDecimal = new BigDecimal(object.toString());
} else if (object instanceof Number) {
bigDecimal = new BigDecimal(object.toString());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for number to bigDecimal, we need to convert through string?

@gopalldb gopalldb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give some more context on what was issue? Any Jira ticket/customer issue

return new BigDecimal(object.toString());
bigDecimal = new BigDecimal(object.toString());
} else if (object instanceof Number) {
bigDecimal = new BigDecimal(object.toString());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do something like this in case of Number instance:

if (number instanceof BigDecimal) {
return (BigDecimal) number;
} else if (number instanceof Long || number instanceof Integer || number instanceof Short || number instanceof Byte) {
return BigDecimal.valueOf(number.longValue());
} else if (number instanceof Double || number instanceof Float) {
// Avoid using new BigDecimal(double) due to precision issues
return BigDecimal.valueOf(number.doubleValue());
} else {
// fallback for other Number types (e.g., AtomicLong)
return new BigDecimal(number.toString());
}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the reason to do this?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because we can directly use the parsed value from different Number types, instead of converting to string and then parsing

@vikrantpuppala

Copy link
Copy Markdown
Collaborator Author

Closing in favour of #856

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants