Untie decimal conversion from arrow metadata#853
Conversation
|
Please ensure that the NEXT_CHANGELOG.md file is updated with any relevant changes. |
There was a problem hiding this comment.
Pull Request Overview
This PR decouples decimal conversion logic from arrow metadata by refactoring relevant methods and tests to use a dedicated ColumnInfo object. Key changes include updating conversion functions to accept ColumnInfo instead of string metadata, adjusting related helper methods in utility classes, and updating the tests accordingly.
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/test/java/com/databricks/jdbc/common/util/DatabricksThriftUtilTest.java | Updated type retrieval test to use ColumnInfo. |
| src/test/java/com/databricks/jdbc/api/impl/converters/ArrowToJavaObjectConverterTest.java | Revised DECIMAL conversion and other tests to pass ColumnInfo. |
| src/test/java/com/databricks/jdbc/api/impl/arrow/InlineChunkProviderTest.java, ArrowResultChunkTest.java, ArrowStreamResult.java, DatabricksResultSetMetaDataTest.java | Adjusted tests for ColumnInfo-based type extraction and conversion. |
| src/main/java/com/databricks/jdbc/common/util/DatabricksTypeUtil.java, DatabricksThriftUtil.java | Refactored helper methods to support ColumnInfo usage. |
| src/main/java/com/databricks/jdbc/api/impl/converters/ArrowToJavaObjectConverter.java | Modified DECIMAL conversion to use ColumnInfo. |
| src/main/java/com/databricks/jdbc/api/impl/arrow/InlineChunkProvider.java, ArrowResultChunk.java, DatabricksResultSetMetaData.java | Updated field conversion and metadata extraction with ColumnInfo. |
Comments suppressed due to low confidence (1)
src/main/java/com/databricks/jdbc/api/impl/converters/ArrowToJavaObjectConverter.java:270
- The conversion method uses 'typePrecision' from ColumnInfo to set the BigDecimal scale, which can be counterintuitive. Consider clarifying the intended use of typePrecision versus typeScale, or renaming the property to better reflect its purpose.
int bigDecimalScale = columnInfo.getTypePrecision().intValue();
| yv.allocateNewSafe(); | ||
| yv.setSafe(0, 1200); | ||
| yv.setValueCount(1); | ||
| ColumnInfo intervalColumnInfo = new ColumnInfo(); |
There was a problem hiding this comment.
[nitpick] Multiple test cases instantiate a new ColumnInfo with default settings repeatedly. Consider extracting a helper method or a constant to generate a default ColumnInfo instance to reduce duplication and improve test readability.
| ColumnInfo intervalColumnInfo = new ColumnInfo(); | |
| ColumnInfo intervalColumnInfo = createDefaultColumnInfo(); |
| return new BigDecimal(object.toString()); | ||
| bigDecimal = new BigDecimal(object.toString()); | ||
| } else if (object instanceof Number) { | ||
| bigDecimal = new BigDecimal(object.toString()); |
There was a problem hiding this comment.
for number to bigDecimal, we need to convert through string?
gopalldb
left a comment
There was a problem hiding this comment.
Can you give some more context on what was issue? Any Jira ticket/customer issue
| return new BigDecimal(object.toString()); | ||
| bigDecimal = new BigDecimal(object.toString()); | ||
| } else if (object instanceof Number) { | ||
| bigDecimal = new BigDecimal(object.toString()); |
There was a problem hiding this comment.
We can do something like this in case of Number instance:
if (number instanceof BigDecimal) {
return (BigDecimal) number;
} else if (number instanceof Long || number instanceof Integer || number instanceof Short || number instanceof Byte) {
return BigDecimal.valueOf(number.longValue());
} else if (number instanceof Double || number instanceof Float) {
// Avoid using new BigDecimal(double) due to precision issues
return BigDecimal.valueOf(number.doubleValue());
} else {
// fallback for other Number types (e.g., AtomicLong)
return new BigDecimal(number.toString());
}
There was a problem hiding this comment.
what's the reason to do this?
There was a problem hiding this comment.
because we can directly use the parsed value from different Number types, instead of converting to string and then parsing
|
Closing in favour of #856 |
Description
Untie decimal conversion from arrow metadata
Testing
Unit tests
Additional Notes to the Reviewer