Skip to content

Fixed getColumns function to return correct output for complex datatypes #974

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
msrathore-db merged 7 commits into from
Sep 9, 2025
Merged

Fixed getColumns function to return correct output for complex datatypes #974

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
77711d8
Fixed getColumns function: SQL_DATATYPE, SQL_DATETIME_SUB, DATA_TYPE,…
msrathore-db Aug 30, 2025
40d7a14
Changed DATA_TYPE to show 12 as code for complex datatype when Enable…
msrathore-db Sep 2, 2025
4effebb
Merge branch 'main' into PECOBLR-838
msrathore-db Sep 3, 2025
0e3beb9
Cleaned tests
msrathore-db Sep 3, 2025
647a163
Updated NEXT_CHANGELOG
msrathore-db Sep 3, 2025
52eda8a
Added tests for variant and struct types in the MetadataResultSetBuil…
msrathore-db Sep 8, 2025
87e2ea9
Merge branch 'main' into PECOBLR-838
msrathore-db Sep 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NEXT_CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,6 @@

### Fixed
- Fixed `ResultSet.getString` for Boolean columns in Metadata result set.
- Fixed `connection.getMetadata().getColumns()` to return the correct SQL data type code for complex type columns.
---
*Note: When making changes, please add your change under the appropriate section with a brief description.*
2 changes: 1 addition & 1 deletion src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSetMetaData.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ public DatabricksResultSetMetaData(
.columnTypeClassName(DatabricksTypeUtil.getColumnTypeClassName(columnTypeName))
.columnType(columnType)
.columnTypeText(
metadataResultSetBuilder.stripTypeName(
metadataResultSetBuilder.stripBaseTypeName(
Copy link
Copy Markdown
Collaborator

@shivam2680 shivam2680 Sep 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is changing stripTypeName to stripBaseTypeName fixing exactly?

Copy link
Copy Markdown
Collaborator

@shivam2680 shivam2680 Sep 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need to change for thrift resultsetmetadata?

Copy link
Copy Markdown
Collaborator Author

@msrathore-db msrathore-db Sep 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So now instead of returning array<INT> it'll simply return the base type that is array.
For thrift, it uses another method but the behaviour is same for that as well.

columnInfo
.getTypeText())) // store base type eg. DECIMAL instead of DECIMAL(7,2)
.typePrecision(precision)
Expand Down
80 changes: 60 additions & 20 deletions src/main/java/com/databricks/jdbc/dbclient/impl/common/MetadataResultSetBuilder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,14 +184,20 @@ List<List<Object>> getRows(
if (typeVal == null) { // safety check
object = null;
} else {
object = getCode(stripTypeName(typeVal));
// Check if complex datatype support is disabled and this is a complex type
if (!ctx.isComplexDatatypeSupportEnabled() && isComplexType(typeVal)) {
object = Types.VARCHAR;
} else {
object = getCode(stripBaseTypeName(typeVal));
}
}
break;
case "SQL_DATETIME_SUB":
// check if typeVal is a date/time related field
if (typeVal != null
&& (typeVal.contains(DATE_TYPE) || typeVal.contains(TIMESTAMP_TYPE))) {
object = getCode(stripTypeName(typeVal));
&& (stripBaseTypeName(typeVal).contains(DATE_TYPE)
Copy link
Copy Markdown
Collaborator

@shivam2680 shivam2680 Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

Copy link
Copy Markdown
Collaborator Author

@msrathore-db msrathore-db Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

|| stripBaseTypeName(typeVal).contains(TIMESTAMP_TYPE))) {
object = getCode(stripBaseTypeName(typeVal));
} else {
object = null;
}
Expand All @@ -218,7 +224,11 @@ List<List<Object>> getRows(
}
} catch (SQLException e) {
if (mappedColumn.getColumnName().equals(DATA_TYPE_COLUMN.getColumnName())) {
object = getCode(stripTypeName(typeVal));
if (!ctx.isComplexDatatypeSupportEnabled() && isComplexType(typeVal)) {
object = Types.VARCHAR;
} else {
object = getCode(stripBaseTypeName(typeVal));
}
} else if (mappedColumn
.getColumnName()
.equals(CHAR_OCTET_LENGTH_COLUMN.getColumnName())) {
Expand Down Expand Up @@ -252,11 +262,12 @@ List<List<Object>> getRows(
if (mappedColumn.getColumnName().equals(COLUMN_TYPE_COLUMN.getColumnName())) {
if (typeVal != null
&& (typeVal.contains(ARRAY_TYPE)
|| typeVal.contains(MAP_TYPE)
|| typeVal.contains(
MAP_TYPE))) { // for complex data types, do not strip type name
STRUCT_TYPE))) { // for complex data types, do not strip type name
object = typeVal;
} else {
object = stripTypeName(typeVal);
object = stripBaseTypeName(typeVal);
}
}
// Set COLUMN_SIZE to 255 if it's not present
Expand Down Expand Up @@ -339,7 +350,7 @@ int getColumnSize(String typeVal) {
if (isTextType(typeVal)) {
return ctx.getDefaultStringColumnLength();
}
String typeName = stripTypeName(typeVal);
String typeName = stripBaseTypeName(typeVal);
switch (typeName) {
case "DECIMAL":
case "NUMERIC":
Expand Down Expand Up @@ -403,13 +414,13 @@ int getBufferLength(String typeVal) {
if (typeVal == null || typeVal.isEmpty()) {
return 0;
}
if (typeVal.contains("ARRAY") || typeVal.contains("MAP")) {
if (typeVal.contains("ARRAY") || typeVal.contains("MAP") || typeVal.contains("STRUCT")) {
return 255;
}
if (isTextType(typeVal)) {
return getColumnSize(typeVal);
}
int sqlType = getCode(stripTypeName(typeVal));
int sqlType = getCode(stripBaseTypeName(typeVal));
return getSizeInBytes(sqlType);
}

Expand Down Expand Up @@ -480,6 +491,22 @@ public String stripBaseTypeName(String typeName) {
return typeName;
}

/**
* Checks if the given type string represents a complex type (ARRAY, MAP, or STRUCT).
*
* @param typeVal The type string to check
* @return true if the type is a complex type, false otherwise
*/
private boolean isComplexType(String typeVal) {
Copy link
Copy Markdown
Collaborator

@gopalldb gopalldb Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what abt variant?

Copy link
Copy Markdown
Collaborator Author

@msrathore-db msrathore-db Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since variant is not a complex type so we do not need to handle it differently

if (typeVal == null) {
return false;
}
String baseType = stripBaseTypeName(typeVal);
return baseType.contains(ARRAY_TYPE)
|| baseType.contains(MAP_TYPE)
|| baseType.contains(STRUCT_TYPE);
}

int getCode(String s) {
switch (s) {
case "STRING":
Expand Down Expand Up @@ -809,30 +836,36 @@ List<List<Object>> getThriftRows(List<List<Object>> rows, List<ResultColumn> col
List<List<Object>> updatedRows = new ArrayList<>();
for (List<Object> row : rows) {
List<Object> updatedRow = new ArrayList<>();
String typeVal = null;
int col_type_index = columns.indexOf(COLUMN_TYPE_COLUMN); // only relevant for getColumns
if (col_type_index != -1) {
typeVal = (String) row.get(col_type_index);
}
for (ResultColumn column : columns) {
if (NULL_COLUMN_COLUMNS.contains(column) || NULL_TABLE_COLUMNS.contains(column)) {
updatedRow.add(null);
continue;
}
Object object;
String typeVal = null;
int col_type_index = columns.indexOf(COLUMN_TYPE_COLUMN); // only relevant for getColumns
if (col_type_index != -1) {
typeVal = (String) row.get(col_type_index);
}
switch (column.getColumnName()) {
case "SQL_DATA_TYPE":
if (typeVal == null) { // safety check
object = null;
} else {
object = getCode(stripTypeName(typeVal));
// Check if complex datatype support is disabled and this is a complex type
if (!ctx.isComplexDatatypeSupportEnabled() && isComplexType(typeVal)) {
object = Types.VARCHAR;
} else {
object = getCode(stripBaseTypeName(typeVal));
}
}
break;
case "SQL_DATETIME_SUB":
// check if typeVal is a date/time related field
if (typeVal != null
&& (typeVal.contains(DATE_TYPE) || typeVal.contains(TIMESTAMP_TYPE))) {
object = getCode(stripTypeName(typeVal));
&& (stripBaseTypeName(typeVal).contains(DATE_TYPE)
|| stripBaseTypeName(typeVal).contains(TIMESTAMP_TYPE))) {
object = getCode(stripBaseTypeName(typeVal));
} else {
object = null;
}
Expand Down Expand Up @@ -870,7 +903,12 @@ List<List<Object>> getThriftRows(List<List<Object>> rows, List<ResultColumn> col
}
}
if (column.getColumnName().equals(DATA_TYPE_COLUMN.getColumnName())) {
object = getCode(stripTypeName(typeVal));
// Check if complex datatype support is disabled and this is a complex type
if (!ctx.isComplexDatatypeSupportEnabled() && isComplexType(typeVal)) {
object = Types.VARCHAR;
} else {
object = getCode(stripBaseTypeName(typeVal));
}
}
if (column.getColumnName().equals(CHAR_OCTET_LENGTH_COLUMN.getColumnName())) {
object = getCharOctetLength(typeVal);
Expand All @@ -889,10 +927,12 @@ List<List<Object>> getThriftRows(List<List<Object>> rows, List<ResultColumn> col
// Handle TYPE_NAME separately for potential modifications
if (column.getColumnName().equals(COLUMN_TYPE_COLUMN.getColumnName())) {
if (typeVal != null
&& (typeVal.contains(ARRAY_TYPE) || typeVal.contains(MAP_TYPE))) {
&& (typeVal.contains(ARRAY_TYPE)
|| typeVal.contains(MAP_TYPE)
|| typeVal.contains(STRUCT_TYPE))) {
object = typeVal;
} else {
object = stripTypeName(typeVal);
object = stripBaseTypeName(typeVal);
}
}
// Set COLUMN_SIZE to 255 if it's not present
Expand Down
42 changes: 42 additions & 0 deletions src/test/java/com/databricks/jdbc/dbclient/impl/common/MetadataResultSetBuilderTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -467,4 +467,46 @@ void testGetTablesResultWithNullTableType() throws SQLException {
}
assertEquals(2, rowCount);
}

@Test
void testComplexTypesReturnVarcharWhenSupportDisabled() throws SQLException {
when(connectionContext.isComplexDatatypeSupportEnabled()).thenReturn(false);

List<ResultColumn> columns =
Arrays.asList(DATA_TYPE_COLUMN, COLUMN_TYPE_COLUMN, SQL_DATA_TYPE_COLUMN);
List<List<Object>> rows =
Comment thread
msrathore-db marked this conversation as resolved.
Arrays.asList(
Arrays.asList(2003, "ARRAY<DATE>", 2003), Arrays.asList(2002, "MAP<STRING,INT>", 2002));
Comment thread
msrathore-db marked this conversation as resolved.
Outdated

List<List<Object>> updatedRows = metadataResultSetBuilder.getThriftRows(rows, columns);

List<Object> arrayRow = updatedRows.get(0);
assertEquals(12, arrayRow.get(0)); // DATA_TYPE should be 12 (VARCHAR)
assertEquals(12, arrayRow.get(2)); // SQL_DATA_TYPE should be 12 (VARCHAR)

List<Object> mapRow = updatedRows.get(1);
assertEquals(12, mapRow.get(0)); // DATA_TYPE should be 12 (VARCHAR)
assertEquals(12, mapRow.get(2)); // SQL_DATA_TYPE should be 12 (VARCHAR)
}

@Test
void testComplexTypesReturnActualCodesWhenSupportEnabled() throws SQLException {
Comment thread
msrathore-db marked this conversation as resolved.
when(connectionContext.isComplexDatatypeSupportEnabled()).thenReturn(true);

List<ResultColumn> columns =
Arrays.asList(DATA_TYPE_COLUMN, COLUMN_TYPE_COLUMN, SQL_DATA_TYPE_COLUMN);
List<List<Object>> rows =
Arrays.asList(
Arrays.asList(2003, "ARRAY<DATE>", 2003), Arrays.asList(2002, "MAP<STRING,INT>", 2002));
Comment thread
msrathore-db marked this conversation as resolved.
Outdated

List<List<Object>> updatedRows = metadataResultSetBuilder.getThriftRows(rows, columns);

List<Object> arrayRow = updatedRows.get(0);
assertEquals(2003, arrayRow.get(0)); // DATA_TYPE should be 2003 (ARRAY)
assertEquals(2003, arrayRow.get(2)); // SQL_DATA_TYPE should be 2003 (ARRAY)

List<Object> mapRow = updatedRows.get(1);
assertEquals(2002, mapRow.get(0)); // DATA_TYPE should be 2002 (MAP)
assertEquals(2002, mapRow.get(2)); // SQL_DATA_TYPE should be 2002 (MAP)
}
}
Loading