Skip to content

Commit c30a51e

Browse files
coderfendercoderfender
committed
int_to_binary
1 parent 9c1dc34 commit c30a51e

1 file changed

Lines changed: 107 additions & 16 deletions

File tree

‎docs/source/user-guide/latest/compatibility.md‎

Lines changed: 107 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -99,17 +99,16 @@ they will be identical to Spark. Unsorted results may have different row orderin
9999

100100
Cast operations in Comet fall into three levels of support:
101101

102-
- **Compatible**: The results match Apache Spark
103-
- **Incompatible**: The results may match Apache Spark for some inputs, but there are known issues where some inputs
102+
- **C (Compatible)**: The results match Apache Spark
103+
- **I (Incompatible)**: The results may match Apache Spark for some inputs, but there are known issues where some inputs
104104
will result in incorrect results or exceptions. The query stage will fall back to Spark by default. Setting
105105
`spark.comet.expression.Cast.allowIncompatible=true` will allow all incompatible casts to run natively in Comet, but this is not
106106
recommended for production use.
107-
- **Unsupported**: Comet does not provide a native version of this cast expression and the query stage will fall back to
107+
- **U (Unsupported)**: Comet does not provide a native version of this cast expression and the query stage will fall back to
108108
Spark.
109+
- **N/A**: Spark does not support this cast.
109110

110-
### Compatible Casts
111-
112-
The following cast operations are generally compatible with Spark except for the differences noted here.
111+
### Legacy Mode
113112

114113
<<<<<<< HEAD
115114

@@ -130,21 +129,113 @@ The following cast operations are generally compatible with Spark except for the
130129

131130
<!-- WARNING! DO NOT MANUALLY MODIFY CONTENT BETWEEN THE BEGIN AND END TAGS -->
132131

133-
<!--BEGIN:COMPAT_CAST_TABLE-->
134-
<!--END:COMPAT_CAST_TABLE-->
135-
136-
### Incompatible Casts
132+
<!--BEGIN:CAST_LEGACY_TABLE-->
133+
<!-- prettier-ignore-start -->
134+
| | binary | boolean | byte | date | decimal | double | float | integer | long | short | string | timestamp |
135+
|---|---|---|---|---|---|---|---|---|---|---|---|---|
136+
| binary | - | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | C | N/A |
137+
| boolean | N/A | - | C | N/A | U | C | C | C | C | C | C | U |
138+
| byte | U | C | - | N/A | C | C | C | C | C | C | C | U |
139+
| date | N/A | U | U | - | U | U | U | U | U | U | C | U |
140+
| decimal | N/A | C | C | N/A | - | C | C | C | C | C | C | U |
141+
| double | N/A | C | C | N/A | I | - | C | C | C | C | C | U |
142+
| float | N/A | C | C | N/A | I | C | - | C | C | C | C | U |
143+
| integer | U | C | C | N/A | C | C | C | - | C | C | C | U |
144+
| long | U | C | C | N/A | C | C | C | C | - | C | C | U |
145+
| short | U | C | C | N/A | C | C | C | C | C | - | C | U |
146+
| string | C | C | C | C | I | C | C | C | C | C | - | I |
147+
| timestamp | N/A | U | U | C | U | U | U | U | C | U | C | - |
148+
<!-- prettier-ignore-end -->
149+
150+
**Notes:**
151+
152+
- **decimal -> string**: There can be formatting differences in some case due to Spark using scientific notation where Comet does not
153+
- **double -> decimal**: There can be rounding differences
154+
- **double -> string**: There can be differences in precision. For example, the input "1.4E-45" will produce 1.0E-45 instead of 1.4E-45
155+
- **float -> decimal**: There can be rounding differences
156+
- **float -> string**: There can be differences in precision. For example, the input "1.4E-45" will produce 1.0E-45 instead of 1.4E-45
157+
- **string -> date**: Only supports years between 262143 BC and 262142 AD
158+
- **string -> decimal**: Does not support fullwidth unicode digits (e.g \\uFF10)
159+
or strings containing null bytes (e.g \\u0000)
160+
- **string -> timestamp**: Not all valid formats are supported
161+
<!--END:CAST_LEGACY_TABLE-->
137162

138-
The following cast operations are not compatible with Spark for all inputs and are disabled by default.
163+
### Try Mode
139164

140165
<!-- WARNING! DO NOT MANUALLY MODIFY CONTENT BETWEEN THE BEGIN AND END TAGS -->
141166

142-
<!--BEGIN:INCOMPAT_CAST_TABLE-->
143-
<!--END:INCOMPAT_CAST_TABLE-->
144-
145-
### Unsupported Casts
167+
<!--BEGIN:CAST_TRY_TABLE-->
168+
<!-- prettier-ignore-start -->
169+
| | binary | boolean | byte | date | decimal | double | float | integer | long | short | string | timestamp |
170+
|---|---|---|---|---|---|---|---|---|---|---|---|---|
171+
| binary | - | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | C | N/A |
172+
| boolean | N/A | - | C | N/A | U | C | C | C | C | C | C | U |
173+
| byte | U | C | - | N/A | C | C | C | C | C | C | C | U |
174+
| date | N/A | U | U | - | U | U | U | U | U | U | C | U |
175+
| decimal | N/A | C | C | N/A | - | C | C | C | C | C | C | U |
176+
| double | N/A | C | C | N/A | I | - | C | C | C | C | C | U |
177+
| float | N/A | C | C | N/A | I | C | - | C | C | C | C | U |
178+
| integer | U | C | C | N/A | C | C | C | - | C | C | C | U |
179+
| long | U | C | C | N/A | C | C | C | C | - | C | C | U |
180+
| short | U | C | C | N/A | C | C | C | C | C | - | C | U |
181+
| string | C | C | C | C | I | C | C | C | C | C | - | I |
182+
| timestamp | N/A | U | U | C | U | U | U | U | C | U | C | - |
183+
<!-- prettier-ignore-end -->
184+
185+
**Notes:**
146186

187+
<<<<<<< HEAD
147188
Any cast not listed in the previous tables is currently unsupported. We are working on adding more. See the
148189
[tracking issue](https://github.com/apache/datafusion-comet/issues/286) for more details.
149190

150-
> > > > > > > 6f8e1c629 (int_to_binary)
191+
> > > > > > > # 6f8e1c629 (int_to_binary)
192+
193+
- **decimal -> string**: There can be formatting differences in some case due to Spark using scientific notation where Comet does not
194+
- **double -> decimal**: There can be rounding differences
195+
- **double -> string**: There can be differences in precision. For example, the input "1.4E-45" will produce 1.0E-45 instead of 1.4E-45
196+
- **float -> decimal**: There can be rounding differences
197+
- **float -> string**: There can be differences in precision. For example, the input "1.4E-45" will produce 1.0E-45 instead of 1.4E-45
198+
- **string -> date**: Only supports years between 262143 BC and 262142 AD
199+
- **string -> decimal**: Does not support fullwidth unicode digits (e.g \\uFF10)
200+
or strings containing null bytes (e.g \\u0000)
201+
- **string -> timestamp**: Not all valid formats are supported
202+
<!--END:CAST_TRY_TABLE-->
203+
204+
### ANSI Mode
205+
206+
<!-- WARNING! DO NOT MANUALLY MODIFY CONTENT BETWEEN THE BEGIN AND END TAGS -->
207+
208+
<!--BEGIN:CAST_ANSI_TABLE-->
209+
<!-- prettier-ignore-start -->
210+
| | binary | boolean | byte | date | decimal | double | float | integer | long | short | string | timestamp |
211+
|---|---|---|---|---|---|---|---|---|---|---|---|---|
212+
| binary | - | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | C | N/A |
213+
| boolean | N/A | - | C | N/A | U | C | C | C | C | C | C | U |
214+
| byte | U | C | - | N/A | C | C | C | C | C | C | C | U |
215+
| date | N/A | U | U | - | U | U | U | U | U | U | C | U |
216+
| decimal | N/A | C | C | N/A | - | C | C | C | C | C | C | U |
217+
| double | N/A | C | C | N/A | I | - | C | C | C | C | C | U |
218+
| float | N/A | C | C | N/A | I | C | - | C | C | C | C | U |
219+
| integer | U | C | C | N/A | C | C | C | - | C | C | C | U |
220+
| long | U | C | C | N/A | C | C | C | C | - | C | C | U |
221+
| short | U | C | C | N/A | C | C | C | C | C | - | C | U |
222+
| string | C | C | C | C | I | C | C | C | C | C | - | I |
223+
| timestamp | N/A | U | U | C | U | U | U | U | C | U | C | - |
224+
<!-- prettier-ignore-end -->
225+
226+
**Notes:**
227+
228+
- **decimal -> string**: There can be formatting differences in some case due to Spark using scientific notation where Comet does not
229+
- **double -> decimal**: There can be rounding differences
230+
- **double -> string**: There can be differences in precision. For example, the input "1.4E-45" will produce 1.0E-45 instead of 1.4E-45
231+
- **float -> decimal**: There can be rounding differences
232+
- **float -> string**: There can be differences in precision. For example, the input "1.4E-45" will produce 1.0E-45 instead of 1.4E-45
233+
- **string -> date**: Only supports years between 262143 BC and 262142 AD
234+
- **string -> decimal**: Does not support fullwidth unicode digits (e.g \\uFF10)
235+
or strings containing null bytes (e.g \\u0000)
236+
- **string -> timestamp**: ANSI mode not supported
237+
<!--END:CAST_ANSI_TABLE-->
238+
239+
See the [tracking issue](https://github.com/apache/datafusion-comet/issues/286) for more details.
240+
241+
> > > > > > > acee9701f (int_to_binary)

0 commit comments

Comments
 (0)