andygrove commented on code in PR #399:
URL: https://github.com/apache/datafusion-comet/pull/399#discussion_r1594041255
##########
core/src/execution/datafusion/expressions/cast.rs:
##########
@@ -191,44 +191,219 @@ macro_rules! cast_int_to_int_macro {
.as_any()
.downcast_ref::<PrimitiveArray<$from_arrow_primitive_type>>()
.unwrap();
+
let spark_int_literal_suffix = match $from_data_type {
&DataType::Int64 => "L",
&DataType::Int16 => "S",
&DataType::Int8 => "T",
_ => "",
};
- let output_array = match $eval_mode {
- EvalMode::Legacy => cast_array
- .iter()
- .map(|value| match value {
- Some(value) => {
- Ok::<Option<$to_native_type>, CometError>(Some(value
as $to_native_type))
- }
- _ => Ok(None),
- })
- .collect::<Result<PrimitiveArray<$to_arrow_primitive_type>,
_>>(),
- _ => cast_array
- .iter()
- .map(|value| match value {
- Some(value) => {
+ let output_array = cast_array
+ .iter()
+ .map(|value| match value {
+ Some(value) => match $eval_mode {
+ EvalMode::Legacy => Ok(Some(value as $to_native_type)),
Review Comment:
I think that we should avoid matching on eval_mode for each value in the
array. The previous approach was to take a different code path for legacy mode.
I see a small regression to existing casts between ints from this change.
```
cast_int_to_int/cast_i32_to_i16
time: [3.1409 µs 3.1437 µs 3.1466 µs]
change: [+2.4650% +2.8884% +3.3150%] (p = 0.00 <
0.05)
Performance has regressed.
```
I will create a PR with this benchmark soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
