0lai0 opened a new issue, #5334:
URL: https://github.com/apache/datafusion-comet/issues/5334
### Describe the bug
`cast_boolean_to_decimal`
(`native/spark-expr/src/conversion_funcs/boolean.rs`) can return `Ok` with a
`Decimal128Array` whose values do not fit the declared precision.
For `true`, the implementation writes the unscaled value `10^scale`. For
`DECIMAL(3,3)`, that is `1000`, which represents `1.000`. But `DECIMAL(3,3)`
only allows `-0.999` to `0.999` (unscaled absolute value ≤ `999`).
The code intends to catch this via `with_precision_and_scale` and remap to
`decimal_overflow_error`, but Arrow's `with_precision_and_scale` (arrow-array
58.4.0, `primitive_array.rs:1615`) only validates type metadata (`scale ≤
precision ≤ 38`). It does **not** validate per-value precision, so the overflow
remap never runs and invalid data is returned as success.
By contrast, `cast_int_to_decimal128_internal` in the same crate checks each
value with `is_validate_decimal_precision`.
This is related to but distinct from #5068.
#5068 describes the same code as "unconditionally raises
`SparkError::NumericValueOutOfRange` when the value does not fit". In fact the
code does not raise at all for this case, it returns invalid data.
### Steps to reproduce
Rust unit test (current behavior returns `Ok`, not an error):
```rust
let array: ArrayRef = Arc::new(BooleanArray::from(vec![Some(true)]));
let result = cast_boolean_to_decimal(&array, 3, 3).unwrap();
// result contains unscaled 1000 labeled as Decimal128(3, 3)
```
Or SQL:
```sql
SELECT CAST(true AS DECIMAL(3,3));
```
Spark rejects this value (ANSI: error; legacy/try: NULL — see #5068).
Comet currently accepts it and produces an invalid decimal.
### Expected behavior
When `10^scale` does not fit the target precision and the batch contains
`true`, Comet should not return a successful array containing that unscaled
value.
Prefer matching Spark: NULL in legacy/try, `NUMERIC_VALUE_OUT_OF_RANGE`
under ANSI (#5068).
At minimum, do not emit a `Decimal128(p, s)` whose values violate precision.
1` (scanning for `true` when it fails) or validate each value with
`is_validate_decimal_precision`.
### Additional context
Found while working on #5095 (delegate int/float/boolean→decimal to Arrow
safe cast).
Existing Rust coverage in `boolean.rs` only exercises `DECIMAL(10,4)`
(single test `test_bool_to_decimal_cast`). The Scala `CometNativeCastSuite`
only exercises non-overflowing targets `DECIMAL(10,2)`, `(14,4)`, `(30,0)`.
That is why this was not caught earlier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
