Smallfu666 opened a new pull request, #5855:
URL: https://github.com/apache/datafusion-comet/pull/5855
## Which issue does this PR close?
Closes #5389.
## Rationale for this change
`FuzzDataGenerator` never produces nulls for boolean, byte, short or int
columns, so fuzz runs
silently skip null handling for those four types.
All four are generated from a `Long` column and then narrowed:
```scala
generateColumn(r, DataTypes.LongType, numRows, options)
.map(_.asInstanceOf[Long].toByte)
```
`asInstanceOf[Long]` on a null goes through `BoxesRunTime.unboxToLong`,
which returns `0L` rather
than throwing. Every null becomes `0`, and the column comes back with no
nulls at all.
## What changes are included in this PR?
Match on the element instead of casting, so nulls survive the narrowing:
```scala
generateColumn(r, DataTypes.LongType, numRows, options).map {
case x: Long => x.toByte
case null => null
}
```
Applied to the boolean, byte, short and int arms. No other type is affected.
## How are these changes tested?
`DataGeneratorSuite` gains `allowNull produces nulls in every type that
honours it`. Its seed is
pinned, and it fails on all four arms before this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
