dwsmith1983 opened a new issue, #7458: URL: https://github.com/apache/datafusion-comet/issues/7458
### Describe the bug #6666 changed `CometLiteral` to read array literal values with `toArray[Any](elementType)` instead of `ArrayData.array`. A constant-folded array such as `sequence(1, 5)` is an `UnsafeArrayData`, and `UnsafeArrayData.array()` throws. Before the change, planning such a query failed with `UNSUPPORTED_CALL`. The test added for that change is the first three queries in `spark/src/test/resources/sql-tests/operators/one_row_relation.sql`. That file has no `-- ConstantFolding: enabled` line, and `CometSqlFileTestSuite` excludes `ConstantFolding` unless the file asks for it (the directive came in #6731, two days after #6666). So `SELECT sequence(1, 5)` there never folds. It runs through the native `CometSequence`, and the `CometLiteral` lines from #6666 are never reached. I couldn't find another test on `main` that folds an array into an `UnsafeArrayData` literal under a Comet operator, so putting `.array` back should go unnoticed. Found in #7449, the `branch-1.1` backport of the fix, which adds a Scala test that does fold: https://github.com/apache/datafusion-comet/pull/7449#pullrequestreview-5754272211 ### Steps to reproduce 1. On `main`, change the two `toArray[Any](...)` calls in `spark/src/main/scala/org/apache/comet/serde/literals.scala` back to `.array`. 2. Run `CometSqlFileTestSuite` for `one_row_relation`. With folding excluded, `sequence(1, 5)` stays a `Sequence` expression, so the reverted lines are never called and the file should still pass. ### Expected behavior A test on `main` fails when the fix is reverted. Either of these would do it: - Add `-- ConstantFolding: enabled` to the queries in `one_row_relation.sql`, or move them into a fixture that has it, such as `expressions/array/folded_array_map_literals.sql`. - Port the test from #7449, which projects `sequence(1, 5)` and `array(sequence(1, 3), sequence(4, 5))` over a Parquet table. The nested query covers the inner `makeListLiteral` call as well. ### Additional context The queries need to read from a table, or run over a `OneRowRelation` with folding on, so that the folded literal sits under an operator Comet converts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
