aryan-212 opened a new pull request, #19984: URL: https://github.com/apache/datafusion/pull/19984
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> - This PR is part of the [Utf8View support](https://github.com/apache/datafusion/issues/10918) epic. It adds `Utf8View` support in the Spark-compat layer. ## Rationale for this change The `SparkConcat` function should accept `Utf8View` and mixed string types in line with the main DataFusion concat. This PR adds that support and follows the same patterns as [DataFusion’s concat](https://github.com/apache/datafusion/blob/main/datafusion/functions/src/string/concat.rs). ## What changes are included in this PR? - Extend the type signature to accept `Utf8View` in addition to `Utf8` and `LargeUtf8` via `TypeSignature::Variadic(vec![Utf8View, Utf8, LargeUtf8])` matching DataFusion’s concat. - In `return_field_from_args`, compute the result type with precedence Utf8View > LargeUtf8 > Utf8. In spark_concat, handle Utf8View and LargeUtf8 in scalar paths (zero-argument and all-NULL). ## Are these changes tested? Yes. - Unit tests: `cargo test --package datafusion-spark function::string::concat::tests`, including `test_concat_utf8view`. - Sqllogictest: `spark/string/concat.slt` includes a “**Utf8View: no extra CAST in plan**” case that uses EXPLAIN and a temporary table to ensure no extra CASTs when using arrow_cast(..., 'Utf8View') with table columns. ## Are there any user-facing changes? - **API:** SparkConcat’s signature is extended to include Utf8View in the variadic list. No breaking changes. _used gpt to rephrase some of these points_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
