Pastoray commented on code in PR #27273: URL: https://github.com/apache/datafusion/pull/27273#discussion_r4543285864
########## datafusion/spark/src/function/string/encode.rs: ########## Review Comment: Not originally, but I have now, using PySpark 4.3.0. The defaults, bytes and error text match. The check also found that my legacy mode aliases were wrong, and I’ve pushed a fix using Java’s real alias sets. Legacy mode still covers only nine charsets, not all of Java’s. ########## datafusion/sqllogictest/test_files/spark/string/encode.slt: ########## Review Comment: It returns NULL, and I checked that this matches Spark 4.3.0: a NULL value with an invalid charset gives NULL, both for a constant charset and per row. A non NULL value with an invalid charset errors, and a NULL charset gives NULL. This is covered in `encode.slt` and in the `test_nulls_are_not_validated` unit test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
