Jefffrey commented on PR #19323: URL: https://github.com/apache/datafusion/pull/19323#issuecomment-3679746916
@rluvaton are we sure on this requirement? I took a look at Spark code and it seems to be nullable, unless I'm reading it wrong/looking at wrong place: https://github.com/apache/spark/blob/20af8bdfb9073be5711cea5df0c5ce0ff168e92c/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala#L107-L114 Also we need to consider when the bit lengths is an array; if there is an invalid bit length it should null, see this example test: ```sql statement ok CREATE TABLE test_table ( expr STRING NOT NULL, bit_length INT NOT NULL ) as VALUES ('foo', 0), ('foo', 999) ; query T select sha2(arrow_cast(expr, 'Utf8'), bit_length) from test_table; ---- 2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae NULL ``` So deriving nullable = false would only work if the bit_length is a constant scalar. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
