andygrove commented on PR #5614: URL: https://github.com/apache/datafusion-comet/pull/5614#issuecomment-5553105326
This merged while I was still working through the review above, so none of it was blocking. I have moved the actionable items into #5712 so they do not get lost: the perf crossover above roughly a thousand elements per row and the missing dispatcher benchmark arm, the unbounded per-batch allocation outside the memory pool, the missing `SequenceBatchTooLarge` test, and the documentation items. To be clear about what I did verify, since it is the reassuring part: all five of Spark's `sequenceLength` failure paths match byte for byte through the native kernel on both 3.5 and 4.1, the fuzz found no divergences, and fourteen downstream consumers of the produced list agree with Spark. Nice work on the error parity, it is exact. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
