pingzh opened a new pull request, #7343: URL: https://github.com/apache/datafusion-comet/pull/7343
## Which issue does this PR close? Closes #6741. Final part of the split of #7117, following #7260, #7261, and #7283. ## Rationale for this change `make_date(6000000, 1, 1)` is a valid Java calendar date, but its epoch day exceeds Date32. Spark raises `ARITHMETIC_OVERFLOW` in both ANSI modes. Comet currently loses the overflow distinction and returns NULL with ANSI disabled or reports an invalid calendar date with ANSI enabled. ## What changes are included in this PR? Separate calendar validation from checked Date32 conversion and propagate overflow through the existing typed Spark error path. Invalid calendar fields retain their ANSI-dependent behavior, including Java's year limits; NULL arguments still skip validation. Add coverage for Date32 endpoints and adjacent overflow, calendar-error precedence, NULL and empty input, and overflowing rows skipped by LIMIT, `RDD.take`, and `mapPartitions`. Document the overflow behavior. ## How are these changes tested? - The new native overflow regression fails before the fix; all 10 focused `make_date` Rust tests pass after it. - Rust formatting and focused Clippy checks (`-D warnings`) pass. - Full JVM reactor compilation and format/style checks pass with Spark 4.1.3 / JDK 17. - All 18 focused Spark tests pass against the freshly rebuilt JNI library: both `make_date` SQL files, temporal tests, and planner safety regressions. - Before merging, request `run-spark-4.1-tests` and `run-all-spark-profiles` for Spark SQL and version coverage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
