andygrove opened a new issue, #1342: URL: https://github.com/apache/datafusion-comet/issues/1342
# Introduction These notes reflect things I am personally involved in or thinking about and may not cover all activities. Feel free to add comments for anything that I missed. # comet-parquet-exec The proof-of-concept work for adding new `native_datafusion` and `native_iceberg_compat` implementations of the Parquet scan has now been merged into the main branch. These implementations can be enabled via the new `COMET_NATIVE_SCAN_IMPL` configuration setting and are disabled by default because they are not yet complete. The focus now is on getting all existing tests to pass with these new scans enabled. Then, we can move on to adding tests for reading complex types (arrays, structs, maps) from Parquet files. # Improving type-checking and test coverage An initial [PR](https://github.com/apache/datafusion-comet/pull/1331) was merged, which refactored QueryPlanSerde.scala to move some inner methods to top-level methods, making them accessible from unit tests, thus making it easier to write unit tests per expression. There is a follow-up [draft PR ](https://github.com/apache/datafusion-comet/pull/1339) to improve test coverage for array expressions. Feedback on the approach is welcome before this is extended to improve tests for all other expressions. # Array expressions There are [several PRs](https://github.com/apache/datafusion-comet/pulls?q=is%3Apr+is%3Aopen+array) ready for review for adding more array expressions. # Upgrade to DataFusion 45 There is a [draft PR](https://github.com/apache/datafusion-comet/pull/1332) to prepare to upgrade to DataFusion 45. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
