comphead commented on PR #5786: URL: https://github.com/apache/datafusion-comet/pull/5786#issuecomment-5732247799
@ErikBPF I didn't check tests, but I would expect the Comet falls back or at least fails in following scenarios: - read duplicated struct from single parquet files - read duplicated struct from multiple parquet files, where struct with duplicated fields is a result of merging schema - tests with merge schema true/false - tests with `spark.read.schema.parquet()` covering combinations when bad struct is in file or in schema, or both, or none. - dont fallback if bad struct exist in parquet but not read I think most of cases following amazing work that @dwsmith1983 made in #5654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
