rgbuilds commented on PR #26195: URL: https://github.com/apache/datafusion/pull/26195#issuecomment-6209962471
@zhuqi-lucas Thanks for the detailed review. I addressed the three actionable points in `c14317f1f` and `5bacdb13b`: - Restored a safe recursive-equality fast path for identical schemas and unchanged nested levels. - Replaced the unguarded presence-map lookup with a defensive lookup and added a focused regression test. - Clarified the container-type invariant by naming the Arrow types whose nested evolution is rejected during schema merging. The duplicate-name issue is valid and also affects the pre-existing top-level logic. Correctly consolidating duplicate fields within a single source has broader semantics than this fix, so I left it unchanged and suggest tracking it separately unless you consider it blocking. I also reran the optimized schema-inference benchmark. Against current `upstream/main`, identical schemas were 2.78–4.99% faster, while evolving nested schemas were 2.11–6.07% slower, adding approximately 0.041–0.947 ms in the tested cases. Formatting and the modified-crate Clippy checks pass. The focused unit, Parquet integration, and SQL logic tests also pass. Ready for another look when convenient. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
