shehabgamin commented on issue #14008: URL: https://github.com/apache/datafusion/issues/14008#issuecomment-2609034408
My apologies @alamb, the DataFusion upgrade from the latest main branch commit is smoother than I initially thought. After investigating the flood of errors, I discovered that many were resolved by simply updating Sail's `serde-arrow` dependency to Arrow `54`. Projects without PyO3 or the `pyarrow` feature in DataFusion should experience a seamless upgrade (as of writing). Projects using PyO3 with the `pyarrow` feature enabled will have varying experiences based on their usage of PyO3. **PyO3 `0.23.3`** DataFusion `45` upgrades from PyO3 `0.22` to `0.23.3`. This is an exciting change, but may introduce significant breaking changes for PyO3 users. Since these changes vary based on PyO3 usage, I'm not listing Sail's specific changes here. Users can refer to the PyO3 migration guide: https://pyo3.rs/v0.23.0/migration **DataFusion** `ValuesExec` is now deprecated. The deprecation message is a bit confusing though. It currently states: "Use `MemoryExec::try_new_as_values` instead", but I think should say: "Use `MemoryExec::try_new_as_values` or `MemoryExec::try_new_from_batches` instead". Or, just simply: "Use `MemoryExec` instead". If you'd like to see these changes, they're in my PR that's testing the regression fixes: https://github.com/lakehq/sail/pull/355 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
