kylebarron commented on issue #1227: URL: https://github.com/apache/datafusion-python/issues/1227#issuecomment-3351913683
> Keep arrow pyarrow: Compiles successfully but doesn't remove PyArrow dependency, so it doesn't solve the problem of making PyArrow optional. The `pyarrow` feature of Rust `arrow` doesn't itself require PyArrow at runtime for data import IIRC. > pyo3-arrow: Hit multiple blockers including requiring updates to 15 or more files importing from `arrow::pyarrow` If maintainers are interested in switching to `pyo3-arrow`, we could have a first PR making this switch. > pyo3-arrow conflicts with abi3 for Python <3.11 You can turn off the buffer protocol support by passing `--no-default-features` > version incompatibility (arrow pyarrow needs pyo3 0.25, pyo3-arrow needs 0.26) There's a [table with version compatibility](https://docs.rs/pyo3-arrow/latest/pyo3_arrow/#version-compatibility). You need to match the version of arrow that DataFusion is using. > and with `default-features = false`, pyo3-arrow doesn't export `PyArrowType` at all I don't know what you mean by `PyArrowType`. Do you mean for the data type? There's nothing called `PyArrowType` in `pyo3-arrow` or `arrow`. You can use [`PyDataType`](https://docs.rs/pyo3-arrow/latest/pyo3_arrow/struct.PyDataType.html). That doesn't depend on the buffer protocol feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
