wForget commented on PR #2244: URL: https://github.com/apache/datafusion-comet/pull/2244#issuecomment-3238626929
> > @comphead do you remember if we looked at OpenDAL originally for HDFS support? > > Yeah, the main concern was limited support for HDFS client https://github.com/Kimahriman/hdfs-native?tab=readme-ov-file#supported-hdfs-settings which can lead to lack of support for IO retries, authentication, etc but this PR uses openDAL as an optional feature, which IMO should be fine The dependency you mentioned corresponds to the [services-hdfs-native](https://github.com/apache/opendal/blob/f00fd1cbc9bb4b0d26f714fb4dae8667e036375c/core/Cargo.toml#L346-L347) feature of OpenDAL, which is a fully native HDFS client. This PR introduces the [service-hdfs](https://github.com/apache/opendal/blob/f00fd1cbc9bb4b0d26f714fb4dae8667e036375c/core/Cargo.toml#L306-L307) feature, which uses [hdrs](https://github.com/Xuanwo/hdrs) crate, a jvm based libhfs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
