Apologies for being late to the discussion but I was busy at Data & AI summit last week. I am supportive of this initiative even though I have not been following this very closely.
Thank you Andrew and Raphael for driving this innovation. Andy. On Wed, Jun 29, 2022 at 11:40 AM Raphael Taylor-Davies <tustv...@apache.org> wrote: > The proposal [1] to switch DataFusion to object_store_rs is looking like > it will be merged in the coming few days, so any concerns/reservations > please speak up! > > Regarding release management, my personal preference would be for it to > managed and released as part of the arrow-rs release process. This will > keep the maintenance burden low by building on a preexisting and mature > release process, whilst permitting closer integration between arrow-rs and > object storage going forward. However, should we decide on something > different, both myself and Andrew are happy to continue to support the > crates evolution regardless, and the hope is that through donating this > project to arrow others may come to share this burden in time > > [1] > https://github.com/apache/arrow-datafusion/pull/2677#issuecomment-1170271295 > > On 2022/06/28 22:52:50 Will Jones wrote: > > Thanks for bringing up this discussion, Andrew! > > > > I started contributing to the crate in the hopes of using it in > DataFusion > > and delta-rs. I've thus far found it to be a high-quality codebase. I'd > > like to see the crate adopted throughout the Rust Arrow ecosystem, and if > > going under Apache governance helps that end, I support that. > > > > My one question is: Do we have maintainers (not sure if it has to be a > > committer or a PMC member) in the Apache project who are ready to take on > > release tasks for the crate? > > > > > > > > On Tue, Jun 28, 2022 at 11:44 AM Andrew Lamb <al...@influxdata.com> > wrote: > > > > > Hello Rust / Arrow community, > > > > > > There is a proposal [1] in DataFusion to switch the abstraction used to > > > read from remote object storage. Due to various reasons, this code is > > > currently in its own crate/library [2] with unclear governance. We > would > > > like to contribute this code to the Apache Arrow project and I am > hoping to > > > gather community feedback on this idea. > > > > > > The full details can be found on [3]. Here is the rationale > > > > > > 1. A common, high quality object store abstraction for communicating > with > > > various remote object stores is useful for a range of projects and > > > usecases. > > > 2. Such a library is directly aligned with the Arrow mission of > providing > > > building blocks for modern high performance analytics systems > > > 3. The clear governance of Apache Arrow offers the best chance to > build a > > > unified and strong community around this crate, hopefully both > increasing > > > its adoption and attracting community contributions for its long term > > > evolution and maintenance > > > > > > Please let us know your thoughts, > > > Andrew > > > > > > > > > [1] https://github.com/apache/arrow-datafusion/issues/2489 > > > [2] https://github.com/influxdata/object_store_rs/issues/41 > > > [3] https://github.com/influxdata/object_store_rs/issues/41 > > > > > >
