Hi Jorge, Thanks for your response.
My current hypothesis is that arrow2 will be donated to Apache Arrow, I > just don't feel comfortable and have the energy doing so right now. Aside from the burden of going through the donation process (maybe other PMCs members are willing to help here?), what would make you comfortable with the donation? Are there specific milestones you are waiting for? If the docs on the arrow2 repo [1] are up-to-date it seems like the main concern is versioning, and sticking faithfully to sem-ver. It sounds like Rust is going to move out of lock-sync versioning with the other implementations. So maybe the gaps are small? There are advantages and disadvantages to having a project hosted by the ASF, so I can understand that it might not be an easy decision. However, as more development is done outside the ASF additional work for IP clearance accumulates if the project ends up getting donated. This seems like it might lead to a cycle where the timing is never quite right. Separately, from an outsider perspective on Rust development, I think a community over code approach applies here. If arrow2 is decided to be the best path forward for DataFusion, continuing to expend effort on maintaining arrow-rs doesn't seem like a good use of people's time given the high degree of overlap. I think it is a bad result if arrow-rs ends up bit-rotting due to lack of maintainers (i.e. I think it would be better to intentionally deprecate it). Hopefully consensus can be reached among the Rust maintainers and we can have clear messaging for users over the future of both projects. Cheers, Micah [1] https://github.com/jorgecarleitao/arrow2#any-plans-to-merge-with-the-apache-arrow-project On Wed, Jan 19, 2022 at 10:19 AM Jorge Cardoso Leitão < [email protected]> wrote: > Hi, > > Thank you for raising this here and for your comments. I am very humbled by > the feedback and adoption that arrow2 got so far. > > My current hypothesis is that arrow2 will be donated to Apache Arrow, I > just don't feel comfortable and have the energy doing so right now. > > Thank you for your understanding, > Jorge > > > On Mon, Jan 17, 2022 at 6:21 PM Wes McKinney <[email protected]> wrote: > > > Sounds good, thanks all and look forward to hearing more about this. > > > > To second what Micah said, a reminder to please engage with civility. > > The ASF's code of conduct is found here [1]. We are all volunteering > > our time to try to do what is best for the developer and user > > communities' long-term health and success. > > > > [1]: https://www.apache.org/foundation/policies/conduct.html > > > > > > On Mon, Jan 17, 2022 at 6:07 AM Andrew Lamb <[email protected]> > wrote: > > > > > > For what it is worth, I personally would likely spend much less time > > > maintaining arrow-rs if datafusion switched to arrow2. That discussion > is > > > happening independently here [1]. > > > > > > [1] https://github.com/apache/arrow-datafusion/issues/1532 > > > > > > On Sun, Jan 16, 2022 at 11:17 PM Micah Kornfield < > [email protected]> > > > wrote: > > > > > > > I agree, Jorge's point of view (and anyone else who has contributed > to > > > > arrow2) is important here. > > > > > > > > One thing that isn't exactly clear to me from the linked issue is how > > much > > > > interest there is in the community for maintaining arrow-rs? How > much > > is a > > > > donation of arrow2 a factor here? > > > > > > > > Also, the last time we discussed this on the mailing list I think > > things > > > > got contentious. Just a reminder please communicate with care. [1] > > has > > > > some good advice on this. > > > > > > > > Thanks, > > > > Micah > > > > > > > > [1] https://community.apache.org/contributors/etiquette > > > > > > > > On Sun, Jan 16, 2022 at 6:51 PM QP Hou <[email protected]> wrote: > > > > > > > > > Hi Wes, > > > > > > > > > > I believe what you mentioned is the plan, i.e. move arrow2 to ASF > in > > > > > the long run when it stabalize on its design/API and could benefit > > > > > from a more rigorous release process. From what I have seen, the > > > > > project is still undergoing major API changes on a monthly basis, > so > > > > > quick releases and fast user feedback is quite valuable. But let's > > > > > hear Jorge's point of view on this first. > > > > > > > > > > On Sun, Jan 16, 2022 at 2:42 PM Wes McKinney <[email protected]> > > > > wrote: > > > > > > > > > > > > Is there a possibility of donating arrow2 to the Arrow project > (at > > > > > > some point)? The main impact to development would be holding > votes > > on > > > > > > releases, but this is probably a good thing long term from a > > > > > > governance standpoint. The answer may be "not right now" and > that's > > > > > > fine. Having many of the same people split across projects with > > > > > > different governance structures is less than ideal. > > > > > > > > > > > > On Fri, Jan 14, 2022 at 1:15 PM Andrew Lamb < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > I wanted to draw your attention to two issues of significance > to > > the > > > > > Rust > > > > > > > Arrow implementations > > > > > > > > > > > > > > Discussion for switching DataFusion to use arrow2: > > > > > > > https://github.com/apache/arrow-datafusion/issues/1532 > > > > > > > > > > > > > > Discussion for what to do with arrow if DataFusion switches to > > use > > > > > arrow2: > > > > > > > https://github.com/apache/arrow-rs/issues/1176 > > > > > > > > > > > > > > The second is likely the most pertinent to people on this > mailing > > > > > list, but > > > > > > > the first is the reason why the second has become important. > > > > > > > > > > > > > > Andrew > > > > > > > > > > > >
