I was wondering, for the benefit of lowering the entry barrier for users and especially future contributions who might find themselves confused by the amount of optional pieces that you can pick when building arrow, would it be reasonable to think of shipping plasma as a separate library? Like arrow-plasma with its own packaging/release cycle? That would also have the benefit of giving us a better understanding of how many people are actually depending on it based on how many people depend on that package.
It's true that there would be an initial burden in separating the codebase and building its own CI/release scripts, but I think it would ease life for people willing to contribute on arrow "ignoring" plasma and it would give plasma the chance to maybe get maintenance outside of the arrow developers from people who might not care about contributing to arrow itself at the moment. On Tue, Jul 13, 2021 at 2:18 PM Neal Richardson <neal.p.richard...@gmail.com> wrote: > Hi Jarek, > Your understanding sounds about right to me. That said, we are still > building and shipping Plasma for those that have come to depend on it and > will continue to do so unless/until it becomes a maintenance burden. But no > one active in the Arrow community is working on Plasma. > > Neal > > On Tue, Jul 13, 2021 at 3:07 AM Jarek Potiuk <pot...@apache.org> wrote: > > > Hello Arrow Community, > > > > We've had a very interesting talk at the Apache Airflow Summit about > > Airflow + Ray (which is really cool BTW and I am looking forward to > > capabilities it will give to Airflow) and we had some discussions that > > followed. From what I understand (maybe I am wrong?) the Plasma which was > > initially developed in Ray, then contributed to Arrow, and then ( > > > https://lists.apache.org/thread.html/r65b2852e4cddb1af8bff06d789bf3822d67777c5dfcd481414acd3d7%40%3Cdev.arrow.apache.org%3E > ) > > forked (?) by Ray and is kind-of abandoned in Arrow and not really > > maintained in Arrow any more (and likely Ray version and Arrow version > are > > not compatible /exchangeable). > > > > Is this correct understanding ? Any more comments or maybe explanation > > what is the relation between Arrow's Plasma and Ray's Plasma? > > > > Just to explain my interest - I am a PMC of Apache Airflow, I am an > > independent Open-Source contributor and advisor, and I am genuinely > > interested in Open-source business models and rationale of stakeholders > and > > how this plays out with individuals and the ASF/PMC and I wanted to > > understand the current state of Plasma :) > > > > J. > > > > >
