Hi David and Antoine, Thanks for your input. On past experience talking to some other Arrow / Spark developers, we anticipate that it would take a long time to get into Spark. Our plan was to build up a user base in the Arrow community before submitting for inclusion to Spark. Is there a place the code can live in the mean time?
Matt Phelps From: Antoine Pitrou <anto...@python.org> Date: Monday, October 17, 2022 at 2:48 PM To: dev@arrow.apache.org <dev@arrow.apache.org> Subject: Re: [DISCUSS] Integrate existing Spark connector for Flight CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Le 17/10/2022 à 21:27, David Li a écrit : > Hey Matt, > > This is cool to see. To be clear, this is an implementation of Spark > DataSourceV2 using Arrow Flight? > > I think the questions I have are: > > - Does this belong under Arrow, or under Spark - I lean towards it being > closer to Spark than Arrow; FWIW, that is my feeling as well. Regards Antoine.
