+1 for getting some kind of async implementation into the spec. I have proposed a few alternative approaches in the PR.
On Fri, Aug 9, 2024 at 1:18 PM Matt Topol <zotthewiz...@gmail.com> wrote: > Hello All, I'd like to discuss the potential addition of an > asynchronous-oriented version of the C Data Stream interface. > > This idea was originally brought up working with ADBC [1] as the current > ADBC interface is inherently synchronous and requires any attempts to be > asynchronous be created/managed/implemented by consumers rather than > providing an asynchronous API itself. As any such API would require needing > to handle interactions with the Arrow C Data Interface, it is better to > upstream such concepts to the Arrow format than create something specific > to ADBC. > > As such, I've created a PR [2] based on the discussions in the ADBC issue, > as a jumping off point for more discussion. I didn't go as far as creating > tests, examples, etc. for it yet as the core design of it might change > based on feedback in this thread and I figured I would wait until I get > more consensus from the community before I invest more time into it. > > Anyways, I'm looking for any and all feedback on the design, structure, > concept, etc. in the hopes of getting this officially added to the Arrow > spec. > > Thanks everyone! > > --Matt > > [1]: https://github.com/apache/arrow-adbc/issues/811 > [2]: https://github.com/apache/arrow/pull/43632 >
