On Thu, Mar 27, 2025 at 1:56 PM Antoine Pitrou <anto...@python.org> wrote:
> > Indeed, it doesn't sound like a terrific use of Arrow maintainer time... > Especially as there's a growing feeling that Flight was not very well > designed, and should perhaps be slowly obsoleted in favor of more > focussed initiative (such as the Arrow-over-HTTP effort that's still not > finished :-)). > > Hi Antoine, I’ve been following the Arrow-over-HTTP discussions on the mailing lists. I’ve noticed that some have raised concerns about the difficulty of replacing Arrow Flight’s full-duplex streaming capabilities without HTTP/2, and I share those concerns. If HTTP/2 is necessary to achieve full-duplex streaming, I’m wondering what advantages it offers over simply continuing with Arrow Flight. I also recognize that HTTP/2 can be used without gRPC, but I’d like to better understand the reasoning behind that approach. If there are limitations in Arrow Flight’s design that make it less suitable for certain use cases, highlighting those areas could be really helpful for the community. Right now, there doesn’t seem to be much guidance indicating where Flight might not be the best fit. If a replacement for Arrow Flight emerges that supports full-duplex streaming of RecordBatches within a well-structured API framework, I’d certainly be open to considering it. Best regards, Rusty
