Ah, yes, thanks for the reminder. That's one of the things that needs to be
addressed for sure.
-David
On Tue, Jan 18, 2022, at 17:48, Supun Kamburugamuve wrote:
> One general observation. I think this implementation uses the polling to
> check the progress. Because of the client server semantic
One general observation. I think this implementation uses the polling to
check the progress. Because of the client server semantics of Arrow Flight,
you may need to use an interrupt based polling like epoll to avoid the busy
looping.
Best,
Supun..
On Tue, Jan 18, 2022 at 8:13 AM David Li wrote:
To follow up on this thread, we now have some new integration tests for
Arrow JS in master. These will help us prevent bundler issues in the
future. The integration tests now bundle a couple example programs (
https://github.com/apache/arrow/tree/master/js/test/bundle) with some
popular bundlers (
Thanks for those results, Yibo! Looks like there's still more room for
improvement here. Yes, things are a little unstable, though I didn't
get that much trouble trying to just start the benchmark - I will need
to find suitable hardware and iron out these issues. Note that I've
only implemented DoG
Some updates.
I tested David's UCX transport patch over 100Gb network. FlightRPC over
UCX/RDMA improves throughput about 50%, with lower and flat latency.
And I think there are chances to improve further. See test report [1].
For the data plane approach, the PoC shared memory data plane also
