Re: Traffic control and cancel detect on flight do_get in python implementation

2023-07-07 Thread David Li
Yes, gRPC calls this feature the BDP probe (bandwidth-delay product) There's a short description of flow control here: https://grpc.io/docs/guides/flow-control/ It's somewhat old, but a gRPC dev suggests there's no way to influence this directly (other than by turning it off!): https://stackov

Re: [VOTE][RUST][DataFusion] Release DataFusion Python Bindings 27.0.0 RC1

2023-07-07 Thread vin jake
+1 (binding) Verified on M1 Mac. Thanks Andy On Thu, Jul 6, 2023, 01:13 Andy Grove wrote: > Hi, > > I would like to propose a release of Apache Arrow DataFusion Python > Bindings, > version 27.0.0. > > This release candidate is based on commit: > 3f81513d6c5fd109bdf8c509f81c0a587924d354 [1] >

Re: Traffic control and cancel detect on flight do_get in python implementation

2023-07-07 Thread Wenbo Hu
Sorry, my bad. It works over time. It seems that the grpc starts with a default window size, then update to a stable value according to options. Wenbo Hu 于2023年7月7日周五 23:32写道: > > Both my server and client are implemented in python now, Java Client > may be in the future. > Back pressure works, b

Re: Traffic control and cancel detect on flight do_get in python implementation

2023-07-07 Thread Wenbo Hu
Both my server and client are implemented in python now, Java Client may be in the future. Back pressure works, but not totally in control. Is there a way to configure how much memory used to read ahead for each grpc stream? For example, the client is calling do_get, then the client wants to limit

Re: [DISCUSS][Format] Draft implementation of string view array format

2023-07-07 Thread Raphael Taylor-Davies
Thus the approach you describe for validating an entire character buffer as UTF-8 then checking offsets will be just as valid for Utf8View arrays as for Utf8 arrays. The difference here is that it is perhaps expected for Utf8View to have gaps in the underlying data that are not referenced as part

Re: Traffic control and cancel detect on flight do_get in python implementation

2023-07-07 Thread David Li
gRPC has backpressure built in. Is your server in Java or Python/C++? If in Java, the server needs to explicitly poll isReady to respect backpressure. (This is rather wasteful, yes, and this will artificially throttle your peak bandwidth because of a long-standing flaw in gRPC-Java.) If in Pyth

Re: [VOTE][RUST][DataFusion] Release DataFusion Python Bindings 27.0.0 RC1

2023-07-07 Thread Jeremy Dyer
+1 (non-binding) Verified on m1 Mac and x86 Ubuntu 22 machines Thanks for churning these out so quickly Andy! -Jeremy Dyer Get Outlook for iOS From: Andrew Lamb Sent: Thursday, July 6, 2023 7:19:26 PM To: dev@arrow.apache.org Subject: Re