Re: [DISCUSS][Format] Draft implementation of string view array format

2023-08-03 Thread Benjamin Kietzman
Hello all, I think that guarantees on masked values are worthwhile to define for more than a single type in isolation. In particular, requiring this exclusively for Utf8View will leave Utf8 and LargeUtf8 as arrays which *may* legally have non-utf8 masked values but cannot be consumed by arrow-rs.

Re: [VOTE][RUST][DataFusion] Release DataFusion Python Bindings 28.0.0 RC1

2023-08-03 Thread Jeremy Dyer
+1 (non-binding) Verified with x86 ubuntu machine and also M1 OSX. Thanks for getting the release together Andy - Jeremy Dyer On Tue, Aug 1, 2023 at 12:30 PM L. C. Hsieh wrote: > +1 (binding) > > Verified on M1 Mac. > > Thanks Andy. > > On Tue, Aug 1, 2023 at 8:06 AM Andy Grove wrote: > > >

Re: [DISCUSS][Format][Flight] Long-running queries support

2023-08-03 Thread David Li
Thanks Kou! For those interested, Spark recently landed a proposal with some similar goals in Spark Connect, calling it "reattachable execution": https://github.com/apache/spark/pull/42228 >From my understanding, Flight is more flexible here, decoupling the query >execution from fetching the r

[DISCUSS] DataFusion changes to timestamp arithmetic and decimal division

2023-08-03 Thread Andrew Lamb
BTW I would like to bring attention to the following PR: [1] It has some non trivial changes: 1. date/time arithmetic is done using Durations (e.g. X milliseconds) rather than Intervals (e.g. "months") which makes behavior consistent 2. Changes the output type of Decimal128 division to avoid vario