Re: [DISCUSS][Format] Draft implementation of string view array format

2023-05-16 Thread Will Jones
Hello Ben, Thanks for your work on this. I think this will be an excellent addition to the format. If I understand correctly, multiple arrays can reference the same buffers in memory, but once they are written to IPC their data buffers will be duplicated. Is that right? Dictionary types have a s

Re: [DISCUSS][Format] Draft implementation of string view array format

2023-05-16 Thread Dewey Dunnington
Very cool! In addition to performance mentioned above, I could see this being useful for the R bindings - we already have a global string pool and a mechanism for keeping a vector of them alive. I don't see the C Data interface in the PR although I may have missed it - is that a part of the propo

Re: [DISCUSS][Format] Draft implementation of string view array format

2023-05-16 Thread Jacob Wujciak
Hello Everyone, I think keeping interoperability with the large ecosystem is a very important goal for arrow so I am overall in favor of this proposal! You mention benchmarks multiple times, are these results published somewhere? Thanks On Tue, May 16, 2023 at 11:39 PM Benjamin Kietzman wrote:

[DISCUSS][Format] Draft implementation of string view array format

2023-05-16 Thread Benjamin Kietzman
Hello all, As previously discussed on this list [1], an UmbraDB/DuckDB/Velox compatible "string view" type could bring several performance benefits to access and authoring of string data in the arrow format [2]. Additionally better interoperability with engines already using this format could be e

Re: [ANNOUNCE] New Arrow committer: Gang Wu

2023-05-16 Thread Gang Wu
Thank you all! I really enjoy being part of the community. Best, Gang On Tue, May 16, 2023 at 4:03 PM Raúl Cumplido wrote: > Congratulations! > > El mar, 16 may 2023 a las 8:12, Jin Shang () > escribió: > > > > Congrats! > > > > On Tue, May 16, 2023 at 9:51 AM Chao Sun wrote: > > > > > Congra

Re: [ANNOUNCE] New Arrow committer: Gang Wu

2023-05-16 Thread Raúl Cumplido
Congratulations! El mar, 16 may 2023 a las 8:12, Jin Shang () escribió: > > Congrats! > > On Tue, May 16, 2023 at 9:51 AM Chao Sun wrote: > > > Congrats Gang! > > > > On Mon, May 15, 2023 at 6:08 PM Jacob Wujciak > > wrote: > > > > > > Congrats! > > > > > > On Mon, May 15, 2023 at 6:53 PM Andrew