Re: Deep copy for ArrayData,Array, Table in C++ API

2020-06-29 Thread Wes McKinney
On Mon, Jun 29, 2020 at 9:33 AM Radu Teodorescu wrote: > > Yes, > I am set for what I need at the moment but since I went for a deepish dive > into the current API, and this has been a recurring use case over the year I > would extend a few proposals, for expanding Take: > 1. Add support for pac

Re: Deep copy for ArrayData,Array, Table in C++ API

2020-06-29 Thread Radu Teodorescu
Yes, I am set for what I need at the moment but since I went for a deepish dive into the current API, and this has been a recurring use case over the year I would extend a few proposals, for expanding Take: 1. Add support for packed indices - three avenues: a) expand Datum.Kind to allow f

Re: Deep copy for ArrayData,Array, Table in C++ API

2020-06-27 Thread Wes McKinney
Efficiently assembling a selection from multiple arrays will require some care -- our current implementation of Take involving ChunkedArray arguments is not too efficient, and they will need some rewriting for efficiency at some point in the future. Using some combination of Concatenate and Take ma

Re: Deep copy for ArrayData,Array, Table in C++ API

2020-06-26 Thread Antoine Pitrou
On Fri, 26 Jun 2020 13:56:26 -0400 Radu Teodorescu wrote: > Looks like Concatenate is my best bet if I am looking at putting together > ranges, certainly doesn’t look as neatly packaged as Take, but this might be > the right tool for this job. Yes, you could Slice the array and then Concatena

Re: Deep copy for ArrayData,Array, Table in C++ API

2020-06-26 Thread Radu Teodorescu
Looks like Concatenate is my best bet if I am looking at putting together ranges, certainly doesn’t look as neatly packaged as Take, but this might be the right tool for this job. > On Jun 26, 2020, at 1:01 PM, Radu Teodorescu > wrote: > > That is fabulous and pretty much it! > Follow up qu

Re: Deep copy for ArrayData,Array, Table in C++ API

2020-06-26 Thread Radu Teodorescu
That is fabulous and pretty much it! Follow up questions: 1. Is there any efficient way to refer to ranges: say I want to take rows 1000-2000 and 4000-5000, feels unwieldy to have to create an index array of 2000 elements and then also the underlying implementation would be less efficient having

Re: Deep copy for ArrayData,Array, Table in C++ API

2020-06-26 Thread Micah Kornfield
This sounds like the Take kernel? On Friday, June 26, 2020, Radu Teodorescu wrote: > (Light weigh topic this time) > Are there any existing functions for deep copying Array,ArrayData or Table > objects in the C++ API? > Ultimately, I am trying to get a bunch of sparse row ranges from a ranges >

Deep copy for ArrayData,Array, Table in C++ API

2020-06-26 Thread Radu Teodorescu
(Light weigh topic this time) Are there any existing functions for deep copying Array,ArrayData or Table objects in the C++ API? Ultimately, I am trying to get a bunch of sparse row ranges from a ranges into a contiguous new Table - I can see how I can copy Buffer and I can implement it all myse