Also, a Google document would be fine to get started. We can transfer it to
Confluence and JIRA after some discussion
On Thu, Oct 4, 2018, 11:45 AM Wes McKinney wrote:
> hi Pearu -- yes, I had thought of this work working on the arrow_gpu
> library. Some time ago I opened
>
> https://issues.apac
hi Pearu -- yes, I had thought of this work working on the arrow_gpu
library. Some time ago I opened
https://issues.apache.org/jira/browse/ARROW-1470
thinking that it would be good to combine the MemoryPool* concept and
the AllocateBuffer concept into a single abstract interface. Such an
interfac
Hi,
Currently, the arrow host memory management includes MemoryPool to
accelerate memory operations (new/free).
Would there be interest in supporting the same concept in CUDA memory
management to reduce the overhead of cudaMalloc/cudaFree?
Best regards,
Pearu
On Wed, Oct 3, 2018 at 11:44 PM Pearu
Hi,
I can make the initial design document from the existing comments.
Do you have examples of some earlier design documents used for similar
purpose? Would shared google docs be OK?
Btw, I also figured out an answer to my original question, here is a
partial codelet for accessing the batch column
What are the action items on this? Sounds like we need to start a
design document. I'm afraid I don't have the bandwidth to champion GPU
functionality at the moment but I will participate in design
discussions and help break down complex tasks into more accessible
JIRA issues.
Thanks
Wes
On Fri, S
Seems like there is a fair bit of work to do to specify APIs and
semantics. I suggest we create a Google document or something
collaborative where we can enumerate and discuss the issues we want to
resolve, and then make a list of the concrete development.
The underlying problem IMHO in ARROW-2446
Hi Wes,
Yes, it makes sense.
If I understand you correctly then defining a device abstraction would also
bring Buffer and CudaBuffer under the same umbrella (that would be opposite
approach to ARROW-2446, btw).
This issue is also related to
https://github.com/dmlc/dlpack/blob/master/include/dl
hi Pearu,
Yes, I think it would be a good idea to develop some tools to make
interacting with device memory using the existing data structures work
seamlessly.
This is all closely related to
https://issues.apache.org/jira/browse/ARROW-2447
I would say step 1 would be defining the device abstrac
Hi,
Consider the following use case:
schema =
cbuf =
cbatch = pa.cuda.read_record_batch(schema, cbuf)
Note that cbatch is pa.RecordBatch instance where data pointers are device
pointers.
for col in cbatch.columns:
# here col is, say, FloatArray, that data pointer is a device pointer
#
