Pearu Peterson created ARROW-5258:
-
Summary: Expose file metadata of dataset pieces to caller
Key: ARROW-5258
URL: https://issues.apache.org/jira/browse/ARROW-5258
Project: Apache Arrow
Pearu Peterson created ARROW-4861:
-
Summary: [C++] Introduce MemoryPool::Memset method.
Key: ARROW-4861
URL: https://issues.apache.org/jira/browse/ARROW-4861
Project: Apache Arrow
Issue Type
Pearu Peterson created ARROW-4825:
-
Summary: MemoryPool is destructed before deallocating its buffers
leads to segfault
Key: ARROW-4825
URL: https://issues.apache.org/jira/browse/ARROW-4825
Project
Pearu Peterson created ARROW-4486:
-
Summary: pyarrow.cuda.Context.foreign_buffer should have a
`base=None` argument
Key: ARROW-4486
URL: https://issues.apache.org/jira/browse/ARROW-4486
Project
Hi,
For host memory, you can use pyarrow.foreign_buffer, see
https://arrow.apache.org/docs/python/generated/pyarrow.foreign_buffer.html
For device memory, one can use pyarrow.cuda.foreign_buffer.
HTH,
Pearu
On Thu, Nov 8, 2018 at 7:53 PM Randy Zwitch
wrote:
> Within OmniSci (MapD), we have
Pearu Peterson created ARROW-3653:
-
Summary: [Python/C++] Support data copying between different GPU
devices
Key: ARROW-3653
URL: https://issues.apache.org/jira/browse/ARROW-3653
Project: Apache
Pearu Peterson created ARROW-3624:
-
Summary: [Python/C++] Support for zero-sized device buffers
Key: ARROW-3624
URL: https://issues.apache.org/jira/browse/ARROW-3624
Project: Apache Arrow
Hi,
While researching CUDA context management code in arrow,
https://github.com/apache/arrow/blob/master/cpp/src/arrow/gpu/cuda_context.cc
,
I noticed the following:
CudaContextImpl has a member own_context_ that seems not being initialized
anywhere. The member own_context_ determines whether the
e,
>> > then our code can dispatch to the correct code paths. At the moment we
>> > can only really detect whether an arrow::Buffer* is a device buffer by
>> > dynamic_cast, and then that is not reliable because we may be a slice
>> > On Fri, Sep 28, 20
gt; > long as a buffer referencing it is aware of the underlying device,
> > then our code can dispatch to the correct code paths. At the moment we
> > can only really detect whether an arrow::Buffer* is a device buffer by
> > dynamic_cast, and then that is not reliable because w
Hi,
pa.cuda.CudaBuffer provides address and size of a device memory, so one can
construct numba.cuda MemoryPointer from a CudaBuffer instance and hence
there exists a way to pass CudaBuffer instances to numba.cuda jitted
functions.
The other way, having a device pointer and size (say, from numba.
Pearu Peterson created ARROW-3354:
-
Summary: read_record_patch interfaces differ in pyarrow and
pyarrow.cuda
Key: ARROW-3354
URL: https://issues.apache.org/jira/browse/ARROW-3354
Project: Apache
e a SlicedBuffer type that delegates
> GetDevice() calls to the parent buffer
>
> Let me know if what I'm saying makes sense. Kou and Antoine probably
> have some thoughts about this also.
>
> - Wes
> On Fri, Sep 28, 2018 at 5:34 AM Pearu Peterson
> wrote:
> >
>
Hi,
Consider the following use case:
schema =
cbuf =
cbatch = pa.cuda.read_record_batch(schema, cbuf)
Note that cbatch is pa.RecordBatch instance where data pointers are device
pointers.
for col in cbatch.columns:
# here col is, say, FloatArray, that data pointer is a device pointer
#
Pearu Peterson created ARROW-3228:
-
Summary: [Python] Immutability of bytes is ignored
Key: ARROW-3228
URL: https://issues.apache.org/jira/browse/ARROW-3228
Project: Apache Arrow
Issue Type
Pearu Peterson created ARROW-3221:
-
Summary: [Python] Add a virtual Slice method to buffers
Key: ARROW-3221
URL: https://issues.apache.org/jira/browse/ARROW-3221
Project: Apache Arrow
Issue
Pearu Peterson created ARROW-3220:
-
Summary: [Python] Add writeat method to writeable NativeFile
Key: ARROW-3220
URL: https://issues.apache.org/jira/browse/ARROW-3220
Project: Apache Arrow
ou have a basic API defined and exposed, so that we can iterate on that.
>
> Regards
>
> Antoine.
>
>
> Le 29/08/2018 à 17:11, Pearu Peterson a écrit :
> > Hi Wes,
> >
> > Thanks for the hints!
> >
> > I have created a very minimal GPU support to pyarr
Hi,
In Arrow C++, various buffer writers define Seek method while in
pyarrow the seek is defined only for buffer readers (for instance,
NativeFile.seek references only rd_file).
So, pyarrow relates 'seekable' strictly to 'readable' file property while
'seekable' would make sense also when a file
Hi,
I'd also like to enforce the raised question, in particular, it would be
very useful to have basic examples of IPC between same or different
languages, including C/C++, Python, Java, etc.
Whatever combination of languages is used, the principles of IPC should be
the same. For instance, in Pyt
ies
> are produced in Circle CI, Travis CI, and Appveyor)?
>
> - Wes
> On Tue, Aug 28, 2018 at 4:34 PM Pearu Peterson
> wrote:
> >
> > Hi,
> >
> > Is there any interest in implementing CudaBuffer support in pyarrow? Are
> > there any obstacles that would p
Hi,
Is there any interest in implementing CudaBuffer support in pyarrow? Are
there any obstacles that would prevent to do this?
I would be willing to give it a try.
As I understand, the entry point for this task would be to patch
pyarrow/includes/libarrow.pxd . Any hints for what to take into ac
sk already.
What do you think?
Best regards,
Pearu
On Wed, Aug 22, 2018 at 11:53 PM, Wes McKinney wrote:
> Hi Pearu,
>
> Seems the formatting of your email got messed up a little bit. Can you
> resend with some more line breaks?
>
> Thanks
>
>
> On Wed, Aug 22, 20
*Hi,The libgdf project defines a column structure that in a simplified form
could be represented astypedef struct {void *data; //
column dataunsigned char *valid; // validity mask // one bit per column
itemsize_t size; // nof itemsenum {INT8, INT16,
Pearu Peterson created ARROW-2944:
-
Summary: Arrow format documentation mentions VectorLayout that
does not exist anymore
Key: ARROW-2944
URL: https://issues.apache.org/jira/browse/ARROW-2944
Project
Pearu Peterson created ARROW-2903:
-
Summary: Setting -DARROW_HDFS=OFF breaks arrow build when linking
against boost libraries
Key: ARROW-2903
URL: https://issues.apache.org/jira/browse/ARROW-2903
26 matches
Mail list logo
