Arrow Build Report for Job nightly-2020-12-17-0
All tasks:
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-17-0
Failed Tasks:
- test-conda-python-3.7-pandas-master:
URL:
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-12-17-0-github-test-conda-
Hi all,
I am wondering if we need to consider the case of an empty buffer for
ArrayData validity buffer.
What happens when we create an ArrayData structure with buffers[0] =
empty_buffer and null_count= -1?
Shouldn't this infer null_count as 0 from this method?
https://github.com/apache/arrow/blob
What is an "empty buffer" here? A buffer with size 0?
Le 17/12/2020 à 18:02, Niranda Perera a écrit :
> Hi all,
>
> I am wondering if we need to consider the case of an empty buffer for
> ArrayData validity buffer.
> What happens when we create an ArrayData structure with buffers[0] =
> empty_
yes! a buffer with size 0. I forgot to mention that the rest of the buffers
in array_data have a non-zero size.
On Thu, Dec 17, 2020 at 12:12 PM Antoine Pitrou wrote:
>
> What is an "empty buffer" here? A buffer with size 0?
>
>
> Le 17/12/2020 à 18:02, Niranda Perera a écrit :
> > Hi all,
> >
>
This doesn't seem to be very useful. Also, multiplying special cases
makes maintenance more difficult and code generally more fragile.
Regards
Antoine.
Le 17/12/2020 à 18:16, Niranda Perera a écrit :
> yes! a buffer with size 0. I forgot to mention that the rest of the buffers
> in array_data
Wouldn't this create an undefined state when determining whether an index
is valid or not?
https://github.com/apache/arrow/blob/8d76312dd397ebe07b71531f6d23b8caa76703dc/cpp/src/arrow/array/array_base.h#L58
On Thu, Dec 17, 2020 at 12:28 PM Antoine Pitrou wrote:
>
> This doesn't seem to be very us
It could certainly error out or read invalid data or leak private data,
but so would any buffer of the wrong size. If you're building a
ArrayData yourself, you're responsible for passing the right data
layout. If you're not sure about it, you can call Validate on the
array, at least during the
So, I figured out the issue here - I had to remove compression from the
pyarrow feather.write_feather(compression='uncompressed'). Is there any way
to read a compressed feather file in arrow js?
See the comment under the first answer here:
https://stackoverflow.com/questions/64629670/how-to-write-a
