[jira] [Created] (ARROW-3704) [Gandiva] Can't build with g++ 8.2.0

2018-11-05 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-3704: --- Summary: [Gandiva] Can't build with g++ 8.2.0 Key: ARROW-3704 URL: https://issues.apache.org/jira/browse/ARROW-3704 Project: Apache Arrow Issue Type: Improveme

Re: Assign/update : NA bitmap vs sentinel

2018-11-05 Thread Jacques Nadeau
On Mon, Nov 5, 2018 at 3:43 PM Matt Dowle wrote: > 1. I see. Good idea. Can we assume bitmap is always present in Arrow then? > I thought I'd seen Wes argue that if there were no NAs, the bitmap doesn't > need to be allocated. Indeed I wasn't worried about the extra storage, > although for 10,00

Re: Assign/update : NA bitmap vs sentinel

2018-11-05 Thread Matt Dowle
1. I see. Good idea. Can we assume bitmap is always present in Arrow then? I thought I'd seen Wes argue that if there were no NAs, the bitmap doesn't need to be allocated. Indeed I wasn't worried about the extra storage, although for 10,000 columns I wonder about the number of vectors. 2. It's onl

Re: Assign/update : NA bitmap vs sentinel

2018-11-05 Thread Jacques Nadeau
A few random thoughts... It seems like you outline three things as possible concerns: 1) Late allocation failure 2) Code complexity, errors. 3) Performance For 1: is the validity bit sizing really too expensive to allocate on initial allocation? In our experience, it is nominal in relationship t

Assign/update : NA bitmap vs sentinel

2018-11-05 Thread Matt Dowle
Hi, (First post to this mailing list.) I tweeted here and Wes invited me to follow up on this list : https://twitter.com/wesmckinn/status/1059440916987961346 Wes - it was great to meet you at Stanford in September. There I mentioned the assign/update aspect which is a downside of bitmap for NA, im

[jira] [Created] (ARROW-3703) [Python] DataFrame.to_parquet crashes if datetime column has time zones

2018-11-05 Thread Diego Argueta (JIRA)
Diego Argueta created ARROW-3703: Summary: [Python] DataFrame.to_parquet crashes if datetime column has time zones Key: ARROW-3703 URL: https://issues.apache.org/jira/browse/ARROW-3703 Project: Apache

[jira] [Created] (ARROW-3702) [R] POSIXct mapped to DateType not TimestampType?

2018-11-05 Thread Javier Luraschi (JIRA)
Javier Luraschi created ARROW-3702: -- Summary: [R] POSIXct mapped to DateType not TimestampType? Key: ARROW-3702 URL: https://issues.apache.org/jira/browse/ARROW-3702 Project: Apache Arrow Is