Re: Next Arrow sync call

2017-08-28 Thread Li Jin
+1. Works for me. On Mon, Aug 28, 2017 at 5:58 PM, Wes McKinney wrote: > We haven't had a sync call in a number of weeks. How about Wednesday, > September 6? Does 12:00 New York Time work for most folks? > > - Wes >

Next Arrow sync call

2017-08-28 Thread Wes McKinney
We haven't had a sync call in a number of weeks. How about Wednesday, September 6? Does 12:00 New York Time work for most folks? - Wes

Re: Reading Parquet datetime column gives different answer in Spark vs PyArrow

2017-08-28 Thread Wes McKinney
see https://issues.apache.org/jira/browse/ARROW-1425 On Mon, Aug 28, 2017 at 12:32 PM, Wes McKinney wrote: > hi Lucas, > > Bryan Cutler, Holden Karau, Li Jin, or someone with deeper knowledge > of the Spark timestamp issue (which is a known, and not a bug per se) > should be able to give some ext

[jira] [Created] (ARROW-1425) [Python] Document semantic differences between Spark timestamps and Arrow timestamps

2017-08-28 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1425: --- Summary: [Python] Document semantic differences between Spark timestamps and Arrow timestamps Key: ARROW-1425 URL: https://issues.apache.org/jira/browse/ARROW-1425 Proj

[jira] [Created] (ARROW-1424) [Python] Initial bindings for libarrow_gpu

2017-08-28 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1424: --- Summary: [Python] Initial bindings for libarrow_gpu Key: ARROW-1424 URL: https://issues.apache.org/jira/browse/ARROW-1424 Project: Apache Arrow Issue Type: New

Re: Reading Parquet datetime column gives different answer in Spark vs PyArrow

2017-08-28 Thread Wes McKinney
hi Lucas, Bryan Cutler, Holden Karau, Li Jin, or someone with deeper knowledge of the Spark timestamp issue (which is a known, and not a bug per se) should be able to give some extra context about this. My understanding is that when you read timezone-naive data in Spark, it is treated as session-

Re: Reading Parquet datetime column gives different answer in Spark vs PyArrow

2017-08-28 Thread Lucas Pickup
Here is the pyspark script I used to see this difference. On Mon, 28 Aug 2017 at 09:20 Lucas Pickup wrote: > Hi all, > > Very sorry if people already responded to this at: > lucas.pic...@microsoft.com There was an INVALID identifier attached to > the end of the reply address for some reason whic

Reading Parquet datetime column gives different answer in Spark vs PyArrow

2017-08-28 Thread Lucas Pickup
Hi all, Very sorry if people already responded to this at: lucas.pic...@microsoft.com There was an INVALID identifier attached to the end of the reply address for some reason which may have caused replies to be lost. I've been messing around with Spark and PyArrow Parquet reading. In my testing I

[jira] [Created] (ARROW-1423) [C++] Create non-owned CudaContext from context handle provided by thirdparty user

2017-08-28 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1423: --- Summary: [C++] Create non-owned CudaContext from context handle provided by thirdparty user Key: ARROW-1423 URL: https://issues.apache.org/jira/browse/ARROW-1423 Projec

Re: Arrow 0.7.0 release timeline

2017-08-28 Thread Wes McKinney
hello again, There is still a lot of work to do for 0.7.0. There are another ~60 JIRAs marked for the 0.7.0 release -- I don't think these will all get done, but we should stretch to do as many as possible. I think we may be able to complete Java/C++ integration tests for (sparse) unions and decim

[jira] [Created] (ARROW-1422) [Format] Add specification document for the serialization scheme used in Python

2017-08-28 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1422: --- Summary: [Format] Add specification document for the serialization scheme used in Python Key: ARROW-1422 URL: https://issues.apache.org/jira/browse/ARROW-1422 Project:

[jira] [Created] (ARROW-1421) [Python] pyarrow.serialize cannot serialize a Python dict input

2017-08-28 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1421: --- Summary: [Python] pyarrow.serialize cannot serialize a Python dict input Key: ARROW-1421 URL: https://issues.apache.org/jira/browse/ARROW-1421 Project: Apache Arrow

[jira] [Created] (ARROW-1420) [C++] Investigate intermittent gflags-related build failures in Appveyor

2017-08-28 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1420: --- Summary: [C++] Investigate intermittent gflags-related build failures in Appveyor Key: ARROW-1420 URL: https://issues.apache.org/jira/browse/ARROW-1420 Project: Apache