Re: Next Arrow sync call

2017-09-05 Thread Wes McKinney
I will send out a Google Hangout link to the mailing list before the call starts. Thanks On Tue, Sep 5, 2017 at 11:41 PM, Rahul Soni wrote: > Second Ankit. > > On Tuesday, September 5, 2017, Ankit Aggarwal < > ankit.ankit.aggar...@gmail.com> wrote: > >> Hi All >> >> Can anyone please tell me how

Re: Next Arrow sync call

2017-09-05 Thread Rahul Soni
Second Ankit. On Tuesday, September 5, 2017, Ankit Aggarwal < ankit.ankit.aggar...@gmail.com> wrote: > Hi All > > Can anyone please tell me how anyone new can join this call? > > Ankit Aggarwal > > On 05-Sep-2017 11:35 PM, "Bryan Cutler" > > wrote: > > Sounds good! I'm going to try to join you gu

Re: Next Arrow sync call

2017-09-05 Thread Ankit Aggarwal
Hi All Can anyone please tell me how anyone new can join this call? Ankit Aggarwal On 05-Sep-2017 11:35 PM, "Bryan Cutler" wrote: Sounds good! I'm going to try to join you guys this time also. On Thu, Aug 31, 2017 at 8:51 AM, Wes McKinney wrote: > Great. > > The next Arrow sync will be at 1

[jira] [Created] (ARROW-1468) [C++] Append to PrimitiveBuilder from std::vector

2017-09-05 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1468: --- Summary: [C++] Append to PrimitiveBuilder from std::vector Key: ARROW-1468 URL: https://issues.apache.org/jira/browse/ARROW-1468 Project: Apache Arrow Issue Ty

[jira] [Created] (ARROW-1467) Fix reset() and allocateNew() in Nullable Value Vectors template

2017-09-05 Thread SIDDHARTH TEOTIA (JIRA)
SIDDHARTH TEOTIA created ARROW-1467: --- Summary: Fix reset() and allocateNew() in Nullable Value Vectors template Key: ARROW-1467 URL: https://issues.apache.org/jira/browse/ARROW-1467 Project: Apache

Re: how to use arrow in project?

2017-09-05 Thread Bryan Cutler
Hello, At the moment for the upcoming release of Spark 2.3, Arrow support is limited to using PySpark DataFrame.toPandas(), so reading Parquet data from Spark does not use Arrow. -Bryan On Wed, Aug 30, 2017 at 2:47 AM, big data wrote: > I want to use Arrow as a middle layer between spark and p

Re: Reading Parquet datetime column gives different answer in Spark vs PyArrow

2017-09-05 Thread Bryan Cutler
Hi Lucas, The assessments from Wes and Li are right on. Just to add to that, and unfortunately make things even more complicated.. Spark does not always use the config "spark.sql.session.timeZone", so it doesn't really help with your example. It would be used if instead you generated timestamps t

[jira] [Created] (ARROW-1466) [C++] Support DecimalArray in arrow::PrettyPrint

2017-09-05 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1466: --- Summary: [C++] Support DecimalArray in arrow::PrettyPrint Key: ARROW-1466 URL: https://issues.apache.org/jira/browse/ARROW-1466 Project: Apache Arrow Issue Typ

[jira] [Created] (ARROW-1465) [C++] Accommodate ABI changes in libhdfs3 from 2.2.31 to 2.3 on conda-forge

2017-09-05 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1465: --- Summary: [C++] Accommodate ABI changes in libhdfs3 from 2.2.31 to 2.3 on conda-forge Key: ARROW-1465 URL: https://issues.apache.org/jira/browse/ARROW-1465 Project: Apac

Re: Next Arrow sync call

2017-09-05 Thread Bryan Cutler
Sounds good! I'm going to try to join you guys this time also. On Thu, Aug 31, 2017 at 8:51 AM, Wes McKinney wrote: > Great. > > The next Arrow sync will be at 16:00 UTC (12:00 New York Time) on > Thursday, September 7. I will send out a Google Hangout link > beforehand > > On Thu, Aug 31, 2017

Extra Arrow debugging context in C++

2017-09-05 Thread Wes McKinney
For anyone doing C, C++ and/or Python development, I want to make sure you see ARROW-1401, which generates "stack traces" from error Statuses: ../src/arrow/ipc/ipc-read-write-test.cc:609: Failure Failed NotImplemented: ../src/arrow/ipc/ipc-read-write-test.cc:574 code: writer->WriteRecordBatch(batc

[jira] [Created] (ARROW-1464) Troubleshooting of build errors

2017-09-05 Thread Wataru Shimizu (JIRA)
Wataru Shimizu created ARROW-1464: - Summary: Troubleshooting of build errors Key: ARROW-1464 URL: https://issues.apache.org/jira/browse/ARROW-1464 Project: Apache Arrow Issue Type: Improvemen

[jira] [Created] (ARROW-1463) [JAVA] Restructure ValueVector hierarchy to minimize compile-time generated code

2017-09-05 Thread Jacques Nadeau (JIRA)
Jacques Nadeau created ARROW-1463: - Summary: [JAVA] Restructure ValueVector hierarchy to minimize compile-time generated code Key: ARROW-1463 URL: https://issues.apache.org/jira/browse/ARROW-1463 Proj