[jira] [Created] (ARROW-7038) Reading from HDFS after ctrl+c(SIGTERM) causes python hangs

2019-10-30 Thread Kevin Jung (Jira)
Kevin Jung created ARROW-7038: - Summary: Reading from HDFS after ctrl+c(SIGTERM) causes python hangs Key: ARROW-7038 URL: https://issues.apache.org/jira/browse/ARROW-7038 Project: Apache Arrow I

[jira] [Created] (ARROW-7037) [C++ ] Compile error on the combination of protobuf >= 3.9 and clang

2019-10-30 Thread Kenta Murata (Jira)
Kenta Murata created ARROW-7037: --- Summary: [C++ ] Compile error on the combination of protobuf >= 3.9 and clang Key: ARROW-7037 URL: https://issues.apache.org/jira/browse/ARROW-7037 Project: Apache Arro

questions about Gandiva

2019-10-30 Thread Yibo Cai
Hi, Arrow cpp integrates Gandiva to provide low level operations on arrow buffers. [1][2] I have some questions, any help is appreciated: - Arrow cpp already has a compute kernel[3], does it duplicate what Gandiva provides? I see a Jira talk about it.[4] - Is Gandiva only for arrow cpp? What ab

[jira] [Created] (ARROW-7036) [C++] Version up ORC to avoid compile errors

2019-10-30 Thread Kenta Murata (Jira)
Kenta Murata created ARROW-7036: --- Summary: [C++] Version up ORC to avoid compile errors Key: ARROW-7036 URL: https://issues.apache.org/jira/browse/ARROW-7036 Project: Apache Arrow Issue Type: T

Re: some questions, please help

2019-10-30 Thread Yibo Cai
Thanks Wes, Micah, your comments are very helpful. Yibo On 10/30/19 10:45 PM, Wes McKinney wrote: On Wed, Oct 30, 2019 at 9:32 AM Micah Kornfield wrote: - I see some SIMD optimizations in arrow go binding, such as vectored sum. [2] But arrow cpp lib doesn't leverage SIMD. [3] Wh

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-10-30-0

2019-10-30 Thread Neal Richardson
https://issues.apache.org/jira/browse/ARROW-7034 (pending +1/merge) will rid us of these meddlesome failures. Neal On Wed, Oct 30, 2019 at 11:25 AM Wes McKinney wrote: > > The failed tasks here are a nuisance. If they can't be fixed, should > they be removed from the nightlies? > > On Wed, Oct 3

[jira] [Created] (ARROW-7035) [R] Default arguments are unclear in write_parquet docs

2019-10-30 Thread Karl Dunkle Werner (Jira)
Karl Dunkle Werner created ARROW-7035: - Summary: [R] Default arguments are unclear in write_parquet docs Key: ARROW-7035 URL: https://issues.apache.org/jira/browse/ARROW-7035 Project: Apache Arrow

[jira] [Created] (ARROW-7034) [CI][Crossbow] Skip known nightly failures

2019-10-30 Thread Neal Richardson (Jira)
Neal Richardson created ARROW-7034: -- Summary: [CI][Crossbow] Skip known nightly failures Key: ARROW-7034 URL: https://issues.apache.org/jira/browse/ARROW-7034 Project: Apache Arrow Issue Typ

Re: [DISCUSS] Result vs Status

2019-10-30 Thread Wes McKinney
Returning to this discussion. Here is my position on the matter since this was brought up on the sync call today * For internal / non-public and pseudo-non-public APIs that have return/out values - Use Result or Status at discretion of the developer, but Result is preferable * For new public A

Re: [VOTE] Clarifications and forward compatibility changes for Dictionary Encoding

2019-10-30 Thread Wes McKinney
I wrote in on the original DISCUSS thread. I believe Antoine is unavailable this week, but hopefully we can drive the discussion to a consensus point next week so we can vote On Sat, Oct 26, 2019 at 12:01 AM Micah Kornfield wrote: > > I think at least the wording was confusing because you raised

Re: [DISCUSS] Dictionary Encoding Clarifications/Future Proofing

2019-10-30 Thread Wes McKinney
Returning to this discussion as there seems to lack consensus in the vote thread Copying Micah's proposals in the VOTE thread here, I wanted to state my opinions so we can discuss further and see where there is potential disagreement 1. It is not required that all dictionary batches occur at the

[jira] [Created] (ARROW-7033) Error in./configure step for jemalloc when building on OSX 10.14.6

2019-10-30 Thread Christian Hudon (Jira)
Christian Hudon created ARROW-7033: -- Summary: Error in./configure step for jemalloc when building on OSX 10.14.6 Key: ARROW-7033 URL: https://issues.apache.org/jira/browse/ARROW-7033 Project: Apache

Re: [VOTE] Release Apache Arrow 0.15.1 - RC0

2019-10-30 Thread Wes McKinney
+1 (binding) * Verified source on Ubuntu 18.04 (using 0.15.1 RC verification script) * Verified wheels on Linux, macOS, and Windows using "verify-release-candidate.sh wheels ..." and verify-release-candidate-wheels.bat * Verified Linux binaries Thanks for fixing the macOS wheel! On Wed, Oct 30,

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-10-30-0

2019-10-30 Thread Wes McKinney
The failed tasks here are a nuisance. If they can't be fixed, should they be removed from the nightlies? On Wed, Oct 30, 2019 at 7:26 AM Crossbow wrote: > > > Arrow Build Report for Job nightly-2019-10-30-0 > > All tasks: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10

Re: Arrow sync call October 30 at 12:00 US/Eastern, 16:00 UTC

2019-10-30 Thread Neal Richardson
Attendees: * Uwe Korn * Micah Kornfield * Praveen Kumar * Wes McKinney * Rok Mihevc * Neal Richardson Discussion: * docker-compose/github-actions (https://github.com/apache/arrow/pull/5589). Needs review, needs to be merged and have followup issues made. Currently too many jobs being run on every

Re: [VOTE] Release Apache Arrow 0.15.1 - RC0

2019-10-30 Thread Krisztián Szűcs
Hi, I've uploaded the correct wheel for CPython 3.7 on macOS, also tested it locally, it works properly. Created a JIRA [1] to test the wheels in the release verification script similarly like we test the linux packages, this should catch both the uploading issues and the linking errors causing mo

[jira] [Created] (ARROW-7032) [Release] Verify python wheels in the release verification script

2019-10-30 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-7032: -- Summary: [Release] Verify python wheels in the release verification script Key: ARROW-7032 URL: https://issues.apache.org/jira/browse/ARROW-7032 Project: Apache A

Re: some questions, please help

2019-10-30 Thread Wes McKinney
On Wed, Oct 30, 2019 at 9:32 AM Micah Kornfield wrote: > > > > > > - I see some SIMD optimizations in arrow go binding, such as vectored > > sum. [2] > > >But arrow cpp lib doesn't leverage SIMD. [3] > > >Why not optimize it in cpp lib so all languages can benefit? > > You're welcome to co

Re: some questions, please help

2019-10-30 Thread Micah Kornfield
> > > - I see some SIMD optimizations in arrow go binding, such as vectored > sum. [2] > >But arrow cpp lib doesn't leverage SIMD. [3] > >Why not optimize it in cpp lib so all languages can benefit? > You're welcome to contribute such optimizations to the C++ library Note that even though

Re: some questions, please help

2019-10-30 Thread Wes McKinney
hi Yibo On Wed, Oct 30, 2019 at 2:16 AM Yibo Cai wrote: > > Hi, > > I'm new to Arrow. Would like to seek for help about some questions. Any > comment is welcomed. > > - About source code tree, my understand is that "cpp" is the core arrow > libraries, "c_glib, go, python, ..." are language bind

[jira] [Created] (ARROW-7031) [Python] Expose the offsets of a ListArray in python

2019-10-30 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-7031: Summary: [Python] Expose the offsets of a ListArray in python Key: ARROW-7031 URL: https://issues.apache.org/jira/browse/ARROW-7031 Project: Apache Arr

AW: State of decimal support in Arrow (from/to Parquet Decimal Logicaltype)

2019-10-30 Thread roman.karlstetter
Hi Wes, the data is indeed not originating from Arrow, so I was looking for how to call the low level WriteBatch API. I figured it out now, it's actually straightforward in the Arrow-API, I just got confused a little with the spec at https://github.com/apache/parquet-format/blob/master/LogicalT

[NIGHTLY] Arrow Build Report for Job nightly-2019-10-30-0

2019-10-30 Thread Crossbow
Arrow Build Report for Job nightly-2019-10-30-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-30-0 Failed Tasks: - docker-clang-format: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-10-30-0-circle-docker-clang-format - docke

[jira] [Created] (ARROW-7030) csv example coredump error

2019-10-30 Thread wjw (Jira)
wjw created ARROW-7030: -- Summary: csv example coredump error Key: ARROW-7030 URL: https://issues.apache.org/jira/browse/ARROW-7030 Project: Apache Arrow Issue Type: Bug Components: C++ Aff

[jira] [Created] (ARROW-7029) [Go] unsafe pointer arithmetic panic w/ Go-1.14-dev

2019-10-30 Thread Sebastien Binet (Jira)
Sebastien Binet created ARROW-7029: -- Summary: [Go] unsafe pointer arithmetic panic w/ Go-1.14-dev Key: ARROW-7029 URL: https://issues.apache.org/jira/browse/ARROW-7029 Project: Apache Arrow

[jira] [Created] (ARROW-7028) Dates in R are different when saved and loaded with parquet

2019-10-30 Thread Sascha (Jira)
Sascha created ARROW-7028: - Summary: Dates in R are different when saved and loaded with parquet Key: ARROW-7028 URL: https://issues.apache.org/jira/browse/ARROW-7028 Project: Apache Arrow Issue Typ

[jira] [Created] (ARROW-7027) [Python] pa.table(..) returns instead of raises error if passing invalid object

2019-10-30 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-7027: Summary: [Python] pa.table(..) returns instead of raises error if passing invalid object Key: ARROW-7027 URL: https://issues.apache.org/jira/browse/ARROW-7027

some questions, please help

2019-10-30 Thread Yibo Cai
Hi, I'm new to Arrow. Would like to seek for help about some questions. Any comment is welcomed. - About source code tree, my understand is that "cpp" is the core arrow libraries, "c_glib, go, python, ..." are language bindings to ease integrating arrow into apps developed by that language. I

[jira] [Created] (ARROW-7026) [Java] Remove assertions in MessageSerializer/vector/writer/reader

2019-10-30 Thread Ji Liu (Jira)
Ji Liu created ARROW-7026: - Summary: [Java] Remove assertions in MessageSerializer/vector/writer/reader Key: ARROW-7026 URL: https://issues.apache.org/jira/browse/ARROW-7026 Project: Apache Arrow Is