[jira] [Created] (ARROW-3749) [GLib] Typos in documentation and test case name

2018-11-09 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-3749: --- Summary: [GLib] Typos in documentation and test case name Key: ARROW-3749 URL: https://issues.apache.org/jira/browse/ARROW-3749 Project: Apache Arrow Issue Typ

[jira] [Created] (ARROW-3750) [R] Pass various wrapped Arrow objects created in Python into R with zero copy via reticulate

2018-11-09 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3750: --- Summary: [R] Pass various wrapped Arrow objects created in Python into R with zero copy via reticulate Key: ARROW-3750 URL: https://issues.apache.org/jira/browse/ARROW-3750

[jira] [Created] (ARROW-3748) [GLib] Add GArrowCSVReader

2018-11-09 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-3748: --- Summary: [GLib] Add GArrowCSVReader Key: ARROW-3748 URL: https://issues.apache.org/jira/browse/ARROW-3748 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-3747) [C++] Flip order of data members in arrow::Decimal128

2018-11-09 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3747: --- Summary: [C++] Flip order of data members in arrow::Decimal128 Key: ARROW-3747 URL: https://issues.apache.org/jira/browse/ARROW-3747 Project: Apache Arrow Issu

[jira] [Created] (ARROW-3746) [Gandiva] [Python] Make it possible to list all functions registered with Gandiva

2018-11-09 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3746: - Summary: [Gandiva] [Python] Make it possible to list all functions registered with Gandiva Key: ARROW-3746 URL: https://issues.apache.org/jira/browse/ARROW-3746 Pro

[jira] [Created] (ARROW-3745) [C++] CMake passes static libraries multiple times to linker

2018-11-09 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3745: --- Summary: [C++] CMake passes static libraries multiple times to linker Key: ARROW-3745 URL: https://issues.apache.org/jira/browse/ARROW-3745 Project: Apache Arrow

Re: [ANNOUNCE] New Arrow PMC member: Krisztián Szűcs

2018-11-09 Thread Srinivas Reddy
Congratulations Krisztián.. -- Srinivas Reddy http://mrsrinivas.com/ (Sent via gmail web) On Fri, 9 Nov 2018 at 03:48, Wes McKinney wrote: > The Project Management Committee (PMC) for Apache Arrow has invited > Krisztián Szűcs to become a PMC member and we are pleased to announce > that h

Re: [ANNOUNCE] New Arrow committers: Romain François, Sebastien Binet, Yosuke Shiro

2018-11-09 Thread Srinivas Reddy
Many congratulations and welcome all :) -- Srinivas Reddy http://mrsrinivas.com/ (Sent via gmail web) On Fri, 9 Nov 2018 at 04:53, Wes McKinney wrote: > On behalf of the Arrow PMC, I'm happy to announce that Romain > François, Sebastien Binet, and Yosuke Shiro have been invited to be > co

Re: [ANNOUNCE] New Arrow PMC member: Krisztián Szűcs

2018-11-09 Thread Bryan Cutler
Congrats Krisztián! On Thu, Nov 8, 2018, 2:00 PM Li Jin Congrats! > > On Thu, Nov 8, 2018 at 4:02 PM Uwe L. Korn wrote: > > > Congratulations Krisztián! > > > > On Thu, Nov 8, 2018, at 9:56 PM, Philipp Moritz wrote: > > > Congrats and welcome Krisztián! > > > > > > On Thu, Nov 8, 2018 at 11:48 A

[jira] [Created] (ARROW-3744) [Ruby] Use garrow_table_to_string() in Arrow::Table#to_s

2018-11-09 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-3744: --- Summary: [Ruby] Use garrow_table_to_string() in Arrow::Table#to_s Key: ARROW-3744 URL: https://issues.apache.org/jira/browse/ARROW-3744 Project: Apache Arrow I

Re: [ANNOUNCE] New Arrow committers: Romain François, Sebastien Binet, Yosuke Shiro

2018-11-09 Thread Bryan Cutler
Congrats to all and welcome! On Thu, Nov 8, 2018, 2:00 PM Li Jin Welcome! > > On Thu, Nov 8, 2018 at 4:01 PM Uwe L. Korn wrote: > > > Welcome to all of you! > > > > On Thu, Nov 8, 2018, at 8:56 PM, Wes McKinney wrote: > > > On behalf of the Arrow PMC, I'm happy to announce that Romain > > > Fran

[jira] [Created] (ARROW-3743) [Ruby] Add support for saving/loading Feather

2018-11-09 Thread Kouhei Sutou (JIRA)
Kouhei Sutou created ARROW-3743: --- Summary: [Ruby] Add support for saving/loading Feather Key: ARROW-3743 URL: https://issues.apache.org/jira/browse/ARROW-3743 Project: Apache Arrow Issue Type:

Re: Assign/update : NA bitmap vs sentinel

2018-11-09 Thread Wes McKinney
hi Matt, On Fri, Nov 9, 2018 at 6:36 PM Matt Dowle wrote: > > On Fri, Nov 9, 2018 at 2:14 PM Wes McKinney wrote: > > > On Fri, Nov 9, 2018 at 4:51 PM Matt Dowle wrote: > > > > > > > There is one database that I'm aware of that uses sentinels _and_ > > > supports complex types with missing values

[jira] [Created] (ARROW-3742) Fix gandiva cython bindings

2018-11-09 Thread Siyuan Zhuang (JIRA)
Siyuan Zhuang created ARROW-3742: Summary: Fix gandiva cython bindings Key: ARROW-3742 URL: https://issues.apache.org/jira/browse/ARROW-3742 Project: Apache Arrow Issue Type: Bug

Re: Assign/update : NA bitmap vs sentinel

2018-11-09 Thread Matt Dowle
On Fri, Nov 9, 2018 at 2:14 PM Wes McKinney wrote: > On Fri, Nov 9, 2018 at 4:51 PM Matt Dowle wrote: > > > > > There is one database that I'm aware of that uses sentinels _and_ > > supports complex types with missing values: Kx's KDB+. > > I read this and was pleased that KDB is being used as a

[jira] [Created] (ARROW-3741) [R] Add support for arrow::compute::Cast to convert Arrow arrays from one type to another

2018-11-09 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3741: --- Summary: [R] Add support for arrow::compute::Cast to convert Arrow arrays from one type to another Key: ARROW-3741 URL: https://issues.apache.org/jira/browse/ARROW-3741

[jira] [Created] (ARROW-3740) [C++] Calling ArrayBuilder::Resize with length smaller than current appended length results in invalid state

2018-11-09 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3740: --- Summary: [C++] Calling ArrayBuilder::Resize with length smaller than current appended length results in invalid state Key: ARROW-3740 URL: https://issues.apache.org/jira/browse/ARRO

Re: Assign/update : NA bitmap vs sentinel

2018-11-09 Thread Matt Dowle
> I can tell you from working in a large advertising company, that it is critical to have a reliable way to distinguish null from not null, and a space had better be not null. What did a single space mean that was so important not to be null? Symbols are recommended in KDB rather than strings. In t

Re: Assign/update : NA bitmap vs sentinel

2018-11-09 Thread Phillip Cloud
The point I was trying to argue is that making the choice of a value from a set of values to use as a sentinel has consequences that are worth thinking about. One consequence of choosing such a value is a (hypothetical) isspace and an isnull (called null in KDB+) function would give indistinguishab

[jira] [Created] (ARROW-3739) [C++] Add option to convert a particular column to timestamps or dates using a passed strptime-compatible string

2018-11-09 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3739: --- Summary: [C++] Add option to convert a particular column to timestamps or dates using a passed strptime-compatible string Key: ARROW-3739 URL: https://issues.apache.org/jira/browse/

[jira] [Created] (ARROW-3738) [C++] Add CSV conversion option to parse ISO8601-like timestamp strings

2018-11-09 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-3738: --- Summary: [C++] Add CSV conversion option to parse ISO8601-like timestamp strings Key: ARROW-3738 URL: https://issues.apache.org/jira/browse/ARROW-3738 Project: Apache A

Re: Assign/update : NA bitmap vs sentinel

2018-11-09 Thread Wes McKinney
On Fri, Nov 9, 2018 at 4:51 PM Matt Dowle wrote: > > > There is one database that I'm aware of that uses sentinels _and_ > supports complex types with missing values: Kx's KDB+. > I read this and was pleased that KDB is being used as a reference. It is a > seriously good database: the gold-standa

Re: Assign/update : NA bitmap vs sentinel

2018-11-09 Thread Matt Dowle
> There is one database that I'm aware of that uses sentinels _and_ supports complex types with missing values: Kx's KDB+. I read this and was pleased that KDB is being used as a reference. It is a seriously good database: the gold-standard in many people's eyes. > This has led to some seriously

[jira] [Created] (ARROW-3737) [CI/Docker/Python] Support running integration tests on multiple python versions

2018-11-09 Thread Krisztian Szucs (JIRA)
Krisztian Szucs created ARROW-3737: -- Summary: [CI/Docker/Python] Support running integration tests on multiple python versions Key: ARROW-3737 URL: https://issues.apache.org/jira/browse/ARROW-3737 Pr

[jira] [Created] (ARROW-3736) [CI/Docker] Ninja test in docker-compose run cpp hangs

2018-11-09 Thread Krisztian Szucs (JIRA)
Krisztian Szucs created ARROW-3736: -- Summary: [CI/Docker] Ninja test in docker-compose run cpp hangs Key: ARROW-3736 URL: https://issues.apache.org/jira/browse/ARROW-3736 Project: Apache Arrow

[jira] [Created] (ARROW-3735) [Python] Proper error handling in _ensure_type

2018-11-09 Thread Krisztian Szucs (JIRA)
Krisztian Szucs created ARROW-3735: -- Summary: [Python] Proper error handling in _ensure_type Key: ARROW-3735 URL: https://issues.apache.org/jira/browse/ARROW-3735 Project: Apache Arrow Issue

Re: Nested data support?

2018-11-09 Thread Joshua Storck
I hope to get around to finishing my PR for reading in the next month or so. The work is mostly complete, it's just that it didn't get merged before the monorepo consolidation. I expect the write side of things to be easier, though it will probably take a month or so of work with testing. On Fri,

Re: Nested data support?

2018-11-09 Thread Wes McKinney
Hi Paul, Yes, we are planning to support it eventually but I estimate there is at least 100 hours of developer time involved in doing it thoroughly and with both write and read support, and with good performance. There is a patch available implementing reads (but not writes yet), but it has perfor

Nested data support?

2018-11-09 Thread Paul Wais
Dear List, I'm writing about the ability to write nested data via pyarrow, as discussed in this StackOverflow posting: https://stackoverflow.com/questions/45341182/nested-data-in-parquet-with-python I looked around JIRA and some of the PRs and I'm a bit confused: is pyarrow supposed to support wr

[jira] [Created] (ARROW-3734) [C++] Linking static zstd library fails on Arch x86-64

2018-11-09 Thread Dimitri Vorona (JIRA)
Dimitri Vorona created ARROW-3734: - Summary: [C++] Linking static zstd library fails on Arch x86-64 Key: ARROW-3734 URL: https://issues.apache.org/jira/browse/ARROW-3734 Project: Apache Arrow