Re: Arrow-Parquet integration location (Was: Arrow cpp travis-ci build broken)

2016-09-06 Thread Uwe Korn
Hello, I'm also in favour of switching the dependency direction between Parquet and Arrow as this would avoid a lot of duplicate code in both projects as well as parquet-cpp profiting from functionality that is available in Arrow. @wesm: go ahead with the JIRAs and I'll add comments or will

[jira] [Resolved] (ARROW-283) [C++] Update arrow_parquet to account for API changes in PARQUET-573

2016-09-06 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-283. Resolution: Fixed Issue resolved by pull request 131 [https://github.com/apache/arrow/pull/131] > [C

Re: Arrow cpp travis-ci build broken

2016-09-06 Thread Wes McKinney
Arrow builds should be fixed now hopefully. I'm rebasing the C++ patch queue. On Tue, Sep 6, 2016 at 6:37 PM, Wes McKinney wrote: > hi Julien, > > I'm very sorry about the inconvenience with this and the delay in > getting it sorted out. I will triage this evening by disabling the > Parquet tests

[jira] [Resolved] (ARROW-284) [C++] Triage builds by disabling Arrow-Parquet module

2016-09-06 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-284. Resolution: Fixed Issue resolved by pull request 132 [https://github.com/apache/arrow/pull/132] > [C

[jira] [Commented] (ARROW-284) [C++] Triage builds by disabling Arrow-Parquet module

2016-09-06 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469393#comment-15469393 ] Wes McKinney commented on ARROW-284: PR: https://github.com/apache/arrow/pull/132 > [

[jira] [Created] (ARROW-284) [C++] Triage builds by disabling Arrow-Parquet module

2016-09-06 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-284: -- Summary: [C++] Triage builds by disabling Arrow-Parquet module Key: ARROW-284 URL: https://issues.apache.org/jira/browse/ARROW-284 Project: Apache Arrow Issue Ty

Re: Arrow-Parquet integration location (Was: Arrow cpp travis-ci build broken)

2016-09-06 Thread Wes McKinney
hi Julien, It makes sense to move the Parquet support for Arrow into Parquet itself and invert the dependency. I had thought that the coupling to Arrow C++'s IO subsystem might be tighter, but the connection between memory allocators and file abstractions is fairly simple: https://github.com/apac

Re: Arrow-Parquet integration location (Was: Arrow cpp travis-ci build broken)

2016-09-06 Thread Brian Bowman
Forgive me if interposing my first post for the Apache Arrow project on this thread is incorrect procedure. What Julien proposes with each storage layer producing Arrow Record Batches is exactly how I envision it working and would certainly make Arrow integration with SAS much more palatable.

Arrow-Parquet integration location (Was: Arrow cpp travis-ci build broken)

2016-09-06 Thread Julien Le Dem
Thanks Wes, No worries, I know you are on top of those things. On a side note, I was wondering if the arrow-parquet integration should be in Parquet instead. Parquet would depend on Arrow and not the other way around. Arrow provides the API and each storage layer (Parquet, Kudu, Cassandra, ...) pro

Re: Arrow cpp travis-ci build broken

2016-09-06 Thread Wes McKinney
hi Julien, I'm very sorry about the inconvenience with this and the delay in getting it sorted out. I will triage this evening by disabling the Parquet tests in Arrow until we get the current problems under control. When we re-enable the Parquet tests in Travis CI I agree we should pin the version

Arrow cpp travis-ci build broken

2016-09-06 Thread Julien Le Dem
The Arrow cpp travis-ci build is broken right now because it depends on parquet-cpp which has changed in an incompatible way. [1] [2] (or so it looks to me) Since parquet-cpp is not released yet it is totally fine to make incompatible API changes. However, we may want to pin the Arrow to Parquet de

[jira] [Assigned] (ARROW-274) Make the MapVector nullable

2016-09-06 Thread Julien Le Dem (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Le Dem reassigned ARROW-274: --- Assignee: Julien Le Dem > Make the MapVector nullable > --- > >

[jira] [Resolved] (ARROW-274) Make the MapVector nullable

2016-09-06 Thread Julien Le Dem (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Le Dem resolved ARROW-274. - Resolution: Fixed Issue resolved by pull request 128 [https://github.com/apache/arrow/pull/128] >