Re: Arrow Support in Parquet Writers

2020-07-06 Thread Wes McKinney
Is there is a path to having an Arrow<->Parquet implementation in Java that does not have a hard dependency on Iceberg? This is a common ask and it seems like it would be a clear community win that would net more contributors than something Iceberg-specific. On Mon, Jul 6, 2020 at 2:54 PM Ryan Blu

Re: random comment

2020-01-06 Thread Wes McKinney
I also joined the dev@ list to keep abreast of what's going on in Iceberg. If all of the development activity is happening on GitHub issues and pull requests, then it seems like all of this activity should be mirrored to _some_ mailing list, for example github@iceberg.a.o. This creates a searchable

Re: Are we going to use Apache JIRA instead of Github issues

2019-08-22 Thread Wes McKinney
. >> >> Anyways, my two cents. >> Thanks. >> >> Cheers, >> Edgar >> >> On Sun, Aug 18, 2019 at 7:43 PM Saisai Shao wrote: >>>> >>>> The issue linking, Fix Version, and assignee features of JIRA are also >>>> h

Re: Are we going to use Apache JIRA instead of Github issues

2019-08-16 Thread Wes McKinney
One significant issue with GitHub issues for ASF projects is that non-committers cannot edit issue or PR metadata (labels, requesting reviews, etc). The lack of formalism around Resolved and Closed states can place an extra communication burden to explain why an issue is closed. Sometimes projects

Re: Future of Iceberg Parquet Reader

2019-05-28 Thread Wes McKinney
ularity of a single file is hidden behind Iceberg business logic, so I would encourage you to make the lower-level single file interface as accessible to general Arrow users as possible. > -Dan > > On Tue, May 28, 2019 at 8:03 AM Wes McKinney wrote: >> >> hi Anton, >> >> O

Re: Future of Iceberg Parquet Reader

2019-05-28 Thread Wes McKinney
hi Anton, On point #5, I would suggest doing the work either in Apache Arrow or in the Parquet Java project -- we are developing both Parquet C++ and Rust codebases within the apache/arrow repository so I think you would find an active community there. I know that there has been a lot of interest

Re: Developing a "dataset" API / framework for Arrow C++ users

2019-02-27 Thread Wes McKinney
t; > > > Uwe > > > > On Mon, Feb 25, 2019, at 10:20 AM, Joel Pfaff wrote: > > > Hello, > > > > > > Thanks for the write-up. > > > > > > Have you considered sharing this document with the Apache Iceberg > > community? > &g

Re: Developing a "dataset" API / framework for Arrow C++ users

2019-02-25 Thread Wes McKinney
ff wrote: > > Hello, > > > > Thanks for the write-up. > > > > Have you considered sharing this document with the Apache Iceberg community? > > > > My feeling is that there are some shared goals here between the two > > projects. > > And while the