Hi Wes, It is really great, that you have open-sourced all this! As far as I understand, you have also open-sourced the engine that can execute relational operators on arrow ? Is it possible to use it as library ? Are you also planning to donate it arrow project at some point?
Sincerely, Michael Shtelma On Thu, Jul 20, 2017 at 10:19 PM, Wes McKinney <wesmck...@gmail.com> wrote: > hi Sven, > > There is a placeholder project in apache/parquet-mr > https://github.com/apache/parquet-mr/tree/master/parquet-arrow. > > It appears in the meantime that Dremio has created a vectorized > Parquet <-> Arrow reader/writer which has just been open sourced under > ASL 2.0: https://github.com/dremio/dremio-oss/tree/master/sabot/ > kernel/src/main/java/com/dremio/exec/store/parquet > > I am sure they are very busy right now, but it may be worth discussing > factoring out this Parquet <-> Arrow interface into a library > component that can be donated to Apache Parquet. > > - Wes > > On Wed, Jul 19, 2017 at 4:28 PM, Sven Wagner-Boysen > <sven.wagner-boy...@signavio.com> wrote: > > Hi, > > > > I started looking into the projects Parquet and Arrow. Looks very > promising > > to me. > > > > I also came across PyArrow and the Parquet-Arrow integration in Python. > Is > > there something similar available for Java? > > > > https://arrow.apache.org/docs/python/parquet.html > > > > Thanks > > Sven >
