[jira] [Created] (ARROW-5144) ParquetDataset and CloudParuqtePiece not serializable

2019-04-08 Thread Martin Durant (JIRA)
Martin Durant created ARROW-5144: Summary: ParquetDataset and CloudParuqtePiece not serializable Key: ARROW-5144 URL: https://issues.apache.org/jira/browse/ARROW-5144 Project: Apache Arrow

[jira] [Created] (ARROW-3247) Support spark array and map types

2018-09-16 Thread Martin Durant (JIRA)
Martin Durant created ARROW-3247: Summary: Support spark array and map types Key: ARROW-3247 URL: https://issues.apache.org/jira/browse/ARROW-3247 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-3246) direct reading/writing of pandas categoricals

2018-09-16 Thread Martin Durant (JIRA)
Martin Durant created ARROW-3246: Summary: direct reading/writing of pandas categoricals Key: ARROW-3246 URL: https://issues.apache.org/jira/browse/ARROW-3246 Project: Apache Arrow Issue

[jira] [Created] (ARROW-3245) Infer index and/or filtering from parquet column statistics

2018-09-16 Thread Martin Durant (JIRA)
Martin Durant created ARROW-3245: Summary: Infer index and/or filtering from parquet column statistics Key: ARROW-3245 URL: https://issues.apache.org/jira/browse/ARROW-3245 Project: Apache Arrow

[jira] [Created] (ARROW-3244) Multi-file parquet loading without scan

2018-09-16 Thread Martin Durant (JIRA)
Martin Durant created ARROW-3244: Summary: Multi-file parquet loading without scan Key: ARROW-3244 URL: https://issues.apache.org/jira/browse/ARROW-3244 Project: Apache Arrow Issue Type

Re: Arrow and oamap [was: Re: Gandiva Initiative]

2018-06-25 Thread Martin Durant
nd would be challenging to use an embedded > system component > > I'm certain these projects can learn from each other -- I have spoken > with Jim (one of the developers of oamap) in the past, so welcome > further discussion here on the mailing list. > > Thanks, > We

Re: Gandiva Initiative

2018-06-25 Thread Martin Durant
1 19:15:20, Jacques Nadeau wrote: > >>>> Hey Guys,> >>>>> >>>> Dremio just open sourced a new framework for processing data in Arrow >>>> data> >>>> structures [1], built on top of the Apache Arrow C++ APIs and leveraging> >>>> LLVM (Apache licensed). It also includes Java APIs that leverage the >>>> Apache> >>>> Arrow Java libraries. I expect the developers who have been working on >>>> this> >>>> will introduce themselves soon. To read more about it, take a look at our> >>>> Ravindra's blog post (he's the lead developer driving this work): [2].> >>>> Hopefully people will find this interesting/useful.> >>>>> >>>> Let us know what you all think!> >>>>> >>>> thanks,> >>>> Jacques> >>>>> >>>>> >>>> [1] https://github.com/dremio/gandiva> >>>> [2] >>>> https://www.dremio.com/announcing-gandiva-initiative-for-apache-arrow/> >>>>> >> β€” Martin Durant martin.dur...@utoronto.ca

Re: How to model massive nested data

2018-05-10 Thread Martin Durant
doing the analysis may not need to go to C++ at all. oamap has POC loaders for arrow and parquet, but it’s original focus was ROOT, from the high-energy physics world. β€” Martin Durant martin.dur...@utoronto.ca

file-system specification

2018-05-09 Thread Martin Durant
will see that there are already some PRs and issues on the repo, please use liberally! Finally, if this project gathers steam, then the spec should be moved to a more prominent and standard location. Thanks, MD β€” Martin Durant martin.dur...@utoronto.ca

[jira] [Created] (ARROW-1322) hdfs: encryption-at-rest and secure transport

2017-08-02 Thread Martin Durant (JIRA)
Martin Durant created ARROW-1322: Summary: hdfs: encryption-at-rest and secure transport Key: ARROW-1322 URL: https://issues.apache.org/jira/browse/ARROW-1322 Project: Apache Arrow Issue

[jira] [Created] (ARROW-1321) hdfs delegation token functions

2017-08-02 Thread Martin Durant (JIRA)
Martin Durant created ARROW-1321: Summary: hdfs delegation token functions Key: ARROW-1321 URL: https://issues.apache.org/jira/browse/ARROW-1321 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-1320) hdfs block locations

2017-08-02 Thread Martin Durant (JIRA)
Martin Durant created ARROW-1320: Summary: hdfs block locations Key: ARROW-1320 URL: https://issues.apache.org/jira/browse/ARROW-1320 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-1319) hdfs methods

2017-08-02 Thread Martin Durant (JIRA)
Martin Durant created ARROW-1319: Summary: hdfs methods Key: ARROW-1319 URL: https://issues.apache.org/jira/browse/ARROW-1319 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-1318) hdfs access with auth

2017-08-02 Thread Martin Durant (JIRA)
Martin Durant created ARROW-1318: Summary: hdfs access with auth Key: ARROW-1318 URL: https://issues.apache.org/jira/browse/ARROW-1318 Project: Apache Arrow Issue Type: Test

[jira] [Created] (ARROW-1317) hdfs environment variables

2017-08-02 Thread Martin Durant (JIRA)
Martin Durant created ARROW-1317: Summary: hdfs environment variables Key: ARROW-1317 URL: https://issues.apache.org/jira/browse/ARROW-1317 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-1316) hdfs connector stand-alone

2017-08-02 Thread Martin Durant (JIRA)
Martin Durant created ARROW-1316: Summary: hdfs connector stand-alone Key: ARROW-1316 URL: https://issues.apache.org/jira/browse/ARROW-1316 Project: Apache Arrow Issue Type: Wish

[jira] [Created] (ARROW-1314) libhdfs installation didn't work - mac

2017-08-02 Thread Martin Durant (JIRA)
Martin Durant created ARROW-1314: Summary: libhdfs installation didn't work - mac Key: ARROW-1314 URL: https://issues.apache.org/jira/browse/ARROW-1314 Project: Apache Arrow Issue

[jira] [Created] (ARROW-1313) libhdfs installation didn't work

2017-08-02 Thread Martin Durant (JIRA)
Martin Durant created ARROW-1313: Summary: libhdfs installation didn't work Key: ARROW-1313 URL: https://issues.apache.org/jira/browse/ARROW-1313 Project: Apache Arrow Issue