will/when Spark/SparkSQL will support ORCFile format

2014-10-08 Thread James Yu
Didn't see anyone asked the question before, but I was wondering if anyone knows if Spark/SparkSQL will support ORCFile format soon? ORCFile is getting more and more popular hi Hive world. Thanks, James

Re: will/when Spark/SparkSQL will support ORCFile format

2014-10-08 Thread James Yu
wrote: > >> James, >> >> Michael at the meetup last night said there was some development >> activity around ORCFiles. >> >> I'm curious though, what are the pros and cons of ORCFiles vs Parquet? >> >> On Wed, Oct 8, 2014 at 10:03 AM, James Yu wr

Re: will/when Spark/SparkSQL will support ORCFile format

2014-10-09 Thread James Yu
more easily and systematically > after this PR is merged. > > On 10/9/14 8:22 AM, James Yu wrote: > >> Thanks Mark! I will keep eye on it. >> >> @Evan, I saw people use both format, so I really want to have Spark >> support >> ORCFile. >> &

Re: will/when Spark/SparkSQL will support ORCFile format

2014-10-09 Thread James Yu
e will be the same > as that for datasources included in the core spark sql library. > > Michael > > On Thu, Oct 9, 2014 at 2:18 PM, James Yu wrote: > >> For performance, will foreign data format support, same as native ones? >> >> Thanks, >> James &