Re: loading in ORC from big compressed file

Thanks Marcin, I worked I uncompressed file and then loaded file in hive table. Now its been quick, few mins. Regards Sanjiv Singh Mob : +091 9990-447-339 On Wed, Jun 22, 2016 at 7:44 AM, Jörn Franke wrote: > > > Marcin is correct : either split up the gzip files in smaller files of at

Re: loading in ORC from big compressed file

Marcin is correct : either split up the gzip files in smaller files of at least on HDFS block or use bzip2 with block compression. What is the original format of the table? > On 22 Jun 2016, at 01:50, Marcin Tustin wrote: > > This is because a GZ file is not splittable at all. Basically, try

Re: loading in ORC from big compressed file

Hi Are you using map-reduce as execution engine? what version of Hive are you on? HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://

Re: loading in ORC from big compressed file

This is because a GZ file is not splittable at all. Basically, try creating this from an uncompressed file, or even better split up the file and put the files in a directory in hdfs/s3/whatever. On Tue, Jun 21, 2016 at 7:45 PM, @Sanjiv Singh wrote: > Hi , > > I have big compressed data file *my_