Re: Merging small files in partitions

Hi Edward,Can we do the same/similar thing for parquet file?Any pointer?Regards,Mohammad On Tuesday, June 16, 2015 2:35 PM, Edward Capriolo wrote: https://github.com/edwardcapriolo/filecrush On Tue, Jun 16, 2015 at 5:05 PM, Chagarlamudi, Prasanth wrote: Hello,I am looking for a

Re: Merging small files in partitions

https://github.com/edwardcapriolo/filecrush On Tue, Jun 16, 2015 at 5:05 PM, Chagarlamudi, Prasanth < prasanth.chagarlam...@epsilon.com> wrote: > Hello, > > I am looking for an optimized way to merge small files in hive partitions > into one big file. > > I came across *Alter Table/Partition Con

Merging small files in partitions

Hello, I am looking for an optimized way to merge small files in hive partitions into one big file. I came across Alter Table/Partition Concatenate https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterTable/PartitionConcatenate. Doc says this only works for