It sounds like you want to look at setting hive.merge.mapredfiles to true in
your hive-site.xml.
Just be aware that it will likely add another map step to your jobs to
consolidate the files.
Matt Tucker
On Nov 8, 2011, at 6:19 PM, Shouguo Li wrote:
> i think that has to do with your config
i think that has to do with your configured block size, check what's your
value for dfs.block.size in /hdfs-site.xml
but just curious, why would number of files matter for your use case?
On Fri, Oct 21, 2011 at 1:18 AM, Vikas Srivastava <
vikas.srivast...@one97.net> wrote:
> Hey All,
>
>
> i hav
Hey All,
i have an issue like i got a table having single partition but in that
partition say around 100 200mb files when i overwrite this into other table
its make 100 files of 20 mb(compressed) what i want is that it should make
only 1 or 2 or 10 file of 200mb or 100mb
means after overwrite
