The merge criteria on dynamic partitons should be per partiton
--------------------------------------------------------------
Key: HIVE-1806
URL: https://issues.apache.org/jira/browse/HIVE-1806
Project: Hive
Issue Type: Bug
Reporter: Ning Zhang
Assignee: Ning Zhang
Currently the criteria of whether a merge job should be fired on dynamic
generated partitions are is the average file size of files across all dynamic
partitions. It is very common that some dynamic partitions contains mostly
large files and some contains mostly small files. Even though the average size
of the total files are larger than the hive.merge.smallfiles.avgsize, we should
merge those partitions containing small files only.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
