Hi, Can you do show create table <TABLE> on your external table and send the sections from
STORED AS ORC TBLPROPERTIES ( onwards please? HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com On 27 February 2016 at 18:59, Jie Zhang <jiezh2...@gmail.com> wrote: > Hi, > > We have an external ORC table which includes ~200 relatively small orc > files (less than 256MB). When querying the table with selective SARG > predicate (explain shows the predicate is qualified pushdown), we expects a > few splits generated with pruning based on predicate condition and only a > few files will be scanned. However, somehow predicate pushdown is not in > effect at all, all the files are scanned in MR job and SARG did not even > show up in the MR job config. > > After digging more in hive code (version 0.14), looks like the split > pruning only happens for the stripes within each file. If the file size is > smaller than default split size, SARG is not considered. Here is the code > we are referring: > > https://github.com/apache/hive/blob/release-0.14.0/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L656 > > > Any idea why SARG is ignored for this scenario? also can split pruning > filter out the files with all stripes not satisfied with SARG condition? > Thanks for any help, really appreciated. > > Jessica >
