> On 2011-06-13 22:57:46, John Sichi wrote: > > ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java, > > line 114 > > <https://reviews.apache.org/r/857/diff/4/?file=20984#file20984line114> > > > > I don't think this should be necessary. We just want to propagate the > > partition column predicate (whatever it is) from the base table query to > > the index table query; partition pruning on the index table query will do > > the rest of the work. > > > > In other words, if the original query had > > > > part_key=<whatever> > > > > we want to preserve that on the index table query. That's what the > > code is already supposed to be doing before your change; was it not working? > > > > Syed Albiz wrote: > This code is to prevent automatic usage from kicking in if the index has > not been built on the partition specified in the partition predicate. (i.e. > if the index has only been built on partition ds=foo, and the query is select > key from src where ds=bar; We do not want to execute an index query in this > case. It seems like adding a test for bitmaps specifically to mirror > index_auto_unused.q(which is where this functionality is tested for Compact > indices) would be a good idea.
The logic for making sure that the necessary index partitions exist is already present in IndexWhereProcessor.checkPartitionsCoveredByIndex. If that's not working, we should fix it; it should not be necessary to change the predicate analyzer at all. - John ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/857/#review825 ----------------------------------------------------------- On 2011-06-14 04:05:43, Syed Albiz wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/857/ > ----------------------------------------------------------- > > (Updated 2011-06-14 04:05:43) > > > Review request for hive and John Sichi. > > > Summary > ------- > > Add support for generating index queries to support automatic usage of bitmap > indexes. This required changing the interface to the IndexHandlers to support > accepting queries on multiple indexes. The compact indexes were modified to > use this new interface as well, although no functional changes were made to > how they work. Only supports AND predicates right now, but it should be > possibly to extend the BitmapQuery interface defined in this patch to easily > support OR predicates as well. Currently benchmarking these changes on a test > cluster. > > > This addresses bug HIVE-2036. > https://issues.apache.org/jira/browse/HIVE-2036 > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 4fba845 > ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java e5ee183 > ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java > 3caa4cc > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java > af9d7b1 > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapOuterQuery.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapQuery.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java > 56e7609 > ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java d64e88b > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java > 268560d > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java > 0873e1a > ql/src/test/queries/clientpositive/index_auto_partitioned.q 5f92f04 > ql/src/test/queries/clientpositive/index_bitmap_auto.q PRE-CREATION > ql/src/test/queries/clientpositive/index_bitmap_auto_partitioned.q > PRE-CREATION > ql/src/test/results/clientpositive/index_auto_partitioned.q.out 05cc84a > ql/src/test/results/clientpositive/index_bitmap_auto.q.out PRE-CREATION > ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out > PRE-CREATION > > Diff: https://reviews.apache.org/r/857/diff > > > Testing > ------- > > Passes unit tests, additional testcase to test automatic bitmap indexing > index_bitmap_auto.q was also added to the TestCliDriver suite. Currently > benchmarking changes on a test cluster. > > > Thanks, > > Syed > >
