> On 2011-06-13 22:57:46, John Sichi wrote: > > ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java, > > line 114 > > <https://reviews.apache.org/r/857/diff/4/?file=20984#file20984line114> > > > > I don't think this should be necessary. We just want to propagate the > > partition column predicate (whatever it is) from the base table query to > > the index table query; partition pruning on the index table query will do > > the rest of the work. > > > > In other words, if the original query had > > > > part_key=<whatever> > > > > we want to preserve that on the index table query. That's what the > > code is already supposed to be doing before your change; was it not working? > >
This code is to prevent automatic usage from kicking in if the index has not been built on the partition specified in the partition predicate. (i.e. if the index has only been built on partition ds=foo, and the query is select key from src where ds=bar; We do not want to execute an index query in this case. It seems like adding a test for bitmaps specifically to mirror index_auto_unused.q(which is where this functionality is tested for Compact indices) would be a good idea. - Syed ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/857/#review825 ----------------------------------------------------------- On 2011-06-11 19:05:42, Syed Albiz wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/857/ > ----------------------------------------------------------- > > (Updated 2011-06-11 19:05:42) > > > Review request for hive and John Sichi. > > > Summary > ------- > > Add support for generating index queries to support automatic usage of bitmap > indexes. This required changing the interface to the IndexHandlers to support > accepting queries on multiple indexes. The compact indexes were modified to > use this new interface as well, although no functional changes were made to > how they work. Only supports AND predicates right now, but it should be > possibly to extend the BitmapQuery interface defined in this patch to easily > support OR predicates as well. Currently benchmarking these changes on a test > cluster. > > > This addresses bug HIVE-2036. > https://issues.apache.org/jira/browse/HIVE-2036 > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 4fba845 > ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java e5ee183 > ql/src/java/org/apache/hadoop/hive/ql/index/IndexPredicateAnalyzer.java > 3caa4cc > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java > af9d7b1 > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapOuterQuery.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapQuery.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java > 56e7609 > ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java d64e88b > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java > 268560d > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java > 0873e1a > ql/src/test/queries/clientpositive/index_auto_partitioned.q 5f92f04 > ql/src/test/queries/clientpositive/index_bitmap_auto.q PRE-CREATION > ql/src/test/queries/clientpositive/index_bitmap_auto_partitioned.q > PRE-CREATION > ql/src/test/results/clientpositive/index_auto_partitioned.q.out 05cc84a > ql/src/test/results/clientpositive/index_bitmap_auto.q.out PRE-CREATION > ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out > PRE-CREATION > > Diff: https://reviews.apache.org/r/857/diff > > > Testing > ------- > > Passes unit tests, additional testcase to test automatic bitmap indexing > index_bitmap_auto.q was also added to the TestCliDriver suite. Currently > benchmarking changes on a test cluster. > > > Thanks, > > Syed > >
