> On 2011-02-27 15:49:28, John Sichi wrote: > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteGBUsingIndex.java, > > line 19 > > <https://reviews.apache.org/r/392/diff/1/?file=10598#file10598line19> > > > > For HIVE-1644, the HMC team is creating a new package > > org.apache.hadoop.hive.ql.optimizer.index. I think any optimizer code > > related to indexing should go in there. > > > > Prajakta Kalmegh wrote: > I understand we just need to place our code in the > 'org.apache.hadoop.hive.ql.optimizer.index' package. Right?
Correct. But for generic optimizer code (e.g. RewriteParseContextGenerator, which is unrelated to indexing) can stay where it is. > On 2011-02-27 15:49:28, John Sichi wrote: > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteGBUsingIndex.java, > > line 51 > > <https://reviews.apache.org/r/392/diff/1/?file=10598#file10598line51> > > > > This code is currently tied to the compact index representation. > > > > We mentioned earlier that we'll need a new index representation > > (summary) instead in order to implement the counts correctly (we should > > leave the compact representation as is). > > > > So: > > > > * until the summary representation is added, we can't enable this > > > > * in general, it would be good to find a way to make this pluggable; > > for example, the bitmap index representation can also be utilized by > > counting the bits, but the rewrite expression would be slightly different > > > > Prajakta Kalmegh wrote: > We have a couple of designs ready for the new index handler. I will post > them on JIRA. > > When you say "pluggable" does it mean that we should be able to change > the rewrite expression depending on which index handler is used? For example, > we will get different rewrite expressions if we use either the new index > handler or the bitmap index handler. Our optimization code should work with > any rewrite expression. Is this understanding correct? Correct. If you look at what's going on in the latest HIVE-1644 patch, they are extending the HiveIndexHandler interface with an optional method for dealing with a WHERE clause. You can do the same for aggregation. It may take a few iterations to get it generic enough, but it helps that we already have the bitmap work in progress so that's two cases to generalize from. > On 2011-02-27 15:49:28, John Sichi wrote: > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteCanApplyProcFactory.java, > > line 291 > > <https://reviews.apache.org/r/392/diff/1/?file=10597#file10597line291> > > > > Need to abstract out this dependency on _c convention. > > Prajakta Kalmegh wrote: > Can you suggest a way using which we can get rid of this dependency? Actually, isn't just checking internalToAlias.get != null already good enough? If so we can just eliminate the _c check. - John ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/392/#review217 ----------------------------------------------------------- On 2011-02-03 16:45:15, John Sichi wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/392/ > ----------------------------------------------------------- > > (Updated 2011-02-03 16:45:15) > > > Review request for hive. > > > Summary > ------- > > Preliminary review. > > > This addresses bug HIVE-1694. > https://issues.apache.org/jira/browse/HIVE-1694 > > > Diffs > ----- > > http://svn.apache.org/repos/asf/hive/trunk/build.xml 1067048 > > http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java > 1067048 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java > 1067048 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java > 1067048 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java > 1067048 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteCanApplyCtx.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteCanApplyProcFactory.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteGBUsingIndex.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteIndexSubqueryCtx.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteIndexSubqueryProcFactory.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteParseContextGenerator.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteRemoveGroupbyCtx.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/RewriteRemoveGroupbyProcFactory.java > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java > 1067048 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java > 1067048 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java > 1067048 > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientnegative/fatal.q > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/ql_rewrite_gbtoidx.q > PRE-CREATION > > http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/ql_rewrite_gbtoidx.q.out > PRE-CREATION > > Diff: https://reviews.apache.org/r/392/diff > > > Testing > ------- > > > Thanks, > > John > >
