----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27247/#review58712 -----------------------------------------------------------
Hi Suhas, thanks a lot for this great work! I haven't looked at all the details, but have some high-level comments just from the first pass. I think if we revert some of the SMB Join stuff and commented code that don't refer to mapjoin, it will be easier to see the core of this change. Thanks! ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkConvertJoinMapJoin.java <https://reviews.apache.org/r/27247/#comment99831> Would it make sense to rename it as SparkMapJoinOptimizer to go more with existing convention? I understand the convention is not very well doc'ed (or even followed). ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkConvertJoinMapJoin.java <https://reviews.apache.org/r/27247/#comment99832> Let's get rid of the commented code in this class (unless its necessary) to make it easier to read. ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkProcContext.java <https://reviews.apache.org/r/27247/#comment99833> We can revert this class, as these changes are not consumed by mapjoin. ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkUtils.java <https://reviews.apache.org/r/27247/#comment99834> We can revert this class as methods are not being used for mapjoin. ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java <https://reviews.apache.org/r/27247/#comment99835> We need to put the check of auto.convert.join before we enable this optimization. It would make it easier this way. ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java <https://reviews.apache.org/r/27247/#comment99836> I don't see this class attached ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java <https://reviews.apache.org/r/27247/#comment99837> Let's revert this code, as its not being used for mapjoin. - Szehon Ho On Oct. 27, 2014, 9:53 p.m., Suhas Satish wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27247/ > ----------------------------------------------------------- > > (Updated Oct. 27, 2014, 9:53 p.m.) > > > Review request for hive, Rui Li, Szehon Ho, and Xuefu Zhang. > > > Repository: hive-git > > > Description > ------- > > This addresses jira HIVE-8616 > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java > 46dcfaf > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkConvertJoinMapJoin.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkProcContext.java > ed88c60 > ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkUtils.java > 8e28887 > ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkWork.java 4f5feca > ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java > 1c663c4 > > Diff: https://reviews.apache.org/r/27247/diff/ > > > Testing > ------- > > > Thanks, > > Suhas Satish > >
