Thanks John, for the comments. Hep Planner, however does not satisfy my requirements. Internally, while enumerating the different join orders, it destroys “bad” query plans. Volcano planner however, is different is this aspect.
I dug around the Hive code, and figured out that when CBO was introduce to Hive it used the Volcano planner. (refer commit id : c3b7c038416d400598bbd9ead3f220101df07aae). The following rules were added to Volcano Planner for join order optimization. planner.addRule(HiveSwapJoinRule.INSTANCE); planner.addRule(HivePushJoinThroughJoinRule.LEFT); planner.addRule(HivePushJoinThroughJoinRule.RIGHT); planner.addRule(HivePullUpProjectsAboveJoinRule.BOTH_PROJECT); planner.addRule(HivePullUpProjectsAboveJoinRule.LEFT_PROJECT); planner.addRule(HivePullUpProjectsAboveJoinRule.RIGHT_PROJECT); planner.addRule(HiveMergeProjectRule.INSTANCE); By changing the existing rules (JoinToMultiJoinRule and LoptOptimizeJoinRule), with the equivalent of the above rules from Calcite, and replacing HepPlanner with VolcanoPlanner, I should be able to get VolcanoPlanner going. Is that right ? I am willing to live with the inefficiency of the Volcano Planner. Thanks Raajay > On Oct 7, 2015, at 11:58 AM, John Pullokkaran <[email protected]> > wrote: > > This would be a broad change. > Hep Planner does enumerate different join orders through > “LoptOptimizeJoinRule". > > Volcano planner is not used as it has scalability issues. > > Thanks > John > From: Raajay <[email protected] <mailto:[email protected]>> > Reply-To: "[email protected] <mailto:[email protected]>" > <[email protected] <mailto:[email protected]>> > Date: Tuesday, October 6, 2015 at 2:06 PM > To: "[email protected] <mailto:[email protected]>" > <[email protected] <mailto:[email protected]>> > Subject: VolcanoPlanner vs HepPlanner > > Hello - > > While surfing the code in "apply" function of CalcitePlannerAction > (CalcitePlanner.java), I see that most rules are passed to a HepPlanner to > arrive at the "least cost" operator tree. > > The HiveVolcanoPlanner although defined and initialized is never invoked to > find the best operator tree. I want to change the planner from Hep to > Volcano, for the following reason: to obtain a collection of costlier join > orders. > > Passing the rules for the Hep Planner as is to HiveVolcano planner does not > help. The query plan is never altered. > > Any ideas, what rules need to be passed to the HiveVolcanoPlanner for > effective CBO ? > > > > Thanks, > Raajay
