Re: Hive optimizer

Its both. Some of the optimizations are rule based and some are cost based. John From: Ashok Kumar mailto:ashok34...@yahoo.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>>, Ashok Kumar mailto:ashok34...@yahoo.com>> Date: Wednesday, February 3, 20

Re: VolcanoPlanner vs HepPlanner

Yes, we used to do VolcanPlanner before but we switched in Hive 14 time frame (2014) due to the performance limitations. I am not sure why you are after VolcanoPlanner; but yes you can get back to VolcanoPlanner and probably get it to work. I know this because I did most of those changes back in

Re: VolcanoPlanner vs HepPlanner

This would be a broad change. Hep Planner does enumerate different join orders through "LoptOptimizeJoinRule". Volcano planner is not used as it has scalability issues. Thanks John From: Raajay mailto:raaja...@gmail.com>> Reply-To: "user@hive.apache.org" mailto:user@

Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan

Congrats Ashutosh! From: Vaibhav Gumashta mailto:vgumas...@hortonworks.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Wednesday, September 16, 2015 at 1:01 PM To: "user@hive.apache.org" mailto:user@hive.apache

Re: CBO - get cost of the plan

I be able to explicitly specify it somewhere, so thats gathering is accurate ? Also, what is the definition of cumulative cost ? Thanks for the help, Raajay On Mon, Aug 24, 2015 at 8:51 PM, John Pullokkaran mailto:jpullokka...@hortonworks.com>> wrote: From the text below it seems li

Re: CBO - get cost of the plan

output_tab (a_day int, a_product string, a_alternate string, >total_sales int); > >-- the query >insert overwrite table output_tab >select >a_day, a_product, b_alternate, (a_sales + b_sales + c_sales) as >total_sales >from >tableA a join tableB b >on a.a_day = b.b_day

Re: CBO - get cost of the plan

In addition to col stats you also need table stats. From: John Pullokkaran mailto:jpullokka...@hortonworks.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Date: Monday, August 24, 2015 at 5:23 PM To: "

Re: CBO - get cost of the plan

Raajay, You don't have col stats hence it assumes 1 for row count. What version of Hive are you on? Thanks John From: Raajay mailto:raaja...@gmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Monday, August 24, 2015 at 5:19 PM To: "

Re: Hive CBO - Calcite Interface

Hi Raajay, #1 No, there is no API for this. #2 If you enable Logging (BaseSemanticAnalyzer) then CalcitePlanner will print out the plan with cost. John From: Raajay mailto:raaja...@gmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Mon

Re: Hive indexing optimization

Index doesn’t seems to be kicking in this case. Please file a bug for this. Thanks John From: Bennie Leo mailto:tben...@hotmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Monday, June 29, 2015 at 5:25 PM To: "user@hive.apache.org

Re: Hive indexing optimization

Could you post explain extended output? From: Bennie Leo mailto:tben...@hotmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Monday, June 29, 2015 at 10:35 AM To: "user@hive.apache.org" mailto:user@hive.apach

Re: Hive indexing optimization

"SELECT StartIp, EndIp, Country FROM ipv4geotable” should have been rewritten as a scan against index table. BitMap Indexes seems to support inequalities (<=, <, >=). Post the explain plan. On 6/26/15, 8:56 PM, "Gopal Vijayaraghavan" wrote: >Hi, > >Hive indexes won¹t really help you speed up t

Re: Cost based optimization

ce code, that are primarily responsible for Query Planning. Thanks, Raajay On Thu, Jun 25, 2015 at 4:45 PM, John Pullokkaran mailto:jpullokka...@hortonworks.com>> wrote: Hive does look in to alternate join orders and pick the best plan that minimizes cost. It uses a greedy algorithm to enume

Re: Hive indexing optimization

Set hive.optimize.index.filter=true; Thanks John From: Bennie Leo mailto:tben...@hotmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Thursday, June 25, 2015 at 5:48 PM To: "user@hive.apache.org" mailto:user

Re: Cost based optimization

Hive does look in to alternate join orders and pick the best plan that minimizes cost. It uses a greedy algorithm to enumerate plan space. Thanks John From: Raajay mailto:raaja...@gmail.com>> Reply-To: "user@hive.apache.org" mailto:user@hive.apache.org>> Date: Thurs

Re: Orc file and Hive Optimiser

en or endorsed by Peridale Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. From: John Pullokka

Re: Orc file and Hive Optimiser

ORC format is transparent to CBO. Currently we are working on a new cost model which might reflect ORC's performance advantages in optimization decisions. Thanks John From: Mich Talebzadeh mailto:m...@peridale.co.uk>> Reply-To: "user@hive.apache.org" mailto:user@hiv

Re: [ANNOUNCE] New Hive Committer - John Pullokkaran

Thanks everyone for all the wishes. On Wed, Oct 1, 2014 at 1:35 PM, Xiaobing Zhou wrote: > Congrats John! > > On Wed, Oct 1, 2014 at 12:54 PM, Matthew McCline > wrote: > >> Congrats! >> >> On Wed, Oct 1, 2014 at 8:19 AM, Carl Steinbach wrote: >> >>

Re: [ANNOUNCE] New Hive Committer - Gunther Hagleitner

Congrats Gunther! On Mon, Jul 22, 2013 at 8:24 AM, Vikram Dixit wrote: > Congrats Gunther! Great news! > > On Jul 21, 2013, at 8:52 PM, Vaibhav Gumashta wrote: > > > Congratulations Gunther! > > > > On Sun, Jul 21, 2013 at 8:36 PM, David Schorow > wrote: > > Cool. Congratulations Gunther. Thi