Re: Hive CBO - issues

tics. I believe you also need auto-convert statement. Hive will not attempt to convert to map-join without this flag. Regards, Vineet Garg On Feb 12, 2019, at 11:48 AM, Venkatesh Selvaraj mailto:venkateshselva...@pinterest.com>> wrote: Hello All, I would like to know if anyone of you faced

Re: Hive CBO - issues

t 11:58 AM Ambud Sharma wrote: > As long as column stats are available for all tables, Hive CBO will try to > do the optimization when generating query plan. > > On Tue, Feb 12, 2019, 11:48 AM Venkatesh Selvaraj < > venkateshselva...@pinterest.com wrote: > >> Hello All, &g

Hive CBO - issues

Hello All, I would like to know if anyone of you faced this issue with HIVE CBO and also would like to get some directions as to how to go about resolving it. We are using Hive 1.2.1. When we were evaluating the benefits of Cost based Optimization(CBO), we stumbled upon this. 1) Queries which

Re: Hive external indexes incorporation into Hive CBO

Kindly provide an example where one can see EXPLAIN SELECT .shows external index usage? That will be great. Choose your table and block size Thanks Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

Re: Hive external indexes incorporation into Hive CBO

I am still not sure why you think they are not used. The main issue is that the block size is usually very large (eg 256 MB compared to kilobytes / sometimes few megabytes in traditional databases) and the indexes refer to blocks. This makes it less likely that you can leverage it for small data

Hive external indexes incorporation into Hive CBO

Hi, As we have discussed this few times, Hive external indexes (as opposed to Store Indexes in ORC tables) are there but are not currently utilised. For Hive to be effective it needs to use these indexes for a variety of reasons and the CBO should leverage these indexes. I am not sure how far we

Hive CBO cost

Hi, Is there any way we can see/get the cost(CPU & I/O units) of the optimized Hive query plan. Once the query is submitted, it gets parsed, multiple plans gets generated, costs get associated with each plan and finally "the lowest cost plan" gets selected before execution. We were looking at

Hive CBO

Hi, Does Hive supports extended CBO options like cpu,io only for tez execution. When I See the HIveDefaultCostModel class (https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java) all cost has been made to zero Cost. Whe

Re: Hive CBO - Calcite Interface

ate: Friday, August 14, 2015 at 12:11 To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>>, "d...@hive.apache.org<mailto:d...@hive.apache.org>" mailto:d...@hive.apache.org>> Subject: Re: Hive CBO - Calcite Interfac

Re: Hive CBO - Calcite Interface

hive.apache.org>> Date: Monday, August 10, 2015 at 8:48 AM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>>, "d...@hive.apache.org<mailto:d...@hive.apache.org>" mailto:d...@hive.apache.org>> Subject: Hive CBO -

Hive CBO - Calcite Interface

For a research project, I am studying the performance gains in Hive queries achieved through cost based optimization. Rather than just study scenarios where CBO is switched on or off, I would like to study the performance across a “collection" of query plans considered by the CBO (Calcite); for exa

Re: hive cbo calciteplanner

+ Dev Regards Sanjiv Singh Mob : +091 9990-447-339 On Wed, Jun 17, 2015 at 6:42 AM, wangzhenhua (G) wrote: > Hi all, > > I'm reading the source code of Hive cbo (CalcaitePlanner), but I find it > hard to follow. > Listed below are some of the questions: > 1. What

hive cbo calciteplanner

Hi all, I'm reading the source code of Hive cbo (CalcaitePlanner), but I find it hard to follow. Listed below are some of the questions: 1. What's the relationship between HepPlanner and HiveVolcanoPlanner? 2. I don't have a clue about these concepts: clusters, traitDef and

Re: Question about bushy join in hive CBO

However, a bit more experimentation and debugging informed us that > LoptOptimizeJoinRule > can actually generate bushy join trees. Problem was we had bugs in our > statistics/cost model which we were feeding to the rule. Once that was > established we switched back to LoptOptimizeJoinRu

Re: Question about bushy join in hive CBO

feeding to the rule. Once that was established we switched back to LoptOptimizeJoinRule. So, in nut shell, hive CBO can and does generate bushy joins. If you have test case where we are not generating bushy join, where we can, please post back. Will be happy to take a look. Thanks, Ashutosh On Fri

Question about bushy join in hive CBO

Hi all, I'm studying CBO code in hive. I have a question about bushy join optimization. Bushy join did get introduced in hive via HIVE-7577 , and played an important role in optimizing several queries in TPCDS benchmark. Somehow I saw the bushy joi