RE: Optimized Hive query

Any progress on this one? Dudu From: Aviral Agarwal [mailto:aviral12...@gmail.com] Sent: Wednesday, June 15, 2016 1:04 PM To: user@hive.apache.org Subject: Re: Optimized Hive query I ok to digging down to the AST Builder class. Can you guys point me to the right class ? Meanwhile "ex

Re: Optimized Hive query

I ok to digging down to the AST Builder class. Can you guys point me to the right class ? Meanwhile "explain (rewrite | logical | extended) ", all are not able to flatten even a basic query of the form: select * from ( select * from ( select c from d) alias_1 ) alias_2 into select c from d Tha

Re: Optimized Hive query

> So I was hoping of using internal Hive CBO to somehow change the AST >generated for the query somehow. Hive does have an "explain rewrite" but that prints out the query before CBO runs. For CBO, you need to dig all the way down to the ASTBuilder class and work upwards from there. Perhaps add

Re: Optimized Hive query

representation of the abstract > syntactic <https://en.wikipedia.org/wiki/Abstract_syntax> structure of source > code <https://en.wikipedia.org/wiki/Source_code> written in a programming > language <https://en.wikipedia.org/wiki/Programming_language>. > > > > &

RE: Optimized Hive query

_syntax> structure of source code<https://en.wikipedia.org/wiki/Source_code> written in a programming language<https://en.wikipedia.org/wiki/Programming_language>. From: Mich Talebzadeh [mailto:mich.talebza...@gmail.com] Sent: Tuesday, June 14, 2016 7:58 PM To: user Subject: Re: Op

Re: Optimized Hive query

…”) > > In no point do we have a “flattened query” > > > > Dudu > > > > *From:* Aviral Agarwal [mailto:aviral12...@gmail.com] > *Sent:* Tuesday, June 14, 2016 10:37 AM > *To:* user@hive.apache.org > *Subject:* Re: Optimized Hive query > > > > Hi,

RE: Optimized Hive query

Subject: Re: Optimized Hive query Hi, Thanks for the replies. I already knew that the optimizer already does that. My usecase is a bit different though. I want to display the flattened query back to the user. So I was hoping of using internal Hive CBO to somehow change the AST generated for the

Re: Optimized Hive query

I presume the user is concerned with performance? The whole use case of a CBO is to take care of queries by finding the optimum access path. otherwise we would have a RBO as is in the old days of Hive. If you are in the more recent version of Hive CBO does the job. However, you may think of mov

Re: Optimized Hive query

Hi, Thanks for the replies. I already knew that the optimizer already does that. My usecase is a bit different though. I want to display the flattened query back to the user. So I was hoping of using internal Hive CBO to somehow change the AST generated for the query somehow. Thanks, Aviral On Tu

Re: Optimized Hive query

> You can see that you get identical execution plans for the nested query >and the flatten one. Wasn't that always though. Back when I started with Hive, before Stinger, it didn't have the identity project remover. To know if your version has this fix, try looking at hive> set hive.optimize.rem

RE: Optimized Hive query

Hi You don’t need to do anything, the optimizer does it for you. You can see that you get identical execution plans for the nested query and the flatten one. Dudu > create multiset table t (i int); > explain select * from t; +---

Re: Optimized Hive query

Yes I want to flatten the query. Also the Insert code is correct. Thanks, Aviral Agarwal On Tue, Jun 14, 2016 at 3:46 AM, Mich Talebzadeh wrote: > you want to flatten the query I understand. > > create temporary table tmp as select c from d; > > INSERT INTO TABLE a > SELECT c from tmp where >

Re: Optimized Hive query

you want to flatten the query I understand. create temporary table tmp as select c from d; INSERT INTO TABLE a SELECT c from tmp where condition Is the INSERT code correct? HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw