回复: different outer join plan between hive 0.9 and hive 0.10

2013-07-03 Thread wzc1989
Hi navis: Thanks for your reply. Currently I'm working on the temporary solution by changing the type of filter mask and doing the performance test. I try to read the patches and source code now and when I get better understanding of the code maybe I can help with this problem :) -- wzc198

Re: 回复: different outer join plan between hive 0.9 and hive 0.10

2013-07-01 Thread Navis류승우
Yes, a little bit. IMHO, these flags could be assigned only for aliases with condition on 'on' clause. Then, I think, even a byte (8 flags) could be enough in most cases. I'll do that if time permits. 2013/7/1 wzc1989 : > hi navis: > look at the patches in (HIVE-3411, HIVE-4206, HIVE-4212, HIVE-

回复: different outer join plan between hive 0.9 and hive 0.10

2013-07-01 Thread wzc1989
hi navis: look at the patches in (HIVE-3411, HIVE-4206, HIVE-4212, HIVE-3464), I understand what you mean by "hive tags rows a filter mask as a short for outer join, which can contain 16 flags. " . I wonder why not choose Long or int which can contain 64/32 tags. Does adding one Long/int in eve

Re: 回复: different outer join plan between hive 0.9 and hive 0.10

2013-05-13 Thread Navis류승우
In short, hive tags rows a filter mask as a short for outer join, which can contain 16 flags. (see HIVE-3411, plz) I'll survey for a solution. 2013/5/14 wzc1989 : > "hive cannot merge joins of 16+ aliases with outer join into single stage." > In our use case we use one table full outer join all o

回复: different outer join plan between hive 0.9 and hive 0.10

2013-05-13 Thread wzc1989
"hive cannot merge joins of 16+ aliases with outer join into single stage." In our use case we use one table full outer join all other table to produce one big table, which may exceed 16 outer join limits and will be split into multi stage under hive 0.10. It become very slow under hive 0.10 whil

Re: 回复: different outer join plan between hive 0.9 and hive 0.10

2013-05-13 Thread Navis류승우
The error message means hive cannot merge joins of 16+ aliases with outer join into single stage. It was 8 way originally (HIVE-3411) but expanded to 16 later. Check https://issues.apache.org/jira/browse/HIVE-3411 for details. 2013/5/14 wzc1989 : > This time i cherry-pick HIVE-3464, HIVE-4212, H

回复: different outer join plan between hive 0.9 and hive 0.10

2013-05-13 Thread wzc1989
This time i cherry-pick HIVE-3464, HIVE-4212, HIVE-4206 and some related commits and the above explain result matches in hive 0.9 and hive 0.10, thanks! But I confuse about this error msg: JOINNODE_OUTERJOIN_MORETHAN_16(10142, "Single join node containing outer join(s) " + "cannot have mo

Re: different outer join plan between hive 0.9 and hive 0.10

2013-03-28 Thread Navis류승우
The problem is mixture of issues (HIVE-3411, HIVE-4209, HIVE-4212, HIVE-3464) and still not completely fixed even in trunk. Will be fixed shortly. 2013/3/29 wzc : > The bug remains even if I apply the patch in HIVE-4206 :( The explain > result hasn't change. > > > 2013/3/28 Navis류승우 >> >> It's

Re: different outer join plan between hive 0.9 and hive 0.10

2013-03-28 Thread wzc
The bug remains even if I apply the patch in HIVE-4206 :( The explain result hasn't change. 2013/3/28 Navis류승우 > It's a bug (https://issues.apache.org/jira/browse/HIVE-4206). > > Thanks for reporting it. > > 2013/3/24 wzc : > > Recently we tried to upgrade our hive from 0.9 to 0.10, but found

Re: different outer join plan between hive 0.9 and hive 0.10

2013-03-27 Thread Navis류승우
It's a bug (https://issues.apache.org/jira/browse/HIVE-4206). Thanks for reporting it. 2013/3/24 wzc : > Recently we tried to upgrade our hive from 0.9 to 0.10, but found some of > our hive queries almost 7 times slow. One of such query consists multiple > table outer join on the same key. By lo

different outer join plan between hive 0.9 and hive 0.10

2013-03-23 Thread wzc
Recently we tried to upgrade our hive from 0.9 to 0.10, but found some of our hive queries almost 7 times slow. One of such query consists multiple table outer join on the same key. By looking into the query, we found the query plans generate by hive 0.9 and hive 0.10 are different. Here is the