[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291024#comment-14291024
]
Lefty Leverenz commented on HIVE-1642:
--
Doc done: The wiki documents all the configur
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093773#comment-14093773
]
Lefty Leverenz commented on HIVE-1642:
--
For the record (and searchability): This adde
Thanks!
On Mon, Dec 27, 2010 at 11:56 PM, Liyin Tang wrote:
> Yes. Only execute one of the them.
>
> On 27 December 2010 23:43, wrote:
>
> > A question about the design doc:
> >
> > "If one of the tables is large and others are small enough to run Map
> Join,
> > then the Conditional Task will
Yes. Only execute one of the them.
On 27 December 2010 23:43, wrote:
> A question about the design doc:
>
> "If one of the tables is large and others are small enough to run Map Join,
> then the Conditional Task will pick the corresponding Map Join Local Task
> to
> run."
> Here you pick one tab
A question about the design doc:
"If one of the tables is large and others are small enough to run Map Join,
then the Conditional Task will pick the corresponding Map Join Local Task to
run."
Here you pick one table as big, hash all other tables into memory by join
key individually. If it works, i
Hi,
If multiple tables join on different join keys, it will be separated into
multiple MapRed Tasks.
Also the threshold of the small table file size means the sum of all the
small table.
There is a documentation and a slide about this feature:
http://www.slideshare.net/aiolos127/join-optimization
Thanks for the reply. I want to get clarification on this feature.
If one of the two joining tables table t1 is smaller than 25M and is
sharded, how does this feature work?
Suppose there are joins on multiple tables such as t1, t2 and t3. If t1 and
t2 are smaller than 25M and co-located with joi
Hi,
How large is t1 and t2 ?
if both of t1 and t2 is larger than 25M (a default threshold), the query
processor will do the common join.
Thanks
Liyin
On 23 December 2010 18:50, wrote:
> Hi,
>
> I set hive.auto.convert.join=true and run the following query:
>
> select t1.foo, count(t2.bar) from
Hi,
I set hive.auto.convert.join=true and run the following query:
select t1.foo, count(t2.bar) from invites t1 join invites t2 on
(t1.foo=t2.foo) group by t1.foo;
I did not see it ran as map side join. Did I miss something? Is there any
precondition for this feature to work?
Thanks.
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain resolved HIVE-1642.
--
Resolution: Fixed
Hadoop Flags: [Reviewed]
Committed. Thanks Liyin
> Convert join queries to map-
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932843#action_12932843
]
Namit Jain commented on HIVE-1642:
--
+1 running tests
> Convert join queries to map-join ba
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: hive-1642_11.patch
When the local task runs out of memory, do NOT print any thing out and jus
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: hive-1642_10.patch
After discussing, we think the function: replaceWithConditionalTask is not
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: hive-1642_9.patch
some minor changes in ConditionalResolverCommonJoin.java
> Convert join qu
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: hive-1642_7.patch
In Task.java
public void replaceWithConditionalTask(ConditionalTask cndTsk,
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932744#action_12932744
]
Namit Jain commented on HIVE-1642:
--
ConditionalResolverCommonJoin
// Iterate the sor
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932741#action_12932741
]
Namit Jain commented on HIVE-1642:
--
ConditionalResolverCommonJoin
// generate file siz
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: hive-1642_6.patch
Remove the getBackupTask interface from all the Conditional Resolver
> Con
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: (was: hive-1642_5.patch)
> Convert join queries to map-join based on size of table/row
> -
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: hive-1642_5.patch
Add more detailed description on configuration xml file
Revert the DriverCon
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: hive-1642_5.patch
Add more descriptions to the configuration files.
Revert the DriverContext.
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932666#action_12932666
]
Namit Jain commented on HIVE-1642:
--
hive-default.xml
477
478 hive.mapjoin.hashtable
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: hive_1642_4.patch
This patch formats the output of local task.
> Convert join queries to map-
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: hive_1642_2.patch
Thanks for the comments.
I have updated the patch according to the review c
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932214#action_12932214
]
Namit Jain commented on HIVE-1642:
--
come more minor comments:
1203 //Qualify the
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932168#action_12932168
]
Namit Jain commented on HIVE-1642:
--
TaskGraphWalker:
if(!(nd instanceof Task)){
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932132#action_12932132
]
Liyin Tang commented on HIVE-1642:
--
There are 2 kinds of backup. 1) task level 2) branch le
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932112#action_12932112
]
Namit Jain commented on HIVE-1642:
--
Let us talk about it - I dont think Tasks need to be se
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931995#action_12931995
]
Liyin Tang commented on HIVE-1642:
--
Thanks for reviewing.
1. I will add these parameters i
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931992#action_12931992
]
Namit Jain commented on HIVE-1642:
--
Can you add more comments for the use of the new TaskWa
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931991#action_12931991
]
Namit Jain commented on HIVE-1642:
--
1. All the new parameters in HiveConf.java need to be a
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931705#action_12931705
]
Liyin Tang commented on HIVE-1642:
--
In the case: A left outer join B right outer join C, A
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931676#action_12931676
]
Ted Yu commented on HIVE-1642:
--
Do the new test cases cover backup task ?
If so, do we know the
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931674#action_12931674
]
Ted Yu commented on HIVE-1642:
--
For this example:
d. Eg. A left outer join B right outer join C
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931620#action_12931620
]
Namit Jain commented on HIVE-1642:
--
great work Liyin - I will take a look
> Convert join q
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HIVE-1642:
-
Attachment: hive_1642_1.patch
> Convert join queries to map-join based on size of table/row
>
[
https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931599#action_12931599
]
Liyin Tang commented on HIVE-1642:
--
I just finished converting common join into map join ba
37 matches
Mail list logo
