[
https://issues.apache.org/jira/browse/HIVE-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YoungYik updated HIVE-2072:
---------------------------
Component/s: (was: CLI)
Description: (was: I create two views from one table of different
partitions, and join these two views in my query, it just use 1 reducer and the
tasks stay in 82% for a long time, and then failed.
So, I use set hive.exec.reducers.max=28 before the execution of query, when
select on joined tables, it works, but still keep saying "Number of reduce
tasks determined at compile time: 1" when on the two joined views:
1)hive -e "create view view_1(uname, login) as select uname,'this30d' from
userlist where domain='$domain' and year=$YYYY and month=$MM and day=$DD and
type='all' and period_days=30;"
2)hive -e "create view view_0(uname, login) as select uname,'last30d' from
userlist where domain='$domain' and year=$YYYY and month=$MM and day=$DD and
type='all' and period_days=30;"
3)hive -e "set mapred.reduce.tasks=28; set; select v0.login,v1.login,count(*)
from view_0 v0 full outer join view_1 v1 group by v0.login,v1.login; set;"
then the output:
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
and at last, it just still keep using 1 reducer in this query.
but if I use "set;" to print the environment, it tells me:
mapred.reduce.tasks=28)
Priority: Trivial (was: Major)
Environment: (was: Linux 2.6.18-128.el5 x86_64 GNU/Linux, JRE
1.6.0_14)
Affects Version/s: (was: 0.6.0)
Issue Type: Test (was: Bug)
Summary: test (was: set hive.exec.reducers.max=<number> is not
working to view)
> test
> ----
>
> Key: HIVE-2072
> URL: https://issues.apache.org/jira/browse/HIVE-2072
> Project: Hive
> Issue Type: Test
> Reporter: YoungYik
> Priority: Trivial
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
