Ryu Kobayashi created HIVE-11353:
------------------------------------
Summary: Map env does not reflect in the Local Map Join
Key: HIVE-11353
URL: https://issues.apache.org/jira/browse/HIVE-11353
Project: Hive
Issue Type: Bug
Reporter: Ryu Kobayashi
mapreduce.map.env is not reflected when the Local Map Join is ran. Following a
sample query:
{code}
hive> set mapreduce.map.env=AAA=111,BBB=222,CCC=333;
hive> select
> reflect("java.lang.System", "getenv", "CCC") as CCC,
> a.AAA,
> b.BBB
> from (
> SELECT
> reflect("java.lang.System", "getenv", "AAA") as AAA
> from
> foo
> ) a
> join (
> select
> reflect("java.lang.System", "getenv", "BBB") as BBB
> from
> foo
> ) b
> limit 1;
Warning: Map Join MAPJOIN[10][bigTable=?] in task 'Stage-3:MAPRED' is a cross
product
Query ID = root_20150716013643_a8ca1539-68ae-4f13-b9fa-7a8b88f01f13
Total jobs = 1
15/07/16 01:36:46 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Execution log at:
/tmp/root/root_20150716013643_a8ca1539-68ae-4f13-b9fa-7a8b88f01f13.log
2015-07-16 01:36:47 Starting to launch local task to process map join;
maximum memory = 477102080
2015-07-16 01:36:48 Dump the side-table for tag: 0 with group count: 1 into
file:
file:/tmp/root/9b900f85-d5e4-4632-90bc-19f4bac516ff/hive_2015-07-16_01-36-43_217_8812243019719259041-1/-local-10003/HashTable-Stage-3/MapJoin-mapfile00--.hashtable
2015-07-16 01:36:48 Uploaded 1 File to:
file:/tmp/root/9b900f85-d5e4-4632-90bc-19f4bac516ff/hive_2015-07-16_01-36-43_217_8812243019719259041-1/-local-10003/HashTable-Stage-3/MapJoin-mapfile00--.hashtable
(282 bytes)
2015-07-16 01:36:48 End of local task; Time Taken: 0.934 sec.
Execution completed successfully
MapredLocal task succeeded
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1436962851556_0015, Tracking URL =
http://hadoop27:8088/proxy/application_1436962851556_0015/
Kill Command = /usr/local/hadoop/bin/hadoop job -kill job_1436962851556_0015
Hadoop job information for Stage-3: number of mappers: 1; number of reducers: 0
2015-07-16 01:36:56,488 Stage-3 map = 0%, reduce = 0%
2015-07-16 01:37:01,656 Stage-3 map = 100%, reduce = 0%, Cumulative CPU 1.28
sec
MapReduce Total cumulative CPU time: 1 seconds 280 msec
Ended Job = job_1436962851556_0015
MapReduce Jobs Launched:
Stage-Stage-3: Map: 1 Cumulative CPU: 1.28 sec HDFS Read: 5428 HDFS Write:
13 SUCCESS
Total MapReduce CPU Time Spent: 1 seconds 280 msec
OK
333 null 222
Time taken: 19.562 seconds, Fetched: 1 row(s)
{code}
The attached patch will include those taken from Hadoop's code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
