Hi,
A very strange thing is happening. I am running the TPC-H benchmark. I have
loaded the tables on HDFS running in pseudo-distributed mode. When i query
one table at a time
select * from customer LIMIT 2; OR
select * from NATION LIMIT 2; results are printed to the cli but as soon as
i try something as simple as
SELECT
c_custkey,
c_name,
c_phone
FROM
customer c
JOIN
nation n
ON (c.c_nationkey = n.n_nationkey)
LIMIT 10;
the following usual log is printed and then it stops on "*Number of reduce
tasks is set to 0 since there's no reduce operator*" as shown below and
then nothing happens. I have to control C twice to get out and start the
hive shell again.
Can someone please help, seems like a very small thing but i am stuck and
would appreciate a prompt response, thanks in advance. A description of
tables is given at the end.
14/07/11 17:25:02 INFO Configuration.deprecation: mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
14/07/11 17:25:02 INFO Configuration.deprecation: mapred.min.split.size is
deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
14/07/11 17:25:02 INFO Configuration.deprecation:
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
mapreduce.reduce.speculative
14/07/11 17:25:02 INFO Configuration.deprecation:
mapred.min.split.size.per.node is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.node
14/07/11 17:25:02 INFO Configuration.deprecation:
mapred.input.dir.recursive is deprecated. Instead, use
mapreduce.input.fileinputformat.input.dir.recursive
14/07/11 17:25:02 INFO Configuration.deprecation:
mapred.min.split.size.per.rack is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.rack
14/07/11 17:25:02 INFO Configuration.deprecation: mapred.max.split.size is
deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
14/07/11 17:25:02 INFO Configuration.deprecation:
mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use
mapreduce.job.committer.setup.cleanup.needed
14/07/11 17:25:03 WARN conf.HiveConf: DEPRECATED: Configuration property
hive.metastore.local no longer has any effect. Make sure to provide a valid
value for hive.metastore.uris if you are connecting to a remote metastore.
Logging initialized using configuration in
jar:file:/usr/local/hive/warehouse/hive-0.13.1/lib/hive-common-0.13.1.jar!/hive-log4j.properties
Putting the global hiverc in $HIVE_HOME/bin/.hiverc is deprecated. Please
use $HIVE_CONF_DIR/.hiverc instead.
Total jobs = 1
14/07/11 17:25:16 WARN conf.Configuration:
file:/tmp/hduser/hive_2014-07-11_17-25-10_520_502061053341218664-1/-local-10006/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
14/07/11 17:25:16 WARN conf.Configuration:
file:/tmp/hduser/hive_2014-07-11_17-25-10_520_502061053341218664-1/-local-10006/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
14/07/11 17:25:16 INFO Configuration.deprecation: mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
14/07/11 17:25:16 INFO Configuration.deprecation: mapred.min.split.size is
deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
14/07/11 17:25:16 INFO Configuration.deprecation:
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
mapreduce.reduce.speculative
14/07/11 17:25:16 INFO Configuration.deprecation:
mapred.min.split.size.per.node is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.node
14/07/11 17:25:16 INFO Configuration.deprecation:
mapred.input.dir.recursive is deprecated. Instead, use
mapreduce.input.fileinputformat.input.dir.recursive
14/07/11 17:25:16 INFO Configuration.deprecation:
mapred.min.split.size.per.rack is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.rack
14/07/11 17:25:16 INFO Configuration.deprecation: mapred.max.split.size is
deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
14/07/11 17:25:16 INFO Configuration.deprecation:
mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use
mapreduce.job.committer.setup.cleanup.needed
14/07/11 17:25:17 WARN conf.HiveConf: DEPRECATED: Configuration property
hive.metastore.local no longer has any effect. Make sure to provide a valid
value for hive.metastore.uris if you are connecting to a remote metastore.
Execution log at:
/tmp/hduser/hduser_20140711172525_68ca10e1-ae38-4fef-a417-ea98cc9edf6f.log
2014-07-11 05:25:17 Starting to launch local task to process map join; maximum
memory = 477364224
2014-07-11 05:25:18 Dump the side-table into file:
file:/tmp/hduser/hive_2014-07-11_17-25-10_520_502061053341218664-1/-local-10003/HashTable-Stage-3/MapJoin-mapfile01--.hashtable
2014-07-11 05:25:18 Uploaded 1 File to:
file:/tmp/hduser/hive_2014-07-11_17-25-10_520_502061053341218664-1/-local-10003/HashTable-Stage-3/MapJoin-mapfile01--.hashtable
(2882 bytes)
2014-07-11 05:25:18 End of local task; Time Taken: 1.168 sec.
Execution completed successfully
MapredLocal task succeeded
Launching Job 1 out of 1
*Number of reduce tasks is set to 0 since there's no reduce operator*
CUSTOMER TABLE
col_name data_type comment
c_custkey int
c_name string
c_address string
c_nationkey int
c_phone string
c_acctbal double
c_mktsegment string
c_comment string
NATION TABLE
col_name data_type comment
n_nationkey int
n_name string
n_regionkey int
n_comment string
both are external tables based on data in hdfs.
Regards,
Sarfraz Rasheed Ramay (DIT)
Dublin, Ireland.
