Somehow (by resetting folder permissions) I got rid of the below error.
But now I'm getting a new error as below. Looks like I'm missing some
configuration, but not sure what and where.
/hive> select count(1) from table1;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:121)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:83)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:76)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:487)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:466)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Job Submission failed with exception 'java.io.IOException(Cannot
initialize Cluster. Please check your configuration for
mapreduce.framework.name and the correspond server addresses.)'
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MapRedTask/
On Tuesday 25 September 2012 05:39 PM, kulkarni.swar...@gmail.com wrote:
The jar is being looked on HDFS as the exception suggests. Run the
following commands:
$ hadoop fs -mkdir //usr/lib/hive/lib/
/$ hadoop fs -put $HIVE_HOME/lib/hive-builtins-0.8.1-cdh4.0.1.jar
///usr/lib/hive/lib/
Your queries should work now.
On Sep 25, 2012, at 6:46 AM, Manish.Bhoge <manish.bh...@target.com
<mailto:manish.bh...@target.com>> wrote:
Sarath,
Is this the external table where you have ran the query? How did you
loaded the table? Because it looks like the error is about the file
related to table than CDH Jar.
Thank You,
Manish
*From:*Sarath [mailto:sarathchandra.jos...@algofusiontech.com]
*Sent:* Tuesday, September 25, 2012 3:48 PM
*To:* user@hive.apache.org <mailto:user@hive.apache.org>
*Subject:* Hive query failing
Hi,
When I run the query "select count(1) from table1;" it fails with the
exception message as below.
/java.io.FileNotFoundException: File does not exist:
/usr/lib/hive/lib/hive-builtins-0.8.1-cdh4.0.1.jar
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:736)
at
org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:208)
at
org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:71)
at
org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:245)
at
org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:283)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:354)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:609)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:604)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:604)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:452)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:710)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Job Submission failed with exception
'java.io.FileNotFoundException(File does not exist:
/usr/lib/hive/lib/hive-builtins-0.8.1-cdh4.0.1.jar)'
Execution failed with exit status: 2/
But the JAR mentioned in the message exists and appropriate R/W
permissions have been set on the folder /usr/lib/hive for the user.
What is going wrong?
Regards,
Sarath.
