Today I'm getting this error again. A Google search brought me back to... you guessed it... my own post. But this time no HDFS corruption. Bounced all services, namenode, jobtracker, datanodes, tasktrackers. Still same error. Here's what it looks like:
*Fsck Output: *FSCK ended at Mon Jun 27 17:02:07 PDT 2011 in 818 milliseconds The filesystem under path '/' is HEALTHY *Hive Query Output: *-bash-3.2$ hive -e "select * from air_client_logs where loglevel = '[FATAL]'" Hive history file=/tmp/hdfs/hive_job_log_hdfs_201106271702_317571839.txt Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201106271658_0002, Tracking URL = http://hadooptest1:50030/jobdetails.jsp?jobid=job_201106271658_0002 Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=hadooptest1:54311 -kill job_201106271658_0002 2011-06-27 17:02:39,874 Stage-1 map = 0%, reduce = 0% 2011-06-27 17:03:02,111 Stage-1 map = 100%, reduce = 100% Ended Job = job_201106271658_0002 with errors java.lang.RuntimeException: Error while reading from task log url at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130) at org.apache.hadoop.hive.ql.exec.ExecDriver.showJobFailDebugInfo(ExecDriver.java:889) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:680) at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:425) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:186) Caused by: java.io.IOException: Server returned HTTP response code: 400 for URL: http://hadooptest14:50060/tasklog?taskid=attempt_201106271658_0002_m_000008_0&all=true at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436) at java.net.URL.openStream(URL.java:1010) at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120) ... 16 more Ended Job = job_201106271658_0002 with exception 'java.lang.RuntimeException(Error while reading from task log url)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask Again, trying to go to " http://hadooptest14:50060/tasklog?taskid=attempt_201106271658_0002_m_000008_0&all=true"; returns that argument attemptid error. What am I doing wrong here? I appear to keep doing it, whatever it is. On Fri, May 6, 2011 at 6:47 PM, Time Less <timelessn...@gmail.com> wrote: > My cluster went corrupt-mode. I wiped it and deleted the Hive metastore and > started over. In the process, I did a "yum upgrade" which probably took me > from CDH3b4 to CDH3u0. Now everytime I submit a Hive query of complexity > requiring a map/reduce job*, I get this error: > > 2011-05-06 18:39:14,533 Stage-1 map = 100%, reduce = 100% > Ended Job = job_201104081532_0509 with errors > java.lang.RuntimeException: Error while reading from task log url > at > org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130) > at > org.apache.hadoop.hive.ql.exec.ExecDriver.showJobFailDebugInfo(ExecDriver.java:889) > at > org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:680) > at > org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123) > .....[snip]..... > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:186) > Caused by: java.io.IOException: Server returned HTTP response code: 400 for > URL: > http://hadooptest3:50060/tasklog?taskid=attempt_201104081532_0509_m_000002_2&all=true > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436) > at java.net.URL.openStream(URL.java:1010) > at > org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120) > ... 16 more > Ended Job = job_201104081532_0509 with exception > 'java.lang.RuntimeException(Error while reading from task log url)' > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.MapRedTask > > It seems to me the key point in here is this: > > Server returned HTTP response code: 400 for URL: > http://hadooptest3:50060/tasklog?taskid=attempt_201104081532_0509_m_000002_2&all=true > > So I submitted that URL to my web browser, which said: > > Problem accessing /tasklog. Reason: Argument attemptid is required > > Does anyone have clues what this means? > > -- > Tim Ellis > Riot Games > * That is to say, simple queries like "select * from table limit 10" return > results just fine. > > -- Tim
