Hi, Downloaded and installed Hadoop 0.20.2-cdh3u2 with Hive. Hive is running from outside the cluster. Am able to create a table and load the data and run: select * from DA where priority='ERROR'; with no issues.
But whenever the following command is run: select count(*) from DA; // this requires a reduce phase the following stack trace is thrown. Please can anybody point me to what I am doing wrong? 2012-01-23 19:06:05,350 WARN org.apache.hadoop.conf.Configuration: /mapred/local/taskTracker/root/jobcache/job_201201231801_0003/job.xml:a attempt to override final parameter: mapred.job.reuse.jvm.num.tasks; Ignoring. 2012-01-23 19:06:05,355 WARN org.apache.hadoop.conf.Configuration: /mapred/local/taskTracker/root/jobcache/job_201201231801_0003/job.xml:a attempt to override final parameter: mapred.child.java.opts; Ignoring. 2012-01-23 19:06:05,391 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library 2012-01-23 19:06:05,498 INFO org.apache.hadoop.mapred.TaskRunner: Creating symlink: /mapred/local/taskTracker/distcache/8198168166430384430_-422349744_222251847/Client1.ssodev-bo.mtv1.vrsn.com/tmp/hive-root/hive_2012-01-23_19-05-46_106_4954344205158962167/-mr-10003/ffdf957a-4ba7-4c4e-a81d-f39832b198e2 <- /mapred/local/taskTracker/root/jobcache/job_201201231801_0003/attempt_201201231801_0003_r_000000_3/work/HIVE_PLANffdf957a-4ba7-4c4e-a81d-f39832b198e2 2012-01-23 19:06:05,513 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /mapred/local/taskTracker/root/jobcache/job_201201231801_0003/jars/.job.jar.crc <- /mapred/local/taskTracker/root/jobcache/job_201201231801_0003/attempt_201201231801_0003_r_000000_3/work/.job.jar.crc 2012-01-23 19:06:05,518 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /mapred/local/taskTracker/root/jobcache/job_201201231801_0003/jars/job.jar <- /mapred/local/taskTracker/root/jobcache/job_201201231801_0003/attempt_201201231801_0003_r_000000_3/work/job.jar 2012-01-23 19:06:05,584 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=SHUFFLE, sessionId= 2012-01-23 19:06:05,639 WARN org.apache.hadoop.conf.Configuration: /mapred/local/taskTracker/root/jobcache/job_201201231801_0003/job.xml:a attempt to override final parameter: mapred.job.reuse.jvm.num.tasks; Ignoring. 2012-01-23 19:06:05,642 WARN org.apache.hadoop.conf.Configuration: /mapred/local/taskTracker/root/jobcache/job_201201231801_0003/job.xml:a attempt to override final parameter: mapred.child.java.opts; Ignoring. 2012-01-23 19:06:05,690 INFO org.apache.hadoop.mapred.ReduceTask: ShuffleRamManager: MemoryLimit=499810304, MaxSingleShuffleLimit=124952576 2012-01-23 19:06:05,699 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201201231801_0003_r_000000_3 Thread started: Thread for merging on-disk files 2012-01-23 19:06:05,699 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201201231801_0003_r_000000_3 Thread waiting: Thread for merging on-disk files 2012-01-23 19:06:05,700 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201201231801_0003_r_000000_3 Need another 1 map output(s) where 0 is already in progress 2012-01-23 19:06:05,700 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201201231801_0003_r_000000_3 Thread started: Thread for merging in memory files 2012-01-23 19:06:05,701 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201201231801_0003_r_000000_3 Scheduled 0 outputs (0 slow hosts and0 dup hosts) 2012-01-23 19:06:05,701 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201201231801_0003_r_000000_3 Thread started: Thread for polling Map Completion Events 2012-01-23 19:06:05,706 FATAL org.apache.hadoop.mapred.Task: attempt_201201231801_0003_r_000000_3 GetMapEventsThread Ignoring exception : java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2806) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2733) 2012-01-23 19:06:05,729 FATAL org.apache.hadoop.mapred.Task: attempt_201201231801_0003_r_000000_3 GetMapEventsThread Ignoring exception : org.apache.hadoop.ipc.RemoteException: java.io.IOException: JvmValidate Failed. Ignoring request from task: attempt_201201231801_0003_r_000000_3, with JvmId: jvm_201201231801_0003_r_-1803805258 at org.apache.hadoop.mapred.TaskTracker.validateJVM(TaskTracker.java:3119) at org.apache.hadoop.mapred.TaskTracker.getMapCompletionEvents(TaskTracker.java:3375) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428) at org.apache.hadoop.ipc.Client.call(Client.java:1107) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226) at $Proxy0.getMapCompletionEvents(Unknown Source) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2769) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2733) 2012-01-23 19:06:05,732 FATAL org.apache.hadoop.mapred.Task: Failed to contact the tasktracker org.apache.hadoop.ipc.RemoteException: java.io.IOException: JvmValidate Failed. Ignoring request from task: attempt_201201231801_0003_r_000000_3, with JvmId: jvm_201201231801_0003_r_-1803805258 at org.apache.hadoop.mapred.TaskTracker.validateJVM(TaskTracker.java:3119) at org.apache.hadoop.mapred.TaskTracker.fatalError(TaskTracker.java:3358) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428) at org.apache.hadoop.ipc.Client.call(Client.java:1107) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226) at $Proxy0.fatalError(Unknown Source) at org.apache.hadoop.mapred.Task.reportFatalError(Task.java:286) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2752)
