In our environment, our data are put on Amazon S3, and data are in RCFile
format. In order to make Hive queries work, we found that we have to change
the hive.optimize.cp to false. Otherwise, some queries will fail.
Now, when we try some complicated queries with multiple subqueries and
joins, we see queries failed again. But if we run the same query with data
on HDFS, everything is OK.
Diagnostic Messages for this Task:
java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing writable 1f 24 30 30 30 33 43 38 38 38 2d 32 42 39 45 2d 31 31
45 33 2d 38 32 44 35 2d 42 36 41 41 31 38 34 34 36 32 33 43 44 85 a4 29 0e
43 6f 6e 71 75 65 73 74 53 6d 61 6c 6c 30 00 18 09 58 50 34 5f 51 75 61 6b
65
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing writable 1f 24 30 30 30 33 43 38 38 38 2d 32 42 39
45 2d 31 31 45 33 2d 38 32 44 35 2d 42 36 41 41 31 38 34 34 36 32 33 43 44
85 a4 29 0e 43 6f 6e 71 75 65 73 74 53 6d 61 6c 6c 30 00 18 09 58 50 34 5f
51 75 61 6b 65
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:539)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
... 8 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:516)
... 9 more
Any insight on this?
Thanks,
Shanzhong
