[
https://issues.apache.org/jira/browse/HIVE-9507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Moustafa Aboul Atta updated HIVE-9507:
--------------------------------------
Description:
I have tweets stored with avro on hdfs with the default twitter status (tweet)
schema.
There's an object called "entities" that contains arrays of structs.
When I run
{{SELECT mytable.*}}
{{FROM tweets}}
{{LATERAL VIEW INLINE(entities.media) mytable}}
I get the exception attached as partial_log.log, however, if I add
{{WHERE entities.media IS NOT NULL}}
it runs perfectly.
was:
I have tweets stored with avro on hdfs with the default twitter status (tweet)
schema.
There's an object called "entities" that contains arrays of structs.
When I run
{{SELECT mytable.*}}
{{FROM tweets}}
{{LATERAL VIEW INLINE(entities.media) mytable}}
I get the exception found hereunder, however if I add
{{WHERE entities.media IS NOT NULL}}
it runs perfectly.
Here's the partial log:
2015-01-29 10:15:00,879 INFO SessionState (SessionState.java:printInfo(824)) -
Status: Running (Executing on YARN cluster with App id
application_1422267635031_0618)
2015-01-29 10:15:00,879 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: -/-
2015-01-29 10:15:02,526 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0/13
2015-01-29 10:15:05,551 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0/13
2015-01-29 10:15:08,722 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0/13
2015-01-29 10:15:12,095 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0/13
2015-01-29 10:15:12,354 INFO log.PerfLogger
(PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=TezRunVertex.Map 1
from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor>
2015-01-29 10:15:12,354 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+5)/13
2015-01-29 10:15:12,557 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6)/13
2015-01-29 10:15:15,691 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6)/13
2015-01-29 10:15:18,892 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-1)/13
2015-01-29 10:15:19,094 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-3)/13
2015-01-29 10:15:19,304 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-5)/13
2015-01-29 10:15:19,507 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-6)/13
2015-01-29 10:15:22,641 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-6)/13
2015-01-29 10:15:24,704 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-8)/13
2015-01-29 10:15:27,735 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-8)/13
2015-01-29 10:15:30,957 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-8)/13
2015-01-29 10:15:34,095 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-8)/13
2015-01-29 10:15:35,138 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-9)/13
2015-01-29 10:15:36,503 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-10)/13
2015-01-29 10:15:36,710 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-11)/13
2015-01-29 10:15:37,971 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-12)/13
2015-01-29 10:15:39,800 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-13)/13
2015-01-29 10:15:41,175 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-14)/13
2015-01-29 10:15:44,414 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-14)/13
2015-01-29 10:15:45,447 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-15)/13
2015-01-29 10:15:47,413 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-16)/13
2015-01-29 10:15:47,618 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-17)/13
2015-01-29 10:15:49,568 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+6,-18)/13
2015-01-29 10:15:51,099 INFO SessionState (SessionState.java:printInfo(824)) -
Map 1: 0(+0,-19)/13
2015-01-29 10:15:51,331 ERROR SessionState (SessionState.java:printError(833))
- Status: Failed
2015-01-29 10:15:51,417 ERROR SessionState (SessionState.java:printError(833))
- Vertex failed, vertexName=Map 1, vertexId=vertex_1422267635031_0618_1_00,
diagnostics=[Task failed, taskId=task_1422267635031_0618_1_00_000000,
diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running
task:java.lang.RuntimeException: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:186)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:138)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:294)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:163)
... 13 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:503)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83)
... 16 more
Caused by: java.lang.NullPointerException
at java.util.ArrayList.<init>(ArrayList.java:164)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDTFInline.process(GenericUDTFInline.java:60)
at
org.apache.hadoop.hive.ql.exec.UDTFOperator.processOp(UDTFOperator.java:108)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator.processOp(LateralViewForwardOperator.java:37)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:493)
... 17 more
], TaskAttempt 1 failed, info=[Error: Failure while running
task:java.lang.RuntimeException: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:186)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:138)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:294)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:163)
... 13 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:503)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83)
... 16 more
Caused by: java.lang.NullPointerException
at java.util.ArrayList.<init>(ArrayList.java:164)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDTFInline.process(GenericUDTFInline.java:60)
at
org.apache.hadoop.hive.ql.exec.UDTFOperator.processOp(UDTFOperator.java:108)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator.processOp(LateralViewForwardOperator.java:37)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:493)
... 17 more
], TaskAttempt 2 failed, info=[Error: Failure while running
task:java.lang.RuntimeException: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:186)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:138)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:294)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:163)
... 13 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:503)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83)
... 16 more
Caused by: java.lang.NullPointerException
at java.util.ArrayList.<init>(ArrayList.java:164)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDTFInline.process(GenericUDTFInline.java:60)
at
org.apache.hadoop.hive.ql.exec.UDTFOperator.processOp(UDTFOperator.java:108)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator.processOp(LateralViewForwardOperator.java:37)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:493)
... 17 more
], TaskAttempt 3 failed, info=[Error: Failure while running
task:java.lang.RuntimeException: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:186)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:138)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:294)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:163)
... 13 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row
{"metadata":{"result_type":"recent","iso_language_code":"it"},"query_id":4013,"data_source_type":1,"search_date":1422300806,"created_at":"Mon
Jan 26 04:31:11 +0000
2015","id":559569212595638274,"id_str":"559569212595638274","text":"@RTA_Dubai
Dubai Metro Escalators Down #Fahidi #BurJuman #Walkathon
#morningblues","source":"<a href=\"http://twitter.com/download/android\";
rel=\"nofollow\">Twitter for
Android</a>","truncated":false,"in_reply_to_status_id_str":null,"in_reply_to_user_id_str":"50548818","in_reply_to_screen_name":"RTA_Dubai","user":{"id":38799882,"id_str":"38799882","name":"Dennis
Menezes","screen_name":"mistaaah","location":"Abu Dhabi","description":"Tech
Lover !!! Rethinking Life Code !!! And Really Big-Picture Thoughts
!!!\r\n\r\nhttp://t.co/LwwU9C0Ce5","url":"http://t.co/TmZwu1hU9Q","entities":{"hashtags":null,"urls":null,"user_mentions":null,"media":null},"_protected":false,"followers_count":109,"friends_count":539,"listed_count":3,"created_at":"Sat
May 09 04:44:51 +0000
2009","favourites_count":76,"time_zone":"Mumbai","geo_enabled":true,"verified":false,"statuses_count":799,"lang":"en","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"000000","profile_background_image_url":"http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai,
United Arab Emirates","country_code":"AE","country":"???????? ???????
???????","bounding_box":{"type":"Polygon","coordinates":[[[54.893974,24.618252],[56.208374,24.618252],[56.208374,25.368671],[54.893974,25.368671]]]}},"contributors":null,"retweeted_status":null,"retweet_count":0,"favorite_count":0,"entities":{"hashtags":[{"text":"Fahidi","indices":[39,46]},{"text":"BurJuman","indices":[47,56]},{"text":"Walkathon","indices":[57,67]},{"text":"morningblues","indices":[68,81]}],"urls":[],"user_mentions":[{"screen_name":"RTA_Dubai","name":"RTA","id":50548818,"id_str":"50548818","indices":[0,10]}],"media":null},"favorited":false,"retweeted":false,"lang":"it","possibly_sensitive":false,"sentiment":{"score":2,"confidence":0.0}}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:503)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83)
... 16 more
Caused by: java.lang.NullPointerException
at java.util.ArrayList.<init>(ArrayList.java:164)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDTFInline.process(GenericUDTFInline.java:60)
at
org.apache.hadoop.hive.ql.exec.UDTFOperator.processOp(UDTFOperator.java:108)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator.processOp(LateralViewForwardOperator.java:37)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at
org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:493)
... 17 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex
vertex_1422267635031_0618_1_00 [Map 1] killed/failed due to:null]
2015-01-29 10:15:51,418 ERROR SessionState (SessionState.java:printError(833))
- DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2015-01-29 10:15:51,418 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(135))
- </PERFLOG method=TezRunDag start=1422519300676 end=1422519351418
duration=50742 from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor>
2015-01-29 10:15:51,430 INFO hooks.ATSHook (ATSHook.java:<init>(87)) - Created
ATS Hook
2015-01-29 10:15:51,430 INFO log.PerfLogger
(PerfLogger.java:PerfLogBegin(108)) - <PERFLOG
method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook
from=org.apache.hadoop.hive.ql.Driver>
2015-01-29 10:15:51,430 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(135))
- </PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook
start=1422519351430 end=1422519351430 duration=0
from=org.apache.hadoop.hive.ql.Driver>
2015-01-29 10:15:51,430 INFO hooks.ATSHook
(ATSHook.java:createPostHookEvent(170)) - Received post-hook notification for
:hadoop_20150129101212_dbf3592e-4f4e-4e97-88da-f16492fcce2e
2015-01-29 10:15:51,431 ERROR ql.Driver (SessionState.java:printError(833)) -
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.tez.TezTask
2015-01-29 10:15:51,431 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(135))
- </PERFLOG method=Driver.execute start=1422519144456 end=1422519351431
duration=206975 from=org.apache.hadoop.hive.ql.Driver>
2015-01-29 10:15:51,431 INFO log.PerfLogger
(PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks
from=org.apache.hadoop.hive.ql.Driver>
2015-01-29 10:15:51,431 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(135))
- </PERFLOG method=releaseLocks start=1422519351431 end=1422519351431
duration=0 from=org.apache.hadoop.hive.ql.Driver>
2015-01-29 10:15:51,431 INFO exec.ListSinkOperator (Operator.java:close(595))
- 8 finished. closing...
2015-01-29 10:15:51,431 INFO exec.ListSinkOperator (Operator.java:close(613))
- 8 Close done
2015-01-29 10:15:51,434 INFO log.PerfLogger
(PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks
from=org.apache.hadoop.hive.ql.Driver>
2015-01-29 10:15:51,434 INFO log.PerfLogger (PerfLogger.java:PerfLogEnd(135))
- </PERFLOG method=releaseLocks start=1422519351434 end=1422519351434
duration=0 from=org.apache.hadoop.hive.ql.Driver>
> Make "LATERAL VIEW inline(expression) mytable" tolerant to nulls
> ----------------------------------------------------------------
>
> Key: HIVE-9507
> URL: https://issues.apache.org/jira/browse/HIVE-9507
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, UDF
> Affects Versions: 0.14.0
> Environment: hdp 2.2
> Windows server 2012 R2 64-bit
> Reporter: Moustafa Aboul Atta
> Assignee: Navis
> Priority: Minor
> Attachments: HIVE-9507.1.patch.txt, parial_log.log
>
>
> I have tweets stored with avro on hdfs with the default twitter status
> (tweet) schema.
> There's an object called "entities" that contains arrays of structs.
> When I run
>
> {{SELECT mytable.*}}
> {{FROM tweets}}
> {{LATERAL VIEW INLINE(entities.media) mytable}}
> I get the exception attached as partial_log.log, however, if I add
> {{WHERE entities.media IS NOT NULL}}
> it runs perfectly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
![http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai](http://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/680295618/3f4db122fd8f3cfe6be2c1003aa30d41.jpeg","profile_background_tile":false,"profile_image_url":"http://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_image_url_https":"https://pbs.twimg.com/profile_images/2175846055/165_17033110245_575825245_1999208_4869_n_normal.jpg","profile_banner_url":null,"profile_link_color":"485C3A","profile_sidebar_border_color":"204207","profile_sidebar_fill_color":"060A00","profile_text_color":"618238","profile_use_background_image":true,"default_profile":false,"default_profile_image":false},"geo":{"type":"Point","coordinates":[25.248241,55.3076]},"coordinates":{"type":"Point","coordinates":[55.3076,25.248241]},"place":{"id":"001907e868d06e24","url":"https://api.twitter.com/1.1/geo/id/001907e868d06e24.json","place_type":"admin","name":"Dubai","full_name":"Dubai){kind=link}