Hi all,

   We have happily been using hive on our CDH install for some time without 
issues.  We recently upgraded from CDH4.2.0 to CDH 4.4.0 and now select queries 
against one table fails with the syslog for the job showing what's below.  
Couple of things to note:

 *   This is a partitioned table
 *   An example failure query:  select count(*) from <table> where <partition 
conditions>
 *   other tables do not appear to be effected however many/most of those have 
small files.  This table's directories have only 1 file which is around 60 GB 
in size
 *   Queries using OOTB hive 0.9.0 pointing at the centralized meta store work 
without issue
 *   It looks like an infinite loop issue where this query usually completes in 
~2 min but we let it run for 45 with the job tracker showing 0% progress on 
mapping (perhaps related to https://issues.apache.org/jira/browse/HIVE-5235)

Have others seen this behavior?

Thanks!!!

2013-11-07 13:58:02,102 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 
Processing alias organic_events for file 
hdfs://nameservice1/events/organic/2013/11/05
2013-11-07 13:58:02,102 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 
forwarding 1 rows
2013-11-07 13:58:02,102 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 
0 forwarding 1 rows
2013-11-07 13:58:02,102 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 
forwarding 1 rows
2013-11-07 13:58:02,103 INFO ExecMapper: ExecMapper: processing 1 rows: used 
memory = 178407888
2013-11-07 13:58:02,104 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 
forwarding 10 rows
2013-11-07 13:58:02,104 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 
0 forwarding 10 rows
2013-11-07 13:58:02,104 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 
forwarding 10 rows
2013-11-07 13:58:02,104 INFO ExecMapper: ExecMapper: processing 10 rows: used 
memory = 178407888
….

Where processing row counts and used memory keeps increasing

Reply via email to