Hello All,
I am running the following hive query in a 10 node cluster on very huge
dataset (6.6 billion records)
create table tst as
select a,b,c,d,......w,
sum(case when x= 'C' then 1 else 0 end) as CS,
sum(case when y = 'I' then 1 else 0 end) as IP,
sum(case when z= 'A' then 1 else 0 end) as AT
from tst_v
group by a,b,c...w ;
where tst_v is a view.
Total no. of mappers: 4565, reducers: 999
CPU time spent(ms): 494,725,960
The mapper function runs successfully but the the reducer fails.Also the
mapper takes painfully long time.
Here is the error from the log:
java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error: Unable to deseralize reduce input key from
x1x128x0x0x0x0x1x254x174x1x49x55x52x46x50x53x52x46x49x46x48x0x1x142x145x93x11x1x128x87x4x73x1x128x32x107x137x1x130x165x214x131x1x49x0x1x51x48x48x120x53x48x0x1x132x11x106x192x1x128x13x178x250x1x128x0x1x0x1x78x86x0x1x55x48x50x0x1x56x57x48x53x52x0x1x50x48x54x0x1x49x51x51x55x51x0x1x48x0x1x48x46x48x0x1x48x0x1x49x55x53x55x52x54x56x55x0x1x48x0x1x0x1x0x1x0x1x0x255
...
Any help on this is appreciated.
Thanks
vr
