Can you paste the logs here? Abhishek
On Sun, May 20, 2012 at 9:36 AM, Balaji Rao <sbalaji...@gmail.com> wrote: > There is only one task attempt that is lurking around and that never > completes. It probably times out or fails after a long time. > > -Balaji > > On Fri, May 18, 2012 at 11:50 PM, Ranjith <ranjith.raghuna...@gmail.com> > wrote: > > This could be a result of the data being skewed. When you look at the > job tracker page are you noticing just a few lurking around? > > > > Thanks, > > Ranjith > > > > On May 18, 2012, at 1:57 PM, Balaji Rao <sbalaji...@gmail.com> wrote: > > > >> I have a simple query running on HIVE using Amazon EMR where the > >> reduce operation seems to be stuck. The table is small with just > >> 300,000+ rows (data_analysis). > >> I intend to work on a 30,000,000 row table once I'm able to fix this. > >> The query (modified to hide some information) is as follows: > >> > >> INSERT OVERWRITE TABLE data_analysis > >> SELECT > >> a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed, > >> hover, hower1, hower5, hower60, > >> Visible, a.x1, a.x2, a.x3, > >> NoViewMeasure, a.x4, a.x5, a.x6, > >> a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27, > a.x28, > >> COUNT(a.auction_id_64), SUM(CASE WHEN b.auction_id_64 IS NULL THEN > >> 1 ELSE 0 END) > >> FROM > >> data_subset_2 a > >> LEFT OUTER JOIN > >> OrderFields b on a.auction_id_64=b.auction_id_64 > >> GROUP BY > >> a.clientid, a.isi, a.os, a.bro, a.engaged, a.iframed, > >> hover, hower1, hower5, hower60, > >> Visible, a.x1, a.x2, a.x3, > >> NoViewMeasure, a.x4, a.x5, a.x6, > >> a.alerttype, a.x7, a.x9, a.x11, a.x13, a.x17, a.x21, a.x27, > a.x28; > >> > >> > >> > >> > >> 2012-05-18 17:49:32,440 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 17:50:32,813 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 17:51:33,176 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 17:52:33,538 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 17:53:33,906 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 17:54:34,270 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 17:55:34,644 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 17:56:35,017 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 17:57:35,387 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 17:58:35,754 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 17:59:36,125 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:00:36,503 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:01:36,873 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:02:36,990 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:03:37,355 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:04:37,719 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:05:38,081 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:06:38,432 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:07:38,777 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:08:39,135 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:09:39,489 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:10:39,852 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:11:40,221 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:12:40,591 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:13:40,962 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:14:41,336 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:15:41,700 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:16:42,070 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:17:42,440 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:18:42,801 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:19:43,156 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:20:43,531 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:21:43,899 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:22:44,255 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:23:44,619 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:24:44,978 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:25:45,347 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:26:45,720 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:27:46,088 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:28:46,446 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:29:46,814 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:30:47,183 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:31:47,546 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:32:47,918 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:33:48,282 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:34:48,408 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:35:48,766 Stage-1 map = 100%, reduce = 97% > >> 2012-05-18 18:36:49,118 Stage-1 map = 100%, reduce = 97% > >> > >> Thanks, > >> Balaji >
