It's difficult to say what's causing the freeze. Was the node rejecting client connections during this time ? Did any of the other nodes log that the node that was freezing was down ? Is there anything else running on the box?
Aaron On 14 Mar 2011, at 07:02, ruslan usifov wrote: > Hello > > I some times see follow behaviour: java on node hung, with follow cpu usage: > > <strange.jpg> > > > With hole in logs where it hung: > > > Here is part of my gc.log (between 19:07 and 19:41 is hole) > > 2011-03-13T19:07:16.339+0300: 64970.498: [GC 64970.498: [ParNew: > 1249324K->96851K(1386112K), 0.0305170 secs] 2867161K->1714688K(600844 > 8K), 0.0307610 secs] [Times: user=0.39 sys=0.00, real=0.03 secs] > 2011-03-13T19:07:20.460+0300: 64974.619: [GC 64974.620: [ParNew: > 1328979K->12323K(1386112K), 0.0453270 secs] 2946816K->1722867K(600844 > 8K), 0.0455710 secs] [Times: user=0.53 sys=0.00, real=0.05 secs] > 2011-03-13T19:07:26.629+0300: 64980.788: [GC 64980.788: [ParNew: > 1244451K->19922K(1386112K), 0.0213010 secs] 2954995K->1730467K(600844 > 8K), 0.0215640 secs] [Times: user=0.26 sys=0.00, real=0.02 secs] > 2011-03-13T19:07:41.081+0300: 64995.241: [GC 64995.241: [ParNew: > 1252050K->29988K(1386112K), 0.0320390 secs] 2962595K->1747546K(600844 > 8K), 0.0323010 secs] [Times: user=0.38 sys=0.01, real=0.03 secs] > 2011-03-13T19:07:56.376+0300: 65010.536: [GC 65010.536: [ParNew: > 1262116K->34260K(1386112K), 0.0401930 secs] 2979674K->1758982K(600844 > 8K), 0.0404350 secs] [Times: user=0.40 sys=0.02, real=0.04 secs] > 2011-03-13T19:41:16.921+0300: 67011.081: [GC 67011.081: [ParNew: > 1266388K->38705K(1386112K), 0.0480950 secs] 2991110K->1775874K(600844 > 8K), 0.0483710 secs] [Times: user=0.37 sys=0.00, real=0.05 secs] > 2011-03-13T19:41:32.116+0300: 67026.276: [GC 67026.276: [ParNew: > 1270833K->44743K(1386112K), 0.0541030 secs] 3008002K->1792109K(600844 > 8K), 0.0543190 secs] [Times: user=0.44 sys=0.02, real=0.05 secs] > 2011-03-13T19:41:39.694+0300: 67033.854: [GC 67033.854: [ParNew: > 1276871K->48249K(1386112K), 0.0496830 secs] 3024237K->1810949K(600844 > 8K), 0.0499810 secs] [Times: user=0.47 sys=0.01, real=0.06 secs] > 2011-03-13T19:41:45.433+0300: 67039.592: [GC 67039.592: [ParNew: > 1280377K->53961K(1386112K), 0.0518250 secs] 3043077K->1829344K(600844 > 8K), 0.0520640 secs] [Times: user=0.61 sys=0.00, real=0.06 secs] > > > > Here is part of my system.log: > > andra/commitlog/CommitLog-1300028248189.log) > INFO [COMMIT-LOG-WRITER] 2011-03-13 19:07:50,067 CommitLog.java (line 437) > Discarding obsolete commit log:CommitLogSegment(/home/cass > andra/commitlog/CommitLog-1300029444680.log) > INFO [ScheduledTasks:1] 2011-03-13 19:41:18,007 Gossiper.java (line 224) > InetAddress /10.24.84.4 is now dead. > INFO [ScheduledTasks:1] 2011-03-13 19:41:18,007 Gossiper.java (line 224) > InetAddress /10.32.59.194 is now dead. > > > root@slv004:~# java -version > java version "1.6.0_24" > Java(TM) SE Runtime Environment (build 1.6.0_24-b07) > Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode) > > > root@slv004:~# uname -a > Linux slv004 2.6.32-24-generic #39-Ubuntu SMP Wed Jul 28 05:14:15 UTC 2010 > x86_64 GNU/Linux > > >
