I encounter the same problem, does anyone have sovled the problem? On Tue, Apr 20, 2010 at 11:03 AM, Ingram Chen <ingramc...@gmail.com> wrote:
> I check system.log both, but there is no exception logged. > > On Tue, Apr 20, 2010 at 10:40, Jonathan Ellis <jbel...@gmail.com> wrote: > >> I don't see csArena-tmp-6-Index.db in the incoming files list. If >> it's not there, that means that it did break out of that while loop. >> >> Did you check both logs for exceptions? >> >> On Mon, Apr 19, 2010 at 9:36 PM, Ingram Chen <ingramc...@gmail.com> >> wrote: >> > Ouch ! I talk too early ! >> > >> > We still suffer same problems after upgrade to 1.6.0_20. >> > >> > In JMX StreamingService, I see several wired incoming/outgoing transfer: >> > >> > In Host A, 192.168.2.87 >> > >> > StreamingService Status: >> > Done with transfer to /192.168.2.88 >> > >> > StreamingService StreamSources: >> > [/192.168.2.88] >> > >> > StreamingService StreamDestinations: >> > [/192.168.2.88] >> > >> > StreamingService getIncomingFiles=192.168.2.88 >> > [ >> > UserState: /var/lib/cassandra/data/UserState/multiMine-tmp-11-Index.db >> > 0/5718, >> > UserState: /var/lib/cassandra/data/UserState/multiMine-tmp-11-Filter.db >> > 0/325, >> > UserState: /var/lib/cassandra/data/UserState/multiMine-tmp-11-Data.db >> > 0/29831, >> > UserState: /var/lib/cassandra/data/UserState/csArena-tmp-13-Index.db >> > 0/47623, >> > >> > ... omit several 0 received pending files..... >> > >> > UserState: /var/lib/cassandra/data/UserState/battleCity2-tmp-19-Data.db >> > 0/355041, >> > >> > UserState: /var/lib/cassandra/data/UserState/mahjong-tmp-12-Data.db >> > 27711/2173906, >> > UserState: /var/lib/cassandra/data/UserState/darkChess-tmp-12-Data.db >> > 27711/18821998, >> > UserState: /var/lib/cassandra/data/UserState/battleCity2-tmp-6-Data.db >> > 27711/743037, >> > UserState: /var/lib/cassandra/data/UserState/big2-tmp-12-Index.db >> > 27711/189214, >> > UserState: >> /var/lib/cassandra/data/UserState/facebookPoker99-tmp-6-Data.db >> > 27711/1892375, >> > UserState: >> /var/lib/cassandra/data/UserState/facebookPoker99-tmp-6-Index.db >> > 27711/143216, >> > UserState: /var/lib/cassandra/data/UserState/csArena-tmp-6-Data.db >> > 27711/201188, >> > UserState: /var/lib/cassandra/data/UserState/darkChess-tmp-12-Index.db >> > 27711/354923, >> > UserState: /var/lib/cassandra/data/UserState/big2-tmp-12-Data.db >> > 27711/1260768, >> > UserState: /var/lib/cassandra/data/UserState/mahjong-tmp-12-Index.db >> > 27711/332649, >> > UserState: /var/lib/cassandra/data/UserState/battleCity2-tmp-6-Index.db >> > 27711/39739 >> > ] >> > >> > lots of files stalled after receiving 27711 bytes. this strange number >> is >> > the length of first file to income, see Host B >> > >> > Host B, 192.168.2.88 >> > >> > StreamingService Status: >> > Receiving stream >> > >> > StreamingService StreamSources: >> > StreamSources: [/192.168.2.87] >> > >> > StreamingService StreamDestinations: >> > [/192.168.2.87] >> > >> > StreamingService getOutgoingFiles=192.168.2.87 >> > [ >> > /var/lib/cassandra/data/UserState/stream/csArena-6-Index.db 27711/27711, >> > /var/lib/cassandra/data/UserState/stream/csArena-6-Filter.db 0/1165, >> > /var/lib/cassandra/data/UserState/stream/csArena-6-Data.db 0/201188, >> > >> > ... omit pending outgoing files .... >> > ] >> > >> > It seems that outgoing files does not terminate properly. and cause the >> > receiver goes into infinite loop to cause busy thread. From thread dump, >> it >> > looks like fc.transferFrom() in IncomingStreamReader never return: >> > >> > while (bytesRead < pendingFile.getExpectedBytes()) { >> > bytesRead += fc.transferFrom(socketChannel, bytesRead, >> > FileStreamTask.CHUNK_SIZE); >> > pendingFile.update(bytesRead); >> > } >> > >> > >> > On Tue, Apr 20, 2010 at 05:48, Rob Coli <rc...@digg.com> wrote: >> >> >> >> On 4/17/10 6:47 PM, Ingram Chen wrote: >> >>> >> >>> after upgrading jdk from 1.6.0_16 to 1.6.0_20, the problem solved. >> >> >> >> FYI, this sounds like it might be : >> >> >> >> https://issues.apache.org/jira/browse/CASSANDRA-896 >> >> >> >> >> http://bugs.sun.com/view_bug.do;jsessionid=60c39aa55d3666c0c84dd70eb826?bug_id=6805775 >> >> >> >> Where garbage collection issues in JVM/JDKs before 7.b70 leads to GC >> >> storming which hoses performance. >> >> >> >> =Rob >> > >> > >> > >> > >> > >> > > > > -- > Ingram Chen > online share order: http://dinbendon.net > blog: http://www.javaworld.com.tw/roller/page/ingramchen > -- Best Regards Anty Rao