Hi Aaron, This was the first error. It occurred a couple of times after this. We did an hardware upgrade on the server and increased the max heap size. Now running fine. Seems that 1.1.1 uses a little more memory, or or data set just grew ;-)
Thank you for your time! 2012/7/3 aaron morton <aa...@thelastpickle.com> > Is this still an issue ? > > It looks like something shut down the messaging service. Was there > anything else in the logs ? > > Cheers > > ----------------- > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 28/06/2012, at 3:49 AM, Robin Verlangen wrote: > > Hi there, > > Today I found one node (running 1.1.1 in a 3 node cluster) being dead for > the third time this week, it died with the following message: > > ERROR [ReadRepairStage:3] 2012-06-27 14:28:30,929 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[ReadRepairStage:3,5,main] > java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has > shut down > at > org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:60) > at java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor.execute(Unknown Source) > at > org.apache.cassandra.net.MessagingService.receive(MessagingService.java:566) > at > org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:439) > at > org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:391) > at > org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:372) > at > org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:460) > at > org.apache.cassandra.service.RowRepairResolver.scheduleRepairs(RowRepairResolver.java:136) > at > org.apache.cassandra.service.RowRepairResolver.resolve(RowRepairResolver.java:94) > at > org.apache.cassandra.service.AsyncRepairCallback$1.runMayThrow(AsyncRepairCallback.java:54) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown > Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at java.lang.Thread.run(Unknown Source) > > Is this a common bug in 1.1.1, or did I get a "race" condition? > > -- > With kind regards, > > Robin Verlangen > *Software engineer* > * > * > W http://www.robinverlangen.nl > E ro...@us2.nl > > Disclaimer: The information contained in this message and attachments is > intended solely for the attention and use of the named addressee and may be > confidential. If you are not the intended recipient, you are reminded that > the information remains the property of the sender. You must not use, > disclose, distribute, copy, print or rely on this e-mail. If you have > received this message in error, please contact the sender immediately and > irrevocably delete this message and any copies. > > > -- With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro...@us2.nl Disclaimer: The information contained in this message and attachments is intended solely for the attention and use of the named addressee and may be confidential. If you are not the intended recipient, you are reminded that the information remains the property of the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If you have received this message in error, please contact the sender immediately and irrevocably delete this message and any copies.
