Hi, Raised here: https://issues.apache.org/jira/browse/CASSANDRA-5737
Am thinking this might be either a kernel bug, or some strange triggering combo on the mmap usage Thanks, Glyn From: aaron morton <aa...@thelastpickle.com<mailto:aa...@thelastpickle.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Date: Monday, 8 July 2013 23:45 To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: Re: CassandraDaemon - recent unsafe memory access operation in compiled Java code Glyn, For tracking purposes could you create an issue on https://issues.apache.org/jira/browse/CASSANDRA and link to the ticket your raised. Just incase something has to change when memmapping. Cheers ----------------- Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 9/07/2013, at 3:50 AM, Glyn Davies <gdav...@omnifone.com<mailto:gdav...@omnifone.com>> wrote: Hi, Yes, this continues without the JNA jar. In fact, the only thing which cured it was a reboot (!) Some serious dark magic going on there, as there were no Java processes running and nothing held the cassandra files open. I found a couple of Java dump texts, and opened an Java bug with one of them: http://bugs.sun.com/view_bug.do?bug_id=9004953 Though it doesn't seem to show up properly yet Glyn From: sankalp kohli <kohlisank...@gmail.com<mailto:kohlisank...@gmail.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Date: Sunday, 7 July 2013 22:26 To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: Re: CassandraDaemon - recent unsafe memory access operation in compiled Java code have u dropped the JNA jar? Looks like the mmap is failing. On Fri, Jul 5, 2013 at 8:15 AM, Glyn Davies <gdav...@omnifone.com<mailto:gdav...@omnifone.com>> wrote: Hi, Just starting to experiment with Cassandra, and have hit an early snag. I'm using 1.2.6 on Ubuntu AWS m1.xlarge instances with the Datastax Community package and have tried using Java versions jdk1.7.0_25 jre1.6.0_45 Also testing with and without libjna-java However, something has triggered a bug in the CassandraDaemon: ERROR [COMMIT-LOG-ALLOCATOR] 2013-07-05 15:00:51,663 CassandraDaemon.java (line 192) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main] java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled Java code at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:126) at org.apache.cassandra.db.commitlog.CommitLogSegment.freshSegment(CommitLogSegment.java:81) at org.apache.cassandra.db.commitlog.CommitLogAllocator.createFreshSegment(CommitLogAllocator.java:250) at org.apache.cassandra.db.commitlog.CommitLogAllocator.access$500(CommitLogAllocator.java:48) at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:104) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at java.lang.Thread.run(Unknown Source) This brought two nodes down out of a three node cluster – using QUORUM write with 3 replicas. Restarting the node replays this error, so I have the system in a 'stable' unstable state – which is probably a good place for trouble shooting. Presumably something a client wrote triggered this situation, and the other third node was to be the final replication point – and is thus still up. Any suggestions on next steps? I've had a good google for the error combinations, but didn't find any hits. Thanks, Glyn ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com<http://www.symanteccloud.com/> ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com<http://www.symanteccloud.com/> ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________
