Adding a swapfile fixed the error, but it doesn't look as though the process is even using the swap file at all.
Lee Parker On Tue, Apr 27, 2010 at 9:49 AM, Eric Hauser <ewhau...@gmail.com> wrote: > Have you read this? > > http://forums.sun.com/thread.jspa?messageID=9734530 > > <http://forums.sun.com/thread.jspa?messageID=9734530>I don't think EC2 > instances have any swap. > > > > On Tue, Apr 27, 2010 at 10:16 AM, Lee Parker <l...@socialagency.com> wrote: > >> Can anyone help with this? It is preventing me from getting backups of >> our cluster. >> >> Lee Parker >> On Mon, Apr 26, 2010 at 10:02 PM, Lee Parker <l...@socialagency.com>wrote: >> >>> I was attempting to get a snapshot on our cassandra nodes. I get the >>> following error every time I run nodetool ... snapshot. >>> >>> Exception in thread "main" java.io.IOException: Cannot run program "ln": >>> java.io.IOException: error=12, Cannot allocate memory >>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459) >>> at >>> org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:221) >>> at >>> org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1060) >>> at org.apache.cassandra.db.Table.snapshot(Table.java:256) >>> at >>> org.apache.cassandra.service.StorageService.takeAllSnapshot(StorageService.java:1005) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>> at java.lang.reflect.Method.invoke(Method.java:597) >>> at >>> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) >>> at >>> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) >>> at >>> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) >>> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120) >>> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262) >>> at >>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836) >>> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761) >>> at >>> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426) >>> at >>> javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) >>> at >>> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264) >>> at >>> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359) >>> at >>> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>> at java.lang.reflect.Method.invoke(Method.java:597) >>> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) >>> at sun.rmi.transport.Transport$1.run(Transport.java:159) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at sun.rmi.transport.Transport.serviceCall(Transport.java:155) >>> at >>> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) >>> at >>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) >>> at >>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:619) >>> Caused by: java.io.IOException: java.io.IOException: error=12, Cannot >>> allocate memory >>> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148) >>> at java.lang.ProcessImpl.start(ProcessImpl.java:65) >>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452) >>> ... 34 more >>> >>> The nodes are both Amazon EC2 Large instances with 7.5G RAM (6 allocated >>> for Java heap) with two cores and only 70G of data in casssandra. They have >>> plenty of available RAM and HD space. Has anyone else run into this error? >>> >>> Lee Parker >>> >> >> >
