move first removes the node from the cluster, then adds it back http://wiki.apache.org/cassandra/Operations#Moving_nodes
If you have 3 nodes and rf 3, removing the node will result in the error you are seeing. There is not enough nodes in the cluster to implement the replication factor. You can drop the RF down to 2 temporarily and then put it back to 3 later, see http://wiki.apache.org/cassandra/Operations#Replication Cheers ----------------- Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 5 Aug 2011, at 03:39, Yan Chunlu wrote: > hi, any help? thanks! > > On Thu, Aug 4, 2011 at 5:02 AM, Yan Chunlu <springri...@gmail.com> wrote: > forgot to mention I am using cassandra 0.7.4 > > > On Thu, Aug 4, 2011 at 5:00 PM, Yan Chunlu <springri...@gmail.com> wrote: > also nothing happens about the streaming: > > nodetool -h node3 netstats > Mode: Normal > Not sending any streams. > Nothing streaming from /10.28.53.11 > Pool Name Active Pending Completed > Commands n/a 0 165086750 > Responses n/a 0 99372520 > > > > On Thu, Aug 4, 2011 at 4:56 PM, Yan Chunlu <springri...@gmail.com> wrote: > sorry the ring info should be this: > > nodetool -h node3 ring > Address Status State Load Owns Token > > > 84944475733633104818662955375549269696 > node1 Up Normal 13.18 GB 81.09% > 52773518586096316348543097376923124102 > node2 Up Normal 22.85 GB 10.48% > 70597222385644499881390884416714081360 > node3 Up Leaving 25.44 GB 8.43% > 84944475733633104818662955375549269696 > > > > On Thu, Aug 4, 2011 at 4:55 PM, Yan Chunlu <springri...@gmail.com> wrote: > I have tried the nodetool move but get the following error.... > > node3:~# nodetool -h node3 move 0 > Exception in thread "main" java.lang.IllegalStateException: replication > factor (3) exceeds number of endpoints (2) > at > org.apache.cassandra.locator.SimpleStrategy.calculateNaturalEndpoints(SimpleStrategy.java:60) > at > org.apache.cassandra.service.StorageService.calculatePendingRanges(StorageService.java:930) > at > org.apache.cassandra.service.StorageService.calculatePendingRanges(StorageService.java:896) > at > org.apache.cassandra.service.StorageService.startLeaving(StorageService.java:1596) > at > org.apache.cassandra.service.StorageService.move(StorageService.java:1734) > at > org.apache.cassandra.service.StorageService.move(StorageService.java:1709) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) > at > com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) > at > com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) > at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120) > at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262) > at > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836) > at > com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761) > at > javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427) > at > javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) > at > javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265) > at > javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360) > at > javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788) > at sun.reflect.GeneratedMethodAccessor108.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) > at sun.rmi.transport.Transport$1.run(Transport.java:159) > at java.security.AccessController.doPrivileged(Native Method) > at sun.rmi.transport.Transport.serviceCall(Transport.java:155) > at > sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > > > > > then nodetool shows the node is leaving.... > > > nodetool -h node3 ring > Address Status State Load Owns Token > > > 84944475733633104818662955375549269696 > node1 Up Normal 13.18 GB 81.09% > 52773518586096316348543097376923124102 > node2 Up Normal 22.85 GB 10.48% > 70597222385644499881390884416714081360 > node3 Up Leaving 25.44 GB 8.43% > 84944475733633104818662955375549269696 > > the log didn't show any error message neither anything abnormal. is there > something wrong? > > > I used to have RF=2, and changed it to RF=3 using cassandra-cli. > > > On Mon, Aug 1, 2011 at 10:22 AM, Yan Chunlu <springri...@gmail.com> wrote: > thanks a lot! I will try the "move". > > > On Mon, Aug 1, 2011 at 7:07 AM, mcasandra <mohitanch...@gmail.com> wrote: > > springrider wrote: > > > > is that okay to do nodetool move before a completely repair? > > > > using this equation? > > def tokens(nodes): > > > > - for x in xrange(nodes): > > - print 2 ** 127 / nodes * x > > > > Yes use that logic to get the tokens. I think it's safe to run move first > and reair later. You are moving some nodes data as is so it's no worse than > what you have right now. > > -- > View this message in context: > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/how-to-solve-one-node-is-in-heavy-load-in-unbalanced-cluster-tp6630827p6639317.html > Sent from the cassandra-u...@incubator.apache.org mailing list archive at > Nabble.com. > > > > > >
