I should probably add.. /etc/hosts had the hostname set to 127.0.1.1.
On Thu, Sep 3, 2015 at 11:00 PM, Jason Lewis <jle...@packetnexus.com> wrote: > I figured this one out. As it turns out, the nodes that I couldn't > connect to, had the hostname set to 127.0.1.1. The listen IP is *not* > that IP. > > Thanks for the logging tip, it helped track it down. > > On Thu, Sep 3, 2015 at 10:43 PM, Jason Lewis <jle...@packetnexus.com> wrote: >> After enabling that option, I'm seeing errors like this on the node I >> can't connect to. >> >> >> Sep 04, 2015 2:35:48 AM sun.rmi.server.UnicastServerRef logCallException >> FINE: RMI TCP Connection(4)-127.0.0.1: [127.0.0.1] exception: >> javax.management.InstanceNotFoundException: >> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=ks,scope=cf,name=PendingTasks >> at >> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095) >> at >> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:643) >> at >> com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678) >> at >> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1448) >> at >> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76) >> at >> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1312) >> at >> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1404) >> at >> javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:641) >> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:497) >> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323) >> at sun.rmi.transport.Transport$1.run(Transport.java:200) >> at sun.rmi.transport.Transport$1.run(Transport.java:197) >> at java.security.AccessController.doPrivileged(Native Method) >> at sun.rmi.transport.Transport.serviceCall(Transport.java:196) >> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) >> at >> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) >> at >> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$79(TCPTransport.java:683) >> at java.security.AccessController.doPrivileged(Native Method) >> at >> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> >> On Wed, Aug 26, 2015 at 2:05 PM, Nate McCall <n...@thelastpickle.com> wrote: >>>> LOCAL_JMX=no >>>> >>>> if [ "$LOCAL_JMX" = "yes" ]; then >>>> JVM_OPTS="$JVM_OPTS -Dcassandra.jmx.local.port=$JMX_PORT >>>> -XX:+DisableExplicitGC" >>>> else >>>> JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT" >>>> JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT" >>>> JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.ssl=false" >>>> JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.authenticate=false" >>>> JVM_OPTS="$JVM_OPTS >>>> >>>> -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password" >>>> fi >>>> >>> >>> Retry with the following option added to your JVM_OPTS: >>> java.rmi.server.logCalls=true >>> >>> This should produce some more information about what is going on. >>> >>> >>> >>> >>> -- >>> ----------------- >>> Nate McCall >>> Austin, TX >>> @zznate >>> >>> Co-Founder & Sr. Technical Consultant >>> Apache Cassandra Consulting >>> http://www.thelastpickle.com
