Jeff, Can you include your entire server.xml for each instance and also your /etc/hosts?
From: Jeff Hall [mailto:jhall....@gmail.com] Sent: Wednesday, August 15, 2012 1:32 PM To: users@tomcat.apache.org Cc: Carrillo, Dan Subject: Re: Unable to start cluster (java.net.BindException: Cannot assign requested address) Dan, thanks for the reply. I don't know why port 49352 was in use when I ran the test. I started the cluster up a second time after making sure all java processes were stopped, and netstat reported no use of port 49352 (the shutdown port for tomcat instance #1). I relaunched it and the log showed my original problem with the bind failing on port 4099: Aug 15, 2012 3:20:57 PM org.apache.catalina.core.AprLifecycleListener init INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /opt/jdk1.6.0_24/jre/lib/amd64/server:/opt/jdk1.6.0_24/jre/lib/amd64:/opt/jdk1.6.0_24/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 1.795: [Full GC (System) 1.796: [CMS: 0K->1655K(240896K), 0.0734030 secs] 12718K->1655K(260032K), [CMS Perm : 9989K->9979K(21248K)], 0.0735830 secs] [Times: user=0.08 sys=0.00, real=0.08 secs] Aug 15, 2012 3:20:58 PM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 1462 ms Aug 15, 2012 3:20:58 PM org.apache.catalina.core.StandardService start INFO: Starting service Catalina Aug 15, 2012 3:20:58 PM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/6.0.32 Aug 15, 2012 3:20:58 PM org.apache.catalina.ha.tcp.SimpleTcpCluster start INFO: Cluster is about to start Aug 15, 2012 3:21:01 PM org.apache.catalina.tribes.transport.ReceiverBase bind INFO: Unable to bind server socket to:/192.168.100.225:4099 throwing error. Aug 15, 2012 3:21:01 PM org.apache.catalina.tribes.transport.nio.NioReceiver start SEVERE: Unable to start cluster receiver java.net.BindException: Cannot assign requested address at sun.nio.ch.Net.bind(Native Method) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52) at org.apache.catalina.tribes.transport.ReceiverBase.bind(ReceiverBase.java:205) at org.apache.catalina.tribes.transport.ReceiverBase.bind(ReceiverBase.java:217) at org.apache.catalina.tribes.transport.ReceiverBase.bind(ReceiverBase.java:217) at org.apache.catalina.tribes.transport.ReceiverBase.bind(ReceiverBase.java:217) ... // I suspect Tomcat is looping looking for a port between 4000 and 4099 here, as the bind at line 217 fails 100 times at org.apache.catalina.tribes.transport.nio.NioReceiver.bind(NioReceiver.java:132) at org.apache.catalina.tribes.transport.nio.NioReceiver.start(NioReceiver.java:102) at org.apache.catalina.tribes.group.ChannelCoordinator.internalStart(ChannelCoordinator.java:142) at org.apache.catalina.tribes.group.ChannelCoordinator.start(ChannelCoordinator.java:97) at org.apache.catalina.tribes.group.ChannelInterceptorBase.start(ChannelInterceptorBase.java:149) at org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.start(MessageDispatchInterceptor.java:147) at org.apache.catalina.tribes.group.ChannelInterceptorBase.start(ChannelInterceptorBase.java:149) at org.apache.catalina.tribes.group.ChannelInterceptorBase.start(ChannelInterceptorBase.java:149) at org.apache.catalina.tribes.group.GroupChannel.start(GroupChannel.java:407) at org.apache.catalina.ha.tcp.SimpleTcpCluster.start(SimpleTcpCluster.java:671) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1043) at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at org.apache.catalina.core.StandardService.start(StandardService.java:525) at org.apache.catalina.core.StandardServer.start(StandardServer.java:754) at org.apache.catalina.startup.Catalina.start(Catalina.java:595) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Aug 15, 2012 3:21:01 PM org.apache.catalina.ha.tcp.SimpleTcpCluster start SEVERE: Unable to start cluster. org.apache.catalina.tribes.ChannelException: java.net.BindException: Cannot assign requested address; No faulty members identified. at org.apache.catalina.tribes.group.ChannelCoordinator.internalStart(ChannelCoordinator.java:169) at org.apache.catalina.tribes.group.ChannelCoordinator.start(ChannelCoordinator.java:97) at org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.start(MessageDispatchInterceptor.java:147) at org.apache.catalina.tribes.group.ChannelInterceptorBase.start(ChannelInterceptorBase.java:149) at org.apache.catalina.tribes.group.ChannelInterceptorBase.start(ChannelInterceptorBase.java:149) at org.apache.catalina.tribes.group.GroupChannel.start(GroupChannel.java:407) at org.apache.catalina.ha.tcp.SimpleTcpCluster.start(SimpleTcpCluster.java:671) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1043) at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at org.apache.catalina.core.StandardService.start(StandardService.java:525) at org.apache.catalina.core.StandardServer.start(StandardServer.java:754) at org.apache.catalina.startup.Catalina.start(Catalina.java:595) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by: java.net.BindException: Cannot assign requested address at sun.nio.ch.Net.bind(Native Method) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52) at org.apache.catalina.tribes.transport.ReceiverBase.bind(ReceiverBase.java:205) at org.apache.catalina.tribes.transport.ReceiverBase.bind(ReceiverBase.java:217) at org.apache.catalina.tribes.transport.ReceiverBase.bind(ReceiverBase.java:217) at org.apache.catalina.tribes.transport.ReceiverBase.bind(ReceiverBase.java:217) at org.apache.catalina.tribes.transport.ReceiverBase.bind(ReceiverBase.java:217) ... at org.apache.catalina.tribes.transport.nio.NioReceiver.bind(NioReceiver.java:132) at org.apache.catalina.tribes.transport.nio.NioReceiver.start(NioReceiver.java:102) at org.apache.catalina.tribes.group.ChannelCoordinator.internalStart(ChannelCoordinator.java:142) ... 18 more Aug 15, 2012 3:21:01 PM org.apache.catalina.startup.Catalina start SEVERE: Catalina.start: LifecycleException: org.apache.catalina.tribes.ChannelException: java.net.BindException: Cannot assign requested address; No faulty members identified. at org.apache.catalina.ha.tcp.SimpleTcpCluster.start(SimpleTcpCluster.java:678) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1043) at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at org.apache.catalina.core.StandardService.start(StandardService.java:525) at org.apache.catalina.core.StandardServer.start(StandardServer.java:754) at org.apache.catalina.startup.Catalina.start(Catalina.java:595) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Aug 15, 2012 3:21:01 PM org.apache.catalina.startup.Catalina start INFO: Server startup in 2767 ms For reference from my original post (sorry, I'm receiving the entire tomcat digest, not individual posts, so I can't reply inline...) ----- Here is my cluster configuration from server.xml: <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8"> <Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true" /> <Channel className="org.apache.catalina.tribes.group.GroupChannel"> <Membership className="org.apache.catalina.tribes.membership.McastService" address="228.0.0.4" port="45564" frequency="500" dropTime="3000" /> <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"> <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" /> </Sender> <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver" address="auto" port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6" /> <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector" /> <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor" /> </Channel> <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter="" /> <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve" /> <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener" /> <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener" /> </Cluster> I checked to make sure multicasting is enabled (ifconfig eth0): eth0 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx inet addr:192.168.200.225 Bcast:192.168.203.255 Mask:255.255.252.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1984991 errors:0 dropped:0 overruns:0 frame:0 TX packets:1941912 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:568159401 (541.8 MiB) TX bytes:569789016 (543.3 MiB) Interrupt:11 Memory:f8000000-f8012800 Is there anything I need to add to /etc/hosts or /etc/hosts.allow? Maybe RackSpace's default rules are somehow blocking access, even though there are no explicit firewall rules blocking 4000-4099. I don't know. I had this same cluster configuration up and running on my MacBook Pro without any issues. But now I am at a loss. Can anyone point me in a direction to investigate further? Thank you! Jeff
