I need help with figuring this issue out - as I cant roll our cloudstack ENV into production.
As of now, the only way to resolve this is to completely recreate the router VMs. This cripples the entire environment. My setup is documented here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Advanced+Network+Tutorial+-+Step+by+Step In the nutshell, I'm running CS4.0 with vSphere 5.0 with Advanced network. I looked at bug CS-14421, which is similar to what I get, however I don't have a secondary DHCP server running that could have conflicted with CS DHCPD. As always, any feedback is appreciated. Thank you -ilya 2012-11-29 11:11:08,201 ERROR [vmware.resource.VmwareResource] (DirectAgent-343:vmha61d.domain.com) DhcpEntryCommand failed due to Exception: java.net.SocketTimeoutException Message: The kexTimeout (60000 ms) expired. java.net.SocketTimeoutException: The kexTimeout (60000 ms) expired. at com.trilead.ssh2.Connection.connect(Connection.java:785) at com.cloud.utils.ssh.SshHelper.sshExecute(SshHelper.java:132) at com.cloud.utils.ssh.SshHelper.sshExecute(SshHelper.java:37) at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:1658) at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:347) at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2012-11-29 11:11:08,203 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-343:null) Seq 40-606472464: Response Received: 2012-11-29 11:11:08,204 DEBUG [agent.transport.Request] (DirectAgent-343:null) Seq 40-606472464: Processing: { Ans: , MgmtId: 345051904793, via: 40, Ver: v1, Flags: 110, [{"Answer":{"result":false,"details":"DhcpEntryCommand failed due to Exception: java.net.SocketTimeoutException\nMessage: The kexTimeout (60000 ms) expired.\n","wait":0}}] } 2012-11-29 11:11:08,204 DEBUG [agent.transport.Request] (Job-Executor-25:job-350) Seq 40-606472464: Received: { Ans: , MgmtId: 345051904793, via: 40, Ver: v1, Flags: 110, { Answer } } 2012-11-29 11:11:08,204 INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-25:job-350) Unable to contact resource. com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:5] is unreachable: Unable to apply dhcp entry on router at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyDhcpEntry(VirtualNetworkApplianceManagerImpl.java:2396) at com.cloud.network.element.VirtualRouterElement.addDhcpEntry(VirtualRouterElement.java:786) at com.cloud.network.NetworkManagerImpl.prepareElement(NetworkManagerImpl.java:2062) at com.cloud.network.NetworkManagerImpl.prepareNic(NetworkManagerImpl.java:2172) at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2113) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:752) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:472) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2929) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2601) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2589) at com.cloud.event.ActionEventCallback.intercept(ActionEventCallback.java:36) at com.cloud.api.commands.DeployVMCmd.execute(DeployVMCmd.java:330) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679)
