Hi Everyone,
Recently my Cloudstack setup machine's got rebooted. After the reboot, the
virtual router is not getting created, due to which I am not able to create
or access any machine. Pasting error log below:
*CloudStack Version: **4.11.3.0*
2020-06-23 20:23:16,599 ERROR [o.c.n.r.d.RouterDeploymentDefinition]
(Work-Job-Executor-17:ctx-f565e932 job-621/job-623 ctx-770b7937)
(logid:6812a0da)
Didn't support redundant virtual router without public network!
2020-06-23 20:23:16,599 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-17:ctx-f565e932 job-621/job-623 ctx-770b7937)
(logid:6812a0da) Unab
le to contact resource.
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1]
is unreachable: Can't find at least one running router!
at
com.cloud.network.element.VirtualRouterElement.prepare(VirtualRouterElement.java:280)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElement(NetworkOrchestrator.java:1296)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(NetworkOrchestrator.java:1631)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(NetworkOrchestrator.java:1565)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1111)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4937)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5100)
at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2020-06-23 20:23:16,621 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-17:ctx-f565e932 job-621/job-623 ctx-770b7937)
(logid:6812a0da) Clea
ning up resources for the vm VM[User|i-2-67-VM] in Starting state
2020-06-23 20:23:16,628 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-17:ctx-f565e932 job-621/job-623 ctx-770b7937)
(logid:6812a0da) Seq 5-62628182318120
96597: Sending { Cmd , MgmtId: 200242944302310, via: 5(master), Ver: v1,
Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBe
foreCleanup":false,"forceStop":false,"volumesToDisconnect":[],"vmName":"i-2-67-VM","executeInSequence":false,"wait":0}}]
}
2020-06-23 20:23:16,836 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-5:null) (logid:) Seq 5-6262818231812096597:
Processing: { Ans: , MgmtId: 200242
944302310, via: 5, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2020-06-23 20:23:16,836 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-17:ctx-f565e932 job-621/job-623 ctx-770b7937)
(logid:6812a0da) Seq 5-62628182318120
96597: Received: { Ans: , MgmtId: 200242944302310, via: 5(master), Ver:
v1, Flags: 10, { StopAnswer } }
2020-06-23 20:23:16,862 ERROR [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-17:ctx-f565e932 job-621/job-623 ctx-770b7937)
(logid:6812a0da) Failed
to remove dhcp-dns entry due to:
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1]
is unreachable: Can't find at least one router!
at
com.cloud.network.element.VirtualRouterElement.removeDhcpEntry(VirtualRouterElement.java:961)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.cleanupNicDhcpDnsEntry(NetworkOrchestrator.java:2933)
at
com.cloud.vm.UserVmManagerImpl.finalizeStop(UserVmManagerImpl.java:4389)
at
com.cloud.vm.VirtualMachineManagerImpl.sendStop(VirtualMachineManagerImpl.java:1485)
at
com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerImpl.java:1519)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1272)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4937)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5100)
at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2020-06-23 20:23:16,876 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-17:ctx-f565e932 job-621/job-623 ctx-770b7937)
(logid:6812a0da) Changi
ng active number of nics for network id=204 on -1
2020-06-23 20:23:16,902 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-17:ctx-f565e932 job-621/job-623 ctx-770b7937)
(logid:6812a0da) Asking
SecurityGroupProvider to release
NicProfile[117-67-90f86a64-4697-47df-a5fb-bef1328f3449-192.168.0.94-vlan://untagged
2020-06-23 20:23:16,902 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-17:ctx-f565e932 job-621/job-623 ctx-770b7937)
(logid:6812a0da) Asking
VirtualRouter to release
NicProfile[117-67-90f86a64-4697-47df-a5fb-bef1328f3449-192.168.0.94-vlan://untagged
2020-06-23 20:23:16,908 WARN [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-17:ctx-f565e932 job-621/job-623 ctx-770b7937)
(logid:6812a0da) Unab
le to release some network resources.
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1]
is unreachable: Can't find at least one router!
at
com.cloud.network.element.VirtualRouterElement.removeDhcpEntry(VirtualRouterElement.java:961)
at
com.cloud.network.element.VirtualRouterElement.release(VirtualRouterElement.java:898)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.releaseNic(NetworkOrchestrator.java:1919)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.release(NetworkOrchestrator.java:1845)
at
com.cloud.vm.VirtualMachineManagerImpl.cleanup(VirtualMachineManagerImpl.java:1567)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1272)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4937)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5100)
at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Can someone please help me with creating the virtual router?
Thanks and Regards,
Jagdish Patil
