server24 opened a new issue #3497: VMs stuck in Stopping state due to NPE at org.apache.cloudstack.network.topology.BasicNetworkVisitor.visit(BasicNetworkVisitor.java:201) URL: https://github.com/apache/cloudstack/issues/3497 Bug Report After upgrading from 4.11.2.0 to 4.11.3.0 every operation that requires deleting an entry in a basic network seems to fail due to a NPE and therefore the VMs are stuck in Stopping state. Possibly caused by commit b2b99ca63eecab6c0be40015ee54c37b55e07bac regarding server/src/main/java/org/apache/cloudstack/network/topology/BasicNetworkTopology.java (but this is just my guess) CloudStack version 4.11.3.0 with Basic Networking How to reproduce: Create a VM and then try to stop or destroy it. The procedure will not complete and the VM stays in "Stopping" state. In the CloudStack Mgm Server the following error message is shown: ### 2019-07-16 13:37:52,781 DEBUG [o.a.c.n.t.BasicNetworkTopology] (AgentManager-Handler-2:null) (logid:) REMOVING DHCP ENTRY RULE 2019-07-16 13:37:52,781 DEBUG [o.a.c.n.t.BasicNetworkTopology] (AgentManager-Handler-2:null) (logid:) Applying dhcp entry in network Ntwk[212|Guest|6] 2019-07-16 13:37:52,784 ERROR [o.a.c.f.m.MessageDispatcher] (AgentManager-Handler-2:null) (logid:) Unexpected exception when calling com.cloud.vm.ClusteredVirtualMachineManagerImpl.HandlePowerStateReport java.lang.reflect.InvocationTargetException at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.cloudstack.framework.messagebus.MessageDispatcher.dispatch(MessageDispatcher.java:75) at org.apache.cloudstack.framework.messagebus.MessageDispatcher.onPublishMessage(MessageDispatcher.java:45) at org.apache.cloudstack.framework.messagebus.MessageBusBase$SubscriptionNode.notifySubscribers(MessageBusBase.java:441) at org.apache.cloudstack.framework.messagebus.MessageBusBase.publish(MessageBusBase.java:178) at com.cloud.vm.VirtualMachinePowerStateSyncImpl.processReport(VirtualMachinePowerStateSyncImpl.java:147) at com.cloud.vm.VirtualMachinePowerStateSyncImpl.processHostVmStatePingReport(VirtualMachinePowerStateSyncImpl.java:68) at com.cloud.vm.VirtualMachineManagerImpl.processCommands(VirtualMachineManagerImpl.java:3071) at com.cloud.agent.manager.AgentManagerImpl.handleCommands(AgentManagerImpl.java:317) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.processRequest(AgentManagerImpl.java:1296) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.doTask(AgentManagerImpl.java:1383) at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:712) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException at org.apache.cloudstack.network.topology.BasicNetworkVisitor.visit(BasicNetworkVisitor.java:201) at com.cloud.network.rules.DhcpEntryRules.accept(DhcpEntryRules.java:64) at org.apache.cloudstack.network.topology.BasicNetworkTopology.applyRules(BasicNetworkTopology.java:390) ... 27 more 2019-07-16 13:37:52,784 WARN [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-2:null) (logid:) Caught: java.lang.RuntimeException: InvocationTargetException when invoking event handler for subject: vm.powerstate at org.apache.cloudstack.framework.messagebus.MessageDispatcher.dispatch(MessageDispatcher.java:84) at org.apache.cloudstack.framework.messagebus.MessageDispatcher.onPublishMessage(MessageDispatcher.java:45) at org.apache.cloudstack.framework.messagebus.MessageBusBase$SubscriptionNode.notifySubscribers(MessageBusBase.java:441) at org.apache.cloudstack.framework.messagebus.MessageBusBase.publish(MessageBusBase.java:178) at com.cloud.vm.VirtualMachinePowerStateSyncImpl.processReport(VirtualMachinePowerStateSyncImpl.java:147) at com.cloud.vm.VirtualMachinePowerStateSyncImpl.processHostVmStatePingReport(VirtualMachinePowerStateSyncImpl.java:68) at com.cloud.vm.VirtualMachineManagerImpl.processCommands(VirtualMachineManagerImpl.java:3071) at com.cloud.agent.manager.AgentManagerImpl.handleCommands(AgentManagerImpl.java:317) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.processRequest(AgentManagerImpl.java:1296) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.doTask(AgentManagerImpl.java:1383) at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:712) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ### Already tried recreating the VR, cleaning and restarting the network etc., but it makes no difference. The problems first appeared after upgrading from CloudStack 4.11.2 to 4.11.3.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
