[ https://issues.apache.org/jira/browse/CLOUDSTACK-5027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sateesh Chodapuneedi resolved CLOUDSTACK-5027. ---------------------------------------------- Resolution: Cannot Reproduce Tested with 4.2 branch, unable to reproduce this. > [VMWARE] Failed to Deploy instance when primary storage of a different zone > are in maintenance mode > --------------------------------------------------------------------------------------------------- > > Key: CLOUDSTACK-5027 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5027 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Storage Controller, VMware > Affects Versions: 4.2.1 > Reporter: Sailaja Mada > Assignee: Sateesh Chodapuneedi > Priority: Critical > Fix For: 4.2.1 > > Attachments: failureloigs.rar > > > Steps: > 1. Configure 2 vMWARE zones each with Zone wide primary storages > 2. Moved Zone 1 primary storages into maintenance mode > 3. Tried to deploy VM on second VMWARE zone which has Primary storage up > state. > Observation: > [VMWARE] Failed to Deploy instance when primary storage of a different zone > are in maintenance mode > 2013-11-03 10:21:07,088 DEBUG [cloud.storage.VolumeManagerImpl] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Checking > if we need to prepare 1 volumes for VM[DomainRouter|r-30-VM] > 2013-11-03 10:21:07,099 DEBUG [storage.image.TemplateDataFactoryImpl] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) template > 8 is already in store:1, type:Image > 2013-11-03 10:21:07,104 DEBUG [storage.datastore.PrimaryDataStoreImpl] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Not > found (templateId:8poolId:6) in template_spool_ref, persisting it > 2013-11-03 10:21:07,121 DEBUG [storage.image.TemplateDataFactoryImpl] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) template > 8 is already in store:6, type:Primary > 2013-11-03 10:21:07,123 DEBUG [storage.volume.VolumeServiceImpl] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Found > template routing-8 in storage pool 6 with VMTemplateStoragePool id: 28 > 2013-11-03 10:21:07,187 DEBUG [agent.manager.AgentManagerImpl] > (AgentManager-Handler-3:null) SeqA 10-30340: Processing Seq 10-30340: { Cmd > , MgmtId: -1, via: 10, Ver: v1, Flags: 11, > [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":17,"_loadInfo":"{\n > \"connections\": []\n}","wait":0}}] } > 2013-11-03 10:21:07,331 DEBUG [storage.volume.VolumeServiceImpl] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Acquire > lock on VMTemplateStoragePool 28 with timeout 3600 seconds > 2013-11-03 10:21:07,333 INFO [storage.volume.VolumeServiceImpl] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) lock is > acquired for VMTemplateStoragePool 28 > 2013-11-03 10:21:07,341 DEBUG [agent.manager.AgentManagerImpl] > (AgentManager-Handler-3:null) SeqA 10-30340: Sending Seq 10-30340: { Ans: , > MgmtId: 94838926819810, via: 10, Ver: v1, Flags: 100010, > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] } > 2013-11-03 10:21:07,373 DEBUG [storage.motion.AncientDataMotionStrategy] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) > copyAsync inspecting src type TEMPLATE copyAsync inspecting dest type TEMPLATE > 2013-11-03 10:21:07,387 DEBUG [storage.motion.AncientDataMotionStrategy] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) copy > object failed: > java.lang.NullPointerException > at > org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:210) > at > org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:411) > at > org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:58) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:446) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:575) > at > com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2567) > at > com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2631) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) > at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2764) > at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1896) > at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1996) > at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1974) > at > com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:189) > at > com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:2043) > at > com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1948) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2129) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:886) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) > at > org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) > at > org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3425) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2985) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2971) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-11-03 10:21:07,479 DEBUG [storage.motion.AncientDataMotionStrategy] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) copy > failed > com.cloud.utils.exception.CloudRuntimeException: > java.lang.NullPointerException > at > org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:231) > at > org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:411) > at > org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:58) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:446) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:575) > at > com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2567) > at > com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2631) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) > at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2764) > at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1896) > at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1996) > at > com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:189) > at > com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:2043) > at > com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1948) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2129) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:886) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) > at > org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) > at > org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3425) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2985) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2971) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-11-03 10:21:07,615 DEBUG [cloud.vm.VirtualMachineManagerImpl] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Cleaning > up resources for the vm VM[DomainRouter|r-30-VM] in Starting state > 2013-11-03 10:21:07,616 DEBUG [agent.transport.Request] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Seq > 9-894185288: Sending { Cmd , MgmtId: 94838926819810, via: 9, Ver: v1, Flags: > 100111, > [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"r-30-VM","wait":0}}] > } > 2013-11-03 10:21:07,617 DEBUG [agent.transport.Request] > (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Seq > 9-894185288: Executing: { Cmd , MgmtId: 94838926819810, via: 9, Ver: v1, > Flags: 100111, > [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"r-30-VM","wait":0}}] > } > 2013-11-03 10:21:07,646 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-148:null) Seq 9-894185288: Executing request > 2013-11-03 10:21:07,647 INFO [vmware.resource.VmwareResource] > (DirectAgent-148:10.102.192.18) Executing resource StopCommand: > {"isProxy":false,"executeInSequence":true,"vmName":"r-30-VM","wait":0} > 2013-11-03 10:21:07,676 DEBUG [vmware.mo.HostMO] > (DirectAgent-148:10.102.192.18) find VM r-30-VM on host -- This message was sent by Atlassian JIRA (v6.1#6144)