[ https://issues.apache.org/jira/browse/CLOUDSTACK-1244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589408#comment-13589408 ]
Sanjeev N commented on CLOUDSTACK-1244: --------------------------------------- I could see following error message in xenserver SMlog: [9572] 2013-02-27 09:51:17.587584 vdi_attach {'sr_uuid': 'd20365c4-2ba0-ec9b-3027-03fa10a27703', 'subtask_of': 'DummyRef:|70c2552c-c742-5489-a7cc-c2b8f40464d1|VDI.attach', 'vdi_ref': 'OpaqueRef:bd0cca02-c40e-3f7d-2890-a4905c36448b', 'vdi_on_boot': 'persist', 'args': ['false'], 'vdi_location': 'systemvm.iso', 'host_ref': 'OpaqueRef:ff76ea88-c8a9-35ba-4ad5-087a2854fba3', 'session_ref': 'OpaqueRef:d94b450c-4530-a3e2-749d-2459c408e2d0', 'device_config': {'legacy_mode': 'true', 'SRmaster': 'true', 'location': '/opt/xensource/packages/iso'}, 'command': 'vdi_attach', 'vdi_allow_caching': 'false', 'sr_ref': 'OpaqueRef:7f4b3f41-07ed-f803-81d1-130167f7a4e0', 'vdi_uuid': '7182e332-eb2d-4532-9652-0d50e5fbdd3d'} [9572] 2013-02-27 09:51:17.587737 Attach & activate [9572] 2013-02-27 09:51:17.614208 Raising exception [144, VDI could not be found] [9572] 2013-02-27 09:51:17.615474 ***** vdi_attach: EXCEPTION SR.SROSError, VDI could not be found File "/opt/xensource/sm/SRCommand.py", line 94, in run return self._run_locked(sr) File "/opt/xensource/sm/SRCommand.py", line 131, in _run_locked return self._run(sr, target) File "/opt/xensource/sm/SRCommand.py", line 163, in _run return target.attach(self.params['sr_uuid'], self.vdi_uuid, writable) File "/opt/xensource/sm/blktap2.py", line 1283, in attach self._attach(sr_uuid, vdi_uuid) File "/opt/xensource/sm/blktap2.py", line 1355, in _attach phy_path = self.target.attach(sr_uuid, vdi_uuid) File "/opt/xensource/sm/blktap2.py", line 970, in attach return self.vdi.attach(sr_uuid, vdi_uuid) File "/opt/xensource/sm/ISOSR", line 424, in attach raise xs_errors.XenError('VDIMissing') File "/opt/xensource/sm/xs_errors.py", line 49, in __init__ raise SR.SROSError(errorcode, errormessage) > fail to push sysmvm.iso onto xen host > -------------------------------------- > > Key: CLOUDSTACK-1244 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1244 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Management Server > Affects Versions: 4.1.0 > Reporter: sadhu suresh > Assignee: Devdeep Singh > Priority: Blocker > Fix For: 4.1.0 > > Attachments: SMlog.8.gz, vmops.rar > > > Env: > host :xen6.0.2 > MS:513f8b98e9b7818d0b37c9641d9021b77a0cfc45 > System VMs unable to start due to MS fail to find the systemvm.iso. That > leads to Runtime Exception > steps: > 1.prepared the setup in dev env > 2.Add the zone,pod,cluster and xen host details and enable the zone > INFO [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) Found a > stopped console proxy, bring it up to running pool. proxy vim id : 2 > WARN [network.element.VpcVirtualRouterElement] (secstorage-1:) Network > Ntwk[200|Public|1] is not associated with any VPC > WARN [network.element.VpcVirtualRouterElement] (consoleproxy-1:) Network > Ntwk[200|Public|1] is not associated with any VPC > WARN [network.element.VpcVirtualRouterElement] (secstorage-1:) Network > Ntwk[202|Control|3] is not associated with any VPC > WARN [network.element.VpcVirtualRouterElement] (secstorage-1:) Network > Ntwk[201|Management|2] is not associated with any VPC > WARN [network.element.VpcVirtualRouterElement] (secstorage-1:) Network > Ntwk[203|Storage|4] is not associated with any VPC > WARN [xen.resource.CitrixResourceBase] (DirectAgent-3:) Catch Exception: > class com.cloud.utils.exception.CloudRuntimeException due to > com.cloud.utils.exception.CloudRuntimeException: can not find systemvmiso > com.cloud.utils.exception.CloudRuntimeException: can not find systemvmiso > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.createPatchVbd(CitrixResourceBase.java:1162) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1307) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:517) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > WARN [xen.resource.CitrixResourceBase] (DirectAgent-3:) Unable to start > s-36-VM due to > com.cloud.utils.exception.CloudRuntimeException: can not find systemvmiso > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.createPatchVbd(CitrixResourceBase.java:1162) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1307) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:517) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > WARN [xen.resource.CitrixResourceBase] (DirectAgent-3:) Unable to clean up > VBD due to > You gave an invalid object reference. The object may have recently been > deleted. The class parameter gives the type of reference given, and the > handle parameter echoes the bad value given. > at com.xensource.xenapi.Types.checkResponse(Types.java:209) > at com.xensource.xenapi.Connection.dispatch(Connection.java:368) > at > com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909) > at com.xensource.xenapi.VBD.unplug(VBD.java:1059) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1114) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1377) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:517) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > INFO [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) Unable to start VM > on Host[-3-Routing] due to Unable to start s-36-VM due to > INFO [xen.resource.CitrixResourceBase] (DirectAgent-4:) VM does not exist on > XenServer4a2aea51-5c83-4176-9be5-79d79318aadb > WARN [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:) > Exception while trying to start secondary storage vim > com.cloud.exception.InsufficientServerCapacityException: Unable to create a > deployment for VM[SecondaryStorageVm|s-36-VM]Scope=interface > com.cloud.dc.DataCenter; id=1 > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:728) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:466) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:459) > at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) > at > org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80) > at > com.cloud.utils.db.TransactionContextBuilder.AroundAnyMethod(TransactionContextBuilder.java:43) > at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:616) > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) > at > org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622) > at > com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:264) > at > com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:689) > at > com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1300) > at > com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:121) > at > com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:52) > at > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > Work around: > if you clear the Tags on xen host parameter and readd the host will resolve > the issue > 1.put the registered host in CS in maintenance mode > 2.remove the host form Cloudstack > 3.perform param-clear on "tags" param > eg: > xe host-param-clear param-name=tags uuid=4a2aea51-5c83-4176-9be5-79d79318aadb > 4.re add the host into cloudstack > will resolve the issue .after performing the workaround system vms are up and > running -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira