Anyone have clue on this ? /Sonali
-----Original Message----- From: Sonali Jadhav [mailto:[email protected]] Sent: Monday, April 6, 2015 5:18 PM To: [email protected] Subject: RE: problem creating instance from template Hi Prashant, 1 - Yeha actually I noticed that, I am not able to create any new VMs from any other template. I just tried creating one Ubuntu instance , got this error in logs, 2015-04-06 13:12:25,300 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-27:ctx-149fc40a job-2588/job-2589) Unable to complete AsyncJobVO {id:2589, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAQ3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 59778234354585, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Apr 06 13:12:13 CEST 2015}, job origin:2588 com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-67-VM]Scope=interface com.cloud.dc.DataCenter; id=1 at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:947) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5195) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5340) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2015-04-06 13:12:25,302 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-27:ctx-149fc40a job-2588/job-2589) Complete async job-2589, jobStatus: FAILED, resultCode: 0, 2015-04-06 13:12:25,396 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-27:ctx-149fc40a job-2588/job-2589) Done executing com.cloud.vm.VmWorkStart for job-2589 2015-04-06 13:12:25,482 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-9:ctx-5e6ac400 job-2588 ctx-0c8863ad) Sync job-2590 execution on object VmWorkJobQueue.67 2015-04-06 13:12:25,486 WARN [c.c.u.d.Merovingian2] (API-Job-Executor-9:ctx-5e6ac400 job-2588 ctx-0c8863ad) Was unable to find lock for the key vm_instance67 and thread id 1697914126 After some time this was 2nd exception about same instance, 2015-04-06 13:12:32,661 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-29:ctx-467f37f1 job-2588/job-2590 ctx-4b4246ef) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[84|vm=67|ROOT]:Can't find template VDI under: 192.168.48.10:/tank/secstore/template/tmpl/1/5 at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume(VolumeOrchestrator.java:1205) at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1257) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:988) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5195) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5340) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) then this, 015-04-06 13:12:44,501 DEBUG [c.c.h.x.r.XenServerStorageProcessor] (DirectAgent-363:ctx-fa031563) Failed to destroy pbd SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", "73b81b8a-ec8f-3423-b2d3-7a0ca1b0ac13")']] at com.xensource.xenapi.Types.checkResponse(Types.java:2006) at com.xensource.xenapi.Connection.dispatch(Connection.java:350) at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456) at com.xensource.xenapi.SR.scan(SR.java:1051) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.createFileSR(Xenserver625StorageProcessor.java:107) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.createFileSr(Xenserver625StorageProcessor.java:133) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.copyTemplateToPrimaryStorage(Xenserver625StorageProcessor.java:167) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:77) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:52) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:547) at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2015-04-06 13:12:44,509 ERROR [c.c.h.x.r.XenServerStorageProcessor] (DirectAgent-363:ctx-fa031563) Failed to forget sr SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", "73b81b8a-ec8f-3423-b2d3-7a0ca1b0ac13")']] at com.xensource.xenapi.Types.checkResponse(Types.java:2006) at com.xensource.xenapi.Connection.dispatch(Connection.java:350) at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456) at com.xensource.xenapi.SR.scan(SR.java:1051) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.createFileSR(Xenserver625StorageProcessor.java:107) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.createFileSr(Xenserver625StorageProcessor.java:133) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.copyTemplateToPrimaryStorage(Xenserver625StorageProcessor.java:167) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:77) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:52) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:547) at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2015-04-06 13:12:44,510 WARN [c.c.h.x.r.XenServerStorageProcessor] (DirectAgent-363:ctx-fa031563) createFileSR failed! due to SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", "73b81b8a-ec8f-3423-b2d3-7a0ca1b0ac13")']] SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", "73b81b8a-ec8f-3423-b2d3-7a0ca1b0ac13")']] at com.xensource.xenapi.Types.checkResponse(Types.java:2006) at com.xensource.xenapi.Connection.dispatch(Connection.java:350) at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456) at com.xensource.xenapi.SR.scan(SR.java:1051) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.createFileSR(Xenserver625StorageProcessor.java:107) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.createFileSr(Xenserver625StorageProcessor.java:133) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.copyTemplateToPrimaryStorage(Xenserver625StorageProcessor.java:167) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:77) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:52) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:547) at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2015-04-06 13:12:44,512 WARN [c.c.h.x.r.XenServerStorageProcessor] (DirectAgent-363:ctx-fa031563) Catch Exception com.cloud.utils.exception.CloudRuntimeException for template due to com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", "73b81b8a-ec8f-3423-b2d3-7a0ca1b0ac13")']] com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", "73b81b8a-ec8f-3423-b2d3-7a0ca1b0ac13")']] at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.createFileSR(Xenserver625StorageProcessor.java:126) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.createFileSr(Xenserver625StorageProcessor.java:133) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.copyTemplateToPrimaryStorage(Xenserver625StorageProcessor.java:167) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:77) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:52) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:547) at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", "73b81b8a-ec8f-3423-b2d3-7a0ca1b0ac13")']] at com.xensource.xenapi.Types.checkResponse(Types.java:2006) at com.xensource.xenapi.Connection.dispatch(Connection.java:350) at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456) at com.xensource.xenapi.SR.scan(SR.java:1051) at com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.createFileSR(Xenserver625StorageProcessor.java:107) ... 21 more 2015-04-06 13:12:44,513 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-363:ctx-fa031563) Seq 1-4183562578850489418: Response Received: 2015-04-06 13:12:44,513 DEBUG [c.c.a.t.Request] (DirectAgent-363:ctx-fa031563) Seq 1-4183562578850489418: Processing: { Ans: , MgmtId: 59778234354585, via: 1, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Catch Exception com.cloud.utils.exception.CloudRuntimeException for template due to com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation(\"VDI\", \"uuid\", \"73b81b8a-ec8f-3423-b2d3-7a0ca1b0ac13\")']]","wait":0}}] } 2015-04-06 13:12:44,513 DEBUG [c.c.a.t.Request] (Work-Job-Executor-28:ctx-42c09fe7 job-2588/job-2590 ctx-0601a974) Seq 1-4183562578850489418: Received: { Ans: , MgmtId: 59778234354585, via: 1, Ver: v1, Flags: 10, { CopyCmdAnswer } } 2015-04-06 13:12:44,616 INFO [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-28:ctx-42c09fe7 job-2588/job-2590 ctx-0601a974) releasing lock for VMTemplateStoragePool 40 2015-04-06 13:12:44,617 WARN [c.c.u.d.Merovingian2] (Work-Job-Executor-28:ctx-42c09fe7 job-2588/job-2590 ctx-0601a974) Was unable to find lock for the key template_spool_ref40 and thread id 1105922582 2015-04-06 13:12:44,617 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-28:ctx-42c09fe7 job-2588/job-2590 ctx-0601a974) Unable to create Vol[84|vm=67|ROOT]:Catch Exception com.cloud.utils.exception.CloudRuntimeException for template due to com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", "73b81b8a-ec8f-3423-b2d3-7a0ca1b0ac13")']] 2015-04-06 13:12:44,617 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-42c09fe7 job-2588/job-2590 ctx-0601a974) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[84|vm=67|ROOT]:Catch Exception com.cloud.utils.exception.CloudRuntimeException for template due to com.cloud.utils.exception.CloudRuntimeException: createFileSR failed! due to SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", "73b81b8a-ec8f-3423-b2d3-7a0ca1b0ac13")']] at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume(VolumeOrchestrator.java:1205) at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1257) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:988) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5195) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5340) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2015-04-06 13:12:44,667 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-42c09fe7 job-2588/job-2590 ctx-0601a974) Cleaning up resources for the vm VM[User|i-2-67-VM] in Starting state 2015-04-06 13:12:41,676 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-28:ctx-42c09fe7 job-2588/job-2590 ctx-0601a974) Checking if we need to prepare 1 volumes for VM[User|i-2-67-VM] 2015-04-06 13:12:33,079 DEBUG [c.c.v.d.VMInstanceDaoImpl] (Work-Job-Executor-29:ctx-467f37f1 job-2588/job-2590 ctx-4b4246ef) Unable to update VM[User|i-2-67-VM]: DB Data={Host=6; State=Starting; updated=5; time=Mon Apr 06 13:12:26 CEST 2015} New Data: {Host=5; State=Starting; updated=6; time=Mon Apr 06 13:12:26 CEST 2015} Stale Data: {Host=4; State=Starting; updated=5; time=Mon Apr 06 13:12:26 CEST 2015} 2015-04-06 13:12:33,079 DEBUG [c.c.v.d.VMInstanceDaoImpl] (Work-Job-Executor-29:ctx-467f37f1 job-2588/job-2590 ctx-4b4246ef) VM i-2-67-VM state has been already been updated to Starting 2015-04-06 13:12:33,079 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-29:ctx-467f37f1 job-2588/job-2590 ctx-4b4246ef) VM state transitted from :Starting to Starting with event: OperationRetryvm's original host id: null new host id: 5 host id before state transition: 4 2015-04-06 13:12:44,913 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-42c09fe7 job-2588/job-2590 ctx-0601a974) Deployment found - P0=VM[User|i-2-67-VM], P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(5)-Storage(Volume(84|ROOT-->Pool(1))] 2015-04-06 13:12:45,013 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-42c09fe7 job-2588/job-2590 ctx-0601a974) VM state transitted from :Starting to Starting with event: OperationRetryvm's original host id: null new host id: 5 host id before state transition: 6 Template its saying not able to find, does exist at that location. I don’t know why its throwing that exception. 2 - My ssvm, routers and storage all are fine. I have few production VMs running , with no issues. Yes my template was in ready , download completed state. 3 - For /usr/local/cloudstack/systemvm/ssvm-check.sh , all check are fine. 4 - info I selected foe template is correct . 5 - of course..... :/ Its weird understating what's causing problem. /Sonali -----Original Message----- From: Prashant s [mailto:[email protected]] Sent: Monday, April 6, 2015 7:30 AM To: [email protected] Subject: Re: problem creating instance from template couple of questions for you Sonali 1. Are you able to build any new vms using any OTHER template that you already have. try to create a test vm 2. cloudstack pit stop checks from web interface : from your ACS interface check if there are any errors ? a ) can check if any of your hosts , b) primary or secondary storage c) systems vms d) router vms are in errored out state ? e) is your template in READY state , download completed .. ? 3. login to system vm and run the ssvm check /usr/local/cloudstack/systemvm/ssvm-check.sh , also while on the ssvm , check the *SSVM Logs* - /var/log/cloudstack/cloud.log 4. when you registered your template did you chose the right OS version etc ... 5.* does your server have the Intel Virtualization Technology (vt-x) or amd-v in BIOS turned on , for windows you really need this on :-0* i can only think of these issues & checks .... On Sat, Apr 4, 2015 at 7:45 AM, Sonali Jadhav <[email protected]> wrote: > [root@XS04 ~]# xe vdi-list uuid=32719160-52d8-4e45-9682-7ab9ada4dad1 > uuid ( RO) : 32719160-52d8-4e45-9682-7ab9ada4dad1 > name-label ( RW): > name-description ( RW): > sr-uuid ( RO): 39311fd4-822d-4ccc-8c5c-bbb7fda4def5 > virtual-size ( RO): 107374182400 > sharable ( RO): false > read-only ( RO): false > > Thanks, > /Sonali > > -----Original Message----- > From: giljae o [mailto:[email protected]] > Sent: Saturday, April 4, 2015 3:11 PM > To: [email protected] > Subject: Re: problem creating instance from template > > Then, > > Can you do the command in XenServer where you are trying to deploy VMs > > xe vdi-list uuid=32719160-52d8-4e45-9682-7ab9ada4dad1 > > James, > > On Sat, Apr 4, 2015 at 6:08 PM, Sonali Jadhav > <[email protected]> > wrote: > > > No, nothing that I am aware of. I have not even created manually. > > > > /Sonali > > > > -----Original Message----- > > From: giljae o [mailto:[email protected]] > > Sent: Saturday, April 4, 2015 2:18 PM > > To: [email protected] > > Subject: Re: problem creating instance from template > > > > Hi > > > > For deploying VM CS needs to download a template from 2nd Storage, > > In the cluster where the template is copied to there the VDI uuid > > was > conflicted. > > Is there any VDI on the pool CS does not know about? > > > > > > > > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org > > .a > > pache.cloudstack.storage.to.TemplateObjectTO":{"path":"template/tmpl > > /2 > > /212/32719160-52d8-4e45-9682-7ab9ada4dad1.vhd","uuid":"34b2b442-81cd > > -4 > > 56b-938d-6670bcfcaf01","id":212,"format":"VHD","accountId":2,"hvm":t > > ru > > e,"displayText":"Windows > > Server 2012 R2 > > Std","imageDataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs:/ > > / > > 192.168.48.10/tank/secstore > > ","_role":"Image"}},"name":"32719160-52d8-4e45-9682-7ab9ada4dad1","h > > yp > > ervisorType":"XenServer"}},"destTO":{"org.apache.cloudstack.storage. > > to > > .TemplateObjectTO":{"uuid":"34b2b442-81cd-456b-938d-6670bcfcaf01","id" > > :212,"format":"VHD","accountId":2,"hvm":true,"displayText":"Windows > > Server 2012 R2 > > > > Std","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryData > > St > > oreTO":{"uuid":"876d490c-a1d4-3bfe-88b7-1bdb2479541b","id":1,"poolTy > > pe > > ":"NetworkFilesystem","host":"192.168.6.194","path":"/tank/primstore > > ", "port":2049,"url":"NetworkFilesystem:// > > 192.168.6.194/tank/primstore/?ROLE=Primary&STOREUUID=876d490c- > > > > > > 2015-04-03 17:10:16,786 DEBUG [c.c.h.x.r.XenServerStorageProcessor] > > (DirectAgent-99:ctx-e6b464d2) Failed to destroy pbd > > SR_BACKEND_FAILURE_40The SR scan failed [opterr=['INTERNAL_ERROR', > > 'Db_exn.Uniqueness_constraint_violation("VDI", "uuid", > > "32719160-52d8-4e45-9682-7ab9ada4dad1")']] > > at com.xensource.xenapi.Types.checkResponse(Types.java:2006) > > at com.xensource.xenapi.Connection.dispatch(Connection.java:350) > > at > > > > com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerC > > on > > nection.dispatch(XenServerConnectionPool.java:456) > > at com.xensource.xenapi.SR.scan(SR.java:1051) > > at > > > > com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.creat > > eF > > ileSR(Xenserver625StorageProcessor.java:107) > > at > > > > com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.creat > > eF > > ileSr(Xenserver625StorageProcessor.java:133) > > at > > > > com.cloud.hypervisor.xen.resource.Xenserver625StorageProcessor.copyT > > em > > plateToPrimaryStorage(Xenserver625StorageProcessor.java:167) > > at > > > > com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execut > > e( > > StorageSubsystemCommandHandlerBase.java:77) > > at > > > > com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handle > > St > > orageCommands(StorageSubsystemCommandHandlerBase.java:52) > > at > > > > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest( > > Ci > > trixResourceBase.java:547) > > at > > > > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest > > (X > > enServer56Resource.java:61) > > at > > > > com.cloud.hypervisor.xen.resource.XenServer610Resource.executeReques > > t( > > XenServer610Resource.java:102) > > at > > > > com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeReq > > ue > > st(XenServer620SP1Resource.java:65) > > at > > > > com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectA > > ge > > ntAttache.java:307) > > at > > > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(M > > an > > agedContextRunnable.java:49) > > at > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.c > > al > > l(DefaultManagedContext.java:56) > > at > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.cal > > lW > > ithContext(DefaultManagedContext.java:103) > > at > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.run > > Wi > > thContext(DefaultManagedContext.java:53) > > at > > > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Man > > ag > > edContextRunnable.java:46) at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:4 > > 71 > > ) at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > at > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask > > .a > > ccess$201(ScheduledThreadPoolExecutor.java:178) > > at > > > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask > > .r > > un(ScheduledThreadPoolExecutor.java:292) > > at > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor > > .j > > ava:1145) > > at > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. > > java:615) at java.lang.Thread.run(Thread.java:744) > > > > > > On Sat, Apr 4, 2015 at 5:21 PM, Sonali Jadhav > > <[email protected]> > > wrote: > > > > > Hi, > > > > > > I have created windows 2012 r2 template. And I am trying to create > > > instance from it. But its failing, I am not able to understand cause. > > > > > > Here is my log file, > > > https://www.dropbox.com/s/4p4pskiy954e697/mgmtTemplateError.log?dl > > > =0 > > > > > > I tried lot of time, somewhere it also said that insufficient > > > capacity on host. Which I would say not true at all. I got 4 > > > xenservers in one > > cluster. > > > > > > Anyone can help me understand issue? > > > > > > /Sonali > > > > > >
