Could you post the complete log output from the point where you press launch? ie. start a tail the second right before you do, that would help us figure out why hosts are put in avoid set.
-- Erik On Tue, Aug 5, 2014 at 1:02 PM, Martin Emrich <[email protected]> wrote: > Hi Dan, > > It is only happening with instances created from this specific template. > > Also, sometimes creating an instance does not work at all, with various > errors. I just tried to recreate the template, and when I try to create an > instance from it, I get (after a minute or so): > > Job failed due to exception Unable to create a deployment for > VM[User|i-2-60-VM] > > In the management-server.log, I get: > > com.cloud.exception.InsufficientServerCapacityException: Unable to create > a deployment for VM[User|i-2-60-VM]Scope=interface > com.cloud.dc.DataCenter; id=1 > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( > VirtualMachineManagerImpl.java:946) > at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart( > VirtualMachineManagerImpl.java:5190) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob( > VmWorkJobHandlerProxy.java:107) > at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob( > VirtualMachineManagerImpl.java:5335) > at com.cloud.vm.VmWorkJobDispatcher.runJob( > VmWorkJobDispatcher.java:102) > at org.apache.cloudstack.framework.jobs.impl. > AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) > at org.apache.cloudstack.managed.context. > ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at org.apache.cloudstack.managed.context.impl. > DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at org.apache.cloudstack.managed.context.impl. > DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at org.apache.cloudstack.managed.context.impl. > DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at org.apache.cloudstack.managed.context. > ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at org.apache.cloudstack.framework.jobs.impl. > AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) > at java.util.concurrent.Executors$RunnableAdapter. > call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > Earlier, I see > > 2014-08-05 12:53:57,646 ERROR [c.c.v.VmWorkJobHandlerProxy] > (Work-Job-Executor-36:ctx-ca0b66c4 job-633/job-635 ctx-0df40702) > Invocation exception, caused by: com.cloud.exception. > InsufficientServerCapacityException: Unable to create a dep > loyment for VM[User|i-2-60-VM]Scope=interface com.cloud.dc.DataCenter; > id=1 > > but there are lots of ressources available. > > Scrolling further up, I see: > > 2014-08-05 12:53:57,095 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-36:ctx-ca0b66c4 job-633/job-635 ctx-0df40702 > FirstFitRoutingAllocator) FirstFitAllocator has 2 hosts to check for > allocation: [Host[-5-Routing], Host[-6-Rou > ting]] > 2014-08-05 12:53:57,098 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-36:ctx-ca0b66c4 job-633/job-635 ctx-0df40702 > FirstFitRoutingAllocator) Found 2 hosts for allocation after > prioritization: [Host[-5-Routing], Host[-6-Routing > ]] > 2014-08-05 12:53:57,098 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-36:ctx-ca0b66c4 job-633/job-635 ctx-0df40702 > FirstFitRoutingAllocator) Looking for speed=1000Mhz, Ram=1024 > 2014-08-05 12:53:57,098 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-36:ctx-ca0b66c4 job-633/job-635 ctx-0df40702 > FirstFitRoutingAllocator) Host name: dev-xen01, hostId: 5 is in avoid set, > skipping this and trying other a > vailable hosts > 2014-08-05 12:53:57,098 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-36:ctx-ca0b66c4 job-633/job-635 ctx-0df40702 > FirstFitRoutingAllocator) Host name: dev-xen02, hostId: 6 is in avoid set, > skipping this and trying other a > vailable hosts > 2014-08-05 12:53:57,098 DEBUG [c.c.a.m.a.i.FirstFitAllocator] > (Work-Job-Executor-36:ctx-ca0b66c4 job-633/job-635 ctx-0df40702 > FirstFitRoutingAllocator) Host Allocator returning 0 suitable hosts > > > so both hosts are on some "avoid" list... At some other attempts, I get > less specific errors. > > All in all, Cloudstack behaves "nondeterministic"... Sometimes an > operation fails, the next day, the same operation succeeds somewhat. > > I also wonder why many of these error log messages are classified as > "DEBUG", which makes it hard to find the source of the problem. Did I > overlook something? > > Thanks, > > Martin > > > Am 05.08.2014 09:39, schrieb Daan Hoogland: > > Martin, is the shutting down happening with the systemvms or guest vms or >> both? >> Do you see this also when you boot a vm of the same template in xen >> directly? >> >> regards, >> >> On Tue, Aug 5, 2014 at 8:37 AM, Martin Emrich <[email protected]> >> wrote: >> >>> Hi! >>> >>> Am 04.08.2014 21:01, schrieb Pierre-Luc Dion: >>> >>> Hi Martin, >>>> >>>> I might not be able to help you much on that, but did you look at >>>> XenCenter >>>> to see if you had something similar like this: >>>> https://issues.apache.org/jira/browse/CLOUDSTACK-7225 >>>> >>>> Basically, SystemVM paused instead of remaining started or perform a >>>> reboot. >>>> >>> >>> >>> Actually, I had this problem, too. After restarting (and possibly >>> upgrading) >>> the SystemVMs with cloudstack-sysvmadm several times, I got them all >>> running. >>> >>> All in all, this is quite "nondeterministic"... >>> >>> Regards, >>> >>> Martin >>> >> >> >> >>
