Hi, These are some of the ones that stand out in management-server.log , prior to InsufficientServerCapacityException.
DEBUG [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-58f2724e) Successfully transitioned to start state for VM[SecondaryStorageVm|s-4728-VM] reservation id = bd3f2fbd-0420-49a6 -b86e-d2c57c7c8204 DEBUG [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-58f2724e) Trying to deploy VM, vm has dcId: 1 and podId: null DEBUG [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-58f2724e) Deploy avoids pods: null, clusters: null, hosts: null I'm not really sure... =========================================================================================================== full log: 2016-02-05 16:57:58,274 DEBUG [c.c.s.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-58f2724e) Zone 1 is ready to launch secondary storage VM 2016-02-05 16:57:58,275 INFO [c.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:ctx-58f2724e) No running secondary storage vms found in datacenter id=1, starting one 2016-02-05 16:57:58,278 INFO [c.c.s.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-58f2724e) Found a stopped secondary storage vm, bring it up to running pool. secStorageVm vm id : 4728 2016-02-05 16:57:58,282 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e) VM state transitted from :Stopped to Starting with event: StartRequestedvm's original host id: null new host id: null h ost id before state transition: null 2016-02-05 16:57:58,282 DEBUG [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-58f2724e) Successfully transitioned to start state for VM[SecondaryStorageVm|s-4728-VM] reservation id = bd3f2fbd-0420-49a6 -b86e-d2c57c7c8204 2016-02-05 16:57:58,381 DEBUG [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-58f2724e) Trying to deploy VM, vm has dcId: 1 and podId: null 2016-02-05 16:57:58,381 DEBUG [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-58f2724e) Deploy avoids pods: null, clusters: null, hosts: null 2016-02-05 16:57:58,383 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) Deploy avoids pods: null, clusters: null, hosts: null 2016-02-05 16:57:58,383 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) DeploymentPlanner allocation algorithm: com.cloud.deploy.FirstFitPlanner@1a94871e 2016-02-05 16:57:58,383 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) Trying to allocate a host and storage pools from dc:1, pod:null,cluster:null, requested cpu: 500, requested r am: 536870912 2016-02-05 16:57:58,383 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) Is ROOT volume READY (pool already allocated)?: No 2016-02-05 16:57:58,383 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:ctx-58f2724e) Searching all possible resources under this Zone: 1 2016-02-05 16:57:58,383 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:ctx-58f2724e) Listing clusters in order of aggregate capacity, that have (atleast one host with) enough CPU and RAM capacity under this Z one: 1 2016-02-05 16:57:58,385 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) Checking resources in Cluster: 4 under Pod: 1 2016-02-05 16:57:58,385 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:4 2016-02-05 16:57:58,386 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) FirstFitAllocator has 1 hosts to check for allocation: [Host[-4-Routing]] 2016-02-05 16:57:58,387 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Found 1 hosts for allocation after prioritization: [Host[-4-Routing]] 2016-02-05 16:57:58,387 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Looking for speed=500Mhz, Ram=512 2016-02-05 16:57:58,390 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Host: 4 has cpu capability (cpu:8, speed:4000) to support requested CPU: 1 and requested speed : 500 2016-02-05 16:57:58,390 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Checking if host: 4 has enough capacity for requested CPU: 500 and requested RAM: 536870912 , cpuOverprovisioningFactor: 1.0 2016-02-05 16:57:58,391 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Hosts's actual total CPU: 32000 and CPU after applying overprovisioning: 32000 2016-02-05 16:57:58,391 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Free CPU: 32000 , Requested CPU: 500 2016-02-05 16:57:58,391 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Free RAM: 15694475264 , Requested RAM: 536870912 2016-02-05 16:57:58,391 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Host has enough CPU and RAM available 2016-02-05 16:57:58,391 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 4, used: 0, reserved: 0, actual total: 32000, total with overp rovisioning: 32000; requested cpu:500,alloc_from_last_host?:false ,considerReservedCapacity?: true 2016-02-05 16:57:58,391 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 4, used: 0, reserved: 0, total: 15694475264; requested mem: 53 6870912,alloc_from_last_host?:false ,considerReservedCapacity?: true 2016-02-05 16:57:58,391 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Found a suitable host, adding to list: 4 2016-02-05 16:57:58,391 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Host Allocator returning 1 suitable hosts 2016-02-05 16:57:58,392 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) Checking suitable pools for volume (Id, Type): (4728,ROOT) 2016-02-05 16:57:58,392 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) We need to allocate new storagepool for this volume 2016-02-05 16:57:58,392 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) Calling StoragePoolAllocators to find suitable pools 2016-02-05 16:57:58,392 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator] (secstorage-1:ctx-58f2724e) LocalStoragePoolAllocator trying to find storage pool to fit the vm 2016-02-05 16:57:58,393 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator] (secstorage-1:ctx-58f2724e) LocalStoragePoolAllocator returning 0 suitable storage pools 2016-02-05 16:57:58,393 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (secstorage-1:ctx-58f2724e) ClusterScopeStoragePoolAllocator looking for storage pool 2016-02-05 16:57:58,393 DEBUG [o.a.c.s.a.ZoneWideStoragePoolAllocator] (secstorage-1:ctx-58f2724e) ZoneWideStoragePoolAllocator to find storage pool 2016-02-05 16:57:58,393 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) No suitable pools found for volume: Vol[4728|vm=4728|ROOT] under cluster: 4 2016-02-05 16:57:58,393 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) No suitable pools found 2016-02-05 16:57:58,393 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) No suitable storagePools found under this Cluster: 4 2016-02-05 16:57:58,394 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) Checking resources in Cluster: 1 under Pod: 1 2016-02-05 16:57:58,394 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:1 2016-02-05 16:57:58,395 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) FirstFitAllocator has 2 hosts to check for allocation: [Host[-6-Routing], From management-server.log : Host[-3-Routing] ] 2016-02-05 16:57:58,396 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Found 2 hosts for allocation after prioritization: [Host[-6-Routing], Host[-3-Routing]] 2016-02-05 16:57:58,396 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Looking for speed=500Mhz, Ram=512 2016-02-05 16:57:58,398 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Host: 6 has cpu capability (cpu:8, speed:4000) to support requested CPU: 1 and requested speed: 500 2016-02-05 16:57:58,399 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Checking if host: 6 has enough capacity for requested CPU: 500 and requested RAM: 536870912 , cpuOverprovisioningFactor: 1.0 2016-02-05 16:57:58,400 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Hosts's actual total CPU: 32000 and CPU after applying overprovisioning: 32000 2016-02-05 16:57:58,400 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Free CPU: 32000 , Requested CPU: 500 2016-02-05 16:57:58,400 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Free RAM: 16768573440 , Requested RAM: 536870912From management-server.log : 2016-02-05 16:57:58,400 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Host has enough CPU and RAM available 2016-02-05 16:57:58,400 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 6, used: 0, reserved: 0, actual total: 32000, total with overprovisioning: 32000; requested cpu:500,alloc_from_last_host?:false ,considerReservedCapacity?: true 2016-02-05 16:57:58,400 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 6, used: 0, reserved: 0, total: 16768573440; requested mem: 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true 2016-02-05 16:57:58,400 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Found a suitable host, adding to list: 6 2016-02-05 16:57:58,402 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Host: 3 has cpu capability (cpu:8, speed:4000) to support requested CPU: 1 and requested speed: 500 2016-02-05 16:57:58,402 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Checking if host: 3 has enough capacity for requested CPU: 500 and requested RAM: 536870912 , cpuOverprovisioningFactor: 1.0 2016-02-05 16:57:58,403 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Hosts's actual total CPU: 32000 and CPU after applying overprovisioning: 32000 2016-02-05 16:57:58,403 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Free CPU: 31500 , Requested CPU: 500 2016-02-05 16:57:58,403 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Free RAM: 15694831616 , Requested RAM: 536870912 2016-02-05 16:57:58,403 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Host has enough CPU and RAM available 2016-02-05 16:57:58,403 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 3, used: 500, reserved: 0, actual total: 32000, total with overprovisioning: 32000; requested cpu:500,alloc_from_last_host?:false ,considerReservedCapacity?: true 2016-02-05 16:57:58,403 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 3, used: 1073741824, reserved: 0, total: 16768573440; requested mem: 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true 2016-02-05 16:57:58,403 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Found a suitable host, adding to list: 3 2016-02-05 16:57:58,403 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (secstorage-1:ctx-58f2724e FirstFitRoutingAllocator) Host Allocator returning 2 suitable hosts 2016-02-05 16:57:58,404 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) Checking suitable pools for volume (Id, Type): (4728,ROOT) 2016-02-05 16:57:58,404 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) We need to allocate new storagepool for this volume 2016-02-05 16:57:58,404 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) Calling StoragePoolAllocators to find suitable pools 2016-02-05 16:57:58,404 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator] (secstorage-1:ctx-58f2724e) LocalStoragePoolAllocator trying to find storage pool to fit the vm 2016-02-05 16:57:58,405 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator] (secstorage-1:ctx-58f2724e) LocalStoragePoolAllocator returning 0 suitable storage pools 2016-02-05 16:57:58,405 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (secstorage-1:ctx-58f2724e) ClusterScopeStoragePoolAllocator looking for storage pool 2016-02-05 16:57:58,405 DEBUG [o.a.c.s.a.ZoneWideStoragePoolAllocator] (secstorage-1:ctx-58f2724e) ZoneWideStoragePoolAllocator to find storage pool 2016-02-05 16:57:58,405 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) No suitable pools found for volume: Vol[4728|vm=4728|ROOT] under cluster: 1 2016-02-05 16:57:58,405 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) No suitable pools found 2016-02-05 16:57:58,405 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) No suitable storagePools found under this Cluster: 1 2016-02-05 16:57:58,405 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (secstorage-1:ctx-58f2724e) Could not find suitable Deployment Destination for this VM under any clusters, returning. 2016-02-05 16:57:58,405 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:ctx-58f2724e) Searching all possible resources under this Zone: 1 2016-02-05 16:57:58,406 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:ctx-58f2724e) Listing clusters in order of aggregate capacity, that have (atleast one host with) enough CPU and RAM capacity under this Zone: 1 2016-02-05 16:57:58,407 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:ctx-58f2724e) Removing from the clusterId list these clusters from avoid set: [1, 4] 2016-02-05 16:57:58,407 DEBUG [c.c.d.FirstFitPlanner] (secstorage-1:ctx-58f2724e) No clusters found after removing disabled clusters and clusters in avoid list, returning. 2016-02-05 16:57:58,530 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-43264e67) Zone 1 is ready to launch console proxy 2016-02-05 16:57:58,607 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-58f2724e) VM state transitted from :Starting to Stopped with event: OperationFailedvm's original host id: null new host id: null host id before state transition: null On Fri, Feb 5, 2016 at 4:38 PM Syafiq Rokman <[email protected]> wrote: > Hello, > Where can I find the full error log? > > On Fri, Feb 5, 2016 at 12:05 PM Suresh Sadhu <[email protected]> > wrote: > >> Hope you have enough resources also check the full error log, it might >> give you the reason for the failure.i.e just before >> InsufficientServerCapacityException. >> >> Regards >> saddhu >> -----Original Message----- >> From: Syafiq Rokman [mailto:[email protected]] >> Sent: Friday, February 5, 2016 9:25 AM >> To: [email protected] >> Subject: Unable to Start SystemVM for SSVM >> >> CS 4.3 Management Server on Ubuntu 14.04 >> >> Error 'Unable to start SystemVM' when I try to start the SSVM. >> >> management-server.log : >> >> >> WARN [c.c.s.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-8b7ec100) >> Exception while trying to start secondary storage vm >> com.cloud.exception.InsufficientServerCapacityException: >> >> Unable to create a deployment for >> VM[SecondaryStorageVm|s-4406-VM]Scope=interface com.cloud.dc.DataCenter; >> id=1 at >> >> com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:922) >> at >> >> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:761) >> at >> >> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:745) >> at >> >> com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:261) >> at >> >> com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:694) >> at >> >> com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1278) >> at >> >> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:123) >> at >> >> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:50) >> at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) >> at >> com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:35) >> at >> com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:88) >> at >> >> com.cloud.vm.SystemVmLoadScanner$1.runInContext(SystemVmLoadScanner.java:79) >> at >> >> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) >> at >> >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) >> at >> >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) >> at >> >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) >> at >> >> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) >> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at >> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) >> at >> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >> at >> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> >> INFO [c.c.s.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-8b7ec100) >> Unable to start secondary storage vm for standby capacity, secStorageVm vm >> Id : 4406, will recycle it and star t a new one INFO >> [c.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:ctx-8b7ec100) >> Primary secondary storage is not even started, wait until next turn >> >
