Hi, Thanks for your response,
I turned on debug logging but nothing new has been discovered. after upgrading from 2.2.14 to 3.0.2 all KVM server with Local Storage behave identically - Unable to re-start the instance ( instance can't be started anymore ) - Make a snapshot for volume. - Can't download the volume For now, back to online such instances I'm can by the following way - import volume of stopped instance as template and the start new instance from Log : libvir: Storage error : Storage volume not found: no storage vol with matching name [root@ /]# virsh vol-list c6f1f032-5aad-4983-bc83-0ec1a6b5e32c | grep 4341f531-bef6-47b6-ae75-2b8d6de63038 4341f531-bef6-47b6-ae75-2b8d6de63038 /var/lib/libvirt/images/4341f531-bef6-47b6-ae75-2b8d6de63038 [root@ /]# qemu-img check /var/lib/libvirt/images/4341f531-bef6-47b6-ae75-2b8d6de63038 No errors were found on the image. Full log output : Log output : http://pastebin.com/dd0Wz22a -- Serg On Jun 4, 2012, at 8:33 PM, Edison Su wrote: > Please turn log level to debug(/etc/cloud/agent/log4j-cloud.xml) on your kvm > host, then restart cloud-agent, take a look at agent log again. > >> -----Original Message----- >> From: Serg [mailto:kernc...@gmail.com] >> Sent: Sunday, June 03, 2012 11:48 AM >> To: cloudstack-dev@incubator.apache.org >> Subject: 3.0.2 - KVM Agent problem? Please help. >> >> Hi, >> >> >> How to debug such problem ? >> All volumes exist and instances running, but can't to >> >> CentOS release 6.2 (Final) >> libvirt-0.9.4-23.el6_2.4.x86_64 >> libvirt-client-0.9.4-23.el6_2.4.x86_64 >> >> >> [root@kvmhost2 snapshot]# tail -10 /var/log/cloud/agent/agent.log >> com.cloud.utils.exception.CloudRuntimeException: >> org.libvirt.LibvirtException: Storage volume not found: no storage vol >> with matching name '/var/lib/libvirt/images/67033fb9-a432-4162-8b22- >> d8a7b2aeb100' >> at >> com.cloud.agent.storage.LibvirtStorageAdaptor.getVolume(LibvirtStorageA >> daptor.java:81) >> ...cut... >> libvir: Storage error : Storage volume not found: no storage vol with >> matching name '/opt/primary/f5b97a37-f8dd-485c-945c-da31d20b9e23' >> ...cut... >> libvir: Storage error : Storage volume not found: no storage vol with >> matching name '/var/lib/libvirt/images/58ba9eed-c864-44f3-a987- >> 9d1eaab182e1' >> >> [root@kvmhost2 snapshot]# tail -10 /var/log/cloud/agent/agent.log | awk >> '{print $15}' | xargs file >> /opt/primary/e371ebb7-8d9e-4c64-83e9-effea15a00e7: Qemu >> Image, Format: Qcow , Version: 2 >> /opt/primary/e371ebb7-8d9e-4c64-83e9-effea15a00e7: Qemu >> Image, Format: Qcow , Version: 2 >> /opt/primary/e55358fa-8dc5-4070-b3b6-3f4bc01b17ae: Qemu >> Image, Format: Qcow , Version: 2 >> /opt/primary/e55358fa-8dc5-4070-b3b6-3f4bc01b17ae: Qemu >> Image, Format: Qcow , Version: 2 >> /var/lib/libvirt/images/d49b318c-8bb4-4bb2-a897-1c40ef234d30: Qemu >> Image, Format: Qcow , Version: 2 >> /var/lib/libvirt/images/d49b318c-8bb4-4bb2-a897-1c40ef234d30: Qemu >> Image, Format: Qcow , Version: 2 >> /var/lib/libvirt/images/193c378c-80ba-4b82-affe-45c9da0ee839: Qemu >> Image, Format: Qcow , Version: 2 >> /var/lib/libvirt/images/193c378c-80ba-4b82-affe-45c9da0ee839: Qemu >> Image, Format: Qcow , Version: 2 >> /var/lib/libvirt/images/58ba9eed-c864-44f3-a987-9d1eaab182e1: Qemu >> Image, Format: Qcow , Version: 2 >> /var/lib/libvirt/images/58ba9eed-c864-44f3-a987-9d1eaab182e1: Qemu >> Image, Format: Qcow , Version: 2 >> >> When I'm try to start instance through UI I'm get the following error : >> UI Error pop-up : Resource [Host:34] is unreachable: Host 34: Unable to >> start instance due to Unable to get answer that is of class >> com.cloud.agent.api.StartAnswer >> >> Agent LOG : >> >> 2012-06-03 20:47:13,397 WARN >> [resource.computing.LibvirtComputingResource] (agentRequest-Handler- >> 2:null) Exception >> com.cloud.utils.exception.CloudRuntimeException: >> org.libvirt.LibvirtException: Storage volume not found: no storage vol >> with matching name '/var/lib/libvirt/images/19d086dd-8229-48c2-aaa7- >> ea3d218dc5dc' >> at >> com.cloud.agent.storage.LibvirtStorageAdaptor.getVolume(LibvirtStorageA >> daptor.java:81) >> at >> com.cloud.agent.storage.LibvirtStorageAdaptor.getPhysicalDisk(LibvirtSt >> orageAdaptor.java:450) >> at >> com.cloud.agent.storage.LibvirtStoragePool.getPhysicalDisk(LibvirtStora >> gePool.java:104) >> at >> com.cloud.agent.resource.computing.LibvirtComputingResource.createVbd(L >> ibvirtComputingResource.java:2593) >> at >> com.cloud.agent.resource.computing.LibvirtComputingResource.execute(Lib >> virtComputingResource.java:2510) >> at >> com.cloud.agent.resource.computing.LibvirtComputingResource.executeRequ >> est(LibvirtComputingResource.java:985) >> at com.cloud.agent.Agent.processRequest(Agent.java:517) >> at >> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:827) >> at com.cloud.utils.nio.Task.run(Task.java:79) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja >> va:1110) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j >> ava:603) >> at java.lang.Thread.run(Thread.java:679) >> 2012-06-03 20:47:13,399{GMT} WARN [cloud.agent.Agent] (agentRequest- >> Handler-2:) Caught: >> java.lang.NullPointerException >> at >> com.cloud.agent.resource.computing.LibvirtComputingResource.cleanupVMNe >> tworks(LibvirtComputingResource.java:3664) >> at >> com.cloud.agent.resource.computing.LibvirtComputingResource.handleVmSta >> rtFailure(LibvirtComputingResource.java:2416) >> at >> com.cloud.agent.resource.computing.LibvirtComputingResource.execute(Lib >> virtComputingResource.java:2539) >> at >> com.cloud.agent.resource.computing.LibvirtComputingResource.executeRequ >> est(LibvirtComputingResource.java:985) >> at com.cloud.agent.Agent.processRequest(Agent.java:517) >> at >> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:827) >> at com.cloud.utils.nio.Task.run(Task.java:79) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja >> va:1110) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j >> ava:603) >> at java.lang.Thread.run(Thread.java:679) >> >> >> >> -- >> Serg >
