Richard, The Cloudstack-agent should populate the libvirt pool-list when it starts up. Have you tried restarting libvirtd and then restarting the Cloudstack-agent?
You may want to turn up debugging on the agent so you get some more detail on what's going on. You can do this by modifying /etc/cloudstack/agent/log4j-cloud.xml See this wiki article for more details: https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug - Si ________________________________________ From: Richard Klein (RSI) <[email protected]> Sent: Friday, April 15, 2016 6:54 PM To: [email protected] Subject: Primary storage not mounted on hosts? I am not sure what happened but our primary storage, which is Gluster, on all our hosts is not mounted anymore. When I do "virsh pool-list" on any host I only see the local pool. Gluster is working fine and there are no problems with it because I can mount the Gluster volume manually on any of the hosts and see the primary storage. Instances that are running can write data to the local volume and pull data from it. But if a VM is stopped it can't start again. I get the "Unable to create a New VM - Error message: Unable to start instance due to Unable to get answer that is of class com.cloud.agent.api.StartAnswer" that I have seen a thread in this mailing list and I am sure its primary storage related. The agent logs on the hosts are issuing the following log snippets which confirm its looking for primary storage: 2016-04-15 18:42:34,838 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:ad8ec05a) Trying to fetch storage pool c3991ea2-b702-3b1b-bfc5-69cb7d928554 from libvirt 2016-04-15 18:45:19,006 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:4c396753) Trying to fetch storage pool c3991ea2-b702-3b1b-bfc5-69cb7d928554 from libvirt 2016-04-15 18:45:49,010 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:4c396753) Trying to fetch storage pool c3991ea2-b702-3b1b-bfc5-69cb7d928554 from libvirt The c3991ea2-b702-3b1b-bfc5-69cb7d928554 is the UUID of our primary storage. We did have some secondary storage issues (NFS) that caused some NFS mounts to secondary storage to hang. The only way to recover was to reboot the host. There were 2 host affected so I put each host in maintenance mode, rebooted and then canceled maintenance mode. I did this one host at a time. It seems like ever since this has happened I have had issues. Is there a way to get the primary storage remounted and added to libvirt pool-list while keeping the VMs up and running? At this point the only idea I have to recover is to power off all VMs, disable primary storage then enable it again. This is a little extreme and is a last resort but I don't know what other options I have. Any suggestions? Richard Klein <[email protected]> RSI 5426 Guadalupe, Suite 100 Austin TX 78751
