Hi, There is a comma (,) after 10.0.32.75 , was it a mistake ?
org.libvirt.LibvirtException: failed to connect to the RADOS monitor on: 10.0.32.71,10.0.32.72,10.0.32.73,10.0.32.74,10.0.32.75,: No such file or directory Have you installed the package "libvirt-daemon-driver-storage-rbd" on the kvm host ? -Wei On Wed, Aug 7, 2024 at 11:27 AM Muhammad Hanis Irfan Mohd Zaid <[email protected]> wrote: > > I'm trying to add a Ceph RBD pool for primary storage use. I've 5 Ceph MONs > in my POC lab. Ping and telnet to all the Ceph MONs with port 6789 works. > > I'm following the steps from this: > - https://docs.ceph.com/en/reef/rbd/rbd-cloudstack/ > - https://rohityadav.cloud/blog/ceph/ > > Agent log when specifying 5 monitors: > 2024-08-07 17:12:34,691 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-3:null) (logid:db5277f2) Attempting to create storage > pool eb5ec036-c08a-3d3d-996d-40968077d391 (RBD) in libvirt > 2024-08-07 17:12:34,706 WARN [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-3:null) (logid:db5277f2) Storage pool > eb5ec036-c08a-3d3d-996d-40968077d391 was not found running in libvirt. Need > to create it. > 2024-08-07 17:12:34,706 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-3:null) (logid:db5277f2) Didn't find an existing > storage pool eb5ec036-c08a-3d3d-996d-40968077d391 by UUID, checking for > pools with duplicate paths > 2024-08-07 17:17:34,738 ERROR [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-3:null) (logid:db5277f2) Failed to create RBD storage > pool: org.libvirt.LibvirtException: failed to connect to the RADOS monitor > on: 10.0.32.71,10.0.32.72,10.0.32.73,10.0.32.74,10.0.32.75,: No such file > or directory > 2024-08-07 17:17:34,739 ERROR [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-3:null) (logid:db5277f2) Failed to create the RBD > storage pool, cleaning up the libvirt secret > 2024-08-07 17:17:34,739 WARN [cloud.agent.Agent] > (agentRequest-Handler-3:null) (logid:db5277f2) Caught: > com.cloud.utils.exception.CloudRuntimeException: Failed to create storage > pool: eb5ec036-c08a-3d3d-996d-40968077d391 > at > com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:743) > at > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:364) > at > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:358) > at > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:42) > at > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:35) > at > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > at > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1929) > at com.cloud.agent.Agent.processRequest(Agent.java:683) > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1106) > at com.cloud.utils.nio.Task.call(Task.java:83) > at com.cloud.utils.nio.Task.call(Task.java:29) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:829) > > Agent log when specifying 1 monitors: > 2024-08-07 17:06:09,791 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-2:null) (logid:c790784b) Attempting to create storage > pool 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 (RBD) in libvirt > 2024-08-07 17:06:09,806 WARN [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-2:null) (logid:c790784b) Storage pool > 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 was not found running in libvirt. Need > to create it. > 2024-08-07 17:06:09,806 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-2:null) (logid:c790784b) Didn't find an existing > storage pool 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 by UUID, checking for > pools with duplicate paths > 2024-08-07 17:11:09,840 ERROR [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-2:null) (logid:c790784b) Failed to create RBD storage > pool: org.libvirt.LibvirtException: failed to connect to the RADOS monitor > on: 10.0.32.71,: No such file or directory > 2024-08-07 17:11:09,840 ERROR [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-2:null) (logid:c790784b) Failed to create the RBD > storage pool, cleaning up the libvirt secret > 2024-08-07 17:11:09,841 WARN [cloud.agent.Agent] > (agentRequest-Handler-2:null) (logid:c790784b) Caught: > com.cloud.utils.exception.CloudRuntimeException: Failed to create storage > pool: 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 > at > com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:743) > at > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:364) > at > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:358) > at > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:42) > at > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:35) > at > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > at > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1929) > at com.cloud.agent.Agent.processRequest(Agent.java:683) > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1106) > at com.cloud.utils.nio.Task.call(Task.java:83) > at com.cloud.utils.nio.Task.call(Task.java:29) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:829) > > Ran these commands on my Ceph node but still no luck. I refer this: > - https://github.com/apache/cloudstack/issues/5741 > > ceph config set mon auth_expose_insecure_global_id_reclaim false > ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false > ceph config set mon auth_allow_insecure_global_id_reclaim false > ceph orch restart mon > > Thanks for the help :)
