Hi, I just tested adding ceph pool on alma9 (it has the same package/version installed), it worked
What's the ceph version ? -Wei On Wed, Aug 7, 2024 at 12:11 PM Muhammad Hanis Irfan Mohd Zaid <[email protected]> wrote: > > I also noticed that comma in the end. I just took a blind eye and expected it > to be as designed. I don't enter any comma in the end: https://ibb.co/N3zMVvc > > Yep, the package is already installed. > > # dnf -y install libvirt-daemon-driver-storage-rbd > Last metadata expiration check: 1:29:57 ago on Wed 07 Aug 2024 04:37:02 PM > +08. > Package libvirt-daemon-driver-storage-rbd-10.0.0-6.6.el9_4.x86_64 is already > installed. > Dependencies resolved. > Nothing to do. > Complete! > > I'm running the KVM host in Rocky Linux 9.4 (Blue Onyx) with CloudStack > 4.19.1.1 > > > > On Wed, 7 Aug 2024 at 18:02, Wei ZHOU <[email protected]> wrote: >> >> Hi, >> >> There is a comma (,) after 10.0.32.75 , was it a mistake ? >> >> org.libvirt.LibvirtException: failed to connect to the RADOS monitor >> on: 10.0.32.71,10.0.32.72,10.0.32.73,10.0.32.74,10.0.32.75,: No such file >> or directory >> >> Have you installed the package "libvirt-daemon-driver-storage-rbd" on >> the kvm host ? >> >> -Wei >> >> On Wed, Aug 7, 2024 at 11:27 AM Muhammad Hanis Irfan Mohd Zaid >> <[email protected]> wrote: >> > >> > I'm trying to add a Ceph RBD pool for primary storage use. I've 5 Ceph MONs >> > in my POC lab. Ping and telnet to all the Ceph MONs with port 6789 works. >> > >> > I'm following the steps from this: >> > - https://docs.ceph.com/en/reef/rbd/rbd-cloudstack/ >> > - https://rohityadav.cloud/blog/ceph/ >> > >> > Agent log when specifying 5 monitors: >> > 2024-08-07 17:12:34,691 INFO [kvm.storage.LibvirtStorageAdaptor] >> > (agentRequest-Handler-3:null) (logid:db5277f2) Attempting to create storage >> > pool eb5ec036-c08a-3d3d-996d-40968077d391 (RBD) in libvirt >> > 2024-08-07 17:12:34,706 WARN [kvm.storage.LibvirtStorageAdaptor] >> > (agentRequest-Handler-3:null) (logid:db5277f2) Storage pool >> > eb5ec036-c08a-3d3d-996d-40968077d391 was not found running in libvirt. Need >> > to create it. >> > 2024-08-07 17:12:34,706 INFO [kvm.storage.LibvirtStorageAdaptor] >> > (agentRequest-Handler-3:null) (logid:db5277f2) Didn't find an existing >> > storage pool eb5ec036-c08a-3d3d-996d-40968077d391 by UUID, checking for >> > pools with duplicate paths >> > 2024-08-07 17:17:34,738 ERROR [kvm.storage.LibvirtStorageAdaptor] >> > (agentRequest-Handler-3:null) (logid:db5277f2) Failed to create RBD storage >> > pool: org.libvirt.LibvirtException: failed to connect to the RADOS monitor >> > on: 10.0.32.71,10.0.32.72,10.0.32.73,10.0.32.74,10.0.32.75,: No such file >> > or directory >> > 2024-08-07 17:17:34,739 ERROR [kvm.storage.LibvirtStorageAdaptor] >> > (agentRequest-Handler-3:null) (logid:db5277f2) Failed to create the RBD >> > storage pool, cleaning up the libvirt secret >> > 2024-08-07 17:17:34,739 WARN [cloud.agent.Agent] >> > (agentRequest-Handler-3:null) (logid:db5277f2) Caught: >> > com.cloud.utils.exception.CloudRuntimeException: Failed to create storage >> > pool: eb5ec036-c08a-3d3d-996d-40968077d391 >> > at >> > com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:743) >> > at >> > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:364) >> > at >> > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:358) >> > at >> > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:42) >> > at >> > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:35) >> > at >> > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) >> > at >> > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1929) >> > at com.cloud.agent.Agent.processRequest(Agent.java:683) >> > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1106) >> > at com.cloud.utils.nio.Task.call(Task.java:83) >> > at com.cloud.utils.nio.Task.call(Task.java:29) >> > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) >> > at >> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) >> > at >> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) >> > at java.base/java.lang.Thread.run(Thread.java:829) >> > >> > Agent log when specifying 1 monitors: >> > 2024-08-07 17:06:09,791 INFO [kvm.storage.LibvirtStorageAdaptor] >> > (agentRequest-Handler-2:null) (logid:c790784b) Attempting to create storage >> > pool 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 (RBD) in libvirt >> > 2024-08-07 17:06:09,806 WARN [kvm.storage.LibvirtStorageAdaptor] >> > (agentRequest-Handler-2:null) (logid:c790784b) Storage pool >> > 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 was not found running in libvirt. Need >> > to create it. >> > 2024-08-07 17:06:09,806 INFO [kvm.storage.LibvirtStorageAdaptor] >> > (agentRequest-Handler-2:null) (logid:c790784b) Didn't find an existing >> > storage pool 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 by UUID, checking for >> > pools with duplicate paths >> > 2024-08-07 17:11:09,840 ERROR [kvm.storage.LibvirtStorageAdaptor] >> > (agentRequest-Handler-2:null) (logid:c790784b) Failed to create RBD storage >> > pool: org.libvirt.LibvirtException: failed to connect to the RADOS monitor >> > on: 10.0.32.71,: No such file or directory >> > 2024-08-07 17:11:09,840 ERROR [kvm.storage.LibvirtStorageAdaptor] >> > (agentRequest-Handler-2:null) (logid:c790784b) Failed to create the RBD >> > storage pool, cleaning up the libvirt secret >> > 2024-08-07 17:11:09,841 WARN [cloud.agent.Agent] >> > (agentRequest-Handler-2:null) (logid:c790784b) Caught: >> > com.cloud.utils.exception.CloudRuntimeException: Failed to create storage >> > pool: 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 >> > at >> > com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:743) >> > at >> > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:364) >> > at >> > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:358) >> > at >> > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:42) >> > at >> > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:35) >> > at >> > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) >> > at >> > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1929) >> > at com.cloud.agent.Agent.processRequest(Agent.java:683) >> > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1106) >> > at com.cloud.utils.nio.Task.call(Task.java:83) >> > at com.cloud.utils.nio.Task.call(Task.java:29) >> > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) >> > at >> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) >> > at >> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) >> > at java.base/java.lang.Thread.run(Thread.java:829) >> > >> > Ran these commands on my Ceph node but still no luck. I refer this: >> > - https://github.com/apache/cloudstack/issues/5741 >> > >> > ceph config set mon auth_expose_insecure_global_id_reclaim false >> > ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false >> > ceph config set mon auth_allow_insecure_global_id_reclaim false >> > ceph orch restart mon >> > >> > Thanks for the help :)
