Hi All - I did some investigation into Ceph RDMA as part of a performance analysis project working with Ceph over Omnipath and NVME.
I wrote up some of the analysis here: https://www.stackhpc.com/ceph-on-the-brain-a-year-with-the-human-brain-project.html <https://www.stackhpc.com/ceph-on-the-brain-a-year-with-the-human-brain-project.html> My conclusion at the time was that Ceph’s RDMA support was not portable across different RDMA-capable network fabrics, but that RoCE worked pretty well. Unfortunately, on the hardware I had available for RoCE testing the network was not the bottleneck, so I didn’t see any compelling advantage. It would be great to do this testing again on a system with the potential to really shine. This work concluded about a year ago, so might be a little out of date. Best wishes, Stig > On 15 Oct 2019, at 13:46, Paul Emmerich <paul.emmer...@croit.io> wrote: > > That's apply/commit latency (the exact same since BlueStore btw, no > point in tracking both). It should not contain any network component. > > Since the path you are optimizing is inter-OSD communication: check > out subop latency, that's the one where this should show up. > > > Paul > > -- > Paul Emmerich > > Looking for help with your Ceph cluster? Contact us at https://croit.io > > croit GmbH > Freseniusstr. 31h > 81247 München > www.croit.io > Tel: +49 89 1896585 90 > > On Tue, Oct 15, 2019 at 2:39 PM <vita...@yourcmc.ru> wrote: >> >> I don't see any changes here... >> >>> There is graph here. It was pure Nautilus before 10-05 and >>> Nautilus+RDMA after. >>> https://nc.avalon.org.ua/s/LptPTEaTeTTyKtD >>> Link expires on Nov 1. >> _______________________________________________ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-le...@ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
