OH.. you can refer to performance related threads on ceph/ceph-devel maillist to get ssd-optimized ceph.conf. the default conf lack of good support on ssd.
On Fri, Mar 24, 2017 at 6:33 AM, Hung-Wei Chiu (邱宏瑋) <hwc...@cs.nctu.edu.tw> wrote: > Hi. > > Basic > [global] > > fsid = 0612cc7e-6239-456c-978b-b4df781fe831 > mon initial members = ceph-1,ceph-2,ceph-3 > mon host = 10.0.0.15,10.0.0.16,10.0.0.17 > osd pool default size = 2 > osd pool default pg num = 1024 > osd pool default pgp num = 1024 > > > RDMA > [global] > > fsid = 0612cc7e-6239-456c-978b-b4df781fe831 > mon initial members = ceph-1,ceph-2,ceph-3 > mon host = 10.0.0.15,10.0.0.16,10.0.0.17 > osd pool default size = 2 > osd pool default pg num = 1024 > osd pool default pgp num = 1024 > ms_type=async+rdma > ms_async_rdma_device_name = mlx4_0 > > > Thanks. > > Best Regards, > > Hung-Wei Chiu(邱宏瑋) > -- > Computer Center, Department of Computer Science > National Chiao Tung University > > 2017-03-24 18:28 GMT+08:00 Haomai Wang <hao...@xsky.com>: > >> the content of ceph.conf ? >> >> On Fri, Mar 24, 2017 at 4:32 AM, Hung-Wei Chiu (邱宏瑋) < >> hwc...@cs.nctu.edu.tw> wrote: >> >>> Hi Deepak. >>> >>> Thansk your reply, >>> >>> I try to use gperf to profile the ceph-osd with basic mode (without >>> RDMA) and you can see the result in the following link. >>> http://imgur.com/a/SJgEL >>> >>> In the gperf result, we can see the whole CPU usage can divided into >>> three significant part, (Network: 36%, FileStore: 17% PG related:29%). >>> Do you have any idea about this? >>> >>> Thanks. >>> >>> >>> Best Regards, >>> >>> Hung-Wei Chiu(邱宏瑋) >>> -- >>> Computer Center, Department of Computer Science >>> National Chiao Tung University >>> >>> 2017-03-24 14:31 GMT+08:00 Deepak Naidu <dna...@nvidia.com>: >>> >>>> >>Now, the kernel can decrease the CPU cycle usages for network I/O >>>> processing because of RDMA enabled (right?). >>>> >>>> Yes, kernel should have comparatively free cycles when using RDMA over >>>> TCP >>>> >>>> >>does it means host can provide more CPU for other processing, such as >>>> Disk I/O ? >>>> >>>> This can be subjective, bcos let's say u have an IO process which takes >>>> 10 mins over TCP then when using RDMA it might be 3 mins as your CPU cycles >>>> are less used by RDMA but if ur application process which provides the RDMA >>>> functionality might have processing cycles as well, which might use CPU >>>> cycles as well, example client or fuse plugins on the host getting it ? >>>> >>>> >>>> -- >>>> Deepak >>>> >>>> > On Mar 23, 2017, at 11:10 PM, Hung-Wei Chiu (邱宏瑋) < >>>> hwc...@cs.nctu.edu.tw> wrote: >>>> > >>>> > Now, the kernel can decrease the CPU cycle usages for network I/O >>>> processing because of RDMA enabled (right?). >>>> > does it means host can provide more CPU for other processing, such as >>>> Disk I/O ? >>>> >>>> ------------------------------------------------------------ >>>> ----------------------- >>>> This email message is for the sole use of the intended recipient(s) and >>>> may contain >>>> confidential information. Any unauthorized review, use, disclosure or >>>> distribution >>>> is prohibited. If you are not the intended recipient, please contact >>>> the sender by >>>> reply email and destroy all copies of the original message. >>>> ------------------------------------------------------------ >>>> ----------------------- >>>> >>> >>> >> >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
