Hi Craig, Are you keeping the filestore, disk and op threads at their default values? or did you also change them?
Cheers > Tuning these values depends on a lot more than just the SSDs and > HDDs. Which kernel and IO scheduler are you using? Does your HBA do > write caching? It also depends on what your goals are. Tuning for a > RadosGW cluster is different that for a RDB cluster. The short > answer is that you are the only person that can can tell you what > your optimal values are. As always, the best benchmark is production > load. > In my small cluster (5 nodes, 44 osds), I'm optimizing to minimize > latency during recovery. When the cluster is healthy, bandwidth and > latency are more than adequate for my needs. Even with journals on > SSDs, I've found that reducing the number of operations and threads > has reduced my average latency. > I use injectargs to try out new values while I monitor cluster > latency. I monitor latency while the cluster is healthy and > recovering. If a change is deemed better, only then will I persist > the change to ceph.conf. This gives me a fallback that any changes > that causes massive problems can be undone with a restart or reboot. > So far, the configs that I've written to ceph.conf are > [global] > mon osd down out interval = 900 > mon osd min down reporters = 9 > mon osd min down reports = 12 > osd pool default flag hashpspool = true > [osd] > osd max backfills = 1 > osd recovery max active = 1 > osd recovery op priority = 1 > I have it on my list to investigate filestore max sync interval . And > now that I've pasted that, I need to revisit the min down > reports/reporters. I have some nodes with 10 OSDs, and I don't want > any one node able to mark the rest of the cluster as down (it > happened once). > On Sat, Nov 22, 2014 at 6:24 AM, Andrei Mikhailovsky < > and...@arhont.com > wrote: > > Hello guys, > > > Could some one comment on the optimal or recommended values of > > various threads values in ceph.conf? > > > At the moment I have the following settings: > > > filestore_op_threads = 8 > > > osd_disk_threads = 8 > > > osd_op_threads = 8 > > > filestore_merge_threshold = 40 > > > filestore_split_multiple = 8 > > > Are these reasonable for a small cluster made of 7.2K SAS disks > > with > > ssd journals with a ratio of 4:1? > > > What are the settings that other people are using? > > > Thanks > > > Andrei > > > _______________________________________________ > > > ceph-users mailing list > > > ceph-users@lists.ceph.com > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
