Hello,
We’ll be going into production with our Ceph cluster shortly and I’m looking
for some advice on the number of PGs per pool we should be using.
We have 216 OSDs totalling 588TB of storage. We’re intending on having several
pools, with not all pools sharing the same replica count - so, some pools will
not have any replicas, others may have one or two replicas depending on need.
It’s not currently possible to reliably estimate which pools will see the most
traffic due to our use case.
Looking through the CERN slides from the London Ceph day[1], they seem to be
using 1-4k PGs per pool over 11 pools (~19500PGs total). It’s not entirely
clear how many OSDs they’ve got, but if they’re running 1 OSD per disk it’ll be
over a thousand. That PG count seems a little low to me, but there are smarter
people than me at CERN so I’m willing to accept it!
For our system, using the formula from the documentation, a single pool spread
over 216 OSDs would require 10800 PGs for a ’size’ of 2. Does 1-4k PGs sound
like a more reasonable number for 200+ OSDs with around 6 pools? Would you need
more PGs for a lower replica count (e.g. 4-6k PGs for a ‘size’ of 0 [2]) or
would it not matter?
Thanks
Dane
[1] - http://www.slideshare.net/Inktank_Ceph/cern-ceph-day-london-2013
[2] - I realise the dangers/stupidity of a replica size of 0, but some of the
data we wish to store just isn’t /that/ important.
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
