I think that would only happen if pg_num for a 3R pool were less than like ~ 1/3 the number of OSDs. Assuming aligned device classes, proper CRUSH rules and topology etc.
Mind you if pg_num is low, the balancer won’t be able to do a great job of distributing data uniformly. If you set pg_num to a non power of two that complicates things as well since you’d have PGs of very different sizes, but this is something rarely seen. Sharing `ceph osd tree` and `ceph osd dump | grep pool` and `ceph osd df` would help. > On Feb 13, 2025, at 1:11 PM, Work Ceph <work.ceph.user.mail...@gmail.com> > wrote: > > Exactly, that is what I am assuming. However, my question is. Can I assume > that the PG number will affect the Max available space that a pool will be > able to use? > > On Thu, Feb 13, 2025 at 3:09 PM Anthony D'Atri <anthony.da...@gmail.com > <mailto:anthony.da...@gmail.com>> wrote: >> Assuming that the pool is replicated, 512 PGs is pretty low if this is the >> only substantial pool on the cluster. When you do `ceph osd df`, if this is >> the only substantial pool, the PGS column at right would average around 12 >> or 13 which is suuuuper low. >> >>> On Feb 13, 2025, at 11:40 AM, Work Ceph <work.ceph.user.mail...@gmail.com >>> <mailto:work.ceph.user.mail...@gmail.com>> wrote: >>> >>> Yes, the bucket that represents the new host is under the ROOT bucket as >>> the others. Also, the OSDs are in the right/expected bucket. >>> >>> I am guessing that the problem is the number of PGs. I have 120 OSDs across >>> all hosts, and I guess that 512 PGS, which is what the pool is using, is >>> not enough. I did not change it yet, because I wanted to understand the >>> effect on PG number in Ceph pool usable volume. >>> >>> On Thu, Feb 13, 2025 at 12:03 PM Anthony D'Atri <anthony.da...@gmail.com >>> <mailto:anthony.da...@gmail.com>> wrote: >>>> Does the new host show up under the proper CRUSH bucket? Do its OSDs? >>>> Send `ceph osd tree` please. >>>> >>>> >>>> >> >>>> >> >>>> >> > Hello guys, >>>> >> > Let's say I have a cluster with 4 nodes with 24 SSDs each, and a >>>> >> single >>>> >> > pool that consumes all OSDs of all nodes. After adding another >>>> >> host, I >>>> >> > noticed that no extra space was added. Can this be a result of >>>> >> the >>>> >> > number >>>> >> > of PGs I am using? >>>> >> > >>>> >> > I mean, when adding more hosts/OSDs, should I always consider >>>> >> increasing >>>> >> > the number of PGs from a pool? >>>> >> > >>>> >> >>>> >> ceph osd tree >>>> >> >>>> >> shows all up and with correct weight? >>>> >> >>>> > >>>> > _______________________________________________ >>>> > ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> >>>> > To unsubscribe send an email to ceph-users-le...@ceph.io >>>> > <mailto:ceph-users-le...@ceph.io> >>>> >> _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
