[ceph-users] MDS stuck at replaying status

2019-04-01 Thread Albert Yue
Hi, This happens after we restart the active MDS, and somehow the standby MDS daemon cannot take over successfully and is stuck at up:replaying. It is showing the following log. Any idea on how to fix this? 2019-04-02 12:54:00.985079 7f6f70670700 1 mds.WXS0023 respawn 2019-04-02 12:54:00.985095

[ceph-users] Best practice for increasing number of pg and pgp

2019-01-29 Thread Albert Yue
Dear Ceph Users, As the number of OSDs increase in our cluster, we reach a point where pg/osd is lower than recommend value and we want to increase it from 4096 to 8192. Somebody recommends that this adjustment should be done in multiple stages, e.g. increase 1024 pg each time. Is this a good pra

Re: [ceph-users] MDS performance issue

2019-01-27 Thread Albert Yue
ect. For > 500G metadata, mds may need 1T or more memory. > > > On Tue, Jan 22, 2019 at 5:48 PM Yan, Zheng wrote: > >> > >> On Tue, Jan 22, 2019 at 10:49 AM Albert Yue > wrote: > >> > > >> > Hi Yan Zheng, > >> > > >> &g

Re: [ceph-users] MDS performance issue

2019-01-22 Thread Albert Yue
But with enough memory on MDS, I can just cache all metadata into memory. Right now there are around 500GB metadata in the ssd. So this is not enough? On Tue, Jan 22, 2019 at 5:48 PM Yan, Zheng wrote: > On Tue, Jan 22, 2019 at 10:49 AM Albert Yue > wrote: > > > > Hi Yan Zhen

Re: [ceph-users] MDS performance issue

2019-01-21 Thread Albert Yue
Hi Yan Zheng, In your opinion, can we resolve this issue by move MDS to a 512GB or 1TB memory machine? On Mon, Jan 21, 2019 at 10:49 PM Yan, Zheng wrote: > On Mon, Jan 21, 2019 at 11:16 AM Albert Yue > wrote: > > > > Dear Ceph Users, > > > > We have set up a ceph

Re: [ceph-users] MDS performance issue

2019-01-20 Thread Albert Yue
Hi Yan Zheng, 1. mds cache limit is set to 64GB 2. we get the size of meta data pool by running `ceph df` and saw meta data pool just used 200MB space. Thanks, On Mon, Jan 21, 2019 at 11:35 AM Yan, Zheng wrote: > On Mon, Jan 21, 2019 at 11:16 AM Albert Yue > wrote: > > > &g

[ceph-users] MDS performance issue

2019-01-20 Thread Albert Yue
Dear Ceph Users, We have set up a cephFS cluster with 6 osd machines, each with 16 8TB harddisk. Ceph version is luminous 12.2.5. We created one data pool with these hard disks and created another meta data pool with 3 ssd. We created a MDS with 65GB cache size. But our users are keep complaining