[ceph-users] Re: Nautilus upgrade causes spike in MDS latency

2020-04-28 Thread Josh Haft
This issue did subside after restarting the original primary daemon and failing back to it. I've since enabled multi-MDS and latencies overall have decreased even further. Thanks for your assistance. On Wed, Apr 15, 2020 at 8:32 AM Josh Haft wrote: > > Thanks for the assistance. >

[ceph-users] CephFS deleted files' space not reclaimed

2019-09-23 Thread Josh Haft
Hi, I've been migrating data from one EC pool to another EC pool: two directories are mounted with ceph.dir.layout.pool file attribute set appropriately, then rsync from old to new and finally, delete the old files. I'm using the kernel client to do this. While the removed files are no longer pres

[ceph-users] Re: CephFS deleted files' space not reclaimed

2019-09-23 Thread Josh Haft
Forgot some important info: I'm running Mimic 13.2.5. On Mon, Sep 23, 2019 at 8:49 AM Josh Haft wrote: > > Hi, > > I've been migrating data from one EC pool to another EC pool: two > directories are mounted with ceph.dir.layout.pool file attribute set > appropriately

[ceph-users] Re: CephFS deleted files' space not reclaimed

2019-09-26 Thread Josh Haft
arnum wrote: > > On Mon, Sep 23, 2019 at 6:50 AM Josh Haft wrote: > > > > Hi, > > > > I've been migrating data from one EC pool to another EC pool: two > > directories are mounted with ceph.dir.layout.pool file attribute set > > appropriately, then rsy

[ceph-users] RGWReshardLock::lock failed to acquire lock ret=-16

2019-10-31 Thread Josh Haft
Hi, Currently running Mimic 13.2.5. We had reports this morning of timeouts and failures with PUT and GET requests to our Ceph RGW cluster. I found these messages in the RGW log: RGWReshardLock::lock failed to acquire lock on bucket_name:bucket_instance ret=-16 NOTICE: resharding operation on buc

[ceph-users] Re: RGWReshardLock::lock failed to acquire lock ret=-16

2020-03-12 Thread Josh Haft
ev cluster, are there any known fixes for this issue included? Regards, Josh On Thu, Oct 31, 2019 at 2:43 PM Josh Haft wrote: > > Hi, > > Currently running Mimic 13.2.5. > > We had reports this morning of timeouts and failures with PUT and GET > requests to our Ceph RGW clu

[ceph-users] Nautilus upgrade causes spike in MDS latency

2020-04-13 Thread Josh Haft
Hi, I upgraded from 13.2.5 to 14.2.6 last week and am now seeing significantly higher latency on various MDS operations. For example, the 2min rate of ceph_mds_server_req_create_latency_sum / ceph_mds_server_req_create_latency_count for an 8hr window last Monday prior to the upgrade was an average

[ceph-users] Re: Nautilus upgrade causes spike in MDS latency

2020-04-13 Thread Josh Haft
On Mon, Apr 13, 2020 at 4:14 PM Gregory Farnum wrote: > > On Mon, Apr 13, 2020 at 1:33 PM Josh Haft wrote: > > > > Hi, > > > > I upgraded from 13.2.5 to 14.2.6 last week and am now seeing > > significantly higher latency on various MDS operation

[ceph-users] Re: Nautilus upgrade causes spike in MDS latency

2020-04-15 Thread Josh Haft
since. I'll 'fail' back to the primary over the weekend to see whether the problem returns. Josh On Wed, Apr 15, 2020 at 2:47 AM Yan, Zheng wrote: > > On Tue, Apr 14, 2020 at 6:17 AM Josh Haft wrote: > > > > On Mon, Apr 13, 2020 at 4:14 PM Gregory Farnum wrote: &g