Re: [ceph-users] HDD OSD 100% busy reading OMAP keys RGW

2019-02-14 Thread Wido den Hollander
On 2/14/19 2:08 PM, Dan van der Ster wrote: > On Thu, Feb 14, 2019 at 12:07 PM Wido den Hollander wrote: >> >> >> >> On 2/14/19 11:26 AM, Dan van der Ster wrote: >>> On Thu, Feb 14, 2019 at 11:13 AM Wido den Hollander wrote: On 2/14/19 10:20 AM, Dan van der Ster wrote: > On Thu.,

Re: [ceph-users] HDD OSD 100% busy reading OMAP keys RGW

2019-02-14 Thread Dan van der Ster
On Thu, Feb 14, 2019 at 12:07 PM Wido den Hollander wrote: > > > > On 2/14/19 11:26 AM, Dan van der Ster wrote: > > On Thu, Feb 14, 2019 at 11:13 AM Wido den Hollander wrote: > >> > >> On 2/14/19 10:20 AM, Dan van der Ster wrote: > >>> On Thu., Feb. 14, 2019, 6:17 a.m. Wido den Hollander >

Re: [ceph-users] HDD OSD 100% busy reading OMAP keys RGW

2019-02-14 Thread Wido den Hollander
On 2/14/19 11:26 AM, Dan van der Ster wrote: > On Thu, Feb 14, 2019 at 11:13 AM Wido den Hollander wrote: >> >> On 2/14/19 10:20 AM, Dan van der Ster wrote: >>> On Thu., Feb. 14, 2019, 6:17 a.m. Wido den Hollander >>> Hi, On a cluster running RGW only I'm running into BlueStore 1

Re: [ceph-users] HDD OSD 100% busy reading OMAP keys RGW

2019-02-14 Thread Dan van der Ster
On Thu, Feb 14, 2019 at 11:13 AM Wido den Hollander wrote: > > On 2/14/19 10:20 AM, Dan van der Ster wrote: > > On Thu., Feb. 14, 2019, 6:17 a.m. Wido den Hollander >> > >> Hi, > >> > >> On a cluster running RGW only I'm running into BlueStore 12.2.11 OSDs > >> being 100% busy sometimes. > >> > >

Re: [ceph-users] HDD OSD 100% busy reading OMAP keys RGW

2019-02-14 Thread Wido den Hollander
On 2/14/19 10:20 AM, Dan van der Ster wrote: > On Thu., Feb. 14, 2019, 6:17 a.m. Wido den Hollander > >> Hi, >> >> On a cluster running RGW only I'm running into BlueStore 12.2.11 OSDs >> being 100% busy sometimes. >> >> This cluster has 85k stale indexes (stale-instances list) and I've been >>

[ceph-users] HDD OSD 100% busy reading OMAP keys RGW

2019-02-13 Thread Wido den Hollander
Hi, On a cluster running RGW only I'm running into BlueStore 12.2.11 OSDs being 100% busy sometimes. This cluster has 85k stale indexes (stale-instances list) and I've been slowly trying to remove them. I noticed that regularly OSDs read their HDD heavily and that device then becomes 100% busy.