Re: [ceph-users] Unexpected increase in the memory usage of OSDs

2019-10-09 Thread Gregory Farnum
On Wed, Oct 9, 2019 at 10:58 AM Vladimir Brik < vladimir.b...@icecube.wisc.edu> wrote: > Best I can tell, automatic cache sizing is enabled and all related > settings are at their default values. > > Looking through cache tunables, I came across > osd_memory_expected_fragmentation, which the docs

Re: [ceph-users] Unexpected increase in the memory usage of OSDs

2019-10-09 Thread Vladimir Brik
Best I can tell, automatic cache sizing is enabled and all related settings are at their default values. Looking through cache tunables, I came across osd_memory_expected_fragmentation, which the docs define as "estimate the percent of memory fragmentation". What's the formula to compute actu

Re: [ceph-users] Unexpected increase in the memory usage of OSDs

2019-10-09 Thread Gregory Farnum
On Mon, Oct 7, 2019 at 7:20 AM Vladimir Brik wrote: > > > Do you have statistics on the size of the OSDMaps or count of them > > which were being maintained by the OSDs? > No, I don't think so. How can I find this information? Hmm I don't know if we directly expose the size of maps. There are p

Re: [ceph-users] Unexpected increase in the memory usage of OSDs

2019-10-07 Thread Vladimir Brik
> Do you have statistics on the size of the OSDMaps or count of them > which were being maintained by the OSDs? No, I don't think so. How can I find this information? Memory consumption started to climb again: https://icecube.wisc.edu/~vbrik/graph-3.png Some more info (not sure if relevant or no

Re: [ceph-users] Unexpected increase in the memory usage of OSDs

2019-10-04 Thread Gregory Farnum
Do you have statistics on the size of the OSDMaps or count of them which were being maintained by the OSDs? I'm not sure why having noout set would change that if all the nodes were alive, but that's my bet. -Greg On Thu, Oct 3, 2019 at 7:04 AM Vladimir Brik wrote: > > And, just as unexpectedly,

Re: [ceph-users] Unexpected increase in the memory usage of OSDs

2019-10-03 Thread Vladimir Brik
And, just as unexpectedly, things have returned to normal overnight https://icecube.wisc.edu/~vbrik/graph-1.png The change seems to have coincided with the beginning of Rados Gateway activity (before, it was essentially zero). I can see nothing in the logs that would explain what happened thoug