At Fri, 1 Dec 2017 14:12:20 -0800, Andres Freund <and...@anarazel.de> wrote in <20171201221220.z5e6wtlpl264w...@alap3.anarazel.de> > On 2017-12-01 17:03:28 -0500, Tom Lane wrote: > > Andres Freund <and...@anarazel.de> writes: > > > On 2017-12-01 16:40:23 -0500, Tom Lane wrote: > > >> I have no faith in either of these proposals, because they both assume > > >> that the problem only arises over the course of many minutes. In the > > >> recent complaint about pg_dump causing relcache bloat, it probably does > > >> not take nearly that long for the bloat to occur. > > > > > To me that's a bit of a different problem than what I was discussing > > > here. It also actually doesn't seem that hard - if your caches are > > > growing fast, you'll continually get hash-resizing of the > > > various. Adding cache-pruning to the resizing code doesn't seem hard, > > > and wouldn't add meaningful overhead. > > > > That's an interesting way to think about it, as well, though I'm not > > sure it's quite that simple. If you tie this to cache resizing then > > the cache will have to grow up to the newly increased size before > > you'll prune it again. That doesn't sound like it will lead to nice > > steady-state behavior. > > Yea, it's not perfect - but if we do pruning both at resize *and* on > regular intervals, like once-a-minute as I was suggesting, I don't think > it's that bad. The steady state won't be reached within seconds, true, > but the negative consequences of only attempting to shrink the cache > upon resizing when the cache size is growing fast anyway doesn't seem > that large. > > I don't think we need to be super accurate here, there just needs to be > *some* backpressure. > > I've had cases in the past where just occasionally blasting the cache > away would've been good enough.
Thank you very much for the valuable suggestions. I still would like to solve this problem and the a-counter-freely-running-in-minute(or several seconds)-resolution and pruning-too-long-unaccessed-entries-on-resizing seems to me to work enough for at least several known bloat cases. This still has a defect that this is not workable for a very quick bloating. I'll try thinking about the remaining issue. If no one has immediate objection to the direction, I'll come up with an implementation. regards, -- Kyotaro Horiguchi NTT Open Source Software Center
