Re: Make autovacuum sort tables in descending order of xid_age

2020-08-02 Thread Michael Paquier
On Mon, Mar 30, 2020 at 09:20:15AM -0700, Mark Dilger wrote: > I have not been working on this issue lately, but as I recall, my > concern was that changing the behavior of autovacuum could introduce > regressions for some users, so we should be careful to get it right > before we rush to release a

Re: Make autovacuum sort tables in descending order of xid_age

2020-03-30 Thread Mark Dilger
> On Mar 30, 2020, at 7:09 AM, David Steele wrote: > > On 1/11/20 12:53 PM, David Fetter wrote: >> I agree that it's a complex situation, and that many different >> approaches will eventually need to be brought to bear. >> What concerns me about introducing a big lump of complexity here is >>

Re: Make autovacuum sort tables in descending order of xid_age

2020-03-30 Thread David Steele
On 1/11/20 12:53 PM, David Fetter wrote: I agree that it's a complex situation, and that many different approaches will eventually need to be brought to bear. What concerns me about introducing a big lump of complexity here is disentangling the effects of each part and of their interaction term

Re: Make autovacuum sort tables in descending order of xid_age

2020-01-11 Thread David Fetter
On Thu, Jan 09, 2020 at 12:23:46PM -0500, Robert Haas wrote: > On Thu, Dec 12, 2019 at 2:26 PM David Fetter wrote: > > > I wonder if you might add information about table size, table changes, > > > and bloat to your RelFrozenXidAge struct and modify rfxa_comparator to > > > use a heuristic to cost

Re: Make autovacuum sort tables in descending order of xid_age

2020-01-10 Thread David Kimura
On Thu, Jan 9, 2020 at 12:07 PM Robert Haas wrote: > > On Thu, Dec 12, 2019 at 2:26 PM David Fetter wrote: > > > I wonder if you might add information about table size, table changes, > > > and bloat to your RelFrozenXidAge struct and modify rfxa_comparator to > > > use a heuristic to cost the (a

Re: Make autovacuum sort tables in descending order of xid_age

2020-01-09 Thread Robert Haas
On Thu, Dec 12, 2019 at 2:26 PM David Fetter wrote: > > I wonder if you might add information about table size, table changes, > > and bloat to your RelFrozenXidAge struct and modify rfxa_comparator to > > use a heuristic to cost the (age, size, bloat, changed) grouping and > > sort on that cost,

Re: Make autovacuum sort tables in descending order of xid_age

2020-01-08 Thread David Kimura
On Tue, Jan 7, 2020 at 12:47 PM David Fetter wrote: > Per a suggestion Christophe made, please find attached a patch to > $Subject: Curious, what's the benefit of autovacuum handling the oldest tables first? If there is a related thread with the discussion, I couldn't find it. > Apart from caref

Re: Make autovacuum sort tables in descending order of xid_age

2019-12-12 Thread Mark Dilger
On 12/12/19 1:35 PM, Mark Dilger wrote: Once the xid age reaches one million, it will start to be the dominant factor. Actually, it doesn't change much from x = 1 to x = 1,000,000 but I was planning to add another factor to the formula and forgot before sending the email. I'll leave that as

Re: Make autovacuum sort tables in descending order of xid_age

2019-12-12 Thread Mark Dilger
On 12/12/19 1:35 PM, Mark Dilger wrote:   Let C = 1.0002065   Let x = xid_age for a table   Let v = clamp(n_dead_tuples / reltuples*2) to max 0.5   Let a = clamp(changes_since_analyze / reltuples) to max 0.5   Let score = C**x + v + a I should hasten to add that this is just a proo

Re: Make autovacuum sort tables in descending order of xid_age

2019-12-12 Thread Mark Dilger
On 12/12/19 11:26 AM, David Fetter wrote: On Thu, Dec 12, 2019 at 08:02:25AM -0800, Mark Dilger wrote: On 11/30/19 2:23 PM, David Fetter wrote: On Sat, Nov 30, 2019 at 10:04:07AM -0800, Mark Dilger wrote: On 11/29/19 2:21 PM, David Fetter wrote: On Fri, Nov 29, 2019 at 07:01:39PM +0100, Da

Re: Make autovacuum sort tables in descending order of xid_age

2019-12-12 Thread David Fetter
On Thu, Dec 12, 2019 at 08:02:25AM -0800, Mark Dilger wrote: > On 11/30/19 2:23 PM, David Fetter wrote: > > On Sat, Nov 30, 2019 at 10:04:07AM -0800, Mark Dilger wrote: > > > On 11/29/19 2:21 PM, David Fetter wrote: > > > > On Fri, Nov 29, 2019 at 07:01:39PM +0100, David Fetter wrote: > > > > > Fol

Re: Make autovacuum sort tables in descending order of xid_age

2019-12-12 Thread Mark Dilger
On 11/30/19 2:23 PM, David Fetter wrote: On Sat, Nov 30, 2019 at 10:04:07AM -0800, Mark Dilger wrote: On 11/29/19 2:21 PM, David Fetter wrote: On Fri, Nov 29, 2019 at 07:01:39PM +0100, David Fetter wrote: Folks, Per a suggestion Christophe made, please find attached a patch to $Subject: A

Re: Make autovacuum sort tables in descending order of xid_age

2019-11-30 Thread David Fetter
On Sat, Nov 30, 2019 at 10:04:07AM -0800, Mark Dilger wrote: > On 11/29/19 2:21 PM, David Fetter wrote: > > On Fri, Nov 29, 2019 at 07:01:39PM +0100, David Fetter wrote: > > > Folks, > > > > > > Per a suggestion Christophe made, please find attached a patch to > > > $Subject: > > > > > > Apart fr

Re: Make autovacuum sort tables in descending order of xid_age

2019-11-30 Thread Mark Dilger
On 11/29/19 2:21 PM, David Fetter wrote: On Fri, Nov 29, 2019 at 07:01:39PM +0100, David Fetter wrote: Folks, Per a suggestion Christophe made, please find attached a patch to $Subject: Apart from carefully fudging with pg_resetwal, and short of running in production for a few weeks, what w

Re: Make autovacuum sort tables in descending order of xid_age

2019-11-29 Thread David Fetter
On Fri, Nov 29, 2019 at 07:01:39PM +0100, David Fetter wrote: > Folks, > > Per a suggestion Christophe made, please find attached a patch to > $Subject: > > Apart from carefully fudging with pg_resetwal, and short of running in > production for a few weeks, what would be some good ways to test th