On Thu, Apr 14, 2022 at 4:19 PM Jim Nasby wrote:
> > - percentage of non-yet-removable vs removable tuples
>
> This'd give you an idea how bad your long-running-transaction problem is.
VACUUM fundamentally works by removing those tuples that are
considered dead according to an XID-based cutoff es
On Mon, Apr 4, 2022 at 8:25 PM Peter Geoghegan wrote:
> Right. The reason I used WARNINGs was because it matches vaguely
> related WARNINGs in vac_update_relstats()'s sibling function,
> vacuum_set_xid_limits().
Okay, pushed the relfrozenxid warning patch.
Thanks
--
Peter Geoghegan
On Mon, Apr 4, 2022 at 8:18 PM Andres Freund wrote:
> The remaining patch are the warnings in vac_update_relstats(), correct? I
> guess one could argue they should be LOG rather than WARNING, but I find the
> project stance on that pretty impractical. So warning's ok with me.
Right. The reason I
Hi,
On 2022-04-04 19:32:13 -0700, Peter Geoghegan wrote:
> On Fri, Apr 1, 2022 at 10:54 AM Peter Geoghegan wrote:
> > I also refined the WARNING patch in v15. It now actually issues
> > WARNINGs (rather than PANICs, which were just a temporary debugging
> > measure in v14).
>
> Going to commit t
On Fri, Apr 1, 2022 at 10:54 AM Peter Geoghegan wrote:
> I also refined the WARNING patch in v15. It now actually issues
> WARNINGs (rather than PANICs, which were just a temporary debugging
> measure in v14).
Going to commit this remaining patch tomorrow, barring objections.
--
Peter Geoghegan
On Sun, Apr 3, 2022 at 12:05 PM Andres Freund wrote:
> Just saw that you committed: Wee! I think this will be a substantial
> improvement for our users.
I hope so! I think that it's much more useful as the basis for future
work than as a standalone thing. Users of Postgres 15 might not notice
a h
Hi,
On 2022-04-01 10:54:14 -0700, Peter Geoghegan wrote:
> On Thu, Mar 31, 2022 at 11:19 AM Peter Geoghegan wrote:
> > The assert is "Assert(diff > 0)", and not "Assert(diff >= 0)".
>
> Attached is v15. I plan to commit the first two patches (the most
> substantial two patches by far) in the next
On Thu, Mar 31, 2022 at 11:19 AM Peter Geoghegan wrote:
> The assert is "Assert(diff > 0)", and not "Assert(diff >= 0)".
Attached is v15. I plan to commit the first two patches (the most
substantial two patches by far) in the next couple of days, barring
objections.
v15 removes this "Assert(diff
On Thu, Mar 31, 2022 at 10:50 AM Andres Freund wrote:
> > So, to be clear: vac_update_relstats() never actually considered the
> > new relfrozenxid value from its vacuumlazy.c caller to be "in the
> > future"?
>
> No, I added separate debug messages for those, and also applied your patch,
> and it
Hi,
On 2022-03-31 10:12:49 -0700, Peter Geoghegan wrote:
> On Wed, Mar 30, 2022 at 9:59 PM Andres Freund wrote:
> > I'm not sure there's a proper bug on HEAD here. I think at worst it can
> > delay
> > the horizon increasing a bunch, by falsely not using an aggressive vacuum
> > when
> > we sho
On Thu, Mar 31, 2022 at 10:11 AM Andres Freund wrote:
> I don't think we should weaken defenses against xids from before relfrozenxid
> in vacuum / amcheck / If anything we should strengthen them.
>
> Isn't it also just plainly required for correctness? We'd not necessarily
> trigger a vacuum
On Wed, Mar 30, 2022 at 9:59 PM Andres Freund wrote:
> I'm not sure there's a proper bug on HEAD here. I think at worst it can delay
> the horizon increasing a bunch, by falsely not using an aggressive vacuum when
> we should have - might even be limited to a single autovacuum cycle.
So, to be cl
Hi,
On 2022-03-31 09:58:18 -0700, Peter Geoghegan wrote:
> On Thu, Mar 31, 2022 at 9:37 AM Andres Freund wrote:
> > The only place that appears to access rd_rel->relfrozenxid outside of DDL is
> > heap_abort_speculative().
>
> I wonder how necessary that really is. Even if the XID is before
> re
On Thu, Mar 31, 2022 at 9:37 AM Andres Freund wrote:
> Perhaps we should explicitly mask out parts of relcache entries in the shared
> init file that we know to be unreliable. I.e. set relfrozenxid, relminmxid to
> Invalid* or such.
That has the advantage of being more honest. If you're going to
Hi,
On 2022-03-30 21:59:15 -0700, Andres Freund wrote:
> On 2022-03-30 21:29:16 -0700, Peter Geoghegan wrote:
> > On Wed, Mar 30, 2022 at 9:20 PM Andres Freund wrote:
> > > Perhaps we should just fetch the horizons from the "local" catalog for
> > > shared
> > > rels?
> >
> > Not sure what you
Hi,
On 2022-03-30 21:29:16 -0700, Peter Geoghegan wrote:
> On Wed, Mar 30, 2022 at 9:20 PM Andres Freund wrote:
> > But the debug elog reports that
> >
> > relfrozenxid updated 714 -> 717
> > relminmxid updated 1 -> 6
> >
> > Tthe problem is that the crashing backend reads the relfrozenxid/relmin
On Wed, Mar 30, 2022 at 9:29 PM Peter Geoghegan wrote:
> > Perhaps we should just fetch the horizons from the "local" catalog for
> > shared
> > rels?
>
> Not sure what you mean.
Wait, you mean use vacrel->relfrozenxid directly? Seems kind of ugly...
--
Peter Geoghegan
On Wed, Mar 30, 2022 at 9:20 PM Andres Freund wrote:
> But the debug elog reports that
>
> relfrozenxid updated 714 -> 717
> relminmxid updated 1 -> 6
>
> Tthe problem is that the crashing backend reads the relfrozenxid/relminmxid
> from the shared relcache init file written by another backend:
W
Hi,
On 2022-03-30 21:11:48 -0700, Peter Geoghegan wrote:
> On Wed, Mar 30, 2022 at 9:04 PM Andres Freund wrote:
> > (gdb) p vacrel->NewRelfrozenXid
> > $3 = 717
> > (gdb) p vacrel->relfrozenxid
> > $4 = 717
> > (gdb) p OldestXmin
> > $5 = 5112
> > (gdb) p aggressive
> > $6 = false
>
> Does this O
Hi,
On 2022-03-30 21:04:07 -0700, Andres Freund wrote:
> On 2022-03-30 20:35:25 -0700, Peter Geoghegan wrote:
> > On Wed, Mar 30, 2022 at 8:28 PM Andres Freund wrote:
> > > I triggered twice now, but it took a while longer the second time.
> >
> > Great.
> >
> > I wonder if you can get an RR reco
On Wed, Mar 30, 2022 at 9:04 PM Andres Freund wrote:
> (gdb) p vacrel->NewRelfrozenXid
> $3 = 717
> (gdb) p vacrel->relfrozenxid
> $4 = 717
> (gdb) p OldestXmin
> $5 = 5112
> (gdb) p aggressive
> $6 = false
Does this OldestXmin seem reasonable at this point in execution, based
on context? Does it
Hi,
On 2022-03-30 20:35:25 -0700, Peter Geoghegan wrote:
> On Wed, Mar 30, 2022 at 8:28 PM Andres Freund wrote:
> > I triggered twice now, but it took a while longer the second time.
>
> Great.
>
> I wonder if you can get an RR recording...
Started it, but looks like it's too slow.
(gdb) p MyPr
Hi,
On 2022-03-30 20:28:44 -0700, Andres Freund wrote:
> I was able to trigger the crash.
>
> cat ~/tmp/pgbench-createdb.sql
> CREATE DATABASE pgb_:client_id;
> DROP DATABASE pgb_:client_id;
>
> pgbench -n -P1 -c 10 -j10 -T100 -f ~/tmp/pgbench-createdb.sql
>
> while I was also running
>
> for
On Wed, Mar 30, 2022 at 8:28 PM Andres Freund wrote:
> I triggered twice now, but it took a while longer the second time.
Great.
I wonder if you can get an RR recording...
--
Peter Geoghegan
Hi,
I was able to trigger the crash.
cat ~/tmp/pgbench-createdb.sql
CREATE DATABASE pgb_:client_id;
DROP DATABASE pgb_:client_id;
pgbench -n -P1 -c 10 -j10 -T100 -f ~/tmp/pgbench-createdb.sql
while I was also running
for i in $(seq 1 100); do echo iteration $i; make -Otarget -C contrib/ -s
in
On Wed, Mar 30, 2022 at 7:37 PM Peter Geoghegan wrote:
> Yeah, a WARNING would be good here. I can write a new version of my
> patch series with a separation patch for that this evening. Actually,
> better make it a PANIC for now...
Attached is v14, which includes a new patch that PANICs like tha
On Wed, Mar 30, 2022 at 7:00 PM Andres Freund wrote:
> Something vaguely like EXPECT_EQ_U32 in regress.c. Maybe
> AssertCmp(type, a, op, b),
>
> Then the assertion could have been something like
>AssertCmp(int32, diff, >, 0)
I'd definitely use them if they were there.
> Does the line number
Hi,
On 2022-03-30 17:50:42 -0700, Peter Geoghegan wrote:
> I tried several times to recreate this issue on CI. No luck with that,
> though -- can't get it to fail again after 4 attempts.
It's really annoying that we don't have Assert variants that show the compared
values, that might make it easi
On Wed, Mar 30, 2022 at 12:01 AM Peter Geoghegan wrote:
> Perhaps something is amiss inside vac_update_relstats(), where the
> boolean flag that indicates that pg_class.relfrozenxid was advanced is
> set:
>
> if (frozenxid_updated)
> *frozenxid_updated = false;
> if (TransactionIdI
On Tue, Mar 29, 2022 at 11:10 PM Justin Pryzby wrote:
>
> + diff = (int32) (vacrel->NewRelfrozenXid -
> vacrel->relfrozenxid);
> + Assert(diff > 0);
>
> Did you see that this crashed on windows cfbot?
>
> https://api.cirrus-ci.com/v1/art
+ diff = (int32) (vacrel->NewRelfrozenXid -
vacrel->relfrozenxid);
+ Assert(diff > 0);
Did you see that this crashed on windows cfbot?
https://api.cirrus-ci.com/v1/artifact/task/4592929254670336/log/tmp_check/postmaster.log
TRAP: Failed
On Tue, Mar 29, 2022 at 11:58 AM Peter Geoghegan wrote:
> > I think I understand what the first paragraph of the header comment
> > for heap_tuple_needs_freeze() is trying to say, but the second one is
> > quite confusing. I think this is again because it veers into talking
> > about what the call
On Tue, Mar 29, 2022 at 10:03 AM Robert Haas wrote:
> I can understand this version of the commit message. Woohoo! I like
> understanding things.
That's good news.
> I think the header comments for FreezeMultiXactId() focus way too much
> on what the caller is supposed to do and not nearly enoug
On Sun, Mar 27, 2022 at 11:24 PM Peter Geoghegan wrote:
> Attached is v12. My current goal is to commit all 3 patches before
> feature freeze. Note that this does not include the more complicated
> patch including with previous revisions of the patch series (the
> page-level freezing work that app
On Thu, Mar 24, 2022 at 2:40 PM Peter Geoghegan wrote:
> > > This is absolutely mandatory in the aggressive case, because otherwise
> > > relfrozenxid advancement might be seen as unsafe. My observation is:
> > > Why should we accept the same race in the non-aggressive case? Why not
> > > do essen
On Thu, Mar 24, 2022 at 1:21 PM Robert Haas wrote:
> > How about: "Set relfrozenxid to oldest extant XID seen by VACUUM"
>
> Sure, that sounds nice.
Cool.
> > What you're saying here boils down to this: it doesn't matter what the
> > visibility map would say right this microsecond (in the aggres
On Thu, Mar 24, 2022 at 3:28 PM Peter Geoghegan wrote:
> But non-aggressive VACUUMs have always been able to do that.
>
> How about: "Set relfrozenxid to oldest extant XID seen by VACUUM"
Sure, that sounds nice.
> Believe it or not, I avoided functional changes in 0002 -- at least in
> one impor
On Thu, Mar 24, 2022 at 10:21 AM Robert Haas wrote:
> You're probably not going to love hearing this, but I think you're
> still explaining things here in ways that are too baroque and hard to
> follow. I do think it's probably better.
There are a lot of dimensions to this work. It's hard to know
On Wed, Mar 23, 2022 at 6:28 PM Peter Geoghegan wrote:
> It would be great if you could take a look v11-0002-*, Robert. Does it
> make sense to you?
You're probably not going to love hearing this, but I think you're
still explaining things here in ways that are too baroque and hard to
follow. I d
On Wed, Mar 23, 2022 at 2:03 PM Thomas Munro wrote:
> Yeah, I found it confusing that DISABLE_PAGE_SKIPPING doesn't disable
> all page skipping, so 3414099c turned out to be not enough.
The proposed change to DISABLE_PAGE_SKIPPING is partly driven by that,
and partly driven by a similar concern a
On Thu, Mar 24, 2022 at 9:59 AM Peter Geoghegan wrote:
> On Wed, Mar 23, 2022 at 1:53 PM Robert Haas wrote:
> > And therefore I favor defining it to mean that we don't
> > skip any work at all.
>
> But even today DISABLE_PAGE_SKIPPING won't do pruning when we cannot
> acquire a cleanup lock on a
On Wed, Mar 23, 2022 at 1:53 PM Robert Haas wrote:
> I see what you mean about it depending on how you define "skipping".
> But I think that DISABLE_PAGE_SKIPPING is intended as a sort of
> emergency safeguard when you really, really don't want to leave
> anything out.
I agree.
> And therefore I
On Wed, Mar 23, 2022 at 4:49 PM Peter Geoghegan wrote:
> On Wed, Mar 23, 2022 at 1:41 PM Robert Haas wrote:
> > It seems to me that if DISABLE_PAGE_SKIPPING doesn't completely
> > disable skipping pages, we have a problem.
>
> It depends on how you define skipping. DISABLE_PAGE_SKIPPING was
> cre
On Wed, Mar 23, 2022 at 1:41 PM Robert Haas wrote:
> It seems to me that if DISABLE_PAGE_SKIPPING doesn't completely
> disable skipping pages, we have a problem.
It depends on how you define skipping. DISABLE_PAGE_SKIPPING was
created at a time when a broader definition of skipping made a lot
mor
On Wed, Mar 23, 2022 at 3:59 PM Peter Geoghegan wrote:
> In other words, since DISABLE_PAGE_SKIPPING doesn't *consistently*
> force lazy_scan_noprune to refuse to process a page on HEAD (it all
> depends on FreezeLimit/vacuum_freeze_min_age), it is logical for
> DISABLE_PAGE_SKIPPING to totally ge
On Sun, Mar 13, 2022 at 9:05 PM Peter Geoghegan wrote:
> Attached is v10. While this does still include the freezing patch,
> it's not in scope for Postgres 15. As I've said, I still think that it
> makes sense to maintain the patch series with the freezing stuff,
> since it's structurally related
On Fri, Feb 25, 2022 at 5:52 PM Peter Geoghegan wrote:
> There is an important practical way in which it makes sense to treat
> 0001 as separate to 0002. It is true that 0001 is independently quite
> useful. In practical terms, I'd be quite happy to just get 0001 into
> Postgres 15, without 0002.
On Tue, Mar 1, 2022 at 1:46 PM Robert Haas wrote:
> I think that this is not really a description of an algorithm -- and I
> think that it is far from clear that the third "in-between" category
> does not need to exist.
But I already described the algorithm. It is very simple
mechanistically -- t
On Sun, Feb 20, 2022 at 3:27 PM Peter Geoghegan wrote:
> > I think that the idea has potential, but I don't think that I
> > understand yet what the *exact* algorithm is.
>
> The algorithm seems to exploit a natural tendency that Andres once
> described in a blog post about his snapshot scalabilit
On Fri, Feb 25, 2022 at 3:26 PM Andres Freund wrote:
> freeze_required_limit, freeze_desired_limit? Or s/limit/cutoff/? Or
> s/limit/below/? I kind of like below because that answers < vs <= which I find
> hard to remember around freezing.
I like freeze_required_limit the most.
> That may be tru
On Fri, Feb 25, 2022 at 3:48 PM Andres Freund wrote:
> I don't see why it matters that OldestXmin and OldestMxact are computed at the
> same time? It's a question of the workload, not vacuum algorithm.
I think it's both.
> OldestMxact inherently lags OldestXmin. OldestMxact can only advance aft
Hi,
On 2022-02-25 15:28:17 -0800, Peter Geoghegan wrote:
> But why should we ever get to the FreezeMultiXactId() loop with the
> stuff from 0002 in place? The whole purpose of the loop is to handle
> cases where we have to remove *some* (not all) XIDs from before
> cutoff_xid that appear in a Mult
On Fri, Feb 25, 2022 at 2:00 PM Peter Geoghegan wrote:
> > Hm. I guess I'll have to look at the code for it. It doesn't immediately
> > "feel" quite right.
>
> I kinda think it might be. Please let me know if you see a problem
> with what I've said.
Oh, wait. I have a better idea of what you mean
Hi,
On 2022-02-25 14:00:12 -0800, Peter Geoghegan wrote:
> On Thu, Feb 24, 2022 at 11:14 PM Andres Freund wrote:
> > I am not a fan of the backstop terminology. It's still the reason we need to
> > do freezing for correctness reasons.
>
> Thanks for the review!
>
> I'm not wedded to that partic
On Thu, Feb 24, 2022 at 11:14 PM Andres Freund wrote:
> I am not a fan of the backstop terminology. It's still the reason we need to
> do freezing for correctness reasons.
Thanks for the review!
I'm not wedded to that particular terminology, but I think that we
need something like it. Open to su
Hi,
On 2022-02-24 20:53:08 -0800, Peter Geoghegan wrote:
> 0002 makes page-level freezing a first class thing.
> heap_prepare_freeze_tuple now has some (limited) knowledge of how this
> works. heap_prepare_freeze_tuple's cutoff_xid argument is now always
> the VACUUM caller's OldestXmin (not its F
On Sun, Feb 20, 2022 at 12:27 PM Peter Geoghegan wrote:
> You've given me a lot of high quality feedback on all of this, which
> I'll work through soon. It's hard to get the balance right here, but
> it's made much easier by this kind of feedback.
Attached is v9. Lots of changes. Highlights:
* M
,
On Sun, Feb 20, 2022 at 7:30 AM Robert Haas wrote:
> Right, so we at least need to add a similar comment to what I proposed
> for MXIDs, and maybe other changes are needed, too.
Agreed.
> > Maybe the way that FreezeLimit/cutoff_xid is overloaded can be fixed
> > here, to make all of this less
On Fri, Feb 18, 2022 at 7:12 PM Peter Geoghegan wrote:
> We have to worry about XIDs from MultiXacts (and xmax values more
> generally). And we have to worry about the case where we start out
> with only xmin frozen (by an earlier VACUUM), and then have to freeze
> xmax too. I believe that we have
On Sat, Feb 19, 2022 at 8:54 PM Andres Freund wrote:
> > Leaving behind disconnected/orphaned heap-only tuples is pretty much
> > pointless anyway, since they'll never be accessible by index scans.
> > Even after a REINDEX, since there is no root item from the heap page
> > to go in the index. (A
On Sat, Feb 19, 2022 at 8:21 PM Andres Freund wrote:
> Why does anything need to happen to it from vacuum's POV? It'll not be a
> problem for freezing etc. Until it's deleted vacuum doesn't need to care.
>
> Probably worth a WARNING, and amcheck definitely needs to detect it, but
> otherwise I
Hi,
On February 19, 2022 7:56:53 PM PST, Peter Geoghegan wrote:
>On Sat, Feb 19, 2022 at 7:47 PM Andres Freund wrote:
>> Non DEAD orphaned versions shouldn't cause a problem in lazy_scan_prune().
>> The
>> problem here is a DEAD orphaned HOT tuples, and those we should be able to
>> delete wit
On Sat, Feb 19, 2022 at 7:47 PM Andres Freund wrote:
> I'm not that sure those are that different... Imagine some corruption leading
> to two hot chains ending in the same tid, which our fancy new secure pruning
> algorithm might detect.
I suppose that's possible, but it doesn't seem all that lik
Hi,
On 2022-02-19 19:31:21 -0800, Peter Geoghegan wrote:
> On Sat, Feb 19, 2022 at 6:16 PM Peter Geoghegan wrote:
> > > Given that heap_surgery's raison d'etre is correcting corruption etc, I
> > > think
> > > it makes sense for it to do as minimal work as possible. Iterating
> > > through a
>
On Sat, Feb 19, 2022 at 7:28 PM Andres Freund wrote:
> If the vacuum can get the cleanup lock due to the adversarial patch, the
> heap_force_kill() doesn't do anything, because the first item is a
> redirect. However if it *can't* get a cleanup lock, heap_force_kill() instead
> targets the root it
On Sat, Feb 19, 2022 at 6:16 PM Peter Geoghegan wrote:
> > Given that heap_surgery's raison d'etre is correcting corruption etc, I
> > think
> > it makes sense for it to do as minimal work as possible. Iterating through a
> > HOT chain would be a problem if you e.g. tried to repair a page with HO
Hi,
On 2022-02-19 19:07:39 -0800, Peter Geoghegan wrote:
> On Sat, Feb 19, 2022 at 7:01 PM Andres Freund wrote:
> > > We can either do that, or we can throw an error concerning corruption
> > > when heap_page_prune notices orphaned tuples. Neither seems
> > > particularly appealing. But it defini
On Sat, Feb 19, 2022 at 7:01 PM Andres Freund wrote:
> It's kind of surprising that this needs this
> 0001-Add-adversarial-ConditionalLockBuff to break. I suspect it's a question
> of hint bits changing due to lazy_scan_noprune(), which then makes
> HeapTupleHeaderIsHotUpdated() have a different r
On Sat, Feb 19, 2022 at 7:01 PM Andres Freund wrote:
> > We can either do that, or we can throw an error concerning corruption
> > when heap_page_prune notices orphaned tuples. Neither seems
> > particularly appealing. But it definitely makes no sense to allow
> > lazy_scan_prune to spin in a futi
Hi,
On 2022-02-19 18:16:54 -0800, Peter Geoghegan wrote:
> On Sat, Feb 19, 2022 at 5:54 PM Andres Freund wrote:
> > How does that cause the endless loop?
>
> Attached is the page image itself, dumped via gdb (and gzip'd). This
> was on recent HEAD (commit 8f388f6f, actually), plus
> 0001-Add-adv
On Sat, Feb 19, 2022 at 5:54 PM Andres Freund wrote:
> How does that cause the endless loop?
Attached is the page image itself, dumped via gdb (and gzip'd). This
was on recent HEAD (commit 8f388f6f, actually), plus
0001-Add-adversarial-ConditionalLockBuff[...]. No other changes. No
defragmenting
Hi,
On 2022-02-19 17:22:33 -0800, Peter Geoghegan wrote:
> Looks like pg_surgery isn't processing HOT chains as whole units,
> which it really should (at least in the context of killing items via
> the heap_force_kill() function). Killing a root item in a HOT chain is
> just hazardous -- disconnec
On Sat, Feb 19, 2022 at 4:22 PM Peter Geoghegan wrote:
> This very much looks like a bug in pg_surgery itself now -- attached
> is a draft fix.
Wait, that's not it either. I jumped the gun -- this isn't sufficient
(though the patch I posted might not be a bad idea anyway).
Looks like pg_surgery
On Sat, Feb 19, 2022 at 3:08 PM Peter Geoghegan wrote:
> It's quite possible that this is nothing more than a bug in my
> adversarial gizmo patch -- since I don't think that
> ConditionalLockBufferForCleanup() can ever fail with a temp buffer
> (though even that's not completely clear right now).
Hi,
(On phone, so crappy formatting and no source access)
On February 19, 2022 3:08:41 PM PST, Peter Geoghegan wrote:
>On Fri, Feb 18, 2022 at 5:00 PM Peter Geoghegan wrote:
>> Another testing strategy occurs to me: we could stress-test the
>> implementation by simulating an environment where t
On Fri, Feb 18, 2022 at 5:00 PM Peter Geoghegan wrote:
> Another testing strategy occurs to me: we could stress-test the
> implementation by simulating an environment where the no-cleanup-lock
> path is hit an unusually large number of times, possibly a fixed
> percentage of the time (like 1%, 5%)
On Fri, Feb 18, 2022 at 2:11 PM Andres Freund wrote:
> I think it'd be good to add a few isolationtest cases for the
> can't-get-cleanup-lock paths. I think it shouldn't be hard using cursors. The
> slightly harder part is verifying that VACUUM did something reasonable, but
> that still should be
On Fri, Feb 18, 2022 at 1:56 PM Robert Haas wrote:
> + * Also maintains *NewRelfrozenxid and *NewRelminmxid, which are the current
> + * target relfrozenxid and relminmxid for the relation. Assumption is that
>
> "maintains" is fuzzy. I think you should be saying something much more
> explicit, a
Hi,
On 2022-02-18 15:54:19 -0500, Robert Haas wrote:
> > Are there any objections to this plan?
>
> I really like the idea of reducing the scope of what is being changed
> here, and I agree that eagerly advancing relfrozenxid carries much
> less risk than the other changes.
Sounds good to me too
Hi,
On 2022-02-18 13:09:45 -0800, Peter Geoghegan wrote:
> 0001 is tricky in the sense that there are a lot of fine details, and
> if you get any one of them wrong the result might be a subtle bug. For
> example, the heap_tuple_needs_freeze() code path is only used when we
> cannot get a cleanup l
On Fri, Feb 18, 2022 at 4:10 PM Peter Geoghegan wrote:
> It does not change any other behavior. It's totally mechanical.
>
> 0001 is tricky in the sense that there are a lot of fine details, and
> if you get any one of them wrong the result might be a subtle bug. For
> example, the heap_tuple_need
On Fri, Feb 18, 2022 at 12:54 PM Robert Haas wrote:
> I'd like to have a clearer idea of exactly what is in each of the
> remaining patches before forming a final opinion.
Great.
> What's tricky about 0001? Does it change any other behavior, either as
> a necessary component of advancing relfroz
On Fri, Feb 18, 2022 at 3:41 PM Peter Geoghegan wrote:
> Concerns about my general approach to this project (and even the
> Postgres 14 VACUUM work) were expressed by Robert and Andres over on
> the "Nonrandom scanned_pages distorts pg_class.reltuples set by
> VACUUM" thread. Some of what was said
On Fri, Feb 11, 2022 at 8:30 PM Peter Geoghegan wrote:
> Attached is v8. No real changes -- just a rebased version.
Concerns about my general approach to this project (and even the
Postgres 14 VACUUM work) were expressed by Robert and Andres over on
the "Nonrandom scanned_pages distorts pg_class.
On Sat, Jan 29, 2022 at 8:42 PM Peter Geoghegan wrote:
> Attached is v7, a revision that overhauls the algorithm that decides
> what to freeze. I'm now calling it block-driven freezing in the commit
> message. Also included is a new patch, that makes VACUUM record zero
> free space in the FSM for
On Mon, Feb 7, 2022 at 12:21 PM Robert Haas wrote:
>
> On Mon, Feb 7, 2022 at 11:43 AM Peter Geoghegan wrote:
> > > That's because, if VACUUM is only ever getting triggered by XID
> > > age advancement and not by bloat, there's no opportunity for your
> > > patch set to advance relfrozenxid any s
On Mon, Feb 7, 2022 at 11:43 AM Peter Geoghegan wrote:
> > That's because, if VACUUM is only ever getting triggered by XID
> > age advancement and not by bloat, there's no opportunity for your
> > patch set to advance relfrozenxid any sooner than we're doing now.
>
> We must distinguish between:
>
On Mon, Feb 7, 2022 at 10:08 AM Robert Haas wrote:
> But ... if I'm not mistaken, in the kind of case that Greg is
> describing, relfrozenxid will be advanced exactly as often as it is
> today.
But what happens today in a scenario like Greg's is pathological,
despite being fairly common (common i
On Fri, Feb 4, 2022 at 10:45 PM Peter Geoghegan wrote:
> > While I've seen all the above cases triggering anti-wraparound cases
> > by far the majority of the cases are not of these pathological forms.
>
> Right - it's practically inevitable that you'll need an
> anti-wraparound VACUUM to advance
On Fri, Feb 4, 2022 at 10:21 PM Greg Stark wrote:
> By far the majority of anti-wraparound vacuums are triggered by tables
> that are very large and so don't trigger regular vacuums for "long
> periods" of time and consistently hit the anti-wraparound threshold
> first.
That's interesting, becaus
On Fri, Feb 4, 2022 at 10:44 PM Peter Geoghegan wrote:
> Right - it's practically inevitable that you'll need an
> anti-wraparound VACUUM to advance relfrozenxid right now. Technically
> it's possible to advance relfrozenxid in any VACUUM, but in practice
> it just never happens on a large table.
On Fri, Feb 4, 2022 at 10:21 PM Greg Stark wrote:
> On Wed, 15 Dec 2021 at 15:30, Peter Geoghegan wrote:
> > My emphasis here has been on making non-aggressive VACUUMs *always*
> > advance relfrozenxid, outside of certain obvious edge cases. And so
> > with all the patches applied, up to and incl
On Wed, 15 Dec 2021 at 15:30, Peter Geoghegan wrote:
>
> My emphasis here has been on making non-aggressive VACUUMs *always*
> advance relfrozenxid, outside of certain obvious edge cases. And so
> with all the patches applied, up to and including the opportunistic
> freezing patch, every autovacuu
On Fri, Feb 4, 2022 at 4:18 PM Robert Haas wrote:
> On Fri, Feb 4, 2022 at 3:31 PM Peter Geoghegan wrote:
> > Application B will already block pruning by VACUUM operations against
> > application A's table, and so effectively blocks recording of the
> > resultant free space in the FSM in your sce
On Fri, Feb 4, 2022 at 3:31 PM Peter Geoghegan wrote:
> Application B will already block pruning by VACUUM operations against
> application A's table, and so effectively blocks recording of the
> resultant free space in the FSM in your scenario. And so application A
> and application B should be c
On Fri, Feb 4, 2022 at 2:45 PM Robert Haas wrote:
> While I agree that there's some case to be made for leaving settled
> pages well enough alone, your criterion for settled seems pretty much
> accidental.
I fully admit that I came up with the FSM heuristic with TPC-C in
mind. But you have to sta
On Sat, Jan 29, 2022 at 11:43 PM Peter Geoghegan wrote:
> When VACUUM sees that all remaining/unpruned tuples on a page are
> all-visible, it isn't just important because of cost control
> considerations. It's deeper than that. It's also treated as a
> tentative signal from the application itself,
On Fri, Feb 4, 2022 at 2:00 PM John Naylor wrote:
> Without having looked at the latest patches, there was something in
> the back of my mind while following the discussion upthread -- the
> proposed opportunistic freezing made a lot more sense if the
> earlier-proposed open/closed pages concept w
On Sat, Jan 29, 2022 at 11:43 PM Peter Geoghegan wrote:
>
> On Thu, Jan 20, 2022 at 2:00 PM Peter Geoghegan wrote:
> > I do see some value in that, too. Though it's not going to be a way of
> > turning off the early freezing stuff, which seems unnecessary (though
> > I do still have work to do on
On Fri, Jan 21, 2022 at 12:07 PM Greg Stark wrote:
> This confuses me. "Transactions per second" is a headline database
> metric that lots of users actually focus on quite heavily -- rather
> too heavily imho.
But transactions per second is for the whole database, not for
individual tables. It's
1 - 100 of 133 matches
Mail list logo
