On Thu, Dec 21, 2023 at 3:58 PM Melanie Plageman
wrote:
>
> On Wed, Dec 13, 2023 at 12:24 PM Robert Haas wrote:
> > On Sat, Dec 9, 2023 at 5:12 AM Melanie Plageman
> > wrote:
> > > The goal is to keep pages frozen for at least target_freeze_duration.
> > > target_freeze_duration is in seconds an
On Thu, Dec 21, 2023 at 10:56 AM Melanie Plageman
wrote:
> Agreed. I plan to test with another distribution. Though, the exercise
> of determining which ones are useful is probably more challenging.
> I imagine we will have to choose one distribution (as opposed to
> supporting different distribut
On Wed, Dec 13, 2023 at 12:24 PM Robert Haas wrote:
>
> Great results.
Thanks!
> On Sat, Dec 9, 2023 at 5:12 AM Melanie Plageman
> wrote:
> > Values can be "removed" from the accumulator by simply decrementing its
> > cardinality and decreasing the sum and sum squared by a value that will
> > n
On 12/21/23 10:56, Melanie Plageman wrote:
On Sat, Dec 9, 2023 at 9:24 AM Joe Conway wrote:
However, even if we assume a more-or-less normal distribution, we should
consider using subgroups in a way similar to Statistical Process
Control[1]. The reasoning is explained in this quote:
The M
On Sat, Dec 9, 2023 at 9:24 AM Joe Conway wrote:
>
> On 12/8/23 23:11, Melanie Plageman wrote:
> >
> > I'd be delighted to receive any feedback, ideas, questions, or review.
>
>
> This is well thought out, well described, and a fantastic improvement in
> my view -- well done!
Thanks, Joe! That me
Great results.
On Sat, Dec 9, 2023 at 5:12 AM Melanie Plageman
wrote:
> Values can be "removed" from the accumulator by simply decrementing its
> cardinality and decreasing the sum and sum squared by a value that will
> not change the mean and standard deviation of the overall distribution.
> To
On 12/8/23 23:11, Melanie Plageman wrote:
On Wed, Nov 8, 2023 at 9:23 PM Melanie Plageman
wrote:
The next step is to devise different heuristics and measure their
efficacy. IMO, the goal of the algorithm it is to freeze pages in a
relation such that we drive early unfreezes/freezes -> 0 and pag
On Wed, Nov 8, 2023 at 9:23 PM Melanie Plageman
wrote:
> The next step is to devise different heuristics and measure their
> efficacy. IMO, the goal of the algorithm it is to freeze pages in a
> relation such that we drive early unfreezes/freezes -> 0 and pages
> frozen/number of pages of a certai
On Wed, Oct 11, 2023 at 8:43 PM Andres Freund wrote:
>
> A rough sketch of a freezing heuristic:
>
> - We concluded that to intelligently control opportunistic freezing we need
> statistics about the number of freezes and unfreezes
>
> - We should track page freezes / unfreezes in shared memor
On Thu, Oct 12, 2023 at 11:50 AM Melanie Plageman
wrote:
> I was under the impression that we decided we still had to consider
> the number of clean pages dirtied as well as the number of pages
> unfrozen. The number of pages frozen and unfrozen over a time period
> gives us some idea of if we are
On Wed, Oct 11, 2023 at 8:43 PM Andres Freund wrote:
>
> Robert, Melanie and I spent an evening discussing this topic around
> pgconf.nyc. Here are, mildly revised, notes from that:
Thanks for taking notes!
> The main thing we are worried about is repeated freezing / unfreezing of
> pages wi
Thanks for these notes.
On Wed, Oct 11, 2023 at 8:43 PM Andres Freund wrote:
> - We also discussed an idea by Robert to track the number of times we need to
> dirty a page when unfreezing and to compare that to the number of pages
> dirtied overall (IIRC), but I don't think we really came to
Hi,
Robert, Melanie and I spent an evening discussing this topic around
pgconf.nyc. Here are, mildly revised, notes from that:
First a few random points that didn't fit with the sketch of an approach
below:
- Are unlogged tables a problem for using LSN based heuristics for freezing?
We conc
On Mon, Oct 2, 2023 at 4:25 PM Robert Haas wrote:
> On Mon, Oct 2, 2023 at 11:37 AM Peter Geoghegan wrote:
> > If no vacuuming against pgbench_accounts is strictly better than some
> > vacuuming (unless it's just to advance relfrozenxid, which can't be
> > avoided), then to what extent is Melanie
On Mon, Oct 2, 2023 at 11:37 AM Peter Geoghegan wrote:
> If no vacuuming against pgbench_accounts is strictly better than some
> vacuuming (unless it's just to advance relfrozenxid, which can't be
> avoided), then to what extent is Melanie's freezing patch obligated to
> not make the situation wor
On Mon, Oct 2, 2023 at 6:26 AM Robert Haas wrote:
> I think it's true that the fact that pgbench does what pgbench does
> makes us think more about that workload than about some other, equally
> plausible workload. It's the test we have, so we end up running it a
> lot. If we had some other test,
On Fri, Sep 29, 2023 at 8:50 PM Peter Geoghegan wrote:
> While pgbench makes a fine stress-test, for the most part its workload
> is highly unrealistic. And yet we seem to think that it's just about
> the most important benchmark of all. If we're not willing to get over
> even small regressions in
On Fri, Sep 29, 2023 at 11:27 AM Robert Haas wrote:
> I think that's true. For me, the issue is what a user is practically
> likely to notice and care about. I submit that on a
> not-particularly-busy system, it would probably be fine to freeze
> aggressively in almost every situation, because you
On Fri, Sep 29, 2023 at 11:27 AM Robert Haas wrote:
> > Even if you're willing to assume that vacuum_freeze_min_age isn't just
> > an arbitrary threshold, this still seems wrong. vacuum_freeze_min_age
> > is applied by VACUUM, at the point that it scans pages. If VACUUM were
> > infinitely fast, a
On Fri, Sep 29, 2023 at 11:57 AM Peter Geoghegan wrote:
> Assuming your concern is more or less limited to those cases where the
> same page could be frozen an unbounded number of times (once or almost
> once per VACUUM), then I think we fully agree. We ought to converge on
> the right behavior ov
On Fri, Sep 29, 2023 at 7:55 AM Robert Haas wrote:
> On Thu, Sep 28, 2023 at 12:03 AM Peter Geoghegan wrote:
> > But isn't the main problem *not* freezing when we could and
> > should have? (Of course the cost of freezing is very relevant, but
> > it's still secondary.)
>
> Perhaps this is all in
On Thu, Sep 28, 2023 at 12:03 AM Peter Geoghegan wrote:
> But isn't the main problem *not* freezing when we could and
> should have? (Of course the cost of freezing is very relevant, but
> it's still secondary.)
Perhaps this is all in how you look at it, but I don't see it this
way. It's easy to
On Wed, Sep 27, 2023 at 7:09 PM Melanie Plageman
wrote:
> At the risk of seeming too execution-focused, I want to try and get more
> specific. Here is a description of an example implementation to test my
> understanding:
>
> In table-level stats, save two numbers: younger_than_cpt/older_than_cpt
On Fri, Sep 8, 2023 at 12:07 AM Andres Freund wrote:
>
> Hi,
>
> On 2023-09-06 10:35:17 -0400, Robert Haas wrote:
> > On Wed, Sep 6, 2023 at 1:09 AM Andres Freund wrote:
> > > Yea, it'd be useful to have a reasonably approximate wall clock time for
> > > the
> > > last modification of a page. We
On Wed, Sep 27, 2023 at 6:35 PM Andres Freund wrote:
> > if insert LSN - RedoRecPtr < insert LSN - page LSN
> > page is older than the most recent checkpoint start, so freeze it
> > regardless of whether or not it would emit an FPI
> >
> > What aggressiveness levels should there be? What sho
On 2023-09-27 19:09:41 -0400, Melanie Plageman wrote:
> On Wed, Sep 27, 2023 at 3:25 PM Robert Haas wrote:
> >
> > On Wed, Sep 27, 2023 at 12:34 PM Andres Freund wrote:
> > > One way to deal with that would be to not track the average age in
> > > LSN-difference-bytes, but convert the value to so
On Wed, Sep 27, 2023 at 5:42 PM Melanie Plageman
wrote:
> On Wed, Sep 27, 2023 at 5:27 PM Peter Geoghegan wrote:
> > What about my idea of holding back when some tuples are already frozen
> > from before? Admittedly that's still a fairly raw idea, but something
> > along those lines seems promisi
On Wed, Sep 27, 2023 at 6:07 PM Andres Freund wrote:
> > I would be sure to look out for new inserts that "unfreeze" pages, too
> > -- ideally you'd have instrumentation that caught that, in order to
> > get a general sense of the extent of the problem in each of your
> > chosen representative wor
Hi,
On 2023-09-27 17:43:04 -0700, Peter Geoghegan wrote:
> On Wed, Sep 27, 2023 at 5:20 PM Melanie Plageman
> wrote:
> > > Can you define "unfreeze"? I don't know if this newly invented term
> > > refers to unsetting a page that was marked all-frozen following (say)
> > > an UPDATE, or if it refe
On Wed, Sep 27, 2023 at 5:20 PM Melanie Plageman
wrote:
> > Can you define "unfreeze"? I don't know if this newly invented term
> > refers to unsetting a page that was marked all-frozen following (say)
> > an UPDATE, or if it refers to choosing to not freeze when the option
> > was available (in t
On Wed, Sep 27, 2023 at 5:27 PM Peter Geoghegan wrote:
>
> On Wed, Sep 27, 2023 at 1:45 PM Andres Freund wrote:
> > I am much more concerned about cases where
> > opportunistic freezing requires an FPI - it'll often *still* be the right
> > choice to freeze the page, but we need a way to prevent
On Wed, Sep 27, 2023 at 7:39 PM Peter Geoghegan wrote:
>
> On Wed, Sep 27, 2023 at 4:09 PM Melanie Plageman
> wrote:
> > At the risk of seeming too execution-focused, I want to try and get more
> > specific. Here is a description of an example implementation to test my
> > understanding:
> >
> >
On Wed, Sep 27, 2023 at 4:09 PM Melanie Plageman
wrote:
> At the risk of seeming too execution-focused, I want to try and get more
> specific. Here is a description of an example implementation to test my
> understanding:
>
> In table-level stats, save two numbers: younger_than_cpt/older_than_cpt
On Wed, Sep 27, 2023 at 3:25 PM Robert Haas wrote:
>
> On Wed, Sep 27, 2023 at 12:34 PM Andres Freund wrote:
> > One way to deal with that would be to not track the average age in
> > LSN-difference-bytes, but convert the value to some age metric at that
> > time. If we e.g. were to convert the b
On Wed, Sep 27, 2023 at 2:26 PM Peter Geoghegan wrote:
> On Wed, Sep 27, 2023 at 1:45 PM Andres Freund wrote:
> > I think we need to make vacuums on large tables much more aggressive than
> > they
> > are now, independent of opportunistic freezing heuristics. It's idiotic that
> > on large table
On Wed, Sep 27, 2023 at 1:45 PM Andres Freund wrote:
> On 2023-09-27 13:14:41 -0700, Peter Geoghegan wrote:
> > As a general rule, I think that we're better of gambling against
> > future FPIs, and then pulling back if we go too far. The fact that we
> > went one VACUUM operation without the workl
Hi,
On 2023-09-27 13:14:41 -0700, Peter Geoghegan wrote:
> On Wed, Sep 27, 2023 at 1:03 PM Andres Freund wrote:
> > I suspect that medium term the better approach would be to be much more
> > aggressive about setting all-visible, including as part of page-level
> > visibility checks, and to deal
On Wed, Sep 27, 2023 at 1:03 PM Andres Freund wrote:
> I suspect that medium term the better approach would be to be much more
> aggressive about setting all-visible, including as part of page-level
> visibility checks, and to deal with the concern of vacuum not processing such
> pages soon by not
On Wed, Sep 27, 2023 at 12:34 PM Andres Freund wrote:
> What do you mean with "always freeze aggressively" - do you mean 'aggressive'
> autovacuums? Or opportunistic freezing being aggressive? I don't know why the
> former would be the case?
I meant the latter.
> > When it grows large enough, we
On Wed, Sep 27, 2023 at 2:36 PM Melanie Plageman
wrote:
> It seems like the ideal freeze pattern for an insert-only table would be
> to freeze as soon as the page is full before any checkpoints which could
> force you to emit an FPI.
Yes. So imagine we have two freeze criteria:
1. Do not ever op
On Wed, Sep 27, 2023 at 11:36 AM Melanie Plageman
wrote:
> One big sticking point for me (brought up elsewhere in this thread, but,
> AFAICT never resolved) is that it seems like the fact that we mark pages
> all-visible even when not freezing them means that no matter what
> heuristic we use, we
Hi,
On 2023-09-27 15:45:06 -0400, Robert Haas wrote:
> > One big sticking point for me (brought up elsewhere in this thread, but,
> > AFAICT never resolved) is that it seems like the fact that we mark pages
> > all-visible even when not freezing them means that no matter what
> > heuristic we use,
On Wed, Sep 27, 2023 at 12:45 PM Robert Haas wrote:
> > One big sticking point for me (brought up elsewhere in this thread, but,
> > AFAICT never resolved) is that it seems like the fact that we mark pages
> > all-visible even when not freezing them means that no matter what
> > heuristic we use,
On Wed, Sep 27, 2023 at 10:46 AM Andres Freund wrote:
> I don't disagree that we should do something in that direction - I just don't
> see the raw number of unfrozen pages being useful in that regard. If you have
> a database where no pages live long, we don't need to freeze
> oppportunistically,
Hi,
On 2023-09-27 10:25:00 -0700, Peter Geoghegan wrote:
> On Wed, Sep 27, 2023 at 10:01 AM Andres Freund wrote:
> > On 2023-09-26 09:07:13 -0700, Peter Geoghegan wrote:
> > I don't think doing this on a system wide basis with a metric like #unfrozen
> > pages is a good idea. It's quite common to
On Wed, Sep 27, 2023 at 10:01 AM Andres Freund wrote:
> On 2023-09-26 09:07:13 -0700, Peter Geoghegan wrote:
> I don't think doing this on a system wide basis with a metric like #unfrozen
> pages is a good idea. It's quite common to have short lived data in some
> tables while also having long-liv
Hi,
On 2023-09-26 09:07:13 -0700, Peter Geoghegan wrote:
> On Tue, Sep 26, 2023 at 8:19 AM Andres Freund wrote:
> > However, I'm not at all convinced doing this on a system wide level is a
> > good
> > idea. Databases do often contain multiple types of workloads at the same
> > time. E.g., we wa
Hi,
(I wrote the first part of the email before Robert and I chatted on a call, I
left it in the email for posterity)
On 2023-09-26 13:49:32 -0400, Robert Haas wrote:
> On Tue, Sep 26, 2023 at 11:11 AM Andres Freund wrote:
> > As long as the most extreme cases are prevented, unnecessarily freezi
On Tue, Sep 26, 2023 at 11:11 AM Andres Freund wrote:
> That I'd like you to expand on "using the RedoRecPtr of the latest checkpoint
> rather than the LSN of the previou vacuum." - I can think of ways of doing so
> that could end up with quite different behaviour...
Yeah, me too. I'm not sure wh
On Tue, Sep 26, 2023 at 8:19 AM Andres Freund wrote:
> However, I'm not at all convinced doing this on a system wide level is a good
> idea. Databases do often contain multiple types of workloads at the same
> time. E.g., we want to freeze aggressively in a database that has the bulk of
> its size
Hi,
On 2023-09-25 14:16:46 -0700, Peter Geoghegan wrote:
> On Mon, Sep 25, 2023 at 11:45 AM Robert Haas wrote:
> I'm surprised that there hasn't been any discussion of the absolute
> amount of system-wide freeze debt on this thread. If 90% of the pages
> in the entire database are frozen, it'll g
Hi,
On 2023-09-25 14:45:07 -0400, Robert Haas wrote:
> On Fri, Sep 8, 2023 at 12:07 AM Andres Freund wrote:
> > > Downthread, I proposed using the RedoRecPtr of the latest checkpoint
> > > rather than the LSN of the previou vacuum. I still like that idea.
> >
> > Assuming that "downthread" refere
On Mon, Sep 25, 2023 at 11:45 AM Robert Haas wrote:
> > The reason I was thinking of using the "lsn at the end of the last vacuum",
> > is
> > that it seems to be more adapative to the frequency of vacuuming.
>
> Yes, but I think it's *too* adaptive. The frequency of vacuuming can
> plausibly be
On Sat, Sep 23, 2023 at 3:53 PM Melanie Plageman
wrote:
> Freeze tuples on a page opportunistically if the page would be totally
> frozen and:
>
> 4. Buffer is already dirty and no FPI is required OR page LSN is older
> than 33% of the LSNs since the last vacuum of the table.
>
> 5. Buffer is a
On Fri, Sep 8, 2023 at 12:07 AM Andres Freund wrote:
> > Downthread, I proposed using the RedoRecPtr of the latest checkpoint
> > rather than the LSN of the previou vacuum. I still like that idea.
>
> Assuming that "downthread" references
> https://postgr.es/m/CA%2BTgmoYb670VcDFbekjn2YQOKF9a7e-kBF
On Sat, Sep 23, 2023 at 3:53 PM Melanie Plageman
wrote:
>
> Workload F:
>
> +--++-++--+
> | algo | WAL GB | cptr bgwriter writes| other reads/writes | IO time AV
> worker|
> +--++-+
On Mon, Aug 28, 2023 at 4:30 PM Melanie Plageman
wrote:
> On Mon, Aug 28, 2023 at 12:26 PM Robert Haas wrote:
> > In row D, your algorithms are all bad, really bad. I don't quite
> > understand how it can be that bad, actually.
>
> So, I realize now that this test was poorly designed. I meant it
Hi,
On 2023-09-07 22:29:04 -0700, Peter Geoghegan wrote:
> On Thu, Sep 7, 2023 at 9:45 PM Andres Freund wrote:
> > I.e. setting an, otherwise unmodified, page all-visible won't trigger an FPI
> > if checksums are disabled, but will FPI with checksums enabled. I think
> > that's
> > a substantial
On Thu, Sep 7, 2023 at 10:46 PM Andres Freund wrote:
> You mean the current heuristic or some new heuristic we're coming up with in
> this thread? If the former, unfortunately I think the current heuristic often
> won't trigger in cases where freezing would be fine, e.g. on an insert-mostly
> (or
Hi,
On 2023-09-07 21:45:22 -0700, Andres Freund wrote:
> In contrast to that, freezing will almost always trigger an FPI (except for
> empty pages, but we imo ought to stop setting empty pages all frozen [1]).
>
>
> Yep, a quick experiment confirms that:
>
> DROP TABLE IF EXISTS foo;
> CREATE T
On Thu, Sep 7, 2023 at 9:45 PM Andres Freund wrote:
> I.e. setting an, otherwise unmodified, page all-visible won't trigger an FPI
> if checksums are disabled, but will FPI with checksums enabled. I think that's
> a substantial difference in WAL volume for insert-only workloads...
Note that all R
Hi,
On 2023-09-06 16:21:31 -0400, Robert Haas wrote:
> On Wed, Sep 6, 2023 at 12:20 PM Peter Geoghegan wrote:
> > If VACUUM freezes too aggressively, then (pretty much by definition)
> > we can be sure that the next VACUUM will scan the same pages -- there
> > may be some scope for VACUUM to "lea
Hi,
On 2023-09-06 10:46:03 -0400, Robert Haas wrote:
> On Fri, Sep 1, 2023 at 9:07 PM Peter Geoghegan wrote:
> > Why not also avoid setting pages all-visible? The WAL records aren't
> > too much smaller than most freeze records these days -- 64 bytes on
> > most systems. I realize that the rules
Hi,
On 2023-09-06 10:35:17 -0400, Robert Haas wrote:
> On Wed, Sep 6, 2023 at 1:09 AM Andres Freund wrote:
> > Yea, it'd be useful to have a reasonably approximate wall clock time for the
> > last modification of a page. We just don't have infrastructure for
> > determining
> > that. We'd need a
On Wed, Sep 6, 2023 at 12:20 PM Peter Geoghegan wrote:
> This was a case where index vacuuming was never required. It's just a
> simple and easy to recreate example of what I think of as a more
> general problem.
OK.
> Why wouldn't we expect a table to have some pages that ought to be
> frozen r
On Wed, Sep 6, 2023 at 7:46 AM Robert Haas wrote:
> It's interesting that when you raised the threshold the rate of
> vacuuming dropped to zero. I haven't seen that behavior. pgbench does
> rely very, very heavily on HOT-pruning, so VACUUM only cleans up a
> small percentage of the dead tuples tha
On Fri, Sep 1, 2023 at 9:07 PM Peter Geoghegan wrote:
> When I reported this a couple of years ago, I noticed that autovacuum
> would spin whenever I set autovacuum_vacuum_scale_factor to 0.02. But
> autovacuum would *never* run (outside of antiwraparound autovacuums)
> when it was set just a litt
On Wed, Sep 6, 2023 at 1:09 AM Andres Freund wrote:
> Yea, it'd be useful to have a reasonably approximate wall clock time for the
> last modification of a page. We just don't have infrastructure for determining
> that. We'd need an LSN->time mapping (xid->time wouldn't be particularly
> useful, I
Hi,
On 2023-08-28 12:26:01 -0400, Robert Haas wrote:
> On Mon, Aug 28, 2023 at 10:00 AM Melanie Plageman
> wrote:
> > For the second goal, I've relied on past data to predict future
> > behavior, so I tried several criteria to estimate the likelihood that a
> > page will not be imminently modifie
On Fri, Sep 1, 2023 at 12:34 PM Robert Haas wrote:
> If the table
> is being vacuumed frequently, then the last-vacuum-LSN will be newer,
> which means we'll freeze *more* aggressively.
Sort of like how if you set autovacuum_vacuum_scale_factor low enough,
with standard pgbench, with heap fill fa
On Tue, Aug 29, 2023 at 10:22 AM Robert Haas wrote:
> Let's assume for a moment that the rate at which the insert LSN is
> advancing is roughly constant over time, so that it serves as a good
> proxy for wall-clock time. Consider four tables A, B, C, and D that
> are, respectively, vacuumed once p
On Mon, Aug 28, 2023 at 4:30 PM Melanie Plageman
wrote:
> By low-velocity, do you mean lower overall TPS? In that case, wouldn't you be
> less likely to run into xid wraparound and thus need less aggressive
> opportunistic freezing?
Yes. But it also means that we've got slack capacity to do extra
On Mon, Aug 28, 2023 at 4:15 PM Melanie Plageman
wrote:
> On Mon, Aug 28, 2023 at 5:06 PM Peter Geoghegan wrote:
> > What I've described in a scheme for deciding to not freeze where that
> > would usually happen -- a scheme for *vetoing* page freezing -- rather
> > than a scheme for deciding to f
On Mon, Aug 28, 2023 at 5:06 PM Peter Geoghegan wrote:
>
> On Mon, Aug 28, 2023 at 1:17 PM Robert Haas wrote:
> > I'm sure this could be implemented, but it's unclear to me why you
> > would expect it to perform well. Freezing a page that has no frozen
> > tuples yet isn't cheaper than freezing o
On Mon, Aug 28, 2023 at 1:17 PM Robert Haas wrote:
> I'm sure this could be implemented, but it's unclear to me why you
> would expect it to perform well. Freezing a page that has no frozen
> tuples yet isn't cheaper than freezing one that does, so for this idea
> to be a win, the presence of froz
On Mon, Aug 28, 2023 at 12:26 PM Robert Haas wrote:
>
> On Mon, Aug 28, 2023 at 10:00 AM Melanie Plageman
> wrote:
> Then there's the question of whether it's the right metric. My first
> reaction is to think that it sounds pretty good. One thing I really
> like about it is that if the table is b
On Mon, Aug 28, 2023 at 3:09 PM Peter Geoghegan wrote:
> I've long emphasized the importance of designs that just try to avoid
> disaster. With that in mind, I wonder: have you thought about
> conditioning page freezing on whether or not there are already some
> frozen tuples on the page? You coul
On Mon, Aug 28, 2023 at 7:00 AM Melanie Plageman
wrote:
> To do this, we need to save that insert LSN
> somewhere. In the attached WIP patch, I saved it in the table stats, for
> now -- knowing that those are not crash-safe.
What patch? You didn't actually attach one.
> Other discarded heuristic
On Mon, Aug 28, 2023 at 7:00 AM Melanie Plageman
wrote:
> I believe that there are two goals that should dictate whether or not we
> should perform opportunistic freezing:
>
> 1. Is it cheap? For example, if the buffer is already dirty, then no
> write amplification occurs, since it must be wr
On Mon, Aug 28, 2023 at 10:00 AM Melanie Plageman
wrote:
> For the second goal, I've relied on past data to predict future
> behavior, so I tried several criteria to estimate the likelihood that a
> page will not be imminently modified. What was most effective was
> Andres' suggestion of comparing
On Fri, Jul 28, 2023 at 3:27 PM Melanie Plageman
wrote:
> On Fri, Jul 28, 2023 at 3:00 PM Peter Geoghegan wrote:
> > > Is this test meant to guard against unnecessary freezing or to avoid
> > > freezing when the cost is too high? That is, are we trying to
> > > determine how likely it is that the
On Fri, Jul 28, 2023 at 5:03 PM Melanie Plageman
wrote:
> When you were working on this, what were the downsides of only having the
> criteria that 1) page would be all_visible/all_frozen and 2) we did prune
> something (i.e. no consideration of FPIs)?
Conditioning freezing on "all_visible && all
On Fri, Jul 28, 2023 at 4:49 PM Peter Geoghegan wrote:
>
> On Fri, Jul 28, 2023 at 4:30 PM Andres Freund wrote:
> > > Put differently, I can't see any reason to care whether pruning
> > > emitted an FPI or not. Either way, it's very unlikely that freezing
> > > needs to do so.
> >
> > +many
> >
>
On Fri, Jul 28, 2023 at 4:30 PM Andres Freund wrote:
> Using FPIs having been generated during pruning as part of the logic to decide
> whether to freeze makes no sense to me. We likely need some heuristic here,
> but I suspect it has to look quite different than the FPI condition.
Why do we need
On Fri, Jul 28, 2023 at 4:30 PM Andres Freund wrote:
> > Put differently, I can't see any reason to care whether pruning
> > emitted an FPI or not. Either way, it's very unlikely that freezing
> > needs to do so.
>
> +many
>
> Using FPIs having been generated during pruning as part of the logic to
Hi,
On 2023-07-28 16:19:47 -0400, Robert Haas wrote:
> On Fri, Jul 28, 2023 at 3:00 PM Peter Geoghegan wrote:
> > > Or are we trying to determine how likely
> > > the freeze record is to emit an FPI and avoid eager freezing when it
> > > isn't worth the cost?
> >
> > No, that's not something that
On Fri, Jul 28, 2023 at 3:00 PM Peter Geoghegan wrote:
> > Or are we trying to determine how likely
> > the freeze record is to emit an FPI and avoid eager freezing when it
> > isn't worth the cost?
>
> No, that's not something that we're doing right now (we just talked
> about doing something lik
On Fri, Jul 28, 2023 at 3:27 PM Melanie Plageman
wrote:
> I see. I don't have an opinion on the "best of a bad situation"
> argument. Though, I think it is worth amending the comment in the code
> to include this explanation.
I think that the user-facing docs should be completely overhauled to
ma
On Fri, Jul 28, 2023 at 3:00 PM Peter Geoghegan wrote:
>
> On Fri, Jul 28, 2023 at 11:13 AM Melanie Plageman
> wrote:
> > if (pagefrz.freeze_required || tuples_frozen == 0 ||
> > (prunestate->all_visible && prunestate->all_frozen &&
> > fpi_before != pgWalUsage.wal_fpi))
> >
On Fri, Jul 28, 2023 at 11:13 AM Melanie Plageman
wrote:
> if (pagefrz.freeze_required || tuples_frozen == 0 ||
> (prunestate->all_visible && prunestate->all_frozen &&
> fpi_before != pgWalUsage.wal_fpi))
> {
>
> I'm trying to understand the condition fpi_before !=
> pgWal
Hi,
While hacking on the pruning and page-level freezing code and am a bit
confused by the test guarding eager freezing [1]:
/*
* Freeze the page when heap_prepare_freeze_tuple indicates that at least
* one XID/MXID from before FreezeLimit/MultiXactCutoff is present. Also
* f
91 matches
Mail list logo
