On Wed, Mar 24, 2021 at 12:10 PM Peter Geoghegan wrote:
>
> On Tue, Mar 23, 2021 at 8:14 AM Masahiko Sawada wrote:
> > By this patch series, btree indexes became like hash indexes in terms
> > of amvacuumcleanup. We do an index scan at btvacuumcleanup() in the
> > two cases: metapage upgrading an
On Tue, Mar 23, 2021 at 8:14 AM Masahiko Sawada wrote:
> By this patch series, btree indexes became like hash indexes in terms
> of amvacuumcleanup. We do an index scan at btvacuumcleanup() in the
> two cases: metapage upgrading and more than 5%
> deleted-but-not-yet-recycled pages. Both cases see
On Mon, Mar 22, 2021 at 7:27 AM Peter Geoghegan wrote:
>
> On Wed, Mar 10, 2021 at 5:34 PM Peter Geoghegan wrote:
> > Here is another bitrot-fix-only revision, v9. Just the recycling patch
> > again.
>
> I committed the final nbtree page deletion patch just now -- the one
> that attempts to make
On Wed, Mar 10, 2021 at 5:34 PM Peter Geoghegan wrote:
> Here is another bitrot-fix-only revision, v9. Just the recycling patch again.
I committed the final nbtree page deletion patch just now -- the one
that attempts to make recycling happen for newly deleted pages. Thanks
for all your work on p
On Mon, Mar 1, 2021 at 7:25 PM Peter Geoghegan wrote:
> Attached is v8 of the patch series, which has new patches. No real
> changes compared to v7 for the first patch, though.
Here is another bitrot-fix-only revision, v9. Just the recycling patch again.
I'll commit this when we get your patch c
On Sun, Mar 7, 2021 at 8:52 PM Masahiko Sawada wrote:
> Yeah, I think that's right.
>
> Perhaps we can do something so that autovacuums triggered by
> autovacuum_vacuum_insert_scale_factor are triggered on only a true
> insert-only case (e.g., by checking if n_dead_tup is 0).
Right -- that's real
On Tue, Mar 2, 2021 at 1:42 PM Peter Geoghegan wrote:
>
> On Mon, Mar 1, 2021 at 8:06 PM Masahiko Sawada wrote:
> > I think that removing vacuum_cleanup_index_scale_factor in the back
> > branches would affect the existing installation much. It would be
> > better to have btree indexes not use th
On Tue, Mar 2, 2021 at 1:06 PM Masahiko Sawada wrote:
>
> On Tue, Mar 2, 2021 at 6:40 AM Peter Geoghegan wrote:
> >
> > On Sun, Feb 28, 2021 at 8:08 PM Masahiko Sawada
> > wrote:
> > > > Even though "the letter of the law" favors removing the
> > > > vacuum_cleanup_index_scale_factor GUC + para
On Mon, Mar 1, 2021 at 8:06 PM Masahiko Sawada wrote:
> I think that removing vacuum_cleanup_index_scale_factor in the back
> branches would affect the existing installation much. It would be
> better to have btree indexes not use this parameter while not changing
> the contents of meta page. That
On Tue, Mar 2, 2021 at 6:40 AM Peter Geoghegan wrote:
>
> On Sun, Feb 28, 2021 at 8:08 PM Masahiko Sawada wrote:
> > > Even though "the letter of the law" favors removing the
> > > vacuum_cleanup_index_scale_factor GUC + param in the way I have
> > > outlined, that is not the only thing that matt
On Mon, Mar 1, 2021 at 1:40 PM Peter Geoghegan wrote:
> > Since it seems not a bug I personally think we don't need to do
> > anything for back branches. But if we want not to trigger an index
> > scan by vacuum_cleanup_index_scale_factor, we could change the default
> > value to a high value (say
On Sun, Feb 28, 2021 at 8:08 PM Masahiko Sawada wrote:
> > Even though "the letter of the law" favors removing the
> > vacuum_cleanup_index_scale_factor GUC + param in the way I have
> > outlined, that is not the only thing that matters -- we must also
> > consider "the spirit of the law".
> > I
On Fri, Feb 26, 2021 at 9:58 AM Peter Geoghegan wrote:
> > If we don't want btvacuumcleanup() to collect index statistics, we can
> > remove vacuum_cleanup_index_scale_factor (at least from btree
> > perspectives), as you mentioned. One thing that may be worth
> > mentioning is that the difference
On Fri, Feb 26, 2021 at 9:58 AM Peter Geoghegan wrote:
>
> On Thu, Feb 25, 2021 at 5:42 AM Masahiko Sawada wrote:
> > btvacuumcleanup() has been playing two roles: recycling deleted pages
> > and collecting index statistics.
>
> Right.
>
> I pushed the VACUUM VERBOSE "index pages newly deleted"
On Thu, Feb 25, 2021 at 5:42 AM Masahiko Sawada wrote:
> btvacuumcleanup() has been playing two roles: recycling deleted pages
> and collecting index statistics.
Right.
I pushed the VACUUM VERBOSE "index pages newly deleted"
instrumentation patch earlier - it really isn't complicated or
controv
On Thu, Feb 25, 2021 at 1:42 PM Peter Geoghegan wrote:
>
> On Mon, Feb 22, 2021 at 2:54 PM Peter Geoghegan wrote:
> > Good point. I think that the structure should make the page deletion
> > triggering condition have only secondary importance -- it is only
> > described at all to be complete and
On Wed, Feb 24, 2021 at 8:13 PM Justin Pryzby wrote:
> I think 8e12f4a25 wasn't quite aggressive enough in its changes, and I had
> another patch laying around. I rebased and came up with this.
See my remarks/questions about vacuum_cleanup_index_scale_factor
addressed to Masahiko from a little e
On Mon, Feb 22, 2021 at 2:54 PM Peter Geoghegan wrote:
> Good point. I think that the structure should make the page deletion
> triggering condition have only secondary importance -- it is only
> described at all to be complete and exhaustive. The
> vacuum_cleanup_index_scale_factor-related thresh
On Mon, Feb 22, 2021 at 02:54:54PM -0800, Peter Geoghegan wrote:
> On Mon, Feb 22, 2021 at 4:21 AM Masahiko Sawada wrote:
> > The 0001 patch looks good to me. In the documentation, I think we need
> > to update the following paragraph in the description of
> > vacuum_cleanup_index_scale_factor:
>
On Tue, Feb 23, 2021 at 7:55 AM Peter Geoghegan wrote:
>
> On Mon, Feb 22, 2021 at 4:21 AM Masahiko Sawada wrote:
> > The 0001 patch looks good to me. In the documentation, I think we need
> > to update the following paragraph in the description of
> > vacuum_cleanup_index_scale_factor:
>
> Good
On Mon, Feb 22, 2021 at 4:21 AM Masahiko Sawada wrote:
> The 0001 patch looks good to me. In the documentation, I think we need
> to update the following paragraph in the description of
> vacuum_cleanup_index_scale_factor:
Good point. I think that the structure should make the page deletion
trigg
On Fri, Feb 19, 2021 at 3:18 PM Masahiko Sawada wrote:
>
> On Fri, Feb 19, 2021 at 3:12 PM Peter Geoghegan wrote:
> >
> > On Thu, Feb 18, 2021 at 3:13 AM Masahiko Sawada
> > wrote:
> > > Agreed. Thanks for your explanation.
> >
> > Attached is v5, which has some of the changes I talked about. C
On Fri, Feb 19, 2021 at 3:12 PM Peter Geoghegan wrote:
>
> On Thu, Feb 18, 2021 at 3:13 AM Masahiko Sawada wrote:
> > Agreed. Thanks for your explanation.
>
> Attached is v5, which has some of the changes I talked about. Changes
> from v4 include:
>
> * Now only updates metapage during btvacuumcl
On Tue, Feb 16, 2021 at 12:26 PM Peter Geoghegan wrote:
>
> On Mon, Feb 15, 2021 at 3:15 AM Masahiko Sawada wrote:
> > Yes. I think this would simplify the problem by resolving almost all
> > problems related to indefinite deferring page recycle.
> >
> > We will be able to recycle almost all just
On Wed, Feb 17, 2021 at 5:41 AM Peter Geoghegan wrote:
>
> On Tue, Feb 16, 2021 at 11:35 AM Peter Geoghegan wrote:
> > Isn't btvacuumcleanup() (or any other amvacuumcleanup() routine)
> > entitled to rely on the bulk delete stats being set in the way I've
> > described? I assumed that that was ok
On Tue, Feb 16, 2021 at 11:35 AM Peter Geoghegan wrote:
> Isn't btvacuumcleanup() (or any other amvacuumcleanup() routine)
> entitled to rely on the bulk delete stats being set in the way I've
> described? I assumed that that was okay in general, but I haven't
> tested parallel VACUUM specifically
On Tue, Feb 16, 2021 at 4:17 AM Masahiko Sawada wrote:
> Ugh, yes, I think it's a bug.
I was actually thinking of a similar bug in nbtree deduplication when
I spotted this one -- see commit 48e12913. The index AM API stuff is
tricky.
> When developing this feature, in an old version patch, we us
On Tue, Feb 16, 2021 at 3:52 PM Peter Geoghegan wrote:
>
> On Mon, Feb 15, 2021 at 7:26 PM Peter Geoghegan wrote:
> > Actually, there is one more reason why I bring up idea 1 now: I want
> > to hear your thoughts on the index AM API questions now, which idea 1
> > touches on. Ideally all of the d
On Mon, Feb 15, 2021 at 7:26 PM Peter Geoghegan wrote:
> Actually, there is one more reason why I bring up idea 1 now: I want
> to hear your thoughts on the index AM API questions now, which idea 1
> touches on. Ideally all of the details around the index AM VACUUM APIs
> (i.e. when and where the
On Mon, Feb 15, 2021 at 3:15 AM Masahiko Sawada wrote:
> Yes. I think this would simplify the problem by resolving almost all
> problems related to indefinite deferring page recycle.
>
> We will be able to recycle almost all just-deleted pages in practice
> especially when btvacuumscan() took a lo
On Sun, Feb 14, 2021 at 3:47 PM Peter Geoghegan wrote:
>
> On Fri, Feb 12, 2021 at 9:04 PM Peter Geoghegan wrote:
> > On Fri, Feb 12, 2021 at 8:38 PM Masahiko Sawada
> > wrote:
> > > I agree that there already are huge problems in that case. But I think
> > > we need to consider an append-only
On Sat, Feb 13, 2021 at 10:47 PM Peter Geoghegan wrote:
> It will be rare. But more importantly, the fact that scenario is now
> an extreme case justifies treating it as an extreme case. We can teach
> _bt_vacuum_needs_cleanup() to recognize it as an extreme case, too. In
> particular, I think tha
On Fri, Feb 12, 2021 at 9:04 PM Peter Geoghegan wrote:
> On Fri, Feb 12, 2021 at 8:38 PM Masahiko Sawada wrote:
> > I agree that there already are huge problems in that case. But I think
> > we need to consider an append-only case as well; after bulk deletion
> > on an append-only table, vacuum d
On Fri, Feb 12, 2021 at 10:27 PM Victor Yegorov wrote:
> I'd like to outline one relevant case.
>
> Quite often bulk deletes are done on a time series data (oldest) and
> effectively
> removes a continuous chunk of data at the (physical) beginning of the table,
> this is especially true for the a
сб, 13 февр. 2021 г. в 05:39, Masahiko Sawada :
> > (BTW, I've been using txid_current() for my own "laptop testing", as a
> > way to work around this issue.)
> >
> > * More generally, if you really can't do recycling of pages that you
> > deleted during the last VACUUM during this VACUUM (perhaps
On Fri, Feb 12, 2021 at 8:38 PM Masahiko Sawada wrote:
> I agree that there already are huge problems in that case. But I think
> we need to consider an append-only case as well; after bulk deletion
> on an append-only table, vacuum deletes heap tuples and index tuples,
> marking some index pages
On Thu, Feb 11, 2021 at 12:10 PM Peter Geoghegan wrote:
>
> On Wed, Feb 10, 2021 at 2:20 AM Masahiko Sawada wrote:
> > Thank you for working on this!
>
> I'm glad that I finally found time for it! It seems like it'll make
> things easier elsewhere.
>
> Attached is v3 of the index. I'll describe t
On Wed, Feb 10, 2021 at 7:10 PM Peter Geoghegan wrote:
> Attached is v3 of the index. I'll describe the changes I made in more
> detail in my response to your points below.
I forget to mention that v3 adds several assertions like this one:
Assert(!_bt_page_recyclable(BufferGetPage(buf)));
These
On Wed, Feb 10, 2021 at 2:20 AM Masahiko Sawada wrote:
> Thank you for working on this!
I'm glad that I finally found time for it! It seems like it'll make
things easier elsewhere.
Attached is v3 of the index. I'll describe the changes I made in more
detail in my response to your points below.
On Tue, Feb 9, 2021 at 11:58 PM Heikki Linnakangas wrote:
> Thanks for picking this up!
I actually had a patch for this in 2019, albeit one that remained in
rough shape until recently. Must have forgotten about it.
> Is it really worth the trouble to maintain 'level' on deleted pages? All
> you
On Wed, Feb 10, 2021 at 10:53 AM Peter Geoghegan wrote:
>
> On Tue, Feb 9, 2021 at 2:14 PM Peter Geoghegan wrote:
> > The first patch teaches nbtree to use 64-bit transaction IDs here, and
> > so makes it impossible to leak deleted nbtree pages. This patch is the
> > nbtree equivalent of commit 6
On 10/02/2021 00:14, Peter Geoghegan wrote:
There is a long standing problem with the way that nbtree page
deletion places deleted pages in the FSM for recycling: The use of a
32-bit XID within the deleted page (in the special
area's/BTPageOpaqueData struct's btpo.xact field) is not robust
agains
On Tue, Feb 9, 2021 at 5:53 PM Peter Geoghegan wrote:
> Here is a plan that allows us to stop storing any kind of XID in the
> metapage in all cases:
Attached is v2, which deals with the metapage 32-bit
XID/btm_oldest_btpo_xact issue using the approach I described earlier.
We don't store an XID i
On Tue, Feb 9, 2021 at 2:14 PM Peter Geoghegan wrote:
> The first patch teaches nbtree to use 64-bit transaction IDs here, and
> so makes it impossible to leak deleted nbtree pages. This patch is the
> nbtree equivalent of commit 6655a729, which made GiST use 64-bit XIDs
> due to exactly the same
There is a long standing problem with the way that nbtree page
deletion places deleted pages in the FSM for recycling: The use of a
32-bit XID within the deleted page (in the special
area's/BTPageOpaqueData struct's btpo.xact field) is not robust
against XID wraparound, which can lead to permanentl
45 matches
Mail list logo
