On Sun, Nov 17, 2019 at 9:18 PM Alexander Korotkov
wrote:
> So, I've put this explanation into README patch. I just change
> designation to better match with Lehman & Yao paper and did some minor
> enchantments.
>
> I'm going to push this patchset if no objections.
So, pushed with minor changes
On Mon, Nov 11, 2019 at 2:42 AM Alexander Korotkov
wrote:
> I'm sorry for late reply. I was busy with various things. Also
> digging into these details took some time. Please find my explanation
> below.
>
> On Wed, Oct 30, 2019 at 2:34 AM Peter Geoghegan wrote:
> > In general, it seems very i
Hi!
I'm sorry for late reply. I was busy with various things. Also
digging into these details took some time. Please find my explanation
below.
On Wed, Oct 30, 2019 at 2:34 AM Peter Geoghegan wrote:
> In general, it seems very important to be clear about exactly how the
> key space works. The
On Sun, Oct 27, 2019 at 12:04 PM Alexander Korotkov
wrote:
> (0001-Fix-deadlock-between-ginDeletePage-and-ginStepRigh-3.patch)
Some thoughts on the first patch:
* How do comparisons of items in each type of B-Tree work?
I would like to see a statement about invariants similar to nbtree's
"subtr
On Mon, Oct 28, 2019 at 2:00 PM Peter Geoghegan wrote:
> On Sun, Oct 27, 2019 at 7:04 PM Alexander Korotkov
> wrote:
> > It tool me a lot of efforts to revise a concurrency section in README.
> > I can't judge the result, but I probably did my best. I'd like to
> > commit this patchset including
Hi Alexander,
On Sun, Oct 27, 2019 at 7:04 PM Alexander Korotkov
wrote:
> It tool me a lot of efforts to revise a concurrency section in README.
> I can't judge the result, but I probably did my best. I'd like to
> commit this patchset including both bug fixes and README update. But
> I'd like y
Hi, Peter!
On Fri, Oct 4, 2019 at 12:05 AM Alexander Korotkov
wrote:
> I'm planning to continue work on README, comments and commit messages.
It tool me a lot of efforts to revise a concurrency section in README.
I can't judge the result, but I probably did my best. I'd like to
commit this patc
On Tue, Oct 1, 2019 at 5:55 AM Alexander Korotkov
wrote:
> On Mon, Sep 30, 2019 at 10:54 PM Peter Geoghegan wrote:
> >
> > On Sun, Sep 29, 2019 at 8:12 AM Alexander Korotkov
> > wrote:
> > > I just managed to reproduce this using two sessions on master branch.
> > >
> > > session 1
> > > ses
On Mon, Sep 30, 2019 at 10:54 PM Peter Geoghegan wrote:
>
> On Sun, Sep 29, 2019 at 8:12 AM Alexander Korotkov
> wrote:
> > I just managed to reproduce this using two sessions on master branch.
> >
> > session 1
> > session 2
>
> Was the involvement of the pending list stuff in Chen's example
On Sun, Sep 29, 2019 at 8:12 AM Alexander Korotkov
wrote:
> I just managed to reproduce this using two sessions on master branch.
>
> session 1
> session 2
Was the involvement of the pending list stuff in Chen's example just a
coincidence? Can you recreate the problem while eliminating that
f
On Mon, Sep 30, 2019 at 11:00 AM Alexander Korotkov
wrote:
> Thank you for pointing. I remember this thread, but don't remember
> details. I'll reread it.
I think that doing this now would be a good idea:
"""
Defensively checking the page type (pending, posting, etc) within
scanPendingInsert()
On Sun, Sep 29, 2019 at 10:38 PM Andrey Borodin wrote:
> As far as I understand deleted page is stamped with
> GinPageSetDeleteXid(page, ReadNewTransactionId());
> It will not be recycled until that Xid is far behind.
That only gets used within posting tree pages, though.
ginInsertCleanup() is co
On Sun, Sep 29, 2019 at 10:53 PM Peter Geoghegan wrote:
> On Sun, Sep 29, 2019 at 7:38 AM Alexander Korotkov
> wrote:
> > Starting from root seems OK for me, because vacuum blocks all
> > concurrent inserts before doing this. But this needs to be properly
> > documented in readme.
>
> I never go
On Mon, Sep 30, 2019 at 8:38 AM Andrey Borodin wrote:
>
> > 29 сент. 2019 г., в 21:27, Alexander Korotkov
> > написал(а):
> >
> > Patch with fix is attached. Idea is simple: ginScanToDelete() now
> > keeps exclusive lock on left page eliminating the need to relock it.
> > So, we preserve left-t
> 29 сент. 2019 г., в 21:27, Alexander Korotkov
> написал(а):
>
> Patch with fix is attached. Idea is simple: ginScanToDelete() now
> keeps exclusive lock on left page eliminating the need to relock it.
> So, we preserve left-to-right locking order and can't deadlock with
> ginStepRight().
On Sun, Sep 29, 2019 at 7:38 AM Alexander Korotkov
wrote:
> Starting from root seems OK for me, because vacuum blocks all
> concurrent inserts before doing this. But this needs to be properly
> documented in readme.
I never got an adequate answer to this closely related question almost
two years
On Sun, Sep 29, 2019 at 6:12 PM Alexander Korotkov
wrote:
> On Sun, Sep 29, 2019 at 5:38 PM Alexander Korotkov
> wrote:
> > On Sun, Sep 29, 2019 at 11:17 AM chenhj wrote:
> > > Does the locking order of autovacuum process(root->right->left) correct?
> > > While insert process lock gin buffer by
On Sun, Sep 29, 2019 at 5:38 PM Alexander Korotkov
wrote:
> On Sun, Sep 29, 2019 at 11:17 AM chenhj wrote:
> > Does the locking order of autovacuum process(root->right->left) correct?
> > While insert process lock gin buffer by order of bottom->top and
> > left->right.
> >
> > 1. vacuum(root->r
Hi!
Thank you for reporting.
On Sun, Sep 29, 2019 at 11:17 AM chenhj wrote:
> Does the locking order of autovacuum process(root->right->left) correct?
> While insert process lock gin buffer by order of bottom->top and left->right.
>
> 1. vacuum(root->right->left):
Starting from root seems OK f
On Sat, Mar 23, 2019 at 11:43 AM Alexander Korotkov
wrote:
> On Fri, Mar 22, 2019 at 11:05 AM Alexander Korotkov
> wrote:
> > On Fri, Mar 22, 2019 at 12:06 AM Alvaro Herrera
> > wrote:
> > > On 2019-Mar-21, Alexander Korotkov wrote:
> > >
> > > > However, I think this still can be backpatched co
On Fri, Mar 22, 2019 at 11:05 AM Alexander Korotkov
wrote:
> On Fri, Mar 22, 2019 at 12:06 AM Alvaro Herrera
> wrote:
> > On 2019-Mar-21, Alexander Korotkov wrote:
> >
> > > However, I think this still can be backpatched correctly. We can
> > > determine whether xlog record data contains deleteX
On Fri, Mar 22, 2019 at 12:06 AM Alvaro Herrera
wrote:
> On 2019-Mar-21, Alexander Korotkov wrote:
>
> > However, I think this still can be backpatched correctly. We can
> > determine whether xlog record data contains deleteXid by its size.
> > See the attached patch. I haven't test this yet. I
On 2019-Mar-21, Alexander Korotkov wrote:
> However, I think this still can be backpatched correctly. We can
> determine whether xlog record data contains deleteXid by its size.
> See the attached patch. I haven't test this yet. I'm going to test
> it. If OK, then push.
Wow, this is so magica
On Thu, 21 Mar 2019 at 15:18, Alexander Korotkov
wrote:
> On Thu, Mar 21, 2019 at 9:26 PM Simon Riggs wrote:
>
>
> > It's been pointed out to me that 52ac6cd2d0cd70e01291e0ac4ee6d068b69bc478
> > introduced a WAL incompatibility that has not been flagged.
> >
> > In ginRedoDeletePage() we use th
On Thu, Mar 21, 2019 at 9:26 PM Simon Riggs wrote:
> On Thu, 13 Dec 2018 at 14:48, Alexander Korotkov wrote:
>> On Thu, Dec 13, 2018 at 10:46 PM Andres Freund wrote:
>> > On 2018-12-13 22:40:59 +0300, Alexander Korotkov wrote:
>> > > It doesn't mater, because we release all locks on every buffer
On Thu, 13 Dec 2018 at 14:48, Alexander Korotkov
wrote:
> On Thu, Dec 13, 2018 at 10:46 PM Andres Freund wrote:
> > On 2018-12-13 22:40:59 +0300, Alexander Korotkov wrote:
> > > It doesn't mater, because we release all locks on every buffer at one
> > > time. The unlock order can have effect on
On Thu, Dec 13, 2018 at 10:40:59PM +0300, Alexander Korotkov wrote:
> On Thu, Dec 13, 2018 at 8:06 PM Andrey Borodin wrote:
> > That's the same variable, one place is definition while other is potential
> > misuse.
> > Seems like these 2 lines [0]
> >
> > + if (BufferIsValid(lbuffer))
> > +
On Thu, Dec 13, 2018 at 10:46 PM Andres Freund wrote:
> On 2018-12-13 22:40:59 +0300, Alexander Korotkov wrote:
> > It doesn't mater, because we release all locks on every buffer at one
> > time. The unlock order can have effect on what waiter will acquire
> > the lock next after ginRedoDeletePag
Hi,
On 2018-12-13 22:40:59 +0300, Alexander Korotkov wrote:
> It doesn't mater, because we release all locks on every buffer at one
> time. The unlock order can have effect on what waiter will acquire
> the lock next after ginRedoDeletePage(). However, I don't see why one
> unlock order is bette
On Thu, Dec 13, 2018 at 8:06 PM Andrey Borodin wrote:
> > if (BufferIsValid(lbuffer))
> > UnlockReleaseBuffer(lbuffer);
> > if (BufferIsValid(pbuffer))
> > UnlockReleaseBuffer(pbuffer);
> > if (BufferIsValid(dbuffer))
> > UnlockReleaseBuffer(dbuffer);
> > ==>
> > if (BufferIsValid(pbuffer))
> > Un
Hi!
> 13 дек. 2018 г., в 17:03, Alexander Korotkov
> написал(а):
>
> Thank you. I've revised your patch and pushed it. As long as two
> other patches in this thread.
That's great! Thanks!
> 13 дек. 2018 г., в 20:12, chjis...@163.com написал(а):
>
>
> hi
> I Have a question. Why the order
Bruce Momjian writes:
> I am seeing this warning in the 9.4 branch:
> ginxlog.c:756:5: warning: ‘lbuffer’ may be used uninitialized
> in this function [-Wmaybe-uninitialized]
I'm also getting that, just in 9.4, but at a different line number:
ginxlog.c: In function 'ginRedoDeletePage
On Thu, Dec 13, 2018 at 03:03:54PM +0300, Alexander Korotkov wrote:
> On Wed, Dec 12, 2018 at 4:08 PM Andrey Borodin wrote:
> > > 12 дек. 2018 г., в 3:26, Alexander Korotkov
> > > написал(а):
> > >
> > > BTW, I still can't see you're setting deleteXid field of
> > > ginxlogDeletePage struct anyw
On Wed, Dec 12, 2018 at 4:08 PM Andrey Borodin wrote:
> > 12 дек. 2018 г., в 3:26, Alexander Korotkov
> > написал(а):
> >
> > BTW, I still can't see you're setting deleteXid field of
> > ginxlogDeletePage struct anywhere.
> Oops. Fixed.
> >
> > Also, I note that protection against re-usage of re
Hi!
> 12 дек. 2018 г., в 3:26, Alexander Korotkov написал(а):
>
> BTW, I still can't see you're setting deleteXid field of
> ginxlogDeletePage struct anywhere.
Oops. Fixed.
>
> Also, I note that protection against re-usage of recently deleted
> pages is implemented differently than it is in B-t
On Tue, Dec 11, 2018 at 1:50 AM Alexander Korotkov wrote:
> On Sun, Dec 9, 2018 at 10:25 PM Alexander Korotkov
> wrote:
> > On Sat, Dec 8, 2018 at 12:48 PM Andrey Borodin wrote:
> > > > 8 дек. 2018 г., в 6:54, Alexander Korotkov
> > > > написал(а):
> > > >
> > > > Yep, please find attached dr
On Tue, Dec 11, 2018 at 10:14 AM Andrey Borodin wrote:
> > 11 дек. 2018 г., в 3:43, Alexander Korotkov
> > написал(а):
> >
> > Attached patch appears to be incomplete. GinPageSetDeleteXid() is
> > called only in ginRedoDeletePage(), so only in recovery, while it
> > should be set during normal
Hi!
> 11 дек. 2018 г., в 3:43, Alexander Korotkov написал(а):
>
> Attached patch appears to be incomplete. GinPageSetDeleteXid() is
> called only in ginRedoDeletePage(), so only in recovery, while it
> should be set during normal work too. deleteXid field of
> ginxlogDeletePage is never set.
On Sun, Dec 9, 2018 at 10:25 PM Alexander Korotkov wrote:
> On Sat, Dec 8, 2018 at 12:48 PM Andrey Borodin wrote:
> > > 8 дек. 2018 г., в 6:54, Alexander Korotkov
> > > написал(а):
> > >
> > > Yep, please find attached draft patch.
> >
> > Patch seems good to me, I'll check it in more detail.
>
On Mon, Dec 10, 2018 at 4:58 PM Andrey Borodin wrote:
> > 10 дек. 2018 г., в 18:56, Andrey Borodin написал(а):
> >
> > PFA this part. In thread about GiST vacuum Heikki was not very happy with
> > reusing PageHeader->pd_prune_xid for this. But we somewhat concluded that
> > this might be OK.
>
> 10 дек. 2018 г., в 18:56, Andrey Borodin написал(а):
>
> PFA this part. In thread about GiST vacuum Heikki was not very happy with
> reusing PageHeader->pd_prune_xid for this. But we somewhat concluded that
> this might be OK.
> Anyway, there's whole page, we can store those bits anywhere.
Hi!
> 10 дек. 2018 г., в 0:25, Alexander Korotkov написал(а):
>>
>> There's a patch above in this thread
>> 0001-Use-correct-locking-protocol-during-GIN-posting-tree.patch where I
>> propose stamping every deleted page with GinPageSetDeleteXid(page,
>> ReadNewTransactionId()); and avoid reusi
On Sat, Dec 8, 2018 at 12:48 PM Andrey Borodin wrote:
> > 8 дек. 2018 г., в 6:54, Alexander Korotkov
> > написал(а):
> >
> > Yep, please find attached draft patch.
>
> Patch seems good to me, I'll check it in more detail.
> The patch gets posting item at FirstOffsetNumber instead of
> btree->ge
Hi!
Thanks, Alexander!
> 8 дек. 2018 г., в 6:54, Alexander Korotkov написал(а):
>
> Yep, please find attached draft patch.
Patch seems good to me, I'll check it in more detail.
The patch gets posting item at FirstOffsetNumber instead of
btree->getLeftMostChild(). This seem OK, since dataGetLef
On Fri, Dec 7, 2018 at 12:50 AM Peter Geoghegan wrote:
> On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov
> wrote:
> > So, algorithm introduced by 218f51584d5 is broken. It tries to
> > guarantee that there are no inserters in the subtree by placing
> > cleanup lock to subtree root, assuming
On Fri, Dec 7, 2018 at 12:14 PM Andrey Borodin wrote:
> > 7 дек. 2018 г., в 2:50, Peter Geoghegan написал(а):
> > On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov
> > wrote:
> >> However, I'd like to note that 218f51584d5 introduces two changes:
> >> 1) Cleanup locking only if there pages to
Hi!
> 7 дек. 2018 г., в 2:50, Peter Geoghegan написал(а):
>
> On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov
> wrote:
>>
>> However, I'd like to note that 218f51584d5 introduces two changes:
>> 1) Cleanup locking only if there pages to delete
>> 2) Cleanup locking only subtree root
>> The
On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov wrote:
> So, algorithm introduced by 218f51584d5 is broken. It tries to
> guarantee that there are no inserters in the subtree by placing
> cleanup lock to subtree root, assuming that inserter holds pins on the
> path from root to leaf. But due
On Thu, Dec 6, 2018 at 10:56 PM Alexander Korotkov wrote:
> On Tue, Dec 4, 2018 at 8:01 PM Andres Freund wrote:
> > On 2018-11-10 17:42:16 -0800, Peter Geoghegan wrote:
> > > On Wed, Nov 7, 2018 at 5:46 PM Peter Geoghegan wrote:
> > > > Teodor: Do you think that the issue is fixable? It looks li
On Tue, Dec 4, 2018 at 8:01 PM Andres Freund wrote:
> On 2018-11-10 17:42:16 -0800, Peter Geoghegan wrote:
> > On Wed, Nov 7, 2018 at 5:46 PM Peter Geoghegan wrote:
> > > Teodor: Do you think that the issue is fixable? It looks like there
> > > are serious issues with the design of 218f51584d5 to
Hi,
On 2018-11-10 17:42:16 -0800, Peter Geoghegan wrote:
> On Wed, Nov 7, 2018 at 5:46 PM Peter Geoghegan wrote:
> > Teodor: Do you think that the issue is fixable? It looks like there
> > are serious issues with the design of 218f51584d5 to me. I don't think
> > the general "there can't be any i
On Fri, Nov 23, 2018 at 2:18 AM Andrey Borodin wrote:
> I found no practical way to fix concept of "subtree-lock". Pre-locking all
> left siblings for cleanup would suffice, but does not look very practical.
> GIN Posting tree has no all useful B-tree inventory like left links and high
> keys fo
Hi!
> 15 нояб. 2018 г., в 17:30, Andrey Borodin написал(а):
>
> I think I can compose patch for consideration.
I found no practical way to fix concept of "subtree-lock". Pre-locking all left
siblings for cleanup would suffice, but does not look very practical.
GIN Posting tree has no all usefu
Hi!
Mail.App somehow borken my reponse with , which
is far beyond of my understanding of what is plain text, that's why I'll quote
all my previous message here. Hope this mail client is OK.
14.11.2018, 23:13, "Andrey Borodin" :
> Hi everyone!
>
> I didn't noticed this thread for too long someho
Hi everyone!I didn't noticed this thread for too long somehow, sorry.8 нояб. 2018 г., в 6:46, Peter Geoghegan написал(а):I don't thinkthe general "there can't be any inserters at this subtree" thing worksgiven that we have to couple buffer locks when moving right for otherreasons. We call ginStepR
On Wed, Nov 7, 2018 at 5:46 PM Peter Geoghegan wrote:
> Teodor: Do you think that the issue is fixable? It looks like there
> are serious issues with the design of 218f51584d5 to me. I don't think
> the general "there can't be any inserters at this subtree" thing works
> given that we have to coup
On Wed, Nov 7, 2018 at 5:46 PM Peter Geoghegan wrote:
> I think that you have to be doing a multi-level delete for a deadlock
> to take place, which isn't particularly likely to coincide with a
> concurrent insertion in general. That may explain why it's taken a
> year to get a high-quality bug re
On Mon, Oct 29, 2018 at 12:04 PM chenhj wrote:
> ## stack of autovacuum:Acquire lock 0x2aaab670dfe4 and hold 0x2aaab4009564
> --
> (gdb) bt
> #0 0x7fe11552379b in do_futex_wait.constprop.1 () from
> /lib64/libpthread.so.0
> #1 0x7fe1155238
On Mon, Oct 29, 2018 at 12:04 PM chenhj wrote:
> #4 0x007095ac in LWLockAcquire (lock=0x2aaab4009564,
> mode=LW_EXCLUSIVE) at lwlock.c:1233
> #5 0x00490a32 in ginStepRight (buffer=6713826, index= out>, lockmode=lockmode@entry=2) at ginbtree.c:174
> #6 0x0049
On Tue, Nov 6, 2018 at 10:05 AM chenhj wrote:
> I analyzed the btree block where lwlock deadlock occurred, as follows:
Thank you for doing this important work.
You're using Postgres 10.2. While that version isn't current with all
GIN bug fixes, it does have this important one:
"Ensure that vac
60 matches
Mail list logo
