Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-11 Thread Tomas Vondra
On 12/11/23 07:12, Amit Kapila wrote: > On Sat, Dec 9, 2023 at 12:16 PM Amit Kapila wrote: >> >> Thanks, I could also reproduce the issue on back branches (tried till >> 12), and the fix works. I'll push this on Monday. >> > > Peter sent one minor suggestion (to write the check differently for >

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-10 Thread Amit Kapila
On Sat, Dec 9, 2023 at 12:16 PM Amit Kapila wrote: > > Thanks, I could also reproduce the issue on back branches (tried till > 12), and the fix works. I'll push this on Monday. > Peter sent one minor suggestion (to write the check differently for easier understanding) offlist which I addressed an

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-08 Thread Amit Kapila
On Fri, Dec 8, 2023 at 7:16 PM Shlok Kyal wrote: > > > Then let's go with the original patch only. BTW, it took almost the > > same time (105 wallclock secs) in my environment (CentOs VM) to run > > tests in src/test/subscription both with and without the patch. I took > > a median of five runs. I

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-08 Thread Shlok Kyal
Hi, > Then let's go with the original patch only. BTW, it took almost the > same time (105 wallclock secs) in my environment (CentOs VM) to run > tests in src/test/subscription both with and without the patch. I took > a median of five runs. I have slightly adjusted the comments and > commit messa

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-08 Thread Amit Kapila
On Thu, Dec 7, 2023 at 11:21 AM Shlok Kyal wrote: > > > I mean to commit the open transaction at the below place in > > wait_for_relation_state_change() > > > > wait_for_relation_state_change() > > { > > ... > > -- commit the xact > > WaitLatch(); > > ... > > } > > > > Then start after the wait is

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-06 Thread Shlok Kyal
Hi, > I mean to commit the open transaction at the below place in > wait_for_relation_state_change() > > wait_for_relation_state_change() > { > ... > -- commit the xact > WaitLatch(); > ... > } > > Then start after the wait is over. This is just to test whether it > improves the difference in regr

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-05 Thread Shlok Kyal
On Tue, 5 Dec 2023 at 17:18, Tomas Vondra wrote: > > On 12/5/23 08:14, Shlok Kyal wrote: > > Hi, > > > >> As for the test results, I very much doubt the differences are not > >> caused simply by random timing variations, or something like that. And I > >> don't understand what "Performance Machine

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-05 Thread Tomas Vondra
On 12/5/23 08:14, Shlok Kyal wrote: > Hi, > >> As for the test results, I very much doubt the differences are not >> caused simply by random timing variations, or something like that. And I >> don't understand what "Performance Machine Linux" is, considering those >> timings are slower than the ot

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-04 Thread Shlok Kyal
Hi, > As for the test results, I very much doubt the differences are not > caused simply by random timing variations, or something like that. And I > don't understand what "Performance Machine Linux" is, considering those > timings are slower than the other two machines. The machine has Total Mem

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-04 Thread Amit Kapila
On Mon, Dec 4, 2023 at 5:30 PM Tomas Vondra wrote: > > On 12/4/23 12:37, Amit Kapila wrote: > > On Sat, Dec 2, 2023 at 9:52 PM Shlok Kyal wrote: > >> > >>> thread. I think you can compare the timing of regression tests in > >>> subscription, with and without the patch to show there is no > >>> re

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-04 Thread Tomas Vondra
On 12/4/23 12:37, Amit Kapila wrote: > On Sat, Dec 2, 2023 at 9:52 PM Shlok Kyal wrote: >> >>> thread. I think you can compare the timing of regression tests in >>> subscription, with and without the patch to show there is no >>> regression. And probably some tests with a large number of tables

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-04 Thread Amit Kapila
On Sat, Dec 2, 2023 at 9:52 PM Shlok Kyal wrote: > > > thread. I think you can compare the timing of regression tests in > > subscription, with and without the patch to show there is no > > regression. And probably some tests with a large number of tables for > > sync with very little data. > > I

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-12-02 Thread Shlok Kyal
Hi, > thread. I think you can compare the timing of regression tests in > subscription, with and without the patch to show there is no > regression. And probably some tests with a large number of tables for > sync with very little data. I have tested the regression test timings for subscription w

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-25 Thread Amit Kapila
On Fri, Nov 24, 2023 at 5:05 PM Shlok Kyal wrote: > > > I tried to reproduce the issue and was able to reproduce it with > > scripts shared by Tomas. > > I tried testing it from PG17 to PG 11. This issue is reproducible for > > each version. > > > > Next I would try to test with the patch in the t

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-24 Thread Shlok Kyal
Hi, > I tried to reproduce the issue and was able to reproduce it with > scripts shared by Tomas. > I tried testing it from PG17 to PG 11. This issue is reproducible for > each version. > > Next I would try to test with the patch in the thread shared by Amit. I have created the v1 patch to resolv

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-23 Thread Shlok Kyal
Hi, I tried to reproduce the issue and was able to reproduce it with scripts shared by Tomas. I tried testing it from PG17 to PG 11. This issue is reproducible for each version. Next I would try to test with the patch in the thread shared by Amit. Thanks, Shlok Kumar Kyal

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-23 Thread Tomas Vondra
On 11/23/23 10:24, Amit Kapila wrote: > On Wed, Nov 22, 2023 at 4:51 PM Tomas Vondra > wrote: >> >> On 11/22/23 11:38, Amit Kapila wrote: >>> >>> Okay. IIUC, what's going on here is that the apply worker acquires >>> AccessShareLock on pg_subscription to update rel state for one of the >>> table

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-23 Thread Amit Kapila
On Wed, Nov 22, 2023 at 4:51 PM Tomas Vondra wrote: > > On 11/22/23 11:38, Amit Kapila wrote: > > > > Okay. IIUC, what's going on here is that the apply worker acquires > > AccessShareLock on pg_subscription to update rel state for one of the > > tables say tbl-1, and then for another table say tb

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-22 Thread Tomas Vondra
On 11/22/23 11:38, Amit Kapila wrote: > On Tue, Nov 21, 2023 at 6:56 PM Tomas Vondra > wrote: >> >> On 11/21/23 14:16, Amit Kapila wrote: >>> On Tue, Nov 21, 2023 at 5:17 PM Tomas Vondra >>> wrote: >>> >>> It seems there is some inconsistency in what you have written for >>> client backends/

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-22 Thread Amit Kapila
On Tue, Nov 21, 2023 at 6:56 PM Tomas Vondra wrote: > > On 11/21/23 14:16, Amit Kapila wrote: > > On Tue, Nov 21, 2023 at 5:17 PM Tomas Vondra > > wrote: > >> > > > > It seems there is some inconsistency in what you have written for > > client backends/tablesync worker vs. apply worker. The above

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-21 Thread Tomas Vondra
On 11/21/23 14:16, Amit Kapila wrote: > On Tue, Nov 21, 2023 at 5:17 PM Tomas Vondra > wrote: >> >> I decided to do some stress-testing of the built-in logical replication, >> as part of the sequence decoding work. And I soon ran into an undetected >> deadlock related to ALTER SUBSCRIPTION ...

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-21 Thread Amit Kapila
On Tue, Nov 21, 2023 at 5:17 PM Tomas Vondra wrote: > > I decided to do some stress-testing of the built-in logical replication, > as part of the sequence decoding work. And I soon ran into an undetected > deadlock related to ALTER SUBSCRIPTION ... REFRESH PUBLICATION :-( > > The attached bash scr

undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

2023-11-21 Thread Tomas Vondra
Hi, I decided to do some stress-testing of the built-in logical replication, as part of the sequence decoding work. And I soon ran into an undetected deadlock related to ALTER SUBSCRIPTION ... REFRESH PUBLICATION :-( The attached bash scripts triggers that in a couple seconds for me. The script l