Re: Add a perl function in Cluster.pm to generate WAL

2024-01-08 Thread Bertrand Drouvot
Hi, On Tue, Jan 09, 2024 at 01:59:08PM +0900, Michael Paquier wrote: > On Mon, Jan 08, 2024 at 08:00:00PM +0300, Alexander Lakhin wrote: > > Yes, I've added (VERBOSE) and also cut down the test to catch the failure > > faster. Thanks Alexander! > > The difference between a successful and a fail

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-08 Thread Michael Paquier
On Mon, Jan 08, 2024 at 08:00:00PM +0300, Alexander Lakhin wrote: > Yes, I've added (VERBOSE) and also cut down the test to catch the failure > faster. > The difference between a successful and a failed run: >     tuples: 1 removed, 15 remain, 0 are dead but not yet removable > [...] >    

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-08 Thread Alexander Lakhin
Hello Bertrand, 08.01.2024 10:34, Bertrand Drouvot wrote: If one is able to reproduce, would it be possible to change the test and launch the vacuum in verbose mode? That way, we could see if this is somehow due to [1] (means something holding global xmin). Yes, I've added (VERBOSE) and also c

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-07 Thread Bertrand Drouvot
Hi, On Mon, Jan 08, 2024 at 09:16:19AM +0900, Michael Paquier wrote: > On Sun, Jan 07, 2024 at 05:00:00PM +0300, Alexander Lakhin wrote: > > Yes, I wrote exactly about that upthread and referenced my previous > > investigation. But what I'm observing now, is that the failure probability > > greatl

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-07 Thread Michael Paquier
On Sun, Jan 07, 2024 at 05:00:00PM +0300, Alexander Lakhin wrote: > Yes, I wrote exactly about that upthread and referenced my previous > investigation. But what I'm observing now, is that the failure probability > greatly increased with c161ab74f, so something really changed in the test > behaviou

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-07 Thread Alexander Lakhin
07.01.2024 10:10, Michael Paquier wrote: On Fri, Jan 05, 2024 at 11:00:00PM +0300, Alexander Lakhin wrote: Your suspicion was proved right. After git show c161ab74f src/test/recovery/t/035_standby_logical_decoding.pl  | git apply -R 20 iterations with 20 tests in parallel performed successfully

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-06 Thread Michael Paquier
On Fri, Jan 05, 2024 at 11:00:00PM +0300, Alexander Lakhin wrote: > Your suspicion was proved right. After > git show c161ab74f src/test/recovery/t/035_standby_logical_decoding.pl  | git > apply -R > 20 iterations with 20 tests in parallel performed successfully for me > (twice). > > So it looks

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-05 Thread Alexander Lakhin
05.01.2024 02:48, Michael Paquier wrote: On Thu, Jan 04, 2024 at 04:00:01PM +0300, Alexander Lakhin wrote: Reproduced here. Did you just make the run slow enough to show the failure with valgrind? Yes, I just run several test instances (with no extra modifications) under Valgrind with paralle

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-04 Thread Michael Paquier
On Thu, Jan 04, 2024 at 04:00:01PM +0300, Alexander Lakhin wrote: > Reproduced here. Did you just make the run slow enough to show the failure with valgrind? > As I can see in the failure logs you referenced, the first problem is: > #   Failed test 'inactiveslot slot invalidation is logged with v

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-04 Thread Alexander Lakhin
Hello Tom, 04.01.2024 02:39, Tom Lane wrote: Buildfarm member skink has failed 3 times in 035_standby_logical_decoding.pl in the last couple of days: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2024-01-03%2020%3A07%3A15 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-03 Thread Michael Paquier
On Wed, Jan 03, 2024 at 06:39:29PM -0500, Tom Lane wrote: > I am not real sure what is happening there, but I see that c161ab74f > changed some details of how that test works, so I wonder if it's > responsible for these failures. The timing isn't a perfect match, > since this commit went in two we

Re: Add a perl function in Cluster.pm to generate WAL

2024-01-03 Thread Tom Lane
Michael Paquier writes: > I have added a comment about pg_logical_emit_message() being in > non-transactional mode and the flush implied by pg_switch_wal() as > that's important, edited a bit the whole, then applied the patch. Buildfarm member skink has failed 3 times in 035_standby_logical_decod

Re: Add a perl function in Cluster.pm to generate WAL

2023-12-20 Thread Michael Paquier
On Wed, Dec 20, 2023 at 12:24:04AM -0300, Euler Taveira wrote: > LGTM. I was eyeing at 020_messages.pl and it has a pg_switch_wal() after a transaction rollbacked. 020_archive_status.pl creates a table, does one segment switch, then a checkpoint (table is used afterwards). Perhaps these could be

Re: Add a perl function in Cluster.pm to generate WAL

2023-12-19 Thread Euler Taveira
On Tue, Dec 19, 2023, at 8:00 PM, Michael Paquier wrote: > On Tue, Dec 19, 2023 at 11:25:50AM +0530, Bharath Rupireddy wrote: > > I used pg_logical_emit_message() in non-transactional mode without > > needing an explicit WAL flush as the pg_switch_wal() does a WAL flush > > at the end [1]. > > Ind

Re: Add a perl function in Cluster.pm to generate WAL

2023-12-19 Thread Michael Paquier
On Tue, Dec 19, 2023 at 11:25:50AM +0530, Bharath Rupireddy wrote: > I used pg_logical_emit_message() in non-transactional mode without > needing an explicit WAL flush as the pg_switch_wal() does a WAL flush > at the end [1]. Indeed, that should be enough to answer my comment. > Attached v4 patch

Re: Add a perl function in Cluster.pm to generate WAL

2023-12-18 Thread Bharath Rupireddy
On Tue, Dec 19, 2023 at 9:51 AM Michael Paquier wrote: > > On Mon, Dec 18, 2023 at 08:48:09AM -0300, Euler Taveira wrote: > > It is cheaper. > > Agreed that this could just use a set of pg_logical_emit_message() > when jumping across N segments. Thanks. I missed the point of using pg_logical_emit

Re: Add a perl function in Cluster.pm to generate WAL

2023-12-18 Thread Michael Paquier
On Mon, Dec 18, 2023 at 08:48:09AM -0300, Euler Taveira wrote: > It is cheaper. Agreed that this could just use a set of pg_logical_emit_message() when jumping across N segments. Another thing that seems quite important to me is to force a flush of WAL with the last segment switch, and the new "f

Re: Add a perl function in Cluster.pm to generate WAL

2023-12-18 Thread Euler Taveira
On Mon, Dec 18, 2023, at 2:39 AM, Bharath Rupireddy wrote: > Rebase needed, attached v3 patch. I think you don't understand the suggestion proposed by Michael and Kyotaro. If you do a comparison with the following SQL commands: euler=# select pg_walfile_name(pg_current_wal_lsn()); pg_walfile

Re: Add a perl function in Cluster.pm to generate WAL

2023-12-17 Thread Bharath Rupireddy
On Wed, Jul 19, 2023 at 4:11 PM Bharath Rupireddy wrote: > > Attached the v2 patch. Thoughts? Rebase needed, attached v3 patch. -- Bharath Rupireddy PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com v3-0001-Add-a-TAP-test-function-to-generate-W

Re: Add a perl function in Cluster.pm to generate WAL

2023-07-19 Thread Bharath Rupireddy
On Fri, Jun 16, 2023 at 8:00 AM Michael Paquier wrote: > > On Thu, Jun 15, 2023 at 01:40:15PM +0900, Kyotaro Horiguchi wrote: > > + "CREATE TABLE tt (); DROP TABLE tt; SELECT > > pg_switch_wal();"); > > > > At least since 11, we can utilize pg_logical_emit_message() for this >

Re: Add a perl function in Cluster.pm to generate WAL

2023-06-15 Thread Kyotaro Horiguchi
Thanks for the comments. At Fri, 16 Jun 2023 11:30:15 +0900, Michael Paquier wrote in > > -$node_primary->safe_psql( > > -'postgres', "create table retain_test(a int); > > - select pg_switch_wal(); > > - insert into ret

Re: Add a perl function in Cluster.pm to generate WAL

2023-06-15 Thread Michael Paquier
On Thu, Jun 15, 2023 at 01:40:15PM +0900, Kyotaro Horiguchi wrote: > + "CREATE TABLE tt (); DROP TABLE tt; SELECT > pg_switch_wal();"); > > At least since 11, we can utilize pg_logical_emit_message() for this > purpose. It's more lightweight and seems appropriate, not only bec

Re: Add a perl function in Cluster.pm to generate WAL

2023-06-14 Thread Kyotaro Horiguchi
Mmm. It seems like the email I thought I'd sent failed to go out. At Sun, 11 Jun 2023 07:14:54 +0530, Bharath Rupireddy wrote in > On Wed, Aug 24, 2022 at 6:42 AM Kyotaro Horiguchi > wrote: > > > > At Tue, 16 Aug 2022 18:40:49 +0200, Alvaro Herrera > > wrote in > > > On 2022-Aug-16, Andrew D

Re: Add a perl function in Cluster.pm to generate WAL

2023-06-10 Thread Bharath Rupireddy
On Wed, Aug 24, 2022 at 6:42 AM Kyotaro Horiguchi wrote: > > At Tue, 16 Aug 2022 18:40:49 +0200, Alvaro Herrera > wrote in > > On 2022-Aug-16, Andrew Dunstan wrote: > > > > > I don't think there's a hard and fast rule about it. Certainly the case > > > would be more compelling if the functions w