Re: Test to dump and restore objects left behind by regression

2025-04-05 Thread Alvaro Herrera
On 2025-Mar-28, Ashutosh Bapat wrote: > However, it's a very painful process to come up with the schedule and > more painful and error prone to maintain it. It could take many days > to come up with the right schedule which can become inaccurate the > moment next SQL file is added OR an existing f

Re: Test to dump and restore objects left behind by regression

2025-04-05 Thread Ashutosh Bapat
On Mon, Mar 24, 2025 at 5:44 PM Alvaro Herrera wrote: > > On 2025-Mar-24, Ashutosh Bapat wrote: > > > One concern I have with directory format is the dumped database is not > > readable. This might make investigating a but identified the test a > > bit more complex. > > Oh, it's readable all right

Re: Test to dump and restore objects left behind by regression

2025-04-05 Thread Ashutosh Bapat
On Thu, Mar 20, 2025 at 10:09 PM Alvaro Herrera wrote: > > On 2025-Mar-20, vignesh C wrote: > > > Will it help the execution time if we use --jobs in case of pg_dump > > and pg_restore wherever supported: > > As I said in another thread, I think we should enable this test to run > without requirin

Re: Test to dump and restore objects left behind by regression

2025-04-05 Thread Michael Paquier
On Thu, Mar 27, 2025 at 06:15:06PM +0100, Alvaro Herrera wrote: > BTW another idea to shorten this tests's runtime might be to try and > identify which of parallel_schedule tests leave objects behind and > create a shorter schedule with only those (a possible implementation > might keep a list of t

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Alvaro Herrera
On 2025-Apr-02, Ashutosh Bapat wrote: > I have closed the CF entry > https://commitfest.postgresql.org/patch/4564/ committed. I will > create another CF entry to park --no-statistics reversal change. That > way, we will know when statistics dump/restore has become stable. No commitfest entry ple

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread vignesh C
On Wed, 2 Apr 2025 at 13:49, Ashutosh Bapat wrote: > > On Tue, Apr 1, 2025 at 10:31 PM Alvaro Herrera > wrote: > > > > On 2025-Apr-01, Ashutosh Bapat wrote: > > > > > Just today morning, I found something which looks like another bug in > > > statistics dump/restore [1]. As Daniel has expressed

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Alvaro Herrera
On 2025-Apr-03, Andres Freund wrote: > I've increased the timeout even further, but I can't say that I am happy about > the slowest test getting even slower. Adding test time in the serially slowest > test is way worse than adding the same time in a concurrent test. Yeah. We discussed strategies

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Ashutosh Bapat
On Thu, Apr 3, 2025 at 1:50 PM Alvaro Herrera wrote: > > On 2025-Apr-03, Ashutosh Bapat wrote: > > > Looks like the problem is in the test itself as pointed out by Jeff in > > [1]. PFA patch fixing the test and enabling statistics back. > > Thanks, pushed. Thanks. -- Best Wishes, Ashutosh Bapat

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Andres Freund
Hi, On 2025-04-04 12:01:16 -0400, Andres Freund wrote: > FWIW, for me 027 is actually considerably faster. In an cassert -O0 build (my > normal development env, I find even -Og too problematic for debugging): > > pg_upgrade/002_pg_upgrade96.61s > recovery/027_stream_regress 66.04s > > After

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Andres Freund
Hi, On 2025-04-03 19:14:02 +0200, Alvaro Herrera wrote: > On 2025-Apr-03, Andres Freund wrote: > > > I've increased the timeout even further, but I can't say that I am happy > > about > > the slowest test getting even slower. Adding test time in the serially > > slowest > > test is way worse tha

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Ashutosh Bapat
On Fri, Apr 4, 2025 at 4:41 PM Ashutosh Bapat wrote: > > On Thu, Apr 3, 2025 at 10:44 PM Alvaro Herrera > wrote: > > > > On 2025-Apr-03, Andres Freund wrote: > > > > > I've increased the timeout even further, but I can't say that I am happy > > > about > > > the slowest test getting even slower

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Ashutosh Bapat
On Thu, Apr 3, 2025 at 10:44 PM Alvaro Herrera wrote: > > On 2025-Apr-03, Andres Freund wrote: > > > I've increased the timeout even further, but I can't say that I am happy > > about > > the slowest test getting even slower. Adding test time in the serially > > slowest > > test is way worse tha

Re: Test to dump and restore objects left behind by regression

2025-04-03 Thread Andres Freund
Hi, On 2025-04-03 10:20:09 +0200, Alvaro Herrera wrote: > On 2025-Apr-03, Ashutosh Bapat wrote: > > > Looks like the problem is in the test itself as pointed out by Jeff in > > [1]. PFA patch fixing the test and enabling statistics back. > > Thanks, pushed. Since then the pg_upgrade tests have be

Re: Test to dump and restore objects left behind by regression

2025-04-03 Thread Alvaro Herrera
On 2025-Apr-03, Ashutosh Bapat wrote: > Looks like the problem is in the test itself as pointed out by Jeff in > [1]. PFA patch fixing the test and enabling statistics back. Thanks, pushed. > A note about variable name changes and introduction of new variables. > We run step 2 between 1 and 3 so

Re: Test to dump and restore objects left behind by regression

2025-04-02 Thread Ashutosh Bapat
On Wed, Apr 2, 2025 at 3:36 PM Ashutosh Bapat wrote: > > > > No commitfest entry please. Better to add an open item on the wiki > > page. > > https://wiki.postgresql.org/wiki/Open_Items > > Posted it on the thread where I have reported the bug. Hopefully, we > will commit both the bug fix and tes

Re: Test to dump and restore objects left behind by regression

2025-04-02 Thread Ashutosh Bapat
On Thu, Apr 3, 2025 at 9:29 AM vignesh C wrote: > > I believe this commitfest entry at [1] can be closed now, as the > buildfarm has been running stably for the past few days. > [1] - https://commitfest.postgresql.org/patch/4956/ I intended to close this but closed another entry by mistake. If po

Re: Test to dump and restore objects left behind by regression

2025-04-02 Thread Ashutosh Bapat
Hi Alvaro, On Wed, Apr 2, 2025 at 2:49 PM Alvaro Herrera wrote: > > On 2025-Apr-02, Ashutosh Bapat wrote: > > > I have closed the CF entry > > https://commitfest.postgresql.org/patch/4564/ committed. I will > > create another CF entry to park --no-statistics reversal change. That > > way, we wi

Re: Test to dump and restore objects left behind by regression

2025-04-02 Thread Ashutosh Bapat
On Tue, Apr 1, 2025 at 10:31 PM Alvaro Herrera wrote: > > On 2025-Apr-01, Ashutosh Bapat wrote: > > > Just today morning, I found something which looks like another bug in > > statistics dump/restore [1]. As Daniel has expressed upthread [2], we > > should go ahead and commit the test even if the

Re: Test to dump and restore objects left behind by regression

2025-04-01 Thread Daniel Gustafsson
> On 1 Apr 2025, at 19:01, Alvaro Herrera wrote: > Thanks! Thanks for taking this one across the finishing line! -- Daniel Gustafsson

Re: Test to dump and restore objects left behind by regression

2025-04-01 Thread Alvaro Herrera
On 2025-Apr-01, Ashutosh Bapat wrote: > Just today morning, I found something which looks like another bug in > statistics dump/restore [1]. As Daniel has expressed upthread [2], we > should go ahead and commit the test even if the bug is not fixed. But > in case it creates a lot of noise and make

Re: Test to dump and restore objects left behind by regression

2025-04-01 Thread Ashutosh Bapat
On Tue, Apr 1, 2025 at 11:52 AM Alvaro Herrera wrote: > > On 2025-Mar-31, Daniel Gustafsson wrote: > > > Given where we are in the cycle, it seems to make sense to stick to using > > the > > schedule we already have rather than invent a new process for generating it, > > and work on that for 19?

Re: Test to dump and restore objects left behind by regression

2025-03-31 Thread Alvaro Herrera
On 2025-Mar-31, Daniel Gustafsson wrote: > Given where we are in the cycle, it seems to make sense to stick to using the > schedule we already have rather than invent a new process for generating it, > and work on that for 19? No objections to that. I'll see about getting this committed during m

Re: Test to dump and restore objects left behind by regression

2025-03-31 Thread Daniel Gustafsson
> On 28 Mar 2025, at 19:12, Alvaro Herrera wrote: > > On 2025-Mar-28, Tom Lane wrote: > >> I think instead of going this direction, we really need to create a >> separately-purposed script that simply creates "one of everything" >> without doing anything else (except maybe loading a little data)

Re: Test to dump and restore objects left behind by regression

2025-03-31 Thread Ashutosh Bapat
On Fri, Mar 28, 2025 at 11:43 PM Alvaro Herrera wrote: > > On 2025-Mar-28, Tom Lane wrote: > > > I think instead of going this direction, we really need to create a > > separately-purposed script that simply creates "one of everything" > > without doing anything else (except maybe loading a little

Re: Test to dump and restore objects left behind by regression

2025-03-31 Thread Ashutosh Bapat
On Mon, Mar 31, 2025 at 5:07 PM Ashutosh Bapat wrote: > The bug related to materialized views has been fixed and now the test passes even if we compare statistics from dumped and restored databases. Hence removing 0003. In the attached patchset I have also addressed Vignesh's below comment On Thu

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Tom Lane
Alvaro Herrera writes: > Hmm, I didn't mean that we'd maintain a separate schedule. I meant that > we'd take the existing schedule, then apply some Perl magic to it that > grep-outs the tests that we know to contribute nothing, and generate a > new schedule file dynamically. We don't need to mai

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Alvaro Herrera
On 2025-Mar-28, Tom Lane wrote: > I think instead of going this direction, we really need to create a > separately-purposed script that simply creates "one of everything" > without doing anything else (except maybe loading a little data). > I believe it'd be a lot easier to remember to add to that

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Ashutosh Bapat
On Fri, Mar 28, 2025 at 12:20 PM Ashutosh Bapat wrote: > > On Fri, Mar 28, 2025 at 7:07 AM Michael Paquier wrote: > > > > On Thu, Mar 27, 2025 at 06:15:06PM +0100, Alvaro Herrera wrote: > > > BTW another idea to shorten this tests's runtime might be to try and > > > identify which of parallel_sch

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Ashutosh Bapat
On Fri, Mar 28, 2025 at 4:05 PM Alvaro Herrera wrote: > > On 2025-Mar-28, Ashutosh Bapat wrote: > > > No, that's losing some information like default installation and the > > same version. > > You don't need to preserve such information. This is just a test name. > People looking for more details

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Alvaro Herrera
On 2025-Mar-28, Ashutosh Bapat wrote: > No, that's losing some information like default installation and the > same version. You don't need to preserve such information. This is just a test name. People looking for more details can grep for the name and they will find the comments. -- Álvaro H

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Ashutosh Bapat
Vignesh and Alvaro On Fri, Mar 28, 2025 at 12:02 PM Ashutosh Bapat wrote: > > > > Maybe > > fail("roundtrip dump/restore of the regression database") > > No, that's losing some information like default installation and the > same version. How about "dump and restore across servers with same Post

Re: Test to dump and restore objects left behind by regression

2025-03-28 Thread Ashutosh Bapat
On Fri, Mar 28, 2025 at 7:07 AM Michael Paquier wrote: > > On Thu, Mar 27, 2025 at 06:15:06PM +0100, Alvaro Herrera wrote: > > BTW another idea to shorten this tests's runtime might be to try and > > identify which of parallel_schedule tests leave objects behind and > > create a shorter schedule w

Re: Test to dump and restore objects left behind by regression

2025-03-27 Thread Ashutosh Bapat
On Thu, Mar 27, 2025 at 10:45 PM Alvaro Herrera wrote: > > On 2025-Mar-27, Ashutosh Bapat wrote: > > > On Thu, Mar 27, 2025 at 6:01 PM vignesh C wrote: > > > > Couple of minor thoughts: > > > 1) I felt this error message is not conveying the error message correctly: > > > + if ($src_node->p

Re: Test to dump and restore objects left behind by regression

2025-03-27 Thread Alvaro Herrera
On 2025-Mar-27, Ashutosh Bapat wrote: > On Thu, Mar 27, 2025 at 6:01 PM vignesh C wrote: > > Couple of minor thoughts: > > 1) I felt this error message is not conveying the error message correctly: > > + if ($src_node->pg_version != $dst_node->pg_version > > + or defined $src

Re: Test to dump and restore objects left behind by regression

2025-03-27 Thread Ashutosh Bapat
On Thu, Mar 27, 2025 at 6:01 PM vignesh C wrote: > > On Tue, 25 Mar 2025 at 16:09, Ashutosh Bapat > wrote: > > > > On Mon, Mar 24, 2025 at 5:44 PM Alvaro Herrera > > wrote: > > > > > > On 2025-Mar-24, Ashutosh Bapat wrote: > > > > > > > One concern I have with directory format is the dumped dat

Re: Test to dump and restore objects left behind by regression

2025-03-27 Thread vignesh C
On Tue, 25 Mar 2025 at 16:09, Ashutosh Bapat wrote: > > On Mon, Mar 24, 2025 at 5:44 PM Alvaro Herrera > wrote: > > > > On 2025-Mar-24, Ashutosh Bapat wrote: > > > > > One concern I have with directory format is the dumped database is not > > > readable. This might make investigating a but ident

Re: Test to dump and restore objects left behind by regression

2025-03-24 Thread Alvaro Herrera
On 2025-Mar-24, Ashutosh Bapat wrote: > One concern I have with directory format is the dumped database is not > readable. This might make investigating a but identified the test a > bit more complex. Oh, it's readable all right. You just need to use `pg_restore -f-` to read it. No big deal.

Re: Test to dump and restore objects left behind by regression

2025-03-24 Thread Daniel Gustafsson
> On 24 Mar 2025, at 10:54, Ashutosh Bapat wrote: > 0003 - same as 0002 in the previous patch set. It excludes statistics > from comparison, otherwise the test will fail because of bug reported > at [1]. Ideally we shouldn't commit this patch so as to test > statistics dump and restore, but in ca

Re: Test to dump and restore objects left behind by regression

2025-03-24 Thread Ashutosh Bapat
On Fri, Mar 21, 2025 at 11:38 PM Alvaro Herrera wrote: > > On 2025-Mar-21, Ashutosh Bapat wrote: > > > I used the same parallelism in pg_restore and pg_dump too. And your > > numbers seem to be similar to mine; slightly less than 20% slowdown. > > But is that slowdown acceptable? From the earlier

Re: Test to dump and restore objects left behind by regression

2025-03-22 Thread Ashutosh Bapat
On Fri, Mar 21, 2025 at 6:04 PM Alvaro Herrera wrote: > > On 2025-Mar-21, Ashutosh Bapat wrote: > > > On Thu, Mar 20, 2025 at 8:37 PM vignesh C wrote: > > > > Should the copyright be only 2025 in this case: > > > The patch was posted in 2024 to this mailing list. So we better > > protect the copy

Re: Test to dump and restore objects left behind by regression

2025-03-21 Thread Alvaro Herrera
On 2025-Mar-21, Ashutosh Bapat wrote: > I used the same parallelism in pg_restore and pg_dump too. And your > numbers seem to be similar to mine; slightly less than 20% slowdown. > But is that slowdown acceptable? From the earlier discussions, it > seems the answer is No. Haven't heard otherwise.

Re: Test to dump and restore objects left behind by regression

2025-03-21 Thread Ashutosh Bapat
On Fri, Mar 21, 2025 at 8:13 PM Alvaro Herrera wrote: > > I passed PROVE_FLAGS="--timer -v" to get the timings and run under > --format=directory. > > Without new test: > ok23400 ms ( 0.00 usr 0.00 sys + 2.84 cusr 1.53 csys = 4.37 CPU) > ok23409 ms ( 0.00 usr 0.01 sys + 2.81 cusr 1.

Re: Test to dump and restore objects left behind by regression

2025-03-21 Thread Alvaro Herrera
I passed PROVE_FLAGS="--timer -v" to get the timings and run under --format=directory. Without new test: ok23400 ms ( 0.00 usr 0.00 sys + 2.84 cusr 1.53 csys = 4.37 CPU) ok23409 ms ( 0.00 usr 0.01 sys + 2.81 cusr 1.53 csys = 4.35 CPU) With new test, under --format=directory: -j2

Re: Test to dump and restore objects left behind by regression

2025-03-21 Thread vignesh C
On Thu, 20 Mar 2025 at 22:09, Alvaro Herrera wrote: > > On 2025-Mar-20, vignesh C wrote: > > > Will it help the execution time if we use --jobs in case of pg_dump > > and pg_restore wherever supported: > > As I said in another thread, I think we should enable this test to run > without requiring a

Re: Test to dump and restore objects left behind by regression

2025-03-21 Thread Alvaro Herrera
On 2025-Mar-21, Ashutosh Bapat wrote: > On Thu, Mar 20, 2025 at 8:37 PM vignesh C wrote: > > Should the copyright be only 2025 in this case: > The patch was posted in 2024 to this mailing list. So we better > protect the copyright since then. I remember a hackers discussion > where a senior mem

Re: Test to dump and restore objects left behind by regression

2025-03-21 Thread Ashutosh Bapat
On Thu, Mar 20, 2025 at 8:37 PM vignesh C wrote: > > Will it help the execution time if we use --jobs in case of pg_dump > and pg_restore wherever supported: > + $src_node->command_ok( > + [ > + 'pg_dump', "-F$format", '--no-sync',

Re: Test to dump and restore objects left behind by regression

2025-03-20 Thread Alvaro Herrera
On 2025-Mar-20, vignesh C wrote: > Will it help the execution time if we use --jobs in case of pg_dump > and pg_restore wherever supported: As I said in another thread, I think we should enable this test to run without requiring any PG_TEST_EXTRA, because otherwise the only way to know about prob

Re: Test to dump and restore objects left behind by regression

2025-03-20 Thread vignesh C
On Wed, 19 Mar 2025 at 17:13, Ashutosh Bapat wrote: > > On Thu, Mar 13, 2025 at 6:10 PM Ashutosh Bapat > wrote: > > > > > > I think the fix is to explicitly pass --lc-monetary to the old cluster > > > and the restored cluster. 003 patch in the attached patch set does > > > that. Please check if i

Re: Test to dump and restore objects left behind by regression

2025-03-19 Thread Ashutosh Bapat
On Thu, Mar 13, 2025 at 6:10 PM Ashutosh Bapat wrote: > > > > I think the fix is to explicitly pass --lc-monetary to the old cluster > > and the restored cluster. 003 patch in the attached patch set does > > that. Please check if it fixes the issue for you. > > > > Additionally we should check tha

Re: Test to dump and restore objects left behind by regression

2025-03-15 Thread Alvaro Herrera
Hello When running these tests, I encounter this strange diff in the dumps, which seems to be that the locale for type money does not match. I imagine the problem is that the locale is not set correctly when initdb'ing one of them? Grepping the regress_log for initdb, I see this: $ grep -B1 'Ru

Re: Test to dump and restore objects left behind by regression

2025-03-13 Thread Ashutosh Bapat
Here are patches missing in the previous email. On Thu, Mar 13, 2025 at 6:09 PM Ashutosh Bapat wrote: > > On Thu, Mar 13, 2025 at 2:12 PM Alvaro Herrera > wrote: > > > > Hello > > > > On 2025-Mar-13, Ashutosh Bapat wrote: > > > > > 1. can you please run the test again and share the dump outputs

Re: Test to dump and restore objects left behind by regression

2025-03-13 Thread Ashutosh Bapat
On Thu, Mar 13, 2025 at 2:12 PM Alvaro Herrera wrote: > > Hello > > On 2025-Mar-13, Ashutosh Bapat wrote: > > > 1. can you please run the test again and share the dump outputs. They > > will be located in a temporary directory with names > > src_dump.sql_adjusted and dest_dump..sql_adjusted. > > A

Re: Test to dump and restore objects left behind by regression

2025-03-13 Thread Alvaro Herrera
Hello On 2025-Mar-13, Ashutosh Bapat wrote: > 1. can you please run the test again and share the dump outputs. They > will be located in a temporary directory with names > src_dump.sql_adjusted and dest_dump..sql_adjusted. Ah, I see the problem :-) The first initdb does this: # Running

Re: Test to dump and restore objects left behind by regression

2025-03-12 Thread Ashutosh Bapat
Hi Alvaro, On Wed, Mar 12, 2025 at 9:39 PM Alvaro Herrera wrote: > > On 2025-Mar-12, Ashutosh Bapat wrote: > > > Does the test pass for you if you don't apply my patches? > > Yes. It also passes if I keep PG_TEST_EXTRA empty. I am not able to reproduce this problem locally. The test uses In

Re: Test to dump and restore objects left behind by regression

2025-03-12 Thread Alvaro Herrera
On 2025-Mar-12, Ashutosh Bapat wrote: > Does the test pass for you if you don't apply my patches? Yes. It also passes if I keep PG_TEST_EXTRA empty. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/

Re: Test to dump and restore objects left behind by regression

2025-03-12 Thread Ashutosh Bapat
On Wed, Mar 12, 2025 at 5:35 PM Alvaro Herrera wrote: > > Hello > > When running these tests, I encounter this strange diff in the dumps, > which seems to be that the locale for type money does not match. I > imagine the problem is that the locale is not set correctly when > initdb'ing one of the

Re: Test to dump and restore objects left behind by regression

2025-03-11 Thread Ashutosh Bapat
On Tue, Feb 25, 2025 at 11:59 AM Ashutosh Bapat wrote: > > On Tue, Feb 11, 2025 at 5:53 PM Ashutosh Bapat > wrote: > > > > Hi Michael, > > > > > > On Sun, Feb 9, 2025 at 1:25 PM Michael Paquier wrote: > > > > > > On Fri, Feb 07, 2025 at 07:11:25AM +0900, Michael Paquier wrote: > > > > Okay, than

Re: Test to dump and restore objects left behind by regression

2025-02-24 Thread Ashutosh Bapat
On Tue, Feb 11, 2025 at 5:53 PM Ashutosh Bapat wrote: > > Hi Michael, > > > On Sun, Feb 9, 2025 at 1:25 PM Michael Paquier wrote: > > > > On Fri, Feb 07, 2025 at 07:11:25AM +0900, Michael Paquier wrote: > > > Okay, thanks for the feedback. We have been relying on diff -u for > > > the parts of t

Re: Test to dump and restore objects left behind by regression

2025-02-11 Thread Ashutosh Bapat
On Wed, Feb 12, 2025 at 5:25 AM Michael Paquier wrote: > > On Tue, Feb 11, 2025 at 12:19:33PM +0530, Ashutosh Bapat wrote: > > Sorry for replying late here. The refactored code in > > 002_compare_backups.pl has a potential to cause confusion even without > > this refactoring. The differences in ta

Re: Test to dump and restore objects left behind by regression

2025-02-11 Thread Michael Paquier
On Tue, Feb 11, 2025 at 12:19:33PM +0530, Ashutosh Bapat wrote: > Sorry for replying late here. The refactored code in > 002_compare_backups.pl has a potential to cause confusion even without > this refactoring. The differences in tablespace paths are adjusted in > compare_files() and not in the ac

Re: Test to dump and restore objects left behind by regression

2025-02-11 Thread Ashutosh Bapat
Hi Michael, On Sun, Feb 9, 2025 at 1:25 PM Michael Paquier wrote: > > On Fri, Feb 07, 2025 at 07:11:25AM +0900, Michael Paquier wrote: > > Okay, thanks for the feedback. We have been relying on diff -u for > > the parts of the tests touched by 0001 for some time now, so if there > > are no obje

Re: Test to dump and restore objects left behind by regression

2025-02-10 Thread Ashutosh Bapat
On Thu, Feb 6, 2025 at 11:32 AM Michael Paquier wrote: > > On Wed, Feb 05, 2025 at 03:28:04PM +0900, Michael Paquier wrote: > > Hmm. I was reading through the patch and there is something that > > clearly stands out IMO: the new compare_dumps(). It is in Utils.pm, > > and it acts as a wrapper of

Re: Test to dump and restore objects left behind by regression

2025-02-08 Thread Michael Paquier
On Fri, Feb 07, 2025 at 07:11:25AM +0900, Michael Paquier wrote: > Okay, thanks for the feedback. We have been relying on diff -u for > the parts of the tests touched by 0001 for some time now, so if there > are no objections I would like to apply 0001 in a couple of days. This part has been appl

Re: Test to dump and restore objects left behind by regression

2025-02-06 Thread Michael Paquier
On Thu, Feb 06, 2025 at 10:43:56AM +0100, Alvaro Herrera wrote: > Great, I've looked at doing something like this in the libpq_pipeline > test for better diff reporting -- what I have uses Test::Differences, > which is pretty neat and usable, but it's not part of the standard > installed perl modul

Re: Test to dump and restore objects left behind by regression

2025-02-06 Thread Alvaro Herrera
On 2025-Feb-06, Michael Paquier wrote: > On Wed, Feb 05, 2025 at 03:28:04PM +0900, Michael Paquier wrote: > > Hmm. I was reading through the patch and there is something that > > clearly stands out IMO: the new compare_dumps(). It is in Utils.pm, > > and it acts as a wrapper of `diff` with its f

Re: Test to dump and restore objects left behind by regression

2025-02-05 Thread Michael Paquier
On Wed, Feb 05, 2025 at 03:28:04PM +0900, Michael Paquier wrote: > Hmm. I was reading through the patch and there is something that > clearly stands out IMO: the new compare_dumps(). It is in Utils.pm, > and it acts as a wrapper of `diff` with its formalized output format. > It is not really abou

Re: Test to dump and restore objects left behind by regression

2025-02-04 Thread Michael Paquier
On Mon, Jan 27, 2025 at 03:04:55PM +0530, Ashutosh Bapat wrote: > PFA patch with rebased on the latest HEAD and conflicts fixed. Thanks for the new patch. Hmm. I was reading through the patch and there is something that clearly stands out IMO: the new compare_dumps(). It is in Utils.pm, and it

Re: Test to dump and restore objects left behind by regression

2025-01-27 Thread Ashutosh Bapat
On Wed, Jan 15, 2025 at 5:59 PM Ashutosh Bapat wrote: > > On Tue, Dec 31, 2024 at 5:24 PM Ashutosh Bapat > wrote: > > > > On Fri, Dec 27, 2024 at 6:17 PM Daniel Gustafsson wrote: > > > > > > > On 20 Dec 2024, at 11:01, Ashutosh Bapat > > > > wrote: > > > > On Wed, Dec 18, 2024 at 7:39 PM Danie

Re: Test to dump and restore objects left behind by regression

2025-01-15 Thread Ashutosh Bapat
On Tue, Dec 31, 2024 at 5:24 PM Ashutosh Bapat wrote: > > On Fri, Dec 27, 2024 at 6:17 PM Daniel Gustafsson wrote: > > > > > On 20 Dec 2024, at 11:01, Ashutosh Bapat > > > wrote: > > > On Wed, Dec 18, 2024 at 7:39 PM Daniel Gustafsson wrote: > > >> > > >>> On 18 Dec 2024, at 12:28, Ashutosh Ba

Re: Test to dump and restore objects left behind by regression

2024-12-31 Thread Ashutosh Bapat
On Fri, Dec 27, 2024 at 6:17 PM Daniel Gustafsson wrote: > > > On 20 Dec 2024, at 11:01, Ashutosh Bapat > > wrote: > > On Wed, Dec 18, 2024 at 7:39 PM Daniel Gustafsson wrote: > >> > >>> On 18 Dec 2024, at 12:28, Ashutosh Bapat > >>> wrote: > > >> + if ( $ENV{PG_TEST_EXTRA} > >> + &

Re: Test to dump and restore objects left behind by regression

2024-12-27 Thread Daniel Gustafsson
> On 20 Dec 2024, at 11:01, Ashutosh Bapat wrote: > On Wed, Dec 18, 2024 at 7:39 PM Daniel Gustafsson wrote: >> >>> On 18 Dec 2024, at 12:28, Ashutosh Bapat >>> wrote: >> + if ( $ENV{PG_TEST_EXTRA} >> + && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/) >> Should this also test that

Re: Test to dump and restore objects left behind by regression

2024-12-20 Thread Ashutosh Bapat
On Wed, Dec 18, 2024 at 7:39 PM Daniel Gustafsson wrote: > > > On 18 Dec 2024, at 12:28, Ashutosh Bapat > > wrote: > > In general I think it's fine to have such an expensive test gated behind a > PG_TEST_EXTRA flag, and since it's only run on demand we might as well run it > for all formats whil

Re: Test to dump and restore objects left behind by regression

2024-12-18 Thread Daniel Gustafsson
> On 18 Dec 2024, at 12:28, Ashutosh Bapat wrote: In general I think it's fine to have such an expensive test gated behind a PG_TEST_EXTRA flag, and since it's only run on demand we might as well run it for all formats while at it. If this ran just once per week in the buildfarm it would still a

Re: Test to dump and restore objects left behind by regression

2024-12-18 Thread Ashutosh Bapat
On Thu, Nov 14, 2024 at 4:16 PM Ashutosh Bapat wrote: > > > But see next > > > > > > > > What's the advantage of testing all the formats? Would that stuff > > > have been able to catch up more issues related to specific format(s) > > > when it came to the compression improvements with inheritance

Re: Test to dump and restore objects left behind by regression

2024-11-07 Thread Ashutosh Bapat
Hi Tom and Michael, Thanks for your inputs. I am replying to all the comments in a single email arranging related comments together. On Thu, Oct 31, 2024 at 11:26 AM Michael Paquier wrote: > > On my laptop, testing the plain format adds roughly 12s, in a test > that now takes 1m20s to run vs 1m

Re: Test to dump and restore objects left behind by regression

2024-10-31 Thread Michael Paquier
On Thu, Oct 31, 2024 at 10:26:01AM -0400, Tom Lane wrote: > I'd be okay with adding it in a form where the default behavior > is to do no additional checking. Whether that's worth maintaining > is hard to say though. In terms of maintenance, it would be nice if we are able to minimize the code ad

Re: Test to dump and restore objects left behind by regression

2024-10-31 Thread Tom Lane
Michael Paquier writes: > On my laptop, testing the plain format adds roughly 12s, in a test > that now takes 1m20s to run vs 1m32s. Enabling regress_dump_formats > and adding three more formats counts for 45s of runtime. For a test > that usually shows up as the last one to finish for a heavily

Re: Test to dump and restore objects left behind by regression

2024-10-31 Thread Michael Paquier
On Mon, Sep 09, 2024 at 03:43:58PM +0530, Ashutosh Bapat wrote: > 894be11adfa60ad1ce5f74534cf5f04e66d51c30 changed the schema in which > objects in test genereated_stored.sql are created. Because of this the > new test added by the patch was failing. Fixed the failure in the > attached. On my lapt

Re: Test to dump and restore objects left behind by regression

2024-09-09 Thread Ashutosh Bapat
On Fri, Jul 12, 2024 at 10:42 AM Ashutosh Bapat wrote: > > I have merged the two patches now. > 894be11adfa60ad1ce5f74534cf5f04e66d51c30 changed the schema in which objects in test genereated_stored.sql are created. Because of this the new test added by the patch was failing. Fixed the failure in

Re: Test to dump and restore objects left behind by regression

2024-07-11 Thread Ashutosh Bapat
On Tue, Jul 9, 2024 at 1:07 PM Michael Paquier wrote: > > On Mon, Jul 08, 2024 at 03:59:30PM +0530, Ashutosh Bapat wrote: > > Before submitting the patch, I looked for all the places which mention > > AdjustUpgrade or AdjustUpgrade.pm to find places where the new module needs > > to be mentioned.

Re: Test to dump and restore objects left behind by regression

2024-07-09 Thread Michael Paquier
On Mon, Jul 08, 2024 at 03:59:30PM +0530, Ashutosh Bapat wrote: > Before submitting the patch, I looked for all the places which mention > AdjustUpgrade or AdjustUpgrade.pm to find places where the new module needs > to be mentioned. But I didn't find any. AdjustUpgrade is not mentioned > in src/te

Re: Test to dump and restore objects left behind by regression

2024-07-08 Thread Ashutosh Bapat
On Fri, Jul 5, 2024 at 10:59 AM Michael Paquier wrote: > On Fri, Jun 28, 2024 at 06:00:07PM +0530, Ashutosh Bapat wrote: > > Here's a description of patches and some notes > > 0001 > > --- > > 1. Per your suggestion the logic to handle dump output differences is > > externalized in PostgreSQL

Re: Test to dump and restore objects left behind by regression

2024-07-04 Thread Michael Paquier
On Fri, Jun 28, 2024 at 06:00:07PM +0530, Ashutosh Bapat wrote: > Here's a description of patches and some notes > 0001 > --- > 1. Per your suggestion the logic to handle dump output differences is > externalized in PostgreSQL::Test::AdjustDump. Instead of eliminating those > differences altoge

Re: Test to dump and restore objects left behind by regression

2024-06-28 Thread Ashutosh Bapat
Sorry for delay, but here's next version of the patchset for review. On Thu, Jun 6, 2024 at 5:07 AM Michael Paquier wrote: > On Wed, Jun 05, 2024 at 05:09:58PM +0530, Ashutosh Bapat wrote: > > Thanks for the suggestion. I didn't understand the dependency with the > > buildfarm module. Will the n

Re: Test to dump and restore objects left behind by regression

2024-06-05 Thread Michael Paquier
On Wed, Jun 05, 2024 at 05:09:58PM +0530, Ashutosh Bapat wrote: > Thanks for the suggestion. I didn't understand the dependency with the > buildfarm module. Will the new module be used in buildfarm separately? I > will work on this soon. Thanks for changing CF entry to WoA. I had some doubts about

Re: Test to dump and restore objects left behind by regression

2024-06-05 Thread Ashutosh Bapat
On Tue, Jun 4, 2024 at 4:28 AM Michael Paquier wrote: > On Fri, Apr 26, 2024 at 06:38:22PM +0530, Ashutosh Bapat wrote: > > Some points for discussion: > > 1. The test still hardcodes the diffs between two dumps. Haven't found a > > better way to do it. I did consider removing the problematic obj

Re: Test to dump and restore objects left behind by regression

2024-06-03 Thread Michael Paquier
On Fri, Apr 26, 2024 at 06:38:22PM +0530, Ashutosh Bapat wrote: > Some points for discussion: > 1. The test still hardcodes the diffs between two dumps. Haven't found a > better way to do it. I did consider removing the problematic objects from > the regression database but thought against it since

Re: Test to dump and restore objects left behind by regression

2024-04-26 Thread Ashutosh Bapat
On Fri, Feb 23, 2024 at 10:46 AM Ashutosh Bapat < ashutosh.bapat@gmail.com> wrote: > On Thu, Feb 22, 2024 at 8:35 PM Tom Lane wrote: > > > > Peter Eisentraut writes: > > > The problem is, we don't really have any end-to-end coverage of > > > > > dump > > > restore > > > dump again > > > comp

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Ashutosh Bapat
On Thu, Feb 22, 2024 at 8:35 PM Tom Lane wrote: > > Peter Eisentraut writes: > > The problem is, we don't really have any end-to-end coverage of > > > dump > > restore > > dump again > > compare the two dumps > > > with a database with lots of interesting objects in it. > > I'm very much against

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Ashutosh Bapat
On Thu, Feb 22, 2024 at 3:50 PM Peter Eisentraut wrote: > > On 22.02.24 11:00, Daniel Gustafsson wrote: > >> On 22 Feb 2024, at 10:55, Ashutosh Bapat > >> wrote: > >> On Thu, Feb 22, 2024 at 3:03 PM Daniel Gustafsson wrote: > > > >> Somebody looking for dump/restore tests wouldn't search > >> s

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Tom Lane
Peter Eisentraut writes: > The problem is, we don't really have any end-to-end coverage of > dump > restore > dump again > compare the two dumps > with a database with lots of interesting objects in it. I'm very much against adding another full run of the core regression tests to support this.

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Peter Eisentraut
On 22.02.24 11:00, Daniel Gustafsson wrote: On 22 Feb 2024, at 10:55, Ashutosh Bapat wrote: On Thu, Feb 22, 2024 at 3:03 PM Daniel Gustafsson wrote: Somebody looking for dump/restore tests wouldn't search src/bin/pg_upgrade, I think. Quite possibly not, but pg_upgrade is already today an i

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Daniel Gustafsson
> On 22 Feb 2024, at 10:55, Ashutosh Bapat wrote: > On Thu, Feb 22, 2024 at 3:03 PM Daniel Gustafsson wrote: > Somebody looking for dump/restore tests wouldn't search > src/bin/pg_upgrade, I think. Quite possibly not, but pg_upgrade is already today an important testsuite for testing pg_dump in

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Ashutosh Bapat
On Thu, Feb 22, 2024 at 3:03 PM Daniel Gustafsson wrote: > > > On 22 Feb 2024, at 10:16, Peter Eisentraut wrote: > > > We have somewhat relied on the pg_upgrade test to provide this testing, but > > we have recently discovered that the dumps in binary-upgrade mode are > > different enough to no

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Daniel Gustafsson
> On 22 Feb 2024, at 10:16, Peter Eisentraut wrote: > We have somewhat relied on the pg_upgrade test to provide this testing, but > we have recently discovered that the dumps in binary-upgrade mode are > different enough to not test the normal dumps well. > > Yes, this test is a bit expensive.

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Peter Eisentraut
On 22.02.24 02:01, Michael Paquier wrote: On Wed, Feb 21, 2024 at 12:18:45PM +0530, Ashutosh Bapat wrote: Even with 1 and 2 the test is useful to detect dump/restore anomalies. I think we should improve 3, but I don't have a good and simpler solution. I didn't find any way to compare two given c

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Ashutosh Bapat
On Thu, Feb 22, 2024 at 6:32 AM Michael Paquier wrote: > > On Wed, Feb 21, 2024 at 12:18:45PM +0530, Ashutosh Bapat wrote: > > Even with 1 and 2 the test is useful to detect dump/restore anomalies. > > I think we should improve 3, but I don't have a good and simpler > > solution. I didn't find any

Re: Test to dump and restore objects left behind by regression

2024-02-21 Thread Michael Paquier
On Wed, Feb 21, 2024 at 12:18:45PM +0530, Ashutosh Bapat wrote: > Even with 1 and 2 the test is useful to detect dump/restore anomalies. > I think we should improve 3, but I don't have a good and simpler > solution. I didn't find any way to compare two given clusters in our > TAP test framework. Bu