Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-11 Thread Alvaro Herrera
Peter Eisentraut wrote: > On 5/3/16 1:25 PM, Alvaro Herrera wrote: > >If we can put together a script that runs test.sh for various versions > >and then verifies the runs, we could use it in both buildfarm and > >coverage. > > Not that that would be useless, but note that the value in this case (a

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-11 Thread Bruce Momjian
On Wed, May 11, 2016 at 09:40:09AM -0400, Peter Eisentraut wrote: > Not that that would be useless, but note that the value in this case (and > most others) comes from having a candidate object in the database before > upgrade that exercises the particular problem, mostly independent of what > vers

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-11 Thread Peter Eisentraut
On 5/3/16 1:25 PM, Alvaro Herrera wrote: If we can put together a script that runs test.sh for various versions and then verifies the runs, we could use it in both buildfarm and coverage. Not that that would be useless, but note that the value in this case (and most others) comes from having a

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-10 Thread Bruce Momjian
On Fri, May 6, 2016 at 03:32:23PM -0400, Tom Lane wrote: > > I think possibly the easiest fix for this is to have pg_upgrade, > > instead of RESETting a nonexistent option, RESET something that's > > still considered to require AccessExclusiveLock. "user_catalog_table" > > would work, looks like;

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-10 Thread Bruce Momjian
On Tue, May 3, 2016 at 12:07:51PM -0400, Tom Lane wrote: > I think possibly the easiest fix for this is to have pg_upgrade, > instead of RESETting a nonexistent option, RESET something that's > still considered to require AccessExclusiveLock. "user_catalog_table" > would work, looks like; though

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-08 Thread Greg Stark
For what it's worth, for my historical sort benchmarks I got Postgres to build right back to 6.5 using modern tools. I have patches if anyone wants them. Pre-7.3 doesn't actually run because we didn't support 64-bit architectures before Tom did the V1 api (there was a set of Alpha patches floating

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-07 Thread Simon Riggs
On 7 May 2016 at 16:49, Tom Lane wrote: > Simon Riggs writes: > > On 3 May 2016 at 18:07, Tom Lane wrote: > >> Or at least, it did until Simon decided that ALTER TABLE RESET > >> doesn't require AccessExclusiveLock. > > > On reflection, this still seems like a good idea. > > Yes, what pg_upgrad

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-07 Thread Tom Lane
Simon Riggs writes: > On 3 May 2016 at 18:07, Tom Lane wrote: >> Or at least, it did until Simon decided that ALTER TABLE RESET >> doesn't require AccessExclusiveLock. > On reflection, this still seems like a good idea. Yes, what pg_upgrade was doing was clearly a hack, and not a very nice one.

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-07 Thread Simon Riggs
On 3 May 2016 at 18:07, Tom Lane wrote: > Or at least, it did until Simon decided that ALTER TABLE RESET > doesn't require AccessExclusiveLock. On reflection, this still seems like a good idea. > Now you get a failure. > Failure condition as an exception to that. > I haven't tried to cons

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-06 Thread Tom Lane
I wrote: > I have no more time to work on this, but I think it needs to be fixed, and > I definitely think we had better put in test coverage when we do fix it. Actually, there is a really easy fix we could put in, which is to decide that optionally_create_toast_tables() is useless and get rid of

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-06 Thread Tom Lane
I wrote: > I haven't tried to construct a pre-9.1 database that would trigger > this, but you can make it happen by applying the attached patch > to create a toast-table-less table in the regression tests, > and then doing "make check" in src/bin/pg_upgrade. You get this: > ... > Restoring databa

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andrew Dunstan
On 05/03/2016 01:58 PM, Alvaro Herrera wrote: Andrew Dunstan wrote: And if this is of any use, here are the dump differences from every live version to git tip, as of this morning. Interesting, thanks. I wonder if some of these diffs could be reduced further by using pg_dump -Fd instead of

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Tom Lane
Andres Freund writes: > On 2016-05-03 12:07:51 -0400, Tom Lane wrote: >> I think possibly the easiest fix for this is to have pg_upgrade, >> instead of RESETting a nonexistent option, RESET something that's >> still considered to require AccessExclusiveLock. "user_catalog_table" >> would work, lo

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andres Freund
On 2016-05-03 13:47:14 -0400, Tom Lane wrote: > I've been thinking of proposing that it's time (not now, at this point, > but for 9.7) to rip out libpq's support for V2 protocol as well as > pg_dump's support for pre-7.4 backends. +1 > There might be an argument for moving pg_dump's cutoff furth

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Alvaro Herrera
Andrew Dunstan wrote: > And if this is of any use, here are the dump differences from every live > version to git tip, as of this morning. Interesting, thanks. I wonder if some of these diffs could be reduced further by using pg_dump -Fd instead of a single text dump -- then internal ordering wo

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Stephen Frost
* Andres Freund (and...@anarazel.de) wrote: > On 2016-05-03 12:07:51 -0400, Tom Lane wrote: > > I think possibly the easiest fix for this is to have pg_upgrade, > > instead of RESETting a nonexistent option, RESET something that's > > still considered to require AccessExclusiveLock. "user_catalog_

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Tom Lane
Stephen Frost writes: > One other point is that pg_dump goes quite a bit farther back than just > what we currently support (or at least, it tries to). I think that, > generally, that's a good thing, but it does mean we have a lot of cases > that don't get tested nearly as much... Yeah. I do pe

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andres Freund
Hi, On 2016-05-03 12:07:51 -0400, Tom Lane wrote: > I think possibly the easiest fix for this is to have pg_upgrade, > instead of RESETting a nonexistent option, RESET something that's > still considered to require AccessExclusiveLock. "user_catalog_table" > would work, looks like; though I'd wan

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Alvaro Herrera
Stephen Frost wrote: > One other point is that pg_dump goes quite a bit farther back than just > what we currently support (or at least, it tries to). I think that, > generally, that's a good thing, but it does mean we have a lot of cases > that don't get tested nearly as much... > > I was able

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andrew Dunstan
On 05/03/2016 01:33 PM, Andrew Dunstan wrote: On 05/03/2016 01:28 PM, Andrew Dunstan wrote: On 05/03/2016 01:21 PM, Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Tom Lane wrote: More generally, though, I wonder how we can have some test coverage on such cases

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andrew Dunstan
On 05/03/2016 01:28 PM, Andrew Dunstan wrote: On 05/03/2016 01:21 PM, Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Tom Lane wrote: More generally, though, I wonder how we can have some test coverage on such cases going forward. Is the patch below too ugly to co

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen Frost wrote: > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > > > Tom Lane wrote: > > > > > > > More generally, though, I wonder how we can have some test coverage > > > > on such cases going forward. Is the patch below too ugly

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andrew Dunstan
On 05/03/2016 01:21 PM, Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Tom Lane wrote: More generally, though, I wonder how we can have some test coverage on such cases going forward. Is the patch below too ugly to commit permanently, and if so, what other idea can

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Tom Lane wrote: > > > More generally, though, I wonder how we can have some test coverage > > on such cases going forward. Is the patch below too ugly to commit > > permanently, and if so, what other idea can you suggest? > > I suggest a build

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Alvaro Herrera
Stephen Frost wrote: > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > > Tom Lane wrote: > > > > > More generally, though, I wonder how we can have some test coverage > > > on such cases going forward. Is the patch below too ugly to commit > > > permanently, and if so, what other idea can yo

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Alvaro Herrera
Tom Lane wrote: > More generally, though, I wonder how we can have some test coverage > on such cases going forward. Is the patch below too ugly to commit > permanently, and if so, what other idea can you suggest? I suggest a buildfarm animal running a custom buildfarm module that exercises the

[HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Tom Lane
There is logic in pg_upgrade plus the backend, mostly added by commit 4c6780fd1, to cope with the corner cases that sometimes arise where the old and new versions have different ideas about whether a given table needs a TOAST table. The more common case is where there's a TOAST table in the old DB