Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-08-03 Thread Bruce Momjian
On Tue, Aug 3, 2021 at 11:13:45AM -0500, Justin Pryzby wrote: > On Tue, Aug 03, 2021 at 12:00:47PM -0400, Bruce Momjian wrote: > > Patch applied through 9.6. > > The comment seems to be a leftover from a copy pasto. > > + /* find hash indexes */ > + res = executeQuery

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-08-03 Thread Justin Pryzby
On Tue, Aug 03, 2021 at 12:00:47PM -0400, Bruce Momjian wrote: > Patch applied through 9.6. The comment seems to be a leftover from a copy pasto. + /* find hash indexes */ + res = executeQueryOrDie(conn, + "

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-08-03 Thread Bruce Momjian
On Fri, Jul 30, 2021 at 12:40:06PM -0400, Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 06:19:56PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > I think we need to first give clear instructions on how to find out if > > > an extension update is available, and then how to update it. I am

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-08-03 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 03:06:45PM -0400, Bruce Momjian wrote: > > I haven't seen it mentioned in the thread, but I think the final phrase > > of this should be a separate step, > > > > > > Copy custom full-text search files > > > > Copy any custom full text search file (dictionary, synonym

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Jan Wieck
On 7/30/21 1:05 PM, Tom Lane wrote: I don't see the need for it anyway. What is different from just putting the update actions into an extension version upgrade script created according to the current rules, and then setting that new extension version as the default version in the extension buil

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Tom Lane
Jan Wieck writes: > An alternative to this would be a recommended version number scheme for > extensions. If extensions were numbered > MAJOR_SERVER.MAJOR_EXTENSION.MINOR_EXTENSION > then pg_upgrade could check the new cluster for the existence of an SQL > script that upgrades the extension

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Jan Wieck
On 7/30/21 7:40 AM, Julien Rouhaud wrote: On Fri, Jul 30, 2021 at 07:33:55AM -0400, Dave Cramer wrote: Ah, ok, got it, thanks. Well I'm not sure how to deal with this. The only thing I can think of is that we add a post_upgrade function to the API. All extensions would have to implement this.

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Jan Wieck
On 7/30/21 7:33 AM, Dave Cramer wrote: Yes, but as I said twice only if the currently installed version is different from the default version.  Otherwise ALTER EXTENSION UPGRADE is a no-op. Ah, ok, got it, thanks. Well I'm not sure how to deal with this. The only thing I can thi

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 06:19:56PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > I think we need to first give clear instructions on how to find out if > > an extension update is available, and then how to update it. I am > > thinking we should supply a query which reports all extensions tha

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Julien Rouhaud
On Fri, Jul 30, 2021 at 07:33:55AM -0400, Dave Cramer wrote: > > Ah, ok, got it, thanks. Well I'm not sure how to deal with this. > The only thing I can think of is that we add a post_upgrade function to the > API. > > All extensions would have to implement this. It seems like a really big hamme

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Dave Cramer
> > Yes, but as I said twice only if the currently installed version is > different > from the default version. Otherwise ALTER EXTENSION UPGRADE is a no-op. > Ah, ok, got it, thanks. Well I'm not sure how to deal with this. The only thing I can think of is that we add a post_upgrade function to

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Julien Rouhaud
On Fri, Jul 30, 2021 at 07:18:56AM -0400, Dave Cramer wrote: > > So my understanding is that upgrade is going to run all of the SQL files > from whatever version the original instance was up to the current version. > > I'm at a loss as to how this would not work ? How do you upgrade your > extens

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Dave Cramer
On Fri, 30 Jul 2021 at 07:07, Julien Rouhaud wrote: > On Fri, Jul 30, 2021 at 06:48:34AM -0400, Dave Cramer wrote: > > > > Well I think that's on the extension author to fix. There's only so much > > pg_upgrade can do here. > > It seems reasonable that upgrading the extension should upgrade the >

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Julien Rouhaud
On Fri, Jul 30, 2021 at 06:48:34AM -0400, Dave Cramer wrote: > > Well I think that's on the extension author to fix. There's only so much > pg_upgrade can do here. > It seems reasonable that upgrading the extension should upgrade the > extension to the latest version. That would only work iff the

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Dave Cramer
On Fri, 30 Jul 2021 at 06:39, Julien Rouhaud wrote: > On Fri, Jul 30, 2021 at 06:03:50AM -0400, Dave Cramer wrote: > > > > What would happen if subsequent to the upgrade "ALTER EXTENSION UPGRADE" > > was executed ? > > If the extension was already up to date on the source cluster then > obviously

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Julien Rouhaud
On Fri, Jul 30, 2021 at 06:03:50AM -0400, Dave Cramer wrote: > > What would happen if subsequent to the upgrade "ALTER EXTENSION UPGRADE" > was executed ? If the extension was already up to date on the source cluster then obviously nothing. Otherwise, the extension would be updated. But unless

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-30 Thread Dave Cramer
On Thu, 29 Jul 2021 at 22:03, Julien Rouhaud wrote: > On Thu, Jul 29, 2021 at 02:14:56PM -0400, Jan Wieck wrote: > > > > I presume that pg_upgrade on a database with that extension installed > would > > silently succeed and have the pg_catalog as well as public (or wherever) > > version of that f

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Julien Rouhaud
On Thu, Jul 29, 2021 at 02:14:56PM -0400, Jan Wieck wrote: > > I presume that pg_upgrade on a database with that extension installed would > silently succeed and have the pg_catalog as well as public (or wherever) > version of that function present. I'll have to run a pg_upgrade with it to be 100

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Alvaro Herrera
On 2021-Jul-29, Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 06:29:11PM -0400, Tom Lane wrote: > > No, because dump/restore does not have this issue. Regular pg_dump just > > issues "CREATE EXTENSION" commands, so you automatically get the target > > server's default version. > > Oh, so pg_up

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
On Thu, 29 Jul 2021 at 18:39, Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 06:29:11PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > On Thu, Jul 29, 2021 at 06:19:56PM -0400, Tom Lane wrote: > > >> I suggested awhile ago that pg_upgrade should look into > > >> pg_available_extensions i

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 06:29:11PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Thu, Jul 29, 2021 at 06:19:56PM -0400, Tom Lane wrote: > >> I suggested awhile ago that pg_upgrade should look into > >> pg_available_extensions in the new cluster, and prepare > >> a script with ALTER EXTENSI

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Tom Lane
Bruce Momjian writes: > On Thu, Jul 29, 2021 at 06:19:56PM -0400, Tom Lane wrote: >> I suggested awhile ago that pg_upgrade should look into >> pg_available_extensions in the new cluster, and prepare >> a script with ALTER EXTENSION UPDATE commands for >> anything that's installed but is not the (

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 06:19:56PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > I think we need to first give clear instructions on how to find out if > > an extension update is available, and then how to update it. I am > > thinking we should supply a query which reports all extensions tha

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Tom Lane
Bruce Momjian writes: > I think we need to first give clear instructions on how to find out if > an extension update is available, and then how to update it. I am > thinking we should supply a query which reports all extensions that can > be upgraded, at least for contrib. I suggested awhile ago

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 05:01:24PM -0400, Tom Lane wrote: > Dave Cramer writes: > > So back to the original motivation for bringing this up. Recall that a > > cluster was upgraded. Everything appeared to work fine, except that the > > definitions of the functions were slightly different enough to

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Tom Lane
Dave Cramer writes: > So back to the original motivation for bringing this up. Recall that a > cluster was upgraded. Everything appeared to work fine, except that the > definitions of the functions were slightly different enough to cause a > fatal issue on restoring a dump from pg_dump. > Since pg

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread David G. Johnston
On Thu, Jul 29, 2021 at 12:28 PM Dave Cramer wrote: > So back to the original motivation for bringing this up. Recall that a > cluster was upgraded. Everything appeared to work fine, except that the > definitions of the functions were slightly different enough to cause a > fatal issue on restorin

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
Dave Cramer On Thu, 29 Jul 2021 at 15:06, Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 01:43:09PM -0400, Álvaro Herrera wrote: > > On 2021-Jul-29, Bruce Momjian wrote: > > > > > + If the old cluster used extensions, whether from > > > + contrib or some other source, it used > > > +

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 02:19:17PM -0400, Álvaro Herrera wrote: > On 2021-Jul-29, Dave Cramer wrote: > > > > I suggest "can be upgraded" rather than "should be upgraded" because > > > we're not making a recommendation, merely stating the fact that it is > > > possible to do so. > > > > Why not rec

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 01:43:09PM -0400, Álvaro Herrera wrote: > On 2021-Jul-29, Bruce Momjian wrote: > > > + If the old cluster used extensions, whether from > > + contrib or some other source, it used > > + shared object files (or DLLs) to implement these extensions, e.g., > > +

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread David G. Johnston
On Thu, Jul 29, 2021 at 11:35 AM Bruce Momjian wrote: > > Oh, and you can't use the same installation procedures as when you > installed the extension because that probably included CREATE EXTENSION. > This really highlights why this is tricky to explain --- we need the > binaries, but not the SQ

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread David G. Johnston
On Thu, Jul 29, 2021 at 11:28 AM Bruce Momjian wrote: > I am sorry but none of your suggestions are exciting me --- they seem to > get into too much detail for the context. > Fair, I still need to consider Alvaro's anyway, but given the amount of general angst surrounding performing a pg_upgrade

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 02:28:55PM -0400, Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 10:00:12AM -0700, David G. Johnston wrote: > > I'm warming up to "should" but maybe add a "why" such as "the old versions > > are > > considered unsupported on the newer server". > > > > I dislike "usually vi

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 10:00:12AM -0700, David G. Johnston wrote: > I'm warming up to "should" but maybe add a "why" such as "the old versions are > considered unsupported on the newer server". > > I dislike "usually via operating system commands", just offload this to the > extension, i.e., "mus

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Alvaro Herrera
On 2021-Jul-29, Dave Cramer wrote: > > I suggest "can be upgraded" rather than "should be upgraded" because > > we're not making a recommendation, merely stating the fact that it is > > possible to do so. > > Why not recommend it? I was going to suggest that we actually run alter > extension upgra

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Jan Wieck
On 7/29/21 2:04 PM, Julien Rouhaud wrote: On Thu, Jul 29, 2021 at 11:46:12AM -0400, Jan Wieck wrote: > I don't have a ready example of such an extension, but if we ever would > convert the backend parts of Slony into an extension, it would be one. FWIW I have an example of such an extension

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
Dave Cramer On Thu, 29 Jul 2021 at 13:43, Alvaro Herrera wrote: > On 2021-Jul-29, Bruce Momjian wrote: > > > + If the old cluster used extensions, whether from > > + contrib or some other source, it used > > + shared object files (or DLLs) to implement these extensions, e.g., > > +

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Julien Rouhaud
On Fri, Jul 30, 2021 at 12:14 AM Bruce Momjian wrote: > > On Thu, Jul 29, 2021 at 11:46:12AM -0400, Jan Wieck wrote: > > > > This assumes that the scripts executed during CREATE EXTENSION have no > > conditional code in them that depends on the server version. Running the > > same SQL script on di

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Alvaro Herrera
On 2021-Jul-29, Bruce Momjian wrote: > + If the old cluster used extensions, whether from > + contrib or some other source, it used > + shared object files (or DLLs) to implement these extensions, e.g., > + pgcrypto.so. Now, shared object files matching > + the new server bina

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
On Thu, 29 Jul 2021 at 13:13, Alvaro Herrera wrote: > On 2021-Jul-29, Dave Cramer wrote: > > > > If the old cluster used extensions those same extensions must be > > > installed in the new cluster via installation procedures specific > > > to, and documented by, each extension. For contrib exten

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Alvaro Herrera
On 2021-Jul-29, Dave Cramer wrote: > > If the old cluster used extensions those same extensions must be > > installed in the new cluster via installation procedures specific > > to, and documented by, each extension. For contrib extensions it is > > usually enough to install the -contrib package

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
> > > > I would simplify the first two sentences to just: > > If the old cluster used extensions those same extensions must be installed > in the new cluster via installation procedures specific to, and documented > by, each extension. For contrib extensions it is usually enough to install > the -

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread David G. Johnston
On Thu, Jul 29, 2021 at 9:55 AM Jan Wieck wrote: > How exactly do you envision that we, the PostgreSQL project, make sure > that extension developers provide those mechanisms in the future? > > I have no suggestions here, and don't really plan to get deeply involved in this area of the project an

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread David G. Johnston
On Thu, Jul 29, 2021 at 9:34 AM Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 12:22:59PM -0400, Dave Cramer wrote: > > On Thu, 29 Jul 2021 at 12:16, Bruce Momjian wrote: > > Can you review the text I just posted? Thanks. I think we are > making > > progress. ;-) > > > > > > I am OK

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Jan Wieck
On 7/29/21 12:44 PM, David G. Johnston wrote: ... - but at present pg_upgrade simply doesn't care and hopes that the extension author's documentation deals with these possibilities in a sane manner. pg_upgrade is not able to address this in a guaranteed, consistent fashion. At this point the

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread David G. Johnston
On Thu, Jul 29, 2021 at 9:28 AM Jan Wieck wrote: > On 7/29/21 12:00 PM, David G. Johnston wrote: > > Ok, looking at the flow again, where exactly would the user even be able > > to execute "CREATE EXTENSION" meaningfully? The relevant databases do > > not exist (not totally sure what happens to

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 12:22:59PM -0400, Dave Cramer wrote: > On Thu, 29 Jul 2021 at 12:16, Bruce Momjian wrote: > Can you review the text I just posted?  Thanks.   I think we are making > progress.  ;-) > > > I am OK with Everything except > > Do not load the schema definitions, > e.g

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Jan Wieck
On 7/29/21 12:00 PM, David G. Johnston wrote: Ok, looking at the flow again, where exactly would the user even be able to execute "CREATE EXTENSION" meaningfully?  The relevant databases do not exist (not totally sure what happens to the postgres database created during the initdb step...) so a

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
Dave Cramer On Thu, 29 Jul 2021 at 12:16, Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 09:00:36AM -0700, David G. Johnston wrote: > > On Thu, Jul 29, 2021 at 8:42 AM Bruce Momjian wrote: > > > > On Thu, Jul 29, 2021 at 11:36:19AM -0400, Dave Cramer wrote: > > > > > > > > >

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 09:00:36AM -0700, David G. Johnston wrote: > On Thu, Jul 29, 2021 at 8:42 AM Bruce Momjian wrote: > > On Thu, Jul 29, 2021 at 11:36:19AM -0400, Dave Cramer wrote: > > > > > >     I have an issue with the fragment "whether they are from contrib" - > my >

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 11:46:12AM -0400, Jan Wieck wrote: > On 7/29/21 11:10 AM, Bruce Momjian wrote: > > On Thu, Jul 29, 2021 at 11:01:43AM -0400, Dave Cramer wrote: > > > Much better, however I'm unclear on whether CREATE EXTENSION is actually > > > executed on the upgraded server. > > > > > >

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 08:39:32AM -0700, David G. Johnston wrote: > On Thu, Jul 29, 2021 at 8:36 AM Dave Cramer wrote: > > > > > I have an issue with the fragment "whether they are from contrib" - my > understanding at this point is that because of the way we package and >

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread David G. Johnston
On Thu, Jul 29, 2021 at 8:42 AM Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 11:36:19AM -0400, Dave Cramer wrote: > > > > > > I have an issue with the fragment "whether they are from contrib" - > my > > understanding at this point is that because of the way we package and > > versio

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
On Thu, 29 Jul 2021 at 11:39, David G. Johnston wrote: > On Thu, Jul 29, 2021 at 8:36 AM Dave Cramer wrote: > >> >> >> I have an issue with the fragment "whether they are from contrib" - my >>> understanding at this point is that because of the way we package and >>> version contrib it should no

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Jan Wieck
On 7/29/21 11:10 AM, Bruce Momjian wrote: On Thu, Jul 29, 2021 at 11:01:43AM -0400, Dave Cramer wrote: Much better, however I'm unclear on whether CREATE EXTENSION is actually executed on the upgraded server. From what I could gather it is not, otherwise the new function definitions should have

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 11:36:19AM -0400, Dave Cramer wrote: > > > I have an issue with the fragment "whether they are from contrib" - my > understanding at this point is that because of the way we package and > version contrib it should not be necessary to copy those shared object >

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 08:28:12AM -0700, David G. Johnston wrote: > On Thu, Jul 29, 2021 at 7:56 AM Bruce Momjian wrote: > > On Wed, Jul 28, 2021 at 09:35:28PM -0700, David G. Johnston wrote: > > On Wed, Jul 28, 2021 at 6:52 PM Bruce Momjian wrote: > > > >     I came up with the

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread David G. Johnston
On Thu, Jul 29, 2021 at 8:36 AM Dave Cramer wrote: > > > I have an issue with the fragment "whether they are from contrib" - my >> understanding at this point is that because of the way we package and >> version contrib it should not be necessary to copy those shared object >> files from the old

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
I have an issue with the fragment "whether they are from contrib" - my > understanding at this point is that because of the way we package and > version contrib it should not be necessary to copy those shared object > files from the old to the new server (maybe, just maybe, with a > qualification t

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread David G. Johnston
On Thu, Jul 29, 2021 at 7:56 AM Bruce Momjian wrote: > On Wed, Jul 28, 2021 at 09:35:28PM -0700, David G. Johnston wrote: > > On Wed, Jul 28, 2021 at 6:52 PM Bruce Momjian wrote: > > > > I came up with the attached patch. > > > > > > Thank you. It is an improvement but I think more could be

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 11:17:52AM -0400, Dave Cramer wrote: > On Thu, 29 Jul 2021 at 11:10, Bruce Momjian wrote: > OK, I think we should be more clear as to what is happening.  Saying they will > be recreated is misleading. > The extension definitions are being copied from the old server to the n

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
On Thu, 29 Jul 2021 at 11:10, Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 11:01:43AM -0400, Dave Cramer wrote: > > > > > > > > > > OK, I went with this new text. There is confusion over install vs > copy, > > and whether this is happening at the operating system level or the > SQL > >

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 11:01:43AM -0400, Dave Cramer wrote: > > > > > OK, I went with this new text.  There is confusion over install vs copy, > and whether this is happening at the operating system level or the SQL > level.  I tried to clarify that, but I am not sure I was success

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
> > OK, I went with this new text. There is confusion over install vs copy, > and whether this is happening at the operating system level or the SQL > level. I tried to clarify that, but I am not sure I was successful. I > also used the word "extension" since this is more common than "custom". >

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Bruce Momjian
On Wed, Jul 28, 2021 at 09:35:28PM -0700, David G. Johnston wrote: > On Wed, Jul 28, 2021 at 6:52 PM Bruce Momjian wrote: > > I came up with the attached patch. > > > Thank you.  It is an improvement but I think more could be done here (not > exactly sure what - though removing the "copy bi

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-29 Thread Dave Cramer
On Thu, 29 Jul 2021 at 00:35, David G. Johnston wrote: > On Wed, Jul 28, 2021 at 6:52 PM Bruce Momjian wrote: > >> I came up with the attached patch. >> > > Thank you. It is an improvement but I think more could be done here (not > exactly sure what - though removing the "copy binaries for cont

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-28 Thread David G. Johnston
On Wed, Jul 28, 2021 at 6:52 PM Bruce Momjian wrote: > I came up with the attached patch. > Thank you. It is an improvement but I think more could be done here (not exactly sure what - though removing the "copy binaries for contrib modules from the old server" seems like a decent second step.)

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-28 Thread Bruce Momjian
On Thu, Jul 15, 2021 at 08:15:57AM -0700, David G. Johnston wrote: > On Thursday, July 15, 2021, David G. Johnston >  My uncertainty revolves around core extensions since it seems odd to tell > the user to overwrite them with versions from an older version of > PostgreSQL. > > Ok. Jus

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thursday, July 15, 2021, Jan Wieck wrote: > On 7/15/21 4:24 PM, David G. Johnston wrote: > > OK, especially as this seems useful outside of pg_upgrade, and if done >> separately is something pg_upgrade could just run as part of its new >> cluster evaluation scripts. Knowing whether an extensi

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Jan Wieck
On 7/15/21 4:24 PM, David G. Johnston wrote: OK, especially as this seems useful outside of pg_upgrade, and if done separately is something pg_upgrade could just run as part of its new cluster evaluation scripts.  Knowing whether an extension is outdated doesn't require the old cluster. Know

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thu, Jul 15, 2021 at 11:52 AM Dave Cramer wrote: > > On Thu, 15 Jul 2021 at 14:31, David G. Johnston < > david.g.johns...@gmail.com> wrote: > >> >> Yep, and IIUC running "CREATE EXTENSION pg_stat_statements VERSION >> '1.5';" works correctly in v13 as does executing >> > While it does work the

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Dave Cramer
On Thu, 15 Jul 2021 at 14:31, David G. Johnston wrote: > On Thu, Jul 15, 2021 at 11:14 AM Jan Wieck wrote: > >> On 7/15/21 1:10 PM, David G. Johnston wrote: >> > ... But while >> > PostgreSQL doesn't really have a choice here - it cannot be expected to >> > subdivide itself - extensions (at leas

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thu, Jul 15, 2021 at 11:14 AM Jan Wieck wrote: > On 7/15/21 1:10 PM, David G. Johnston wrote: > > ... But while > > PostgreSQL doesn't really have a choice here - it cannot be expected to > > subdivide itself - extensions (at least external ones - PostGIS is one I > > have in mind presently) -

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Bruce Momjian
On Thu, Jul 15, 2021 at 02:14:28PM -0400, Jan Wieck wrote: > On 7/15/21 1:10 PM, David G. Johnston wrote: > > ... But while PostgreSQL doesn't really have a choice here - it cannot > > be expected to subdivide itself - extensions (at least external ones - > > PostGIS is one I have in mind presently

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Jan Wieck
On 7/15/21 1:10 PM, David G. Johnston wrote: ... But while PostgreSQL doesn't really have a choice here - it cannot be expected to subdivide itself - extensions (at least external ones - PostGIS is one I have in mind presently) - can and often do attempt to support multiple versions of Postgre

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thu, Jul 15, 2021 at 9:58 AM Jan Wieck wrote: > On 7/15/21 12:46 PM, David G. Johnston wrote: > > > I am an application developer who operates on the principle of "change > > only one thing at a time". > > Which pg_upgrade by definition isn't. It is bringing ALL the changes > from one major ve

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Jan Wieck
On 7/15/21 12:46 PM, David G. Johnston wrote: I am an application developer who operates on the principle of "change only one thing at a time". Which pg_upgrade by definition isn't. It is bringing ALL the changes from one major version to the target version, which may be multiple at once. In

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thu, Jul 15, 2021 at 9:36 AM Jan Wieck wrote: > > I am a bit confused here. From the previous exchange I get the feeling > that you haven't created and maintained a single extension that survived > a single version upgrade of itself or PostgreSQL (in the latter case > requiring code changes to

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Jan Wieck
On 7/15/21 12:31 PM, Robert Eckhardt wrote:  I don’t know if this is a terrible flaw in pg_upgrade, it is a terrible flaw in the overall Postgres experience. +1 (that is the actual problem here) -- Jan Wieck

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Jan Wieck
On 7/15/21 12:25 PM, David G. Johnston wrote: I would say that it probably should be "--upgrade-extension=aaa --upgrade_extension=bbb" though if we are going to the effort to offer something. I am a bit confused here. From the previous exchange I get the feeling that you haven't created and

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Dave Cramer
On Thu, 15 Jul 2021 at 12:25, David G. Johnston wrote: > On Thu, Jul 15, 2021 at 9:16 AM Dave Cramer wrote: > >> Eh, and >>> pg_upgrade [other switches] --upgrade-extensions >>> sounds good too ... >>> >> >> Ultimately I believe this is the solution, however we still need to teach >> extension

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Robert Eckhardt
On Thu, Jul 15, 2021 at 8:43 AM Dave Cramer mailto:davecra...@gmail.com>> wrote: On Thu, 15 Jul 2021 at 11:29, David G. Johnston mailto:david.g.johns...@gmail.com>> wrote: I’m not familiar with what hoops extensions jump through to facilitate upgrades but even if it was as simple as “create ex

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thu, Jul 15, 2021 at 9:16 AM Dave Cramer wrote: > Eh, and >> pg_upgrade [other switches] --upgrade-extensions >> sounds good too ... >> > > Ultimately I believe this is the solution, however we still need to teach > extensions how to upgrade themselves or emit a message saying they can't, >

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Dave Cramer
On Thu, 15 Jul 2021 at 12:13, Alvaro Herrera wrote: > On 2021-Jul-15, Alvaro Herrera wrote: > > > On 2021-Jul-15, Dave Cramer wrote: > > > > > Well IMHO the status quo is terrible. Perhaps you have a suggestion on > how > > > to make it better ? > > > > I thought the suggestion of having pg_upgra

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thu, Jul 15, 2021 at 8:43 AM Dave Cramer wrote: > > On Thu, 15 Jul 2021 at 11:29, David G. Johnston < > david.g.johns...@gmail.com> wrote: > >> >> I’m not familiar with what hoops extensions jump through to facilitate >> upgrades but even if it was as simple as “create extension upgrade” I >>

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Alvaro Herrera
On 2021-Jul-15, Alvaro Herrera wrote: > On 2021-Jul-15, Dave Cramer wrote: > > > Well IMHO the status quo is terrible. Perhaps you have a suggestion on how > > to make it better ? > > I thought the suggestion of having pg_upgrade emit a file with a list of > all extensions needing upgrade in eac

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Alvaro Herrera
On 2021-Jul-15, Dave Cramer wrote: > Well IMHO the status quo is terrible. Perhaps you have a suggestion on how > to make it better ? I thought the suggestion of having pg_upgrade emit a file with a list of all extensions needing upgrade in each database was a fairly decent one. -- Álvaro Herre

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Dave Cramer
On Thu, 15 Jul 2021 at 11:29, David G. Johnston wrote: > On Thursday, July 15, 2021, Dave Cramer wrote: > >> >> I'm thinking at this point we need something a bit more sophisticated like >> >> ALTER EXTENSION ... UPGRADE. And the extension knows how to upgrade >> itself. >> > > I’m not familiar

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thursday, July 15, 2021, Dave Cramer wrote: > > I'm thinking at this point we need something a bit more sophisticated like > > ALTER EXTENSION ... UPGRADE. And the extension knows how to upgrade itself. > I’m not familiar with what hoops extensions jump through to facilitate upgrades but even

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Dave Cramer
On Thu, 15 Jul 2021 at 11:15, David G. Johnston wrote: > On Thursday, July 15, 2021, David G. Johnston > wrote: > >> On Thursday, July 15, 2021, Dave Cramer wrote: >> >>> >>> Install any custom shared object files (or DLLs) used by the old >>> cluster >>> into the new cluster, e.g.,

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thursday, July 15, 2021, Dave Cramer wrote: > Well clearly my suggestion was not clear if you interpreted that as over > writing them with versions from an older version of PostgreSQL. > >> >> Ignoring my original interpretation as being moot; the section immediately preceding your edit says t

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thursday, July 15, 2021, David G. Johnston wrote: > On Thursday, July 15, 2021, Dave Cramer wrote: > >> >> Install any custom shared object files (or DLLs) used by the old >> cluster >> into the new cluster, e.g., pgcrypto.so, >> whether they are from contrib >> - or som

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Dave Cramer
On Thu, 15 Jul 2021 at 11:01, David G. Johnston wrote: > On Thursday, July 15, 2021, Dave Cramer wrote: > >> >> As a first step I propose the following >> >> diff --git a/doc/src/sgml/ref/pgupgrade.sgml >> b/doc/src/sgml/ref/pgupgrade.sgml >> index a83c63cd98..f747a4473a 100644 >> --- a/doc/src/

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread David G. Johnston
On Thursday, July 15, 2021, Dave Cramer wrote: > > As a first step I propose the following > > diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade. > sgml > index a83c63cd98..f747a4473a 100644 > --- a/doc/src/sgml/ref/pgupgrade.sgml > +++ b/doc/src/sgml/ref/pgupgrade.sgml >

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-15 Thread Dave Cramer
Dave Cramer On Wed, 14 Jul 2021 at 15:43, Tom Lane wrote: > Dave Cramer writes: > > On Wed, 14 Jul 2021 at 15:09, David G. Johnston < > david.g.johns...@gmail.com> > > wrote: > >> "Install ... files used by the old cluster" (which must be binary > >> compatible with the new cluster as noted el

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-14 Thread Tom Lane
Dave Cramer writes: > On Wed, 14 Jul 2021 at 15:09, David G. Johnston > wrote: >> "Install ... files used by the old cluster" (which must be binary >> compatible with the new cluster as noted elsewhere on that page) supports >> the claim that it is the old cluster's version that is going to resul

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-14 Thread David G. Johnston
On Wed, Jul 14, 2021 at 12:21 PM Dave Cramer wrote: > > On Wed, 14 Jul 2021 at 15:09, David G. Johnston < > david.g.johns...@gmail.com> wrote: > >> >> Something like, "... because the installed extensions will be copied from >> the old cluster during the upgrade." >> > > This is still rather opaq

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-14 Thread Dave Cramer
On Wed, 14 Jul 2021 at 15:09, David G. Johnston wrote: > On Wed, Jul 14, 2021 at 11:59 AM Dave Cramer wrote: > >> >> >> On Wed, 14 Jul 2021 at 14:47, David G. Johnston < >> david.g.johns...@gmail.com> wrote: >> >>> On Wednesday, July 14, 2021, Dave Cramer wrote: >>> Notice the up

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-14 Thread David G. Johnston
On Wed, Jul 14, 2021 at 11:59 AM Dave Cramer wrote: > > > On Wed, 14 Jul 2021 at 14:47, David G. Johnston < > david.g.johns...@gmail.com> wrote: > >> On Wednesday, July 14, 2021, Dave Cramer wrote: >> >>> >>> >>> Notice the upgraded version is 1.5 and the new version is 1.8 >>> >>> I would think

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-07-14 Thread Dave Cramer
On Wed, 14 Jul 2021 at 14:47, David G. Johnston wrote: > On Wednesday, July 14, 2021, Dave Cramer wrote: > >> >> >> Notice the upgraded version is 1.5 and the new version is 1.8 >> >> I would think somewhere in the upgrade of the schema there should have >> been a create extension pg_stat_statem

  1   2   >