Re: add timing information to pg_upgrade

2023-08-24 Thread Nathan Bossart
On Wed, Aug 23, 2023 at 09:35:20AM -0700, Nathan Bossart wrote: > On Tue, Aug 22, 2023 at 07:06:23AM -0700, Nathan Bossart wrote: >> On Tue, Aug 22, 2023 at 11:49:33AM +0200, Peter Eisentraut wrote: >>> Let's change MESSAGE_WIDTH to 62 in v16, and then pursue the larger >>> restructuring leisurely.

Re: add timing information to pg_upgrade

2023-08-23 Thread Nathan Bossart
On Tue, Aug 22, 2023 at 07:06:23AM -0700, Nathan Bossart wrote: > On Tue, Aug 22, 2023 at 11:49:33AM +0200, Peter Eisentraut wrote: >> Let's change MESSAGE_WIDTH to 62 in v16, and then pursue the larger >> restructuring leisurely. > > Sounds good. I plan to commit this within the next couple of d

Re: add timing information to pg_upgrade

2023-08-22 Thread Nathan Bossart
On Tue, Aug 22, 2023 at 11:49:33AM +0200, Peter Eisentraut wrote: > Let's change MESSAGE_WIDTH to 62 in v16, and then pursue the larger > restructuring leisurely. Sounds good. I plan to commit this within the next couple of days. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: add timing information to pg_upgrade

2023-08-22 Thread Peter Eisentraut
On 01.08.23 17:45, Nathan Bossart wrote: On Tue, Aug 01, 2023 at 09:46:02AM +0200, Peter Eisentraut wrote: On 31.07.23 20:37, Nathan Bossart wrote: - prep_status("Checking for incompatible \"aclitem\" data type in user tables"); + prep_status("Checking for \"aclitem\" data type in

Re: add timing information to pg_upgrade

2023-08-02 Thread Nathan Bossart
On Wed, Aug 02, 2023 at 09:09:14AM -0700, Nathan Bossart wrote: > On Wed, Aug 02, 2023 at 01:02:53PM +0530, Bharath Rupireddy wrote: >> On Wed, Aug 2, 2023 at 12:45 PM Peter Eisentraut >> wrote: >>> I think we should change the output format to be more like initdb, like >>> >>> Doing somethi

Re: add timing information to pg_upgrade

2023-08-02 Thread Nathan Bossart
On Wed, Aug 02, 2023 at 01:02:53PM +0530, Bharath Rupireddy wrote: > On Wed, Aug 2, 2023 at 12:45 PM Peter Eisentraut wrote: >> I think we should change the output format to be more like initdb, like >> >> Doing something ... ok >> >> without horizontally aligning all the "ok"s. > > While th

Re: add timing information to pg_upgrade

2023-08-02 Thread Nathan Bossart
On Wed, Aug 02, 2023 at 09:14:06AM +0200, Peter Eisentraut wrote: > On 01.08.23 18:00, Nathan Bossart wrote: >> One of the main purposes of this thread is to gauge interest. I'm hoping >> there are other developers who are interested in reducing >> pg_upgrade-related downtime, and it seemed like i

Re: add timing information to pg_upgrade

2023-08-02 Thread Peter Eisentraut
On 02.08.23 10:30, Bharath Rupireddy wrote: Moreover, the ts command gives me the timestamps for each of the messages printed, so an extra step is required to calculate the time taken for an operation. There is "ts -i" for that.

Re: add timing information to pg_upgrade

2023-08-02 Thread Bharath Rupireddy
On Wed, Aug 2, 2023 at 12:44 PM Peter Eisentraut wrote: > > On 01.08.23 18:00, Nathan Bossart wrote: > > One of the main purposes of this thread is to gauge interest. I'm hoping > > there are other developers who are interested in reducing > > pg_upgrade-related downtime, and it seemed like it'd

Re: add timing information to pg_upgrade

2023-08-02 Thread Bharath Rupireddy
On Wed, Aug 2, 2023 at 12:45 PM Peter Eisentraut wrote: > > On 01.08.23 17:45, Nathan Bossart wrote: > > The message is too long, so there's no space between it and the "ok" > > message: > > > > Checking for incompatible "aclitem" data type in user tablesok > > > > Instead of altering the me

Re: add timing information to pg_upgrade

2023-08-02 Thread Peter Eisentraut
On 01.08.23 17:45, Nathan Bossart wrote: The message is too long, so there's no space between it and the "ok" message: Checking for incompatible "aclitem" data type in user tablesok Instead of altering the messages, we could bump MESSAGE_WIDTH from 60 to 62 or 64. Do you prefer that ap

Re: add timing information to pg_upgrade

2023-08-02 Thread Peter Eisentraut
On 01.08.23 18:00, Nathan Bossart wrote: One of the main purposes of this thread is to gauge interest. I'm hoping there are other developers who are interested in reducing pg_upgrade-related downtime, and it seemed like it'd be nice to have built-in functionality for measuring the step times ins

Re: add timing information to pg_upgrade

2023-08-01 Thread Jacob Champion
On Tue, Aug 1, 2023 at 9:00 AM Nathan Bossart wrote: > >> On 1 Aug 2023, at 09:45, Peter Eisentraut wrote: > >> But who would use that, other than, you know, you, right now? /me raises hand Or at least, me back when I was hacking on pg_upgrade performance. This, or something like it, would have

Re: add timing information to pg_upgrade

2023-08-01 Thread Nathan Bossart
On Tue, Aug 01, 2023 at 09:58:24AM +0200, Daniel Gustafsson wrote: >> On 1 Aug 2023, at 09:45, Peter Eisentraut wrote: >> On 28.07.23 01:51, Nathan Bossart wrote: > >>> This information can be used to better understand where the time is going >>> and to validate future improvements. >> >> But wh

Re: add timing information to pg_upgrade

2023-08-01 Thread Nathan Bossart
On Tue, Aug 01, 2023 at 09:46:02AM +0200, Peter Eisentraut wrote: > On 31.07.23 20:37, Nathan Bossart wrote: >> -prep_status("Checking for incompatible \"aclitem\" data type in user >> tables"); >> +prep_status("Checking for \"aclitem\" data type in user tables"); > > Why these changes?

Re: add timing information to pg_upgrade

2023-08-01 Thread Daniel Gustafsson
> On 1 Aug 2023, at 09:45, Peter Eisentraut wrote: > On 28.07.23 01:51, Nathan Bossart wrote: >> This information can be used to better understand where the time is going >> and to validate future improvements. > > But who would use that, other than, you know, you, right now? > > I think the pg

Re: add timing information to pg_upgrade

2023-08-01 Thread Peter Eisentraut
On 31.07.23 20:37, Nathan Bossart wrote: - prep_status("Checking for incompatible \"aclitem\" data type in user tables"); + prep_status("Checking for \"aclitem\" data type in user tables"); Why these changes? I think this is losing precision about what it's doing.

Re: add timing information to pg_upgrade

2023-08-01 Thread Peter Eisentraut
On 28.07.23 01:51, Nathan Bossart wrote: I've been looking into some options for reducing the amount of downtime required for pg_upgrade, and $SUBJECT seemed like something that would be worthwhile independent of that effort. The attached work-in-progress patch adds the elapsed time spent in eac

Re: add timing information to pg_upgrade

2023-07-31 Thread Nathan Bossart
On Mon, Jul 31, 2023 at 11:34:57AM +0530, Bharath Rupireddy wrote: > Either of "Checking for \"aclitem\" data type usage" or "Checking for > \"aclitem\" data type in user tables" seems okay to me, however, I > prefer the second wording. Okay. I used the second wording for all the data type check

Re: add timing information to pg_upgrade

2023-07-30 Thread Bharath Rupireddy
On Sun, Jul 30, 2023 at 2:44 AM Nathan Bossart wrote: > > On Sat, Jul 29, 2023 at 12:17:40PM +0530, Bharath Rupireddy wrote: > > While on this, I noticed a thing unrelated to your patch that there's > > no space between the longest status message of size 60 bytes and ok - > > 'Checking for incompa

Re: add timing information to pg_upgrade

2023-07-29 Thread Nathan Bossart
On Sat, Jul 29, 2023 at 12:17:40PM +0530, Bharath Rupireddy wrote: > While on this, I noticed a thing unrelated to your patch that there's > no space between the longest status message of size 60 bytes and ok - > 'Checking for incompatible "aclitem" data type in user tablesok > 23.932 ms'. I think

Re: add timing information to pg_upgrade

2023-07-28 Thread Bharath Rupireddy
On Sat, Jul 29, 2023 at 12:17 AM Nathan Bossart wrote: > > On Fri, Jul 28, 2023 at 10:38:14AM -0700, Nathan Bossart wrote: > > I'm okay with simply adding the time. However, I wonder if we want to > > switch to seconds, minutes, hours, etc. if the step takes longer. This > > would add a bit of c

Re: add timing information to pg_upgrade

2023-07-28 Thread Nathan Bossart
On Fri, Jul 28, 2023 at 10:38:14AM -0700, Nathan Bossart wrote: > I'm okay with simply adding the time. However, I wonder if we want to > switch to seconds, minutes, hours, etc. if the step takes longer. This > would add a bit of complexity, but it would improve human readability. > Alternatively

Re: add timing information to pg_upgrade

2023-07-28 Thread Nathan Bossart
On Fri, Jul 28, 2023 at 01:10:14PM +0530, Bharath Rupireddy wrote: > How about using INSTR_TIME_SET_CURRENT, INSTR_TIME_SUBTRACT and > INSTR_TIME_GET_MILLISEC macros instead of gettimeofday and explicit > calculations? That seems like a good idea. > +report_status(PG_REPORT, "ok (took %ld ms)

Re: add timing information to pg_upgrade

2023-07-28 Thread Bharath Rupireddy
On Fri, Jul 28, 2023 at 5:21 AM Nathan Bossart wrote: > > This information can be used to better understand where the time is going > and to validate future improvements. I'm open to suggestions on formatting > the timing information, assuming folks are interested in this idea. > > Thoughts? +1