Re: improve performance of pg_dump --binary-upgrade

2024-07-03 Thread Nathan Bossart
Committed. -- nathan

Re: improve performance of pg_dump --binary-upgrade

2024-05-17 Thread Nathan Bossart
rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From ded5e61ff631c2d02835fdba941068dcd86741ce Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Thu, 18 Apr 2024 15:15:19 -0500 Subject: [PATCH v5 1/2] Remove is_index parameter from binary_upgrade_set_pg_class_oids(). --

Re: improve performance of pg_dump --binary-upgrade

2024-04-22 Thread Nathan Bossart
I noticed that there are some existing examples of this sort of thing in pg_dump (e.g., commit d5e8930), so I adjusted the patch to match the surrounding style a bit better. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 80dd3233730422298ffe3b4523a7c58d7e9b55b9 Mon Sep 17 00:

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Nathan Bossart
On Thu, Apr 18, 2024 at 10:33:08PM +0200, Daniel Gustafsson wrote: > From a read-through they look good, a very nice performance improvement in an > important path. I think it would be nice with some comments on the > BinaryUpgradeClassOids struct (since the code using it is thousands of lines > a

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Daniel Gustafsson
> On 18 Apr 2024, at 22:28, Nathan Bossart wrote: > > On Thu, Apr 18, 2024 at 10:23:01AM -0500, Nathan Bossart wrote: >> On Thu, Apr 18, 2024 at 09:24:53AM +0200, Daniel Gustafsson wrote: >>> That does indeed seem like a saner approach. Since we look up the relkind >>> we >>> can also remove th

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Nathan Bossart
On Thu, Apr 18, 2024 at 10:23:01AM -0500, Nathan Bossart wrote: > On Thu, Apr 18, 2024 at 09:24:53AM +0200, Daniel Gustafsson wrote: >> That does indeed seem like a saner approach. Since we look up the relkind we >> can also remove the is_index parameter to binary_upgrade_set_pg_class_oids >> sinc

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Nathan Bossart
On Thu, Apr 18, 2024 at 09:24:53AM +0200, Daniel Gustafsson wrote: >> On 18 Apr 2024, at 06:17, Nathan Bossart wrote: > >> The attached work-in-progress patch speeds up 'pg_dump --binary-upgrade' >> for this case. Instead of executing the query in every call to the >> function, we can execute it

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Nathan Bossart
On Thu, Apr 18, 2024 at 02:08:28AM -0400, Corey Huinker wrote: > Bar-napkin math tells me in a worst-case architecture and braindead byte > alignment, we'd burn 64 bytes per struct, so the 100K tables cited would be > about 6.25MB of memory. That doesn't seem too terrible. > The obvious low-memor

Re: improve performance of pg_dump --binary-upgrade

2024-04-18 Thread Daniel Gustafsson
> On 18 Apr 2024, at 06:17, Nathan Bossart wrote: > The attached work-in-progress patch speeds up 'pg_dump --binary-upgrade' > for this case. Instead of executing the query in every call to the > function, we can execute it once during the first call and store all the > required information in a

Re: improve performance of pg_dump --binary-upgrade

2024-04-17 Thread Michael Paquier
On Thu, Apr 18, 2024 at 02:08:28AM -0400, Corey Huinker wrote: > Bar-napkin math tells me in a worst-case architecture and braindead byte > alignment, we'd burn 64 bytes per struct, so the 100K tables cited would be > about 6.25MB of memory. > > The obvious low-memory alternative would be to make

Re: improve performance of pg_dump --binary-upgrade

2024-04-17 Thread Corey Huinker
> > One downside of this approach is the memory usage. This was more-or-less > > Bar-napkin math tells me in a worst-case architecture and braindead byte alignment, we'd burn 64 bytes per struct, so the 100K tables cited would be about 6.25MB of memory. The obvious low-memory alternative would be