Re: Experience and feedback on pg_restore --data-only

2025-04-05 Thread Adrian Klaver
On 3/24/25 08:51, Dimitrios Apostolou wrote: On Mon, 24 Mar 2025, Adrian Klaver wrote: On 3/24/25 07:24, Dimitrios Apostolou wrote:  On Sun, 23 Mar 2025, Laurenz Albe wrote:  On Thu, 2025-03-20 at 23:48 +0100, Dimitrios Apostolou wrote:  Performance issues: (important as my db size is >5TB)

Re: Experience and feedback on pg_restore --data-only

2025-04-05 Thread shammat
Dimitrios Apostolou schrieb am 20.03.2025 um 23:48: > Things that made my life hard: > > * plenty of permission denials for both ALTER OWNER or SET SESSION >   AUTHORIZATION (depending on command line switches).  Both of these >   require superuser privilege, but in my case this is not really neede

Re: Experience and feedback on pg_restore --data-only

2025-04-05 Thread Adrian Klaver
On 3/24/25 07:24, Dimitrios Apostolou wrote: On Sun, 23 Mar 2025, Laurenz Albe wrote: On Thu, 2025-03-20 at 23:48 +0100, Dimitrios Apostolou wrote: Performance issues: (important as my db size is >5TB) * WAL writes: I didn't manage to avoid writing to the WAL, despite having    setting wal_

Re: Experience and feedback on pg_restore --data-only

2025-04-05 Thread Adrian Klaver
On 3/20/25 15:48, Dimitrios Apostolou wrote: Rationale: When restoring a backup in an emergency situation, it's fine to run pg_restore as superuser and get an exact replica of the dumped db. AFAICT pg_restore (without --data-only) is optimised for such case. But pg_dump/restore can be used as a

Re: Experience and feedback on pg_restore --data-only

2025-04-05 Thread Dimitrios Apostolou
Hi Laurenz, On Sun, 23 Mar 2025, Laurenz Albe wrote: On Thu, 2025-03-20 at 23:48 +0100, Dimitrios Apostolou wrote: Performance issues: (important as my db size is >5TB) * WAL writes: I didn't manage to avoid writing to the WAL, despite having    setting wal_level=minimal. I even wrote my own

Re: Experience and feedback on pg_restore --data-only

2025-03-25 Thread Adrian Klaver
On 3/24/25 09:15, Dimitrios Apostolou wrote: Hi Ron, I read your reply in the mailing list archives as I'm not subscribed to the list, and I'm copy-pasting a response here. Please include me as a recipient in further replies. Why are you regularly having emergencies requiring the restoration o

Re: Experience and feedback on pg_restore --data-only

2025-03-24 Thread Laurenz Albe
On Mon, 2025-03-24 at 16:51 +0100, Dimitrios Apostolou wrote: > Laurenz informed me that I could avoid writing to the WAL if I "create and > load the table in a single transaction". > I haven't tried, but here is what I would do to try --single-transaction: > > Transaction 1: manually issuing all

Re: Experience and feedback on pg_restore --data-only

2025-03-24 Thread Laurenz Albe
On Mon, 2025-03-24 at 15:24 +0100, Dimitrios Apostolou wrote: > By the way do you see potential problems with using --single-transaction > to restore billion-rows tables? No. Yours, Laurenz Albe

Re: Experience and feedback on pg_restore --data-only

2025-03-24 Thread Dimitrios Apostolou
Hi Ron, I read your reply in the mailing list archives as I'm not subscribed to the list, and I'm copy-pasting a response here. Please include me as a recipient in further replies. Why are you regularly having emergencies requiring the restoration of multi-TB tables to databases with lots of cr

Re: Experience and feedback on pg_restore --data-only

2025-03-24 Thread Dimitrios Apostolou
On Sun, 23 Mar 2025, Adrian Klaver wrote: On 3/20/25 15:48, Dimitrios Apostolou wrote: * plenty of permission denials for both ALTER OWNER or SET SESSION   AUTHORIZATION (depending on command line switches).  Both of these   require superuser privilege, but in my case this is not really n

Re: Experience and feedback on pg_restore --data-only

2025-03-24 Thread Ron Johnson
Why are you regularly having emergencies requiring the restoration of multi-TB tables to databases with lots of cruft? Fixing that would go a long way towards eliminating your problems with pg_restore. On Mon, Mar 24, 2025 at 11:51 AM Dimitrios Apostolou wrote: > On Mon, 24 Mar 2025, Adrian Kla

Re: Experience and feedback on pg_restore --data-only

2025-03-24 Thread Dimitrios Apostolou
On Mon, 24 Mar 2025, Adrian Klaver wrote: On 3/24/25 07:24, Dimitrios Apostolou wrote: On Sun, 23 Mar 2025, Laurenz Albe wrote: On Thu, 2025-03-20 at 23:48 +0100, Dimitrios Apostolou wrote: Performance issues: (important as my db size is >5TB) * WAL writes: I didn't manage to avoid writ

Re: Experience and feedback on pg_restore --data-only

2025-03-24 Thread Dimitrios Apostolou
On Sun, 23 Mar 2025, Laurenz Albe wrote: On Thu, 2025-03-20 at 23:48 +0100, Dimitrios Apostolou wrote: Performance issues: (important as my db size is >5TB) * WAL writes: I didn't manage to avoid writing to the WAL, despite having    setting wal_level=minimal. I even wrote my own function to A

Re: Experience and feedback on pg_restore --data-only

2025-03-24 Thread Laurenz Albe
On Thu, 2025-03-20 at 23:48 +0100, Dimitrios Apostolou wrote: > Performance issues: (important as my db size is >5TB) > > * WAL writes: I didn't manage to avoid writing to the WAL, despite having >    setting wal_level=minimal. I even wrote my own function to ALTER all >    tables to UNLOGGED, but

Re: Experience and feedback on pg_restore --data-only

2025-03-21 Thread Dimitrios Apostolou
On Thu, 20 Mar 2025, Dimitrios Apostolou wrote: Rationale: When restoring a backup in an emergency situation, it's fine to run pg_restore as superuser and get an exact replica of the dumped db. AFAICT pg_restore (without --data-only) is optimised for such case. But pg_dump/restore can be used

Re: Experience and feedback on pg_restore --data-only

2025-03-21 Thread Ron Johnson
On Fri, Mar 21, 2025 at 2:36 PM Dimitrios Apostolou wrote: > On Thu, 20 Mar 2025, Dimitrios Apostolou wrote: > > > Rationale: > > > > When restoring a backup in an emergency situation, it's fine to run > > pg_restore as superuser and get an exact replica of the dumped db. > How often do you have