Re: pg_restore causing ENOSPACE on the WAL partition

2025-04-11 Thread Dimitrios Apostolou
Happened again even with max_wal_size=32GB on the 128GB WAL partition. At this point I'm quite sure what happens: + a checkpoint is going on for several minutes (because of I/O contention on the archival drive) + meanwhile data keeps coming in through several processes of pg_restore + data is

Re: pg_restore causing ENOSPACE on the WAL partition

2025-04-10 Thread Dimitrios Apostolou
Forgot an important setting I have during pg_restore: On Thu, 10 Apr 2025, Dimitrios Apostolou wrote: max_wal_size=64GB max_replication_slots = 0 max_logical_replication_workers = 0 max_wal_senders = 0 wal_level = minimal autovacuum = off checkpoint_completion_target = 0 so the checkpoint sh

Re: pg_restore enhancements

2023-11-23 Thread Ron Johnson
Thanks for the explanation. On Thu, Nov 23, 2023 at 10:55 AM Tom Lane wrote: > Ron Johnson writes: > > On Thu, Nov 23, 2023 at 3:37 AM Laurenz Albe > > wrote: > >> You can avoidwriting WAL if you set "wal_level = minimal", restart > >> PostgreSQL > >> and restore the dump with the --single-tra

Re: pg_restore enhancements

2023-11-23 Thread Tom Lane
Ron Johnson writes: > On Thu, Nov 23, 2023 at 3:37 AM Laurenz Albe > wrote: >> You can avoidwriting WAL if you set "wal_level = minimal", restart >> PostgreSQL >> and restore the dump with the --single-transaction option. > Why does "--single-transaction" prevent WAL writes? I'd expect _more_ >

Re: pg_restore enhancements

2023-11-23 Thread Ron Johnson
On Thu, Nov 23, 2023 at 3:37 AM Laurenz Albe wrote: [snip] > > You can avoidwriting WAL if you set "wal_level = minimal", restart > PostgreSQL > and restore the dump with the --single-transaction option. > Why does "--single-transaction" prevent WAL writes? I'd expect _more_ pg_wal growth from

Re: pg_restore enhancements

2023-11-23 Thread Laurenz Albe
On Wed, 2023-11-22 at 16:55 +, Efrain J. Berdecia wrote: > Thanks, the issue we've run into, which I guess could be really a setup > issue, with running a COPY command while executing pg_restore,  > is that if we are restoring a large table (bigger than 500GB) our WAL > directory can grow to b

Re: pg_restore enhancements

2023-11-22 Thread Ron Johnson
On Wed, Nov 22, 2023 at 2:28 PM Tom Lane wrote: > "Efrain J. Berdecia" writes: > > Thanks, the issue we've run into, which I guess could be really a setup > issue, with running a COPY command while executing pg_restore, is that if > we are restoring a large table (bigger than 500GB) our WAL dire

Re: pg_restore enhancements

2023-11-22 Thread Tom Lane
"Efrain J. Berdecia" writes: > Thanks, the issue we've run into, which I guess could be really a setup > issue, with running a COPY command while executing pg_restore, is that if we > are restoring a large table (bigger than 500GB) our WAL directory can grow to > be very large. > I would think

Re: pg_restore enhancements

2023-11-22 Thread Efrain J. Berdecia
Thanks, the issue we've run into, which I guess could be really a setup issue, with running a COPY command while executing pg_restore, is that if we are restoring a large table (bigger than 500GB) our WAL directory can grow to be very large. I would think that if the pg_restore or COPY command w

Re: pg_restore enhancements

2023-11-22 Thread Adrian Klaver
On 11/22/23 05:25, Efrain J. Berdecia wrote: After working for a site where we are constantly doing logical pg_dump to refresh environments I've come to miss features available in other RDBMS' refresh/restore utilities. Someone could point me in the right direction otherwise, but pg_restore s

Re: pg_restore enhancements

2023-11-22 Thread Efrain J. Berdecia
Thanks, I'm trying to gage the interest on such a feature enhancement.  Up to now I have not actively contributed to the Postgres Project but this is itching my rusty programming fingers lol Thanks,Efrain J. Berdecia On Wednesday, November 22, 2023 at 08:28:18 AM EST, David G. Johnston wro

Re: pg_restore enhancements

2023-11-22 Thread David G. Johnston
On Wednesday, November 22, 2023, Efrain J. Berdecia wrote: > > Thanks in advance for any suggestions or the green light to post this to > the PG-developer group :-) > If you aren’t offering up a patch for these it isn’t developer material and belongs right here. David J.

Re: pg_restore unexpected end of file

2023-09-08 Thread Adrian Klaver
On 9/7/23 23:29, Les wrote: I'm trying to migrate a database from version 11 to version 15. I have created a dump file on v11: pg_dump --dbname=not_telling -Fc --no-owner > dump --no-owner is ignored for non-text outputs. It is handled on the pg_restore end. Then I was trying to import i

Re: pg_restore mostly idle on restoring a large number of tables

2023-07-15 Thread Tom Lane
Boris Sagadin writes: > restoring a 1.5TB database with about 800k tables on i3.4xlarge AWS > instace, PgSQL V12.15 on Ubuntu. > Running pg_restore with -j 16, I noticed the pg_restore is busy for an hour > or so with IO at 80%+ and then most of processes start idling and only a > few doing some

Re: pg_restore mostly idle on restoring a large number of tables

2023-07-15 Thread Ron
On 7/13/23 02:41, Boris Sagadin wrote: Hi, restoring a 1.5TB database with about 800k tables on i3.4xlarge AWS instace, PgSQL V12.15 on Ubuntu. Running pg_restore with -j 16, I noticed the pg_restore is busy for an hour or so with IO at 80%+ and then most of processes start idling and only

Re: pg_restore remap schema

2022-11-17 Thread Fabrice Chapuis
Postgres allows us to rename a schema with the command *alter schema schema_orig rename to schema_dest*. however the definition of functions belonging to the original schema are not modified. CREATE OR REPLACE FUNCTION foo.fcount() RETURNS integer AS $$ declare v_count integer; BEGIN

Re: pg_restore remap schema

2022-11-16 Thread Thomas Kellerer
Tom Lane schrieb am 08.08.2022 um 20:22: > Guillaume Lelarge writes: >> Le lun. 8 août 2022 à 18:28, Fabrice Chapuis a >>> Is a development in progress to add this option > >> Nope, never heard of someone working on this. > > People have asked for such a thing before, but it'd be quite difficult

Re: pg_restore remap schema

2022-11-16 Thread Tom Lane
=?UTF-8?Q?St=C3=A9phane_Tachoires?= writes: > And could it be at pg_dump level ? > It seems more aware of what is doing... pg_dump would be in a slightly better position, but only slightly. It still has no idea about the contents of function bodies. Also, if it's acceptable to do this at dump ti

Re: pg_restore remap schema

2022-11-16 Thread Stéphane Tachoires
Hi And could it be at pg_dump level ? It seems more aware of what is doing... Stephane Le mer. 16 nov. 2022 à 15:42, Tom Lane a écrit : > Guillaume Lelarge writes: > > Le mer. 16 nov. 2022 à 13:08, Fabrice Chapuis > a > > écrit : > >> I worked on the pg_dump source code to add remap schema fu

Re: pg_restore remap schema

2022-11-16 Thread Tom Lane
Guillaume Lelarge writes: > Le mer. 16 nov. 2022 à 13:08, Fabrice Chapuis a > écrit : >> I worked on the pg_dump source code to add remap schema functionality to >> use it internally where I work. This is a first version that allows to >> remap tables, views and sequences (only to export schema).

Re: pg_restore remap schema

2022-11-16 Thread Guillaume Lelarge
Hi, Le mer. 16 nov. 2022 à 13:08, Fabrice Chapuis a écrit : > Hi, > I worked on the pg_dump source code to add remap schema functionality to > use it internally where I work. This is a first version that allows to > remap tables, views and sequences (only to export schema). Is this > development

Re: pg_restore remap schema

2022-11-16 Thread Fabrice Chapuis
Hi, I worked on the pg_dump source code to add remap schema functionality to use it internally where I work. This is a first version that allows to remap tables, views and sequences (only to export schema). Is this development likely to interest the PG community and to continue this development fur

Re: pg_restore error on function

2022-11-03 Thread Ron
On 11/3/22 09:57, Adrian Klaver wrote: On 11/3/22 07:45, Ron wrote: On 11/3/22 09:28, Post Gresql wrote: Hello I first successfully ran pg_dump -U postgres -b -E UTF8 -f my_dump --format=custom -n my_schema --no-owner -v my_db What was the *complete* pg_dump command? I'm going to say

Re: pg_restore error on function

2022-11-03 Thread Tom Lane
"David G. Johnston" writes: > On Thu, Nov 3, 2022 at 8:39 AM Post Gresql wrote: >> Could it be that the >> create function . >> in the dump file does not implicitly create the schema as a >> create table . >> would? > I don't know where you got the idea that a schema is implicitly created via >

Re: pg_restore error on function

2022-11-03 Thread Adrian Klaver
On 11/3/22 08:38, Post Gresql wrote: On 2022-11-03 15:43, Adrian Klaver wrote: On 11/3/22 07:28, Post Gresql wrote: Hello I first successfully ran pg_dump -U postgres -b -E UTF8 -f my_dump --format=custom -n my_schema --no-owner -v my_db but then pg_restore --single-transaction -v -U pos

Re: pg_restore error on function

2022-11-03 Thread David G. Johnston
On Thu, Nov 3, 2022 at 8:39 AM Post Gresql wrote: > > On 2022-11-03 15:43, Adrian Klaver wrote: > > On 11/3/22 07:28, Post Gresql wrote: > >> Hello > >> > >> I first successfully ran > >> > >> pg_dump -U postgres -b -E UTF8 -f my_dump --format=custom -n my_schema > --no-owner -v my_db > >> > >> b

Re: pg_restore error on function

2022-11-03 Thread Post Gresql
On 2022-11-03 15:43, Adrian Klaver wrote: On 11/3/22 07:28, Post Gresql wrote: Hello I first successfully ran pg_dump -U postgres -b -E UTF8 -f my_dump --format=custom -n my_schema --no-owner -v my_db but then pg_restore --single-transaction -v -U postgres -O -e -d my_other_db my_dump fa

Re: pg_restore error on function

2022-11-03 Thread Post Gresql
On 2022-11-03 15:57, Adrian Klaver wrote: On 11/3/22 07:45, Ron wrote: On 11/3/22 09:28, Post Gresql wrote: Hello I first successfully ran pg_dump -U postgres -b -E UTF8 -f my_dump --format=custom -n my_schema --no-owner -v my_db What was the *complete* pg_dump command? I'm going to

Re: pg_restore error on function

2022-11-03 Thread Adrian Klaver
On 11/3/22 07:45, Ron wrote: On 11/3/22 09:28, Post Gresql wrote: Hello I first successfully ran pg_dump -U postgres -b -E UTF8 -f my_dump --format=custom -n my_schema --no-owner -v my_db What was the *complete* pg_dump command? I'm going to say the above. -- Adrian Klaver adrian.kl

Re: pg_restore error on function

2022-11-03 Thread Ron
On 11/3/22 09:28, Post Gresql wrote: Hello I first successfully ran pg_dump -U postgres -b -E UTF8 -f my_dump --format=custom -n my_schema --no-owner -v my_db but then pg_restore --single-transaction -v -U postgres -O -e -d my_other_db my_dump failed with pg_restore: connecting to databas

Re: pg_restore error on function

2022-11-03 Thread Adrian Klaver
On 11/3/22 07:28, Post Gresql wrote: Hello I first successfully ran pg_dump -U postgres -b -E UTF8 -f my_dump --format=custom -n my_schema --no-owner -v my_db but then pg_restore --single-transaction -v -U postgres -O -e -d my_other_db my_dump failed with pg_restore: connecting to databas

Re: pg_restore 12 "permission denied for schema" errors

2022-10-22 Thread Tom Lane
Ron writes: > On 10/20/22 10:02, Adrian Klaver wrote: >> 3) What if you run without --jobs? > It runs without error.  Add "--jobs=2" and the errors appear. That's ... suggestive ... but not suggestive enough. Can you create a self-contained test case? It probably doesn't depend much at all on

Re: pg_restore 12 "permission denied for schema" errors

2022-10-22 Thread Ron
On 10/22/22 17:06, Adrian Klaver wrote: On 10/22/22 14:45, Ron wrote: On 10/22/22 16:29, Adrian Klaver wrote: To pseudo for me. What file exactly is: pg_restore --jobs=X --no-owner $NEWDB restoring? And how was that file created? Knowing this might help get at why the more straight forwa

Re: pg_restore 12 "permission denied for schema" errors

2022-10-22 Thread Adrian Klaver
On 10/22/22 14:45, Ron wrote: On 10/22/22 16:29, Adrian Klaver wrote: To pseudo for me. What file exactly is: pg_restore --jobs=X --no-owner $NEWDB restoring? And how was that file created? Knowing this might help get at why the more straight forward method does not work. This is what

Re: pg_restore 12 "permission denied for schema" errors

2022-10-22 Thread Ron
On 10/22/22 16:29, Adrian Klaver wrote: On 10/22/22 14:02, Ron wrote: On 10/22/22 12:00, Adrian Klaver wrote: On 10/22/22 09:41, Ron wrote: On 10/22/22 11:20, Adrian Klaver wrote: On 10/20/22 14:34, Ron wrote: On 10/20/22 10:02, Adrian Klaver wrote: On 10/20/22 06:20, Ron wrote: On 10/20/2

Re: pg_restore 12 "permission denied for schema" errors

2022-10-22 Thread Adrian Klaver
On 10/22/22 14:02, Ron wrote: On 10/22/22 12:00, Adrian Klaver wrote: On 10/22/22 09:41, Ron wrote: On 10/22/22 11:20, Adrian Klaver wrote: On 10/20/22 14:34, Ron wrote: On 10/20/22 10:02, Adrian Klaver wrote: On 10/20/22 06:20, Ron wrote: On 10/20/22 00:12, Tom Lane wrote: I was afrai

Re: pg_restore 12 "permission denied for schema" errors

2022-10-22 Thread Ron
On 10/22/22 12:00, Adrian Klaver wrote: On 10/22/22 09:41, Ron wrote: On 10/22/22 11:20, Adrian Klaver wrote: On 10/20/22 14:34, Ron wrote: On 10/20/22 10:02, Adrian Klaver wrote: On 10/20/22 06:20, Ron wrote: On 10/20/22 00:12, Tom Lane wrote: I was afraid you were going to say that.

Re: pg_restore 12 "permission denied for schema" errors

2022-10-22 Thread Adrian Klaver
On 10/22/22 09:41, Ron wrote: On 10/22/22 11:20, Adrian Klaver wrote: On 10/20/22 14:34, Ron wrote: On 10/20/22 10:02, Adrian Klaver wrote: On 10/20/22 06:20, Ron wrote: On 10/20/22 00:12, Tom Lane wrote: I was afraid you were going to say that. The work-around is to: pg_dump $SRCDB --s

Re: pg_restore 12 "permission denied for schema" errors

2022-10-22 Thread Ron
On 10/22/22 11:20, Adrian Klaver wrote: On 10/20/22 14:34, Ron wrote: On 10/20/22 10:02, Adrian Klaver wrote: On 10/20/22 06:20, Ron wrote: On 10/20/22 00:12, Tom Lane wrote: I ran "pg_dumpall --globals-only --no-role-passwords" on the source instance, and applied it to the new instance bef

Re: pg_restore 12 "permission denied for schema" errors

2022-10-22 Thread Adrian Klaver
On 10/20/22 14:34, Ron wrote: On 10/20/22 10:02, Adrian Klaver wrote: On 10/20/22 06:20, Ron wrote: On 10/20/22 00:12, Tom Lane wrote: I ran "pg_dumpall --globals-only --no-role-passwords" on the source instance, and applied it to the new instance before doing the pg_restore. If I hadn't do

Re: pg_restore 12 "permission denied for schema" errors

2022-10-20 Thread Ron
On 10/20/22 10:02, Adrian Klaver wrote: On 10/20/22 06:20, Ron wrote: On 10/20/22 00:12, Tom Lane wrote: Frank Gard writes: Am 20.10.22 um 02:58 schrieb Ron: pg_restore -v --create --clean --jobs=`nproc` -Fd --dbname=template1 $DB What database do you want to restore your data into? As far a

Re: pg_restore 12 "permission denied for schema" errors

2022-10-20 Thread Ron
On 10/20/22 10:02, Adrian Klaver wrote: On 10/20/22 06:20, Ron wrote: On 10/20/22 00:12, Tom Lane wrote: Frank Gard writes: Am 20.10.22 um 02:58 schrieb Ron: pg_restore -v --create --clean --jobs=`nproc` -Fd --dbname=template1 $DB What database do you want to restore your data into? As far a

Re: pg_restore 12 "permission denied for schema" errors

2022-10-20 Thread Adrian Klaver
On 10/20/22 06:20, Ron wrote: On 10/20/22 00:12, Tom Lane wrote: Frank Gard writes: Am 20.10.22 um 02:58 schrieb Ron: pg_restore -v --create --clean --jobs=`nproc` -Fd --dbname=template1 $DB What database do you want to restore your data into? As far as I know your pg_restore command would i

Re: pg_restore 12 "permission denied for schema" errors

2022-10-20 Thread Ron
On 10/20/22 00:12, Tom Lane wrote: Frank Gard writes: Am 20.10.22 um 02:58 schrieb Ron: pg_restore -v --create --clean --jobs=`nproc` -Fd --dbname=template1 $DB What database do you want to restore your data into? As far as I know your pg_restore command would import the data into template1,

Re: pg_restore 12 "permission denied for schema" errors

2022-10-20 Thread Frank Gard
Hi Tom, Am 20.10.22 um 07:12 schrieb Tom Lane: Frank Gard writes: Am 20.10.22 um 02:58 schrieb Ron: pg_restore -v --create --clean --jobs=`nproc` -Fd --dbname=template1 $DB What database do you want to restore your data into? As far as I know your pg_restore command would import the data in

Re: pg_restore 12 "permission denied for schema" errors

2022-10-19 Thread Tom Lane
Frank Gard writes: > Am 20.10.22 um 02:58 schrieb Ron: >> pg_restore -v --create --clean --jobs=`nproc` -Fd --dbname=template1 $DB > What database do you want to restore your data into? As far as I know your > pg_restore command would import the data into template1, right? No --- my ears went u

Re: pg_restore 12 "permission denied for schema" errors

2022-10-19 Thread Frank Gard
Hi Ron, Am 20.10.22 um 02:58 schrieb Ron: RDS Postgresql 12.11 Backed up a database from one RDS instance, and now am trying to restore it to a new instance.  (If you're wondering why, it's to restore in an instance with less disk space.) Here are the commands: export PGHOST=${RDSENV}.x

Re: pg_restore creates public schema?

2022-10-08 Thread Adrian Klaver
On 10/7/22 20:14, Ron wrote: On 10/7/22 15:31, Alban Hertroys wrote: Can you create an SSH tunnel to the new machine from the VM, then pipe that to an SSH connection from a machine that does have enough space to dump? Interesting.  (It's above my ssh expertise, though.) An example using p

Re: pg_restore creates public schema?

2022-10-07 Thread Ron
On 10/7/22 15:31, Alban Hertroys wrote: On 6 Oct 2022, at 22:52, Ron wrote: On 10/6/22 12:46, Christophe Pettus wrote: On Oct 6, 2022, at 10:44, Ron wrote: Sadly, that VM doesn't have nearly enough disk space to hold the backup folder. Use file mode, and stream the output via scp/ssh to a di

Re: pg_restore creates public schema?

2022-10-07 Thread Adrian Klaver
On 10/7/22 10:11, Ron wrote: And the RHEL8 server can't talk to the RHEL6 server? Give me /some/ credit for thinking of that first... Also give me credit for not assuming what you have done and instead confirming it. I have been in involved in or followed multiple threads on this list wher

Re: pg_restore creates public schema?

2022-10-07 Thread Alban Hertroys
> On 6 Oct 2022, at 22:52, Ron wrote: > > On 10/6/22 12:46, Christophe Pettus wrote: >>> On Oct 6, 2022, at 10:44, Ron wrote: >>> Sadly, that VM doesn't have nearly enough disk space to hold the backup >>> folder. >> Use file mode, and stream the output via scp/ssh to a different machine? >

Re: pg_restore creates public schema?

2022-10-07 Thread Ron
On 10/6/22 17:01, Adrian Klaver wrote: On 10/6/22 2:03 PM, Ron wrote: On 10/6/22 14:35, Adrian Klaver wrote: On 10/6/22 09:46, Ron wrote: On 10/6/22 10:20, Tom Lane wrote: Because installing new software on production servers requires hurdles (Service Now change ticket approved by the app

Re: pg_restore creates public schema?

2022-10-06 Thread Adrian Klaver
On 10/6/22 1:54 PM, Ron wrote: On 10/6/22 14:32, Adrian Klaver wrote: On 10/6/22 10:46, Christophe Pettus wrote: On Oct 6, 2022, at 10:44, Ron wrote: Sadly, that VM doesn't have nearly enough disk space to hold the backup folder. Use file mode, and stream the output via scp/ssh to a diff

Re: pg_restore creates public schema?

2022-10-06 Thread Adrian Klaver
On 10/6/22 2:03 PM, Ron wrote: On 10/6/22 14:35, Adrian Klaver wrote: On 10/6/22 09:46, Ron wrote: On 10/6/22 10:20, Tom Lane wrote: Because installing new software on production servers requires hurdles (Service Now change ticket approved by the application support manager, Delivery Serv

Re: pg_restore creates public schema?

2022-10-06 Thread Ron
On 10/6/22 14:35, Adrian Klaver wrote: On 10/6/22 09:46, Ron wrote: On 10/6/22 10:20, Tom Lane wrote: Ron writes: On 10/6/22 09:49, Tom Lane wrote: Ron writes: pg_dump 9.6.24 You realize that that version's been out of support for a year? Yes, which is why I'm dumping from an EOL cluster,

Re: pg_restore creates public schema?

2022-10-06 Thread Ron
On 10/6/22 14:32, Adrian Klaver wrote: On 10/6/22 10:46, Christophe Pettus wrote: On Oct 6, 2022, at 10:44, Ron wrote: Sadly, that VM doesn't have nearly enough disk space to hold the backup folder. Use file mode, and stream the output via scp/ssh to a different machine? Or Plan B: 1)

Re: pg_restore creates public schema?

2022-10-06 Thread Ron
On 10/6/22 12:46, Christophe Pettus wrote: On Oct 6, 2022, at 10:44, Ron wrote: Sadly, that VM doesn't have nearly enough disk space to hold the backup folder. Use file mode, and stream the output via scp/ssh to a different machine? I thought of that, too.  Unfortunately, the ssh version in R

Re: pg_restore creates public schema?

2022-10-06 Thread Adrian Klaver
On 10/6/22 09:46, Ron wrote: On 10/6/22 10:20, Tom Lane wrote: Ron writes: On 10/6/22 09:49, Tom Lane wrote: Ron writes: pg_dump 9.6.24 You realize that that version's been out of support for a year? Yes, which is why I'm dumping from an EOL cluster, and restoring to a supported version.

Re: pg_restore creates public schema?

2022-10-06 Thread Adrian Klaver
On 10/6/22 10:46, Christophe Pettus wrote: On Oct 6, 2022, at 10:44, Ron wrote: Sadly, that VM doesn't have nearly enough disk space to hold the backup folder. Use file mode, and stream the output via scp/ssh to a different machine? Or Plan B: 1) Use pg_dump 9.6.24 on existing(going EOL

Re: pg_restore creates public schema?

2022-10-06 Thread Christophe Pettus
> On Oct 6, 2022, at 10:44, Ron wrote: > Sadly, that VM doesn't have nearly enough disk space to hold the backup > folder. Use file mode, and stream the output via scp/ssh to a different machine?

Re: pg_restore creates public schema?

2022-10-06 Thread Ron
On 10/6/22 11:48, Christophe Pettus wrote: On Oct 6, 2022, at 09:46, Ron wrote: Because installing new software on production servers requires hurdles (Service Now change ticket approved by the application support manager, Delivery Service Manager, Engineering Change Board, and a one week lead

Re: pg_restore creates public schema?

2022-10-06 Thread Christophe Pettus
> On Oct 6, 2022, at 09:46, Ron wrote: > Because installing new software on production servers requires hurdles > (Service Now change ticket approved by the application support manager, > Delivery Service Manager, Engineering Change Board, and a one week lead time > before installing during

Re: pg_restore creates public schema?

2022-10-06 Thread Ron
On 10/6/22 10:20, Tom Lane wrote: Ron writes: On 10/6/22 09:49, Tom Lane wrote: Ron writes: pg_dump 9.6.24 You realize that that version's been out of support for a year? Yes, which is why I'm dumping from an EOL cluster, and restoring to a supported version. But why are you using the dea

Re: pg_restore creates public schema?

2022-10-06 Thread Tom Lane
Ron writes: > On 10/6/22 09:49, Tom Lane wrote: >> Ron writes: >>> pg_dump 9.6.24 >> You realize that that version's been out of support for a year? > Yes, which is why I'm dumping from an EOL cluster, and restoring to a > supported version. But why are you using the dead version's pg_dump? Y

Re: pg_restore creates public schema?

2022-10-06 Thread Ron
On 10/6/22 09:49, Tom Lane wrote: Ron writes: pg_dump 9.6.24 You realize that that version's been out of support for a year? Yes, which is why I'm dumping from an EOL cluster, and restoring to a supported version. Why does pg_restore explicitly create "public" even though public is autom

Re: pg_restore creates public schema?

2022-10-06 Thread Tom Lane
Ron writes: > pg_dump 9.6.24 You realize that that version's been out of support for a year? > Why does pg_restore explicitly create "public" even though public is > automatically created when the database is created? We fixed that in v11 (see 5955d9341). Evidently the fix requires an updated

Re: pg_restore remap schema

2022-08-08 Thread Fabrice Chapuis
Thank you for your reply. sed is a solution for making substitutions on plain text. But if we work with directory mode to use parallelism, I don't see how to proceed to make a schema remap. Editing the toc file in text mode does not work neither. Regards, Fabrice On Mon, Aug 8, 2022 at 9:27 PM M

Re: pg_restore remap schema

2022-08-08 Thread Marcos Pegoraro
> > People have asked for such a thing before, but it'd be quite difficult > to do reliably --- particularly inside function bodies, which aren't > normally parsed at all during a dump/restore. If you're willing to > accept a 95% solution, running the pg_restore output through "sed" > would likely

Re: pg_restore remap schema

2022-08-08 Thread Tom Lane
Guillaume Lelarge writes: > Le lun. 8 août 2022 à 18:28, Fabrice Chapuis a >> Is a development in progress to add this option > Nope, never heard of someone working on this. People have asked for such a thing before, but it'd be quite difficult to do reliably --- particularly inside function bo

Re: pg_restore remap schema

2022-08-08 Thread Guillaume Lelarge
Hi, Le lun. 8 août 2022 à 18:28, Fabrice Chapuis a écrit : > Hello, > I can't find an option with pg_restore to rename an exported schema > schema1 -> schema2 > That's because it doesn't exist :) > Is a development in progress to add this option > > Nope, never heard of someone working on thi

Re: pg_restore depending on user functions

2021-11-15 Thread Дмитрий Иванов
Yes, it is. I continue to extract data as promised, but I think I see some pattern. "chicken or egg" To work with the NPGSQL library, I created a cast. They are created after the views in which I use them. Here is the order 279: CREATE TYPE bpd.cclass_prop 4646: CREATE VIEW bpd.vclass_prop 4784: CR

Re: pg_restore depending on user functions

2021-11-15 Thread Дмитрий Иванов
Yes, it is. I'll leave behind the lost features, views, table definitions, and dependent objects. It will take some time. thanks, for the help. пн, 15 нояб. 2021 г. в 20:49, Tom Lane : > =?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JjQstCw0L3QvtCy?= > writes: > > Is it ok to attach a 5MB data schema or is it

Re: pg_restore depending on user functions

2021-11-15 Thread Дмитрий Иванов
Yes, it is. I did so (--schema-only). Removing unnecessary definitions. That is, you do not need to attach files? пн, 15 нояб. 2021 г. в 20:49, Tom Lane : > =?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JjQstCw0L3QvtCy?= > writes: > > Is it ok to attach a 5MB data schema or is it not possible? Copy one by > on

Re: pg_restore depending on user functions

2021-11-15 Thread Tom Lane
=?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JjQstCw0L3QvtCy?= writes: > Is it ok to attach a 5MB data schema or is it not possible? Copy one by one > to a letter? Is it smaller if you omit the data (-s switch)? Shouldn't be relevant here. regards, tom lane

Re: pg_restore depending on user functions

2021-11-15 Thread Дмитрий Иванов
Thanks for the feedback! Is it ok to attach a 5MB data schema or is it not possible? Copy one by one to a letter? It would be nice if in the end I could rebuild the base without hindrance. Thank you in advance. Restore Windows 10 PostgreSQL 12.9, compiled by Visual C++ build 1914, 64-bit -> Debian

Re: pg_restore depending on user functions

2021-11-14 Thread Tom Lane
=?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JjQstCw0L3QvtCy?= writes: > вс, 14 нояб. 2021 г. в 23:46, Tom Lane : >> This is not "details", this is an evidence-free assertion. Please show >> a concrete example of problematic functions. > Ok, I'll do it. > [ example ] This is pretty far from being a self-cont

Re: pg_restore depending on user functions

2021-11-14 Thread Дмитрий Иванов
Ok, I'll do it. > Сreating function A before function B results in a compilation error. On my part, this is an incorrect assumption. There are no compilation errors here. she just didn't recover from the first pass. вс, 14 нояб. 2021 г. в 23:46, Tom Lane : > =?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JjQstCw

Re: pg_restore depending on user functions

2021-11-14 Thread Дмитрий Иванов
Ok, I'll do it. вс, 14 нояб. 2021 г. в 23:46, Tom Lane : > =?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JjQstCw0L3QvtCy?= > writes: > > вс, 14 нояб. 2021 г. в 22:31, Tom Lane : > >> Usually this is caused by being careless about search_path assumptions > >> in your functions ... but with no details, it's impo

Re: pg_restore depending on user functions

2021-11-14 Thread Tom Lane
=?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JjQstCw0L3QvtCy?= writes: > вс, 14 нояб. 2021 г. в 22:31, Tom Lane : >> Usually this is caused by being careless about search_path assumptions >> in your functions ... but with no details, it's impossible to say >> anything with certainty. > No, in this case it is n

Re: pg_restore depending on user functions

2021-11-14 Thread Дмитрий Иванов
Спасибо за ваш отзыв. вс, 14 нояб. 2021 г. в 22:31, Tom Lane : > =?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JjQstCw0L3QvtCy?= > writes: > > 1. I can determine the dependencies of the custom functions when creating > > the pg_dump? I did not find anything suitable for SUPPORT, probably this > is > > not the

Re: pg_restore depending on user functions

2021-11-14 Thread Tom Lane
=?UTF-8?B?0JTQvNC40YLRgNC40Lkg0JjQstCw0L3QvtCy?= writes: > 1. I can determine the dependencies of the custom functions when creating > the pg_dump? I did not find anything suitable for SUPPORT, probably this is > not the case? > To recreate all of the functionality, I had to run schema recovery tw

Re: pg_restore (fromuser -> touser)

2021-08-03 Thread Peter J. Holzer
On 2021-07-27 09:43:45 +0800, Ben Madin wrote: > Hi - we have had to do such operations quite commonly, if you want to automate > / stream such a change I would suggest that you look into sed. Off the top of > my head, consider something like this: > > pg_dump --format=p --schema=schema_a original

Re: pg_restore (fromuser -> touser)

2021-07-26 Thread Ben Madin
Hi - we have had to do such operations quite commonly, if you want to automate / stream such a change I would suggest that you look into sed. Off the top of my head, consider something like this: pg_dump --format=p --schema=schema_a original_database | sed 's/ schema_a./ schema_b./g' | psql new_da

Re: pg_restore (fromuser -> touser)

2021-07-26 Thread Mayan
Thanks for the responses everyone and the reality check of how new features get in. I will look into the suggestion by Vijaykumar in more detail and try it out. Thanks again, Mayan On Sun, Jul 25, 2021 at 10:11 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Sun, Jul 25, 2021 at

Re: pg_restore (fromuser -> touser)

2021-07-25 Thread David G. Johnston
On Sun, Jul 25, 2021 at 8:02 PM Ganesh Korde wrote: > You just need to export dump without any privileges. And while restoring > dump use the new role. > You should read the body of the original email and not just the subject line. The actual question pertains to schemas - which IIUC are tightl

Re: pg_restore (fromuser -> touser)

2021-07-25 Thread Ganesh Korde
You just need to export dump without any privileges. And while restoring dump use the new role. Regards, Ganesh Korde. On Mon, 26 Jul 2021, 1:01 am David G. Johnston, wrote: > On Sun, Jul 25, 2021 at 8:39 AM Mayan wrote: > >> Thanks for your reply. Is this something that I can request as a fea

Re: pg_restore (fromuser -> touser)

2021-07-25 Thread David G. Johnston
On Sun, Jul 25, 2021 at 8:39 AM Mayan wrote: > Thanks for your reply. Is this something that I can request as a feature > add? I don't think it should be too much of effort (based on my limited > source code knowledge), but I'm not familiar with the process to request a > feature. > You just did

Re: pg_restore (fromuser -> touser)

2021-07-25 Thread Vijaykumar Jain
On Sun, 25 Jul 2021 at 21:09, Mayan wrote: > Thanks for your reply. Is this something that I can request as a feature > add? I don't think it should be too much of effort (based on my limited > source code knowledge), but I'm not familiar with the process to request a > feature. > although there

Re: pg_restore (fromuser -> touser)

2021-07-25 Thread Mayan
Thanks for your reply. Is this something that I can request as a feature add? I don't think it should be too much of effort (based on my limited source code knowledge), but I'm not familiar with the process to request a feature. Thanks, Mayan On Fri, Jul 23, 2021, 10:58 PM Ninad Shah wrote: > T

Re: pg_restore (fromuser -> touser)

2021-07-23 Thread Ninad Shah
This is true. Such a feature is not available in PostgreSQL. What you need to do is you have to take a structure dump, and change the schema name as per required. And, then, you may copy the data. Regards, Ninad Shah On Fri, 23 Jul 2021 at 23:08, Mayan wrote: > Hi, > > > > I had a general que

Re: pg_restore - generated column - not populating

2021-02-23 Thread Santosh Udupi
Sure. I will try that. On Tue, Feb 23, 2021 at 4:42 PM Adrian Klaver wrote: > On 2/23/21 4:25 PM, Santosh Udupi wrote: > > Yes, this is what we have been doing now:- Backup using pg_dump, create > > the new database at the destination, manually create the tables which > > give problems, and the

Re: pg_restore - generated column - not populating

2021-02-23 Thread Adrian Klaver
On 2/23/21 4:25 PM, Santosh Udupi wrote: Yes,  this is what we have been doing now:- Backup using pg_dump, create the new database at the destination, manually create the tables which give problems, and then do the pg_restore. Another solution for us is to Well that introduces another error,

Re: pg_restore - generated column - not populating

2021-02-23 Thread Santosh Udupi
Yes, this is what we have been doing now:- Backup using pg_dump, create the new database at the destination, manually create the tables which give problems, and then do the pg_restore. Another solution for us is to backup using pg_dump that comes with pgAdmin (Windows), rsync it to the destination

Re: pg_restore - generated column - not populating

2021-02-23 Thread Adrian Klaver
On 2/23/21 12:57 PM, Santosh Udupi wrote: So that it makes it a lot easier for the application logic just to collect json fields and update in one column "info" instead of including multiple columns in the insert/update statements. I doubt it, but then again this why I don't answer Postgres/JS

Re: pg_restore - generated column - not populating

2021-02-23 Thread Santosh Udupi
So that it makes it a lot easier for the application logic just to collect json fields and update in one column "info" instead of including multiple columns in the insert/update statements. On Tue, Feb 23, 2021 at 12:20 PM Adrian Klaver wrote: > On 2/23/21 12:15 PM, Santosh Udupi wrote: > > Here

Re: pg_restore - generated column - not populating

2021-02-23 Thread Adrian Klaver
On 2/23/21 12:15 PM, Santosh Udupi wrote: Here is my table structure. I will try to get the pg_dump output for this table in both the versions. create table tbl_main( item_id int GENERATED ALWAYS AS IDENTITY, - operating_offices i

Re: pg_restore - generated column - not populating

2021-02-23 Thread Santosh Udupi
Here is my table structure. I will try to get the pg_dump output for this table in both the versions. create table tbl_main( item_id int GENERATED ALWAYS AS IDENTITY, - operating_offices int [] GENERATED ALWAYS AS ( nullif(array[(info->>'o')::in

Re: pg_restore - generated column - not populating

2021-02-23 Thread Tom Lane
Santosh Udupi writes: > Both are different versions > The following works: > Version: pg_dump (pgAdmin Windows) version:13.1 > The following does not work: (Does not populate the generated column > values) > Version: pg_dump (PostgreSQL v13.2 on Ubuntu 20.04) version 13.2 Hmm ... well, that wo

Re: pg_restore - generated column - not populating

2021-02-23 Thread Santosh Udupi
Both are different versions The following works: Version: pg_dump (pgAdmin Windows) version:13.1 Method: Backup using Windows connecting to the remote host: Command: pg_dump -Fc -p 5432 -h -d mydb > mydb.backup1 -U postgres Restore locally on Ubuntu : pg_restore -d mydb-restore mydb.backu

Re: pg_restore - generated column - not populating

2021-02-23 Thread Tom Lane
Santosh Udupi writes: > Right. pg_dump under the pgAdmin runtime folder works perfectly. pg_dump > in postgres13 (ubuntu) does not work. Exact same syntax. So, are these identical pg_dump versions? We did fix some things in this area in 13.2. regards, tom lane

  1   2   >