Re: [PERFORM] pg_dump and thousands of schemas

2012-06-12 Thread Tatsuo Ishii
> On Tue, Jun 12, 2012 at 1:54 AM, Tatsuo Ishii wrote: >>> On Sun, Jun 10, 2012 at 4:47 PM, Jeff Janes wrote: On Wed, May 30, 2012 at 2:06 AM, Tatsuo Ishii wrote: >> Yeah, Jeff's experiments indicated that the remaining bottleneck is lock >> management in the server.  What I fixed s

Re: [PERFORM] pg_dump and thousands of schemas

2012-06-12 Thread Jeff Janes
On Tue, Jun 12, 2012 at 1:54 AM, Tatsuo Ishii wrote: >> On Sun, Jun 10, 2012 at 4:47 PM, Jeff Janes wrote: >>> On Wed, May 30, 2012 at 2:06 AM, Tatsuo Ishii wrote: > Yeah, Jeff's experiments indicated that the remaining bottleneck is lock > management in the server.  What I fixed so far

Re: [PERFORM] Performance of pg_basebackup

2012-06-12 Thread Shaun Thomas
On 06/12/2012 09:57 AM, Magnus Hagander wrote: If you have a single tablespace you can have pg_basebackup write the output to stdout and then pipe that through pigz. Yeah, I saw that. Unfortunately we have tiered storage and hence two tablespaces. :( To be fair, my current process cheats by

Re: [PERFORM] Performance of pg_basebackup

2012-06-12 Thread Magnus Hagander
On Tue, Jun 12, 2012 at 4:54 PM, Shaun Thomas wrote: > Hey everyone, > > I was wondering if anyone has found a way to get pg_basebackup to be... > faster. Currently we do our backups something like this: > > tar -c -I pigz -f /db/backup_-mm-dd.tar.gz -C /db pgdata > > Which basically calls pig

[PERFORM] Performance of pg_basebackup

2012-06-12 Thread Shaun Thomas
Hey everyone, I was wondering if anyone has found a way to get pg_basebackup to be... faster. Currently we do our backups something like this: tar -c -I pigz -f /db/backup_-mm-dd.tar.gz -C /db pgdata Which basically calls pigz to do parallel compression because with RAIDs and ioDrives al

Re: [PERFORM] pg_dump and thousands of schemas

2012-06-12 Thread Tatsuo Ishii
> On Sun, Jun 10, 2012 at 4:47 PM, Jeff Janes wrote: >> On Wed, May 30, 2012 at 2:06 AM, Tatsuo Ishii wrote: Yeah, Jeff's experiments indicated that the remaining bottleneck is lock management in the server.  What I fixed so far on the pg_dump side should be enough to let partial d