Re: [PERFORM] pg_dump and thousands of schemas

2012-05-25 Thread Tatsuo Ishii
> "Hugo " writes: >> If anyone has more suggestions, I would like to hear them. Thank you! > > Provide a test case? > > We recently fixed a couple of O(N^2) loops in pg_dump, but those covered > extremely specific cases that might or might not have anything to do > with what you're seeing. The

Re: [PERFORM] Parallel (concurrent) inserts?

2012-05-25 Thread Ivan Voras
On 26 May 2012 01:36, Jeff Janes wrote: > On Fri, May 25, 2012 at 3:04 PM, Ivan Voras wrote: >> Hello, >> >> I'm wondering if there is ia document describing which guarantees (if >> any) PostgreSQL makes about concurrency for various operations? Speaking >> in general (i.e. IO can handle it, numb

Re: [PERFORM] Parallel (concurrent) inserts?

2012-05-25 Thread Jeff Janes
On Fri, May 25, 2012 at 3:04 PM, Ivan Voras wrote: > Hello, > > I'm wondering if there is ia document describing which guarantees (if > any) PostgreSQL makes about concurrency for various operations? Speaking > in general (i.e. IO can handle it, number of CPU cores and client > threads is optimal)

Re: [PERFORM] Parallel (concurrent) inserts?

2012-05-25 Thread Josh Berkus
> I'm wondering if there is ia document describing which guarantees (if > any) PostgreSQL makes about concurrency for various operations? Speaking > in general (i.e. IO can handle it, number of CPU cores and client > threads is optimal), are fully concurrent operations (independant and > non-block

[PERFORM] Parallel (concurrent) inserts?

2012-05-25 Thread Ivan Voras
Hello, I'm wondering if there is ia document describing which guarantees (if any) PostgreSQL makes about concurrency for various operations? Speaking in general (i.e. IO can handle it, number of CPU cores and client threads is optimal), are fully concurrent operations (independant and non-blocking

Re: [PERFORM] pg_dump and thousands of schemas

2012-05-25 Thread Jeff Janes
On Fri, May 25, 2012 at 1:02 PM, Tom Lane wrote: > Jeff Janes writes: >> For dumping entire databases, It looks like the biggest problem is >> going to be LockReassignCurrentOwner in the server.  And that doesn't >> seem to be easy to fix, as any change to it to improve pg_dump will >> risk degra

Re: [PERFORM] pg_dump and thousands of schemas

2012-05-25 Thread Tom Lane
Jeff Janes writes: > For dumping entire databases, It looks like the biggest problem is > going to be LockReassignCurrentOwner in the server. And that doesn't > seem to be easy to fix, as any change to it to improve pg_dump will > risk degrading normal use cases. > If we want to be able to effic

Re: [PERFORM] pg_dump and thousands of schemas

2012-05-25 Thread Jeff Janes
On Fri, May 25, 2012 at 9:56 AM, Tom Lane wrote: > Jeff Janes writes: > >> For dumping entire databases, It looks like the biggest problem is >> going to be LockReassignCurrentOwner in the server.  And that doesn't >> seem to be easy to fix, as any change to it to improve pg_dump will >> risk deg

Re: [PERFORM] pg_dump and thousands of schemas

2012-05-25 Thread Tom Lane
Jeff Janes writes: > There is an operation in pg_dump which is O(#_of_schemata_in_db * > #_of_table_in_db), or something like that. > The attached very crude patch reduces that to > O(log_of_#_of_schemata_in_db * #_of_table_in_db) > I was hoping this would be a general improvement. It doesn't se

Re: [PERFORM] pg_dump and thousands of schemas

2012-05-25 Thread Jeff Janes
On Fri, May 25, 2012 at 8:18 AM, Bruce Momjian wrote: > On Fri, May 25, 2012 at 10:41:23AM -0400, Tom Lane wrote: >> "Hugo " writes: >> > If anyone has more suggestions, I would like to hear them. Thank you! >> >> Provide a test case? >> >> We recently fixed a couple of O(N^2) loops in pg_dump, b

[PERFORM] Millions of relations (from Maximum number of sequences that can be created)

2012-05-25 Thread Greg Spiegelberg
On Fri, May 25, 2012 at 9:04 AM, Craig James wrote: > On Fri, May 25, 2012 at 4:58 AM, Greg Spiegelberg > wrote: > >> On Sun, May 13, 2012 at 10:01 AM, Craig James >> wrote: >> >>> >>> On Sun, May 13, 2012 at 1:12 AM, Віталій Тимчишин >>> wrote: >>> The sequences AFAIK are account

Re: [PERFORM] pg_dump and thousands of schemas

2012-05-25 Thread Jeff Janes
On Thu, May 24, 2012 at 8:54 PM, Bruce Momjian wrote: > On Thu, May 24, 2012 at 08:20:34PM -0700, Jeff Janes wrote: >> pg_dump itself seems to have a lot of quadratic portions (plus another >> one on the server which it hits pretty heavily), and it hard to know >> where to start addressing them.

Re: [PERFORM] pg_dump and thousands of schemas

2012-05-25 Thread Bruce Momjian
On Fri, May 25, 2012 at 10:41:23AM -0400, Tom Lane wrote: > "Hugo " writes: > > If anyone has more suggestions, I would like to hear them. Thank you! > > Provide a test case? > > We recently fixed a couple of O(N^2) loops in pg_dump, but those covered > extremely specific cases that might or mig

Re: [PERFORM] Maximum number of sequences that can be created

2012-05-25 Thread Craig James
On Fri, May 25, 2012 at 4:58 AM, Greg Spiegelberg wrote: > On Sun, May 13, 2012 at 10:01 AM, Craig James wrote: > >> >> On Sun, May 13, 2012 at 1:12 AM, Віталій Тимчишин wrote: >> >>> >>> The sequences AFAIK are accounted as relations. Large list of relations >>> may slowdown different system util

Re: [PERFORM] pg_dump and thousands of schemas

2012-05-25 Thread Tom Lane
"Hugo " writes: > If anyone has more suggestions, I would like to hear them. Thank you! Provide a test case? We recently fixed a couple of O(N^2) loops in pg_dump, but those covered extremely specific cases that might or might not have anything to do with what you're seeing. The complainant was

Re: [PERFORM] Maximum number of sequences that can be created

2012-05-25 Thread Greg Spiegelberg
On Sun, May 13, 2012 at 10:01 AM, Craig James wrote: > > On Sun, May 13, 2012 at 1:12 AM, Віталій Тимчишин wrote: > >> >> The sequences AFAIK are accounted as relations. Large list of relations >> may slowdown different system utilities like vacuuming (or may not, depends >> on queries and indexe