Re: [HACKERS] Reducing Catalog Locking

2014-10-31 Thread Simon Riggs
On 31 October 2014 14:49, Andres Freund wrote: > On 2014-10-31 10:02:28 -0400, Tom Lane wrote: >> Andres Freund writes: >> > On 2014-10-31 09:48:52 -0400, Tom Lane wrote: >> >> But more to the point, this seems like optimizing pg_dump startup by >> >> adding overhead everywhere else, which doesn'

Re: [HACKERS] Reducing Catalog Locking

2014-10-31 Thread Simon Riggs
On 31 October 2014 13:03, Robert Haas wrote: >> Given these are catalog tables, we aren't doing much to them that >> requires a strong lock. Specifically, only CLUSTER and VACUUM FULL >> touch those tables like that. When we do that, pretty much everything >> else hangs, cos you can't get much do

Re: [HACKERS] Reducing Catalog Locking

2014-10-31 Thread Andres Freund
On 2014-10-31 10:02:28 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-10-31 09:48:52 -0400, Tom Lane wrote: > >> But more to the point, this seems like optimizing pg_dump startup by > >> adding overhead everywhere else, which doesn't really sound like a > >> great tradeoff to me. > >

Re: [HACKERS] Reducing Catalog Locking

2014-10-31 Thread Simon Riggs
On 31 October 2014 13:39, Tom Lane wrote: > I doubt that this can ever be safe, because it will effectively assume > that all operations on catalog tables are done by code that knows that it > is accessing a catalog. > What about manual DML, or even DDL, on a catalog? I've never really understo

Re: [HACKERS] Reducing Catalog Locking

2014-10-31 Thread Tom Lane
Andres Freund writes: > On 2014-10-31 09:48:52 -0400, Tom Lane wrote: >> But more to the point, this seems like optimizing pg_dump startup by >> adding overhead everywhere else, which doesn't really sound like a >> great tradeoff to me. > Well, it'd finally make pg_dump "correct" under concurrent

Re: [HACKERS] Reducing Catalog Locking

2014-10-31 Thread Robert Haas
On Fri, Oct 31, 2014 at 9:54 AM, Andres Freund wrote: >> But more to the point, this seems like optimizing pg_dump startup by >> adding overhead everywhere else, which doesn't really sound like a >> great tradeoff to me. > > Well, it'd finally make pg_dump "correct" under concurrent DDL. That's >

Re: [HACKERS] Reducing Catalog Locking

2014-10-31 Thread Andres Freund
On 2014-10-31 09:48:52 -0400, Tom Lane wrote: > Robert Haas writes: > > On a related note, I've previously had the thought that it would be > > nice to have a "big DDL lock" - that is, a lock that prevents > > concurrent DDL without preventing anything else - so that pg_dump > > could get just tha

Re: [HACKERS] Reducing Catalog Locking

2014-10-31 Thread Tom Lane
Robert Haas writes: > On a related note, I've previously had the thought that it would be > nice to have a "big DDL lock" - that is, a lock that prevents > concurrent DDL without preventing anything else - so that pg_dump > could get just that one lock and then not worry about the state of the > w

Re: [HACKERS] Reducing Catalog Locking

2014-10-31 Thread Tom Lane
Simon Riggs writes: > Recent work on parallel query has opened my eyes to exactly how > frequently we request locks on various catalog tables. (Attached file > is a lock analysis on a representative Pg server). > Given these are catalog tables, we aren't doing much to them that > requires a stron

Re: [HACKERS] Reducing Catalog Locking

2014-10-31 Thread Robert Haas
On Fri, Oct 31, 2014 at 6:35 AM, Simon Riggs wrote: > Recent work on parallel query has opened my eyes to exactly how > frequently we request locks on various catalog tables. (Attached file > is a lock analysis on a representative Pg server). That analysis is interesting. > Given these are catal

[HACKERS] Reducing Catalog Locking

2014-10-31 Thread Simon Riggs
Recent work on parallel query has opened my eyes to exactly how frequently we request locks on various catalog tables. (Attached file is a lock analysis on a representative Pg server). Given these are catalog tables, we aren't doing much to them that requires a strong lock. Specifically, only CLUS