Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-16 Thread Robert Haas
On Sun, Jan 15, 2012 at 11:03 AM, Hitoshi Harada wrote: > On Sat, Jan 14, 2012 at 6:48 AM, Robert Haas wrote: >> On Sat, Jan 14, 2012 at 5:25 AM, Hitoshi Harada wrote: >>> The patch looks ok, though I wonder if we could have a way to release >>> the lock on namespace much before the end of trans

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-15 Thread Hitoshi Harada
On Sat, Jan 14, 2012 at 6:48 AM, Robert Haas wrote: > On Sat, Jan 14, 2012 at 5:25 AM, Hitoshi Harada wrote: >> The patch looks ok, though I wonder if we could have a way to release >> the lock on namespace much before the end of transaction. > > Well, that wold kind of miss the point, wouldn't i

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-14 Thread Robert Haas
On Sat, Jan 14, 2012 at 5:25 AM, Hitoshi Harada wrote: > The patch looks ok, though I wonder if we could have a way to release > the lock on namespace much before the end of transaction. Well, that wold kind of miss the point, wouldn't it? I mean, the race is that the process dropping the schema

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-14 Thread Hitoshi Harada
On Sat, Jan 14, 2012 at 2:25 AM, Hitoshi Harada wrote: > On Fri, Jan 13, 2012 at 2:05 PM, Robert Haas wrote: >> On Tue, Nov 29, 2011 at 10:10 AM, Robert Haas wrote: >>> >>> I have plans to try to improve this, but it's one of those things that >>> I care about more than the people who write the

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-14 Thread Hitoshi Harada
On Fri, Jan 13, 2012 at 2:05 PM, Robert Haas wrote: > On Tue, Nov 29, 2011 at 10:10 AM, Robert Haas wrote: >> >> I have plans to try to improve this, but it's one of those things that >> I care about more than the people who write the checks do, so it >> hasn't quite gotten to the top of the prio

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-13 Thread Robert Haas
On Tue, Nov 29, 2011 at 10:10 AM, Robert Haas wrote: > On Tue, Nov 29, 2011 at 3:37 AM, Daniel Farina wrote: >> Hmm, just to prod this thread: has any fix for this been committed? >> After Nikhil confirmed that this bug could cause pg_dump to not be >> able to operate without direct catalog surge

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-29 Thread Robert Haas
On Tue, Nov 29, 2011 at 3:37 AM, Daniel Farina wrote: > Hmm, just to prod this thread: has any fix for this been committed? > After Nikhil confirmed that this bug could cause pg_dump to not be > able to operate without direct catalog surgery I have encountered this > bug (and treated its symptoms

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-29 Thread Daniel Farina
On Mon, Nov 14, 2011 at 12:07 PM, Robert Haas wrote: > On Mon, Nov 14, 2011 at 2:26 PM, Alvaro Herrera > wrote: >> Excerpts from Robert Haas's message of lun nov 14 15:56:43 -0300 2011: >> >>> Well, it looks to me like there are three different places that we >>> need to nail down: RangeVarGetAnd

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Robert Haas
On Mon, Nov 14, 2011 at 2:26 PM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of lun nov 14 15:56:43 -0300 2011: > >> Well, it looks to me like there are three different places that we >> need to nail down: RangeVarGetAndCheckCreationNamespace() is used for >> relations (except that

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Alvaro Herrera
Excerpts from Robert Haas's message of lun nov 14 15:56:43 -0300 2011: > Well, it looks to me like there are three different places that we > need to nail down: RangeVarGetAndCheckCreationNamespace() is used for > relations (except that a few places call RangeVarGetCreationNamespace > directly, w

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Robert Haas
On Mon, Nov 14, 2011 at 12:48 PM, Nikhil Sontakke wrote: >> So it's probably going to take a while to get this >> completely nailed down, but we can keep chipping away at it. > > Agreed. So are you planning to commit this change? Or we want some more > objects to be fixed? Last I looked at this, w

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Nikhil Sontakke
> So it's probably going to take a while to get this > completely nailed down, but we can keep chipping away at it. > > Agreed. So are you planning to commit this change? Or we want some more objects to be fixed? Last I looked at this, we will need locking to be done while creating tables, views, t

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Robert Haas
On Mon, Nov 14, 2011 at 11:48 AM, Nikhil Sontakke wrote: >> If all you need to do is lock a schema, you can just call >> LockDatabaseObject(NamespaceRelationId, namespace_oid, 0, >> AccessShareLock); there's no need to fake up an objectaddress just to >> take a lock.  But I think that's not really

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Nikhil Sontakke
> If all you need to do is lock a schema, you can just call > LockDatabaseObject(NamespaceRelationId, namespace_oid, 0, > AccessShareLock); there's no need to fake up an objectaddress just to > take a lock. But I think that's not really all you need to do, > because somebody could drop the namespa

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Robert Haas
On Thu, Nov 10, 2011 at 7:00 AM, Nikhil Sontakke wrote: > PFA, a patch against git head. We take the AccessShareLock lock on the > schema in DefineRelation now. Note that we do not want to interlock with > other concurrent creations in the schema. We only want to interlock with > deletion activity

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-10 Thread Nikhil Sontakke
> > Continuing in gdb, also completes the creation of c2 table without any > > errors. We are now left with a dangling entry in pg_class along with all > the > > corresponding data files in our data directory. The problem becomes > worse if > > c2 was created using a TABLESPACE. Now dropping of tha

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-10 Thread Daniel Farina
On Wed, Nov 9, 2011 at 1:56 AM, Nikhil Sontakke wrote: > Hi, > > Consider the following sequence of events: > > s1 #> CREATE SCHEMA test_schema; > > s1 #> CREATE TABLE test_schema.c1(x int); > > Now open another session s2 and via gdb issue a breakpoint on > heap_create_with_catalog() which is cal

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-10 Thread Nikhil Sontakke
> Um ... why would we do this only for tables, and not for creations of > other sorts of objects that belong to schemas? > > Right, we need to do it for other objects like functions etc. too. > Also, if we are going to believe that this is a serious problem, what > of ALTER ... SET SCHEMA? > > I

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-10 Thread Tom Lane
Nikhil Sontakke writes: > PFA, a patch against git head. We take the AccessShareLock lock on the > schema in DefineRelation now. Um ... why would we do this only for tables, and not for creations of other sorts of objects that belong to schemas? Also, if we are going to believe that this is a se

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-10 Thread Nikhil Sontakke
Hi, > Ok, understood. > > PFA, a patch against git head. We take the AccessShareLock lock on the schema in DefineRelation now. Note that we do not want to interlock with other concurrent creations in the schema. We only want to interlock with deletion activity. So even performance wise this shoul

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-09 Thread Nikhil Sontakke
> > Yeah thanks, that does the object locking. For pre-9.1 versions, we will > > need a similar solution. I encountered the issue on 8.3.x.. > > I don't think we should back-patch a fix of this type. There is a lot > of cruftiness of this type scattered throughout the code base, and if > we start

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-09 Thread Robert Haas
On Wed, Nov 9, 2011 at 9:51 AM, Nikhil Sontakke wrote: > Yeah thanks, that does the object locking. For pre-9.1 versions, we will > need a similar solution. I encountered the issue on 8.3.x.. I don't think we should back-patch a fix of this type. There is a lot of cruftiness of this type scatter

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-09 Thread Nikhil Sontakke
> > We definitely need some interlocking to handle this. For lack of better > > APIs, we could do a LockDatabaseObject() call in AccessShareLock mode on > the > > namespace and release the same on completion of the creation of the > object. > > > > Thoughts? > > In general, we've been reluctant to

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-09 Thread Robert Haas
On Wed, Nov 9, 2011 at 4:56 AM, Nikhil Sontakke wrote: > Consider the following sequence of events: > > s1 #> CREATE SCHEMA test_schema; > > s1 #> CREATE TABLE test_schema.c1(x int); > > Now open another session s2 and via gdb issue a breakpoint on > heap_create_with_catalog() which is called by D

[HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-09 Thread Nikhil Sontakke
Hi, Consider the following sequence of events: s1 #> CREATE SCHEMA test_schema; s1 #> CREATE TABLE test_schema.c1(x int); Now open another session s2 and via gdb issue a breakpoint on heap_create_with_catalog() which is called by DefineRelation(). s2 #> CREATE TABLE test_schema.c2(y int); The