Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-28 Thread Simon Riggs
On Wed, 2010-07-28 at 15:24 +0300, Peter Eisentraut wrote: > On tor, 2010-07-15 at 10:24 +0100, Simon Riggs wrote: > > Patch to reduce lock levels for > > ALTER TABLE > > CREATE TRIGGER > > CREATE RULE > > Tried this out, but $subject is still the case. The problem is that > ATRewriteCatalogs

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-28 Thread Simon Riggs
On Wed, 2010-07-28 at 15:24 +0300, Peter Eisentraut wrote: > On tor, 2010-07-15 at 10:24 +0100, Simon Riggs wrote: > > Patch to reduce lock levels for > > ALTER TABLE > > CREATE TRIGGER > > CREATE RULE > > Tried this out, but $subject is still the case. The problem is that > ATRewriteCatalogs

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-28 Thread Peter Eisentraut
On tor, 2010-07-15 at 10:24 +0100, Simon Riggs wrote: > Patch to reduce lock levels for > ALTER TABLE > CREATE TRIGGER > CREATE RULE Tried this out, but $subject is still the case. The problem is that ATRewriteCatalogs() calls AlterTableCreateToastTable() based on what it thinks the subcomman

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-19 Thread Robert Haas
On Mon, Jul 19, 2010 at 2:46 AM, Simon Riggs wrote: > On Sun, 2010-07-18 at 22:47 -0400, Robert Haas wrote: >>  But it seems >> that it's far from clear what to do about it, and it's not the job of >> this patch to fix it anyway. > > Agreed. > >> Regarding the actual patch, it looks mostly good.  

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Simon Riggs
On Sun, 2010-07-18 at 22:47 -0400, Robert Haas wrote: > But it seems > that it's far from clear what to do about it, and it's not the job of > this patch to fix it anyway. Agreed. > Regarding the actual patch, it looks mostly good. Questions: > > 1. Why in rewriteSupport.c are we adding a call

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Robert Haas
On Sun, Jul 18, 2010 at 1:20 PM, Tom Lane wrote: > Andres Freund writes: >> On Sunday 18 July 2010 18:02:26 Simon Riggs wrote: >>> Then I think the fix is to look at the xmin values on all of the tables >>> used during planning and ensure that we only use constraint-based >>> optimisations in a s

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Andres Freund
On Sunday 18 July 2010 19:20:25 Tom Lane wrote: > Andres Freund writes: > > On Sunday 18 July 2010 18:02:26 Simon Riggs wrote: > >> Then I think the fix is to look at the xmin values on all of the tables > >> used during planning and ensure that we only use constraint-based > >> optimisations in a

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Tom Lane
Andres Freund writes: > On Sunday 18 July 2010 18:02:26 Simon Riggs wrote: >> Then I think the fix is to look at the xmin values on all of the tables >> used during planning and ensure that we only use constraint-based >> optimisations in a serializable transaction when our top xmin is later >> th

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Andres Freund
On Sunday 18 July 2010 18:02:26 Simon Riggs wrote: > On Sun, 2010-07-18 at 17:28 +0200, Andres Freund wrote: > > Unfortunately the same issue exists with constraint exclusion - and we > > can hardly disable that for serializable transactions... > > Then I think the fix is to look at the xmin value

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Simon Riggs
On Sun, 2010-07-18 at 17:28 +0200, Andres Freund wrote: > Unfortunately the same issue exists with constraint exclusion - and we > can hardly disable that for serializable transactions... Then I think the fix is to look at the xmin values on all of the tables used during planning and ensure that

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-18 Thread Andres Freund
On Saturday 17 July 2010 09:55:37 Simon Riggs wrote: > On Fri, 2010-07-16 at 23:03 +0200, Andres Freund wrote: > > Sure its not that bad, but at least it needs to get documented imho. > > Likely others should chime in here ;-) > > Don't understand you. This is a clear bug in join removal, test cas

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-17 Thread Simon Riggs
On Fri, 2010-07-16 at 20:45 -0400, Tom Lane wrote: > Andres Freund writes: > > Just to help me: The primary reasons for using SnapshotNow is speed and in > > some cases correctness (referential integrity). Right? Any other reasons? > > Well, the main point for system catalog accesses is that you

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-17 Thread Simon Riggs
On Fri, 2010-07-16 at 23:03 +0200, Andres Freund wrote: > Sure its not that bad, but at least it needs to get documented imho. > Likely others should chime in here ;-) Don't understand you. This is a clear bug in join removal, test case attached, a minor rework of your original test case. > What

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Tom Lane
Andres Freund writes: > Just to help me: The primary reasons for using SnapshotNow is speed and in > some cases correctness (referential integrity). Right? Any other reasons? Well, the main point for system catalog accesses is that you *must* have an up-to-date view of the table schemas. As an

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
On Saturday 17 July 2010 01:53:24 Robert Haas wrote: > On Jul 16, 2010, at 6:01 PM, Tom Lane wrote: > > Andres Freund writes: > >> What could the join removal path (and similar places) *possibly* do > >> against such a case? Without stopping to use SnapshotNow I dont see any > >> way :-( > > > >

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Robert Haas
On Jul 16, 2010, at 6:01 PM, Tom Lane wrote: > Andres Freund writes: >> What could the join removal path (and similar places) *possibly* do against >> such a case? Without stopping to use SnapshotNow I dont see any way :-( > > But the planner, along with most of the rest of the backend, *does*

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Tom Lane
Andres Freund writes: > What could the join removal path (and similar places) *possibly* do against > such a case? Without stopping to use SnapshotNow I dont see any way :-( But the planner, along with most of the rest of the backend, *does* use SnapshotNow when examining the system catalogs. I

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
On Friday 16 July 2010 22:24:32 Simon Riggs wrote: > On Fri, 2010-07-16 at 21:38 +0200, Andres Freund wrote: > > boom > > Your test case would still occur in the case where the first query had > not been executed against the same table. So the test case illustrates a > failing of join removal, not

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Simon Riggs
On Fri, 2010-07-16 at 21:38 +0200, Andres Freund wrote: > boom Your test case would still occur in the case where the first query had not been executed against the same table. So the test case illustrates a failing of join removal, not of this patch. -- Simon Riggs www.2ndQuadrant.com

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
On Friday 16 July 2010 21:15:44 Simon Riggs wrote: > On Fri, 2010-07-16 at 21:10 +0200, Andres Freund wrote: > > On Friday 16 July 2010 20:41:44 Andres Freund wrote: > > > >> ! */ > > > >> !case AT_AddColumn: /* may > > > >> rewrite heap,

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
On Friday 16 July 2010 21:12:33 Simon Riggs wrote: > On Fri, 2010-07-16 at 20:41 +0200, Andres Freund wrote: > > You argue above that you cant change SET [NOT] NULL to be less > > restrictive because it might change plans - isnt that true for some of > > the above cases as well? > > > > For exampl

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Simon Riggs
On Fri, 2010-07-16 at 21:10 +0200, Andres Freund wrote: > On Friday 16 July 2010 20:41:44 Andres Freund wrote: > > >> ! */ > > >> !case AT_AddColumn: /* may > > >> rewrite heap, in some cases and visible to SELECT */ ! > > >>

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Simon Riggs
On Fri, 2010-07-16 at 20:41 +0200, Andres Freund wrote: > You argue above that you cant change SET [NOT] NULL to be less > restrictive because it might change plans - isnt that true for some of the > above cases as well? > > For example UNIQUE/PRIMARY might make join removal possible - which cou

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
On Friday 16 July 2010 20:41:44 Andres Freund wrote: > >> ! */ > >> !case AT_AddColumn: /* may > >> rewrite heap, in some cases and visible to SELECT */ ! > >>case AT_DropColumn: /* change > >> visi

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-16 Thread Andres Freund
Hi Simon, Your patch implements part of a feature I desire greatly - thanks! Some comments: On Thursday 15 July 2010 11:24:27 Simon Riggs wrote: >> ! LOCKMODE >> ! AlterTableGreatestLockLevel(List *cmds) >> ! { >> !ListCell *lcmd; >> !LOCKMODE lockmode = ShareUpdateExclusiveLock; /* d

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-15 Thread Josh Berkus
On 7/7/10 6:04 PM, Cédric Villemain wrote: > I just faced production issue where it is impossible to alter table to > adjust autovacuum settings in a pg8.4. (5K tps, 260M rows table, lock > too much) We could try to resolve the COMMENT ON issue with the same mechanism. What we need is a table lock

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-15 Thread Simon Riggs
On Thu, 2010-07-08 at 07:16 +0100, Simon Riggs wrote: > I'll take my previous patch through to completion now Patch to reduce lock levels for ALTER TABLE CREATE TRIGGER CREATE RULE I've completely re-analyzed the required lock levels for sub-commands, so lock levels can now also be these, if

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-11 Thread Simon Riggs
On Fri, 2010-07-09 at 15:03 -0400, Robert Haas wrote: > On Fri, Jul 9, 2010 at 1:18 PM, Simon Riggs wrote: > > On Fri, 2010-07-09 at 13:04 -0400, Robert Haas wrote: > >> Tom asked what happens when two transactions attempt to do concurrent > >> actions on the same table. Your response was that we

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-10 Thread Bruce Momjian
Robert Haas wrote: > On Wed, Jul 7, 2010 at 9:04 PM, C?dric Villemain > wrote: > >>> > I assume this did not get done for 9.0. ?Do we want a TODO item? > >>> > >>> Yes. > >> > >> Added: > >> > >> ? ? ? ?Reduce locking required for ALTER commands > > > > I just faced production issue where it is im

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-09 Thread Robert Haas
On Fri, Jul 9, 2010 at 1:18 PM, Simon Riggs wrote: > On Fri, 2010-07-09 at 13:04 -0400, Robert Haas wrote: >> Tom asked what happens when two transactions attempt to do concurrent >> actions on the same table.  Your response was that we should handle it >> like CREATE INDEX, and handle the update

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-09 Thread Simon Riggs
On Fri, 2010-07-09 at 13:04 -0400, Robert Haas wrote: > Tom asked what happens when two transactions attempt to do concurrent > actions on the same table. Your response was that we should handle it > like CREATE INDEX, and handle the update of the pg_class row > non-transactionally. But of cours

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-09 Thread Robert Haas
On Thu, Jul 8, 2010 at 5:09 PM, Simon Riggs wrote: > On Thu, 2010-07-08 at 06:08 -0400, Robert Haas wrote: >> On Thu, Jul 8, 2010 at 2:16 AM, Simon Riggs wrote: >> > On Wed, 2010-07-07 at 22:26 -0400, Robert Haas wrote: >> >> Rereading the thread, I'm a bit confused by why we're proposing to use

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-08 Thread Simon Riggs
On Thu, 2010-07-08 at 06:08 -0400, Robert Haas wrote: > On Thu, Jul 8, 2010 at 2:16 AM, Simon Riggs wrote: > > On Wed, 2010-07-07 at 22:26 -0400, Robert Haas wrote: > >> Rereading the thread, I'm a bit confused by why we're proposing to use > >> a SHARE lock; it seems to me that a self-conflicting

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-08 Thread Robert Haas
On Thu, Jul 8, 2010 at 2:16 AM, Simon Riggs wrote: > On Wed, 2010-07-07 at 22:26 -0400, Robert Haas wrote: >> Rereading the thread, I'm a bit confused by why we're proposing to use >> a SHARE lock; it seems to me that a self-conflicting lock type would >> simplify things.  There's a bunch of discu

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-07 Thread Simon Riggs
On Wed, 2010-07-07 at 22:26 -0400, Robert Haas wrote: > Rereading the thread, I'm a bit confused by why we're proposing to use > a SHARE lock; it seems to me that a self-conflicting lock type would > simplify things. There's a bunch of discussion on the thread about > how to handle pg_class updat

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-07 Thread Robert Haas
On Wed, Jul 7, 2010 at 9:04 PM, Cédric Villemain wrote: >>> > I assume this did not get done for 9.0.  Do we want a TODO item? >>> >>> Yes. >> >> Added: >> >>        Reduce locking required for ALTER commands > > I just faced production issue where it is impossible to alter table to > adjust autov

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-07 Thread Cédric Villemain
2010/3/3 Bruce Momjian : > Peter Eisentraut wrote: >> On m?n, 2010-02-22 at 10:32 -0500, Bruce Momjian wrote: >> > Simon Riggs wrote: >> > > On Mon, 2009-10-19 at 12:56 -0300, Alvaro Herrera wrote: >> > > > Simon Riggs wrote: >> > > > > >> > > > > On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera w

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-03-02 Thread Bruce Momjian
Peter Eisentraut wrote: > On m?n, 2010-02-22 at 10:32 -0500, Bruce Momjian wrote: > > Simon Riggs wrote: > > > On Mon, 2009-10-19 at 12:56 -0300, Alvaro Herrera wrote: > > > > Simon Riggs wrote: > > > > > > > > > > On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: > > > > > > Tom Lane wrote

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-03-01 Thread Peter Eisentraut
On mån, 2010-02-22 at 10:32 -0500, Bruce Momjian wrote: > Simon Riggs wrote: > > On Mon, 2009-10-19 at 12:56 -0300, Alvaro Herrera wrote: > > > Simon Riggs wrote: > > > > > > > > On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: > > > > > Tom Lane wrote: > > > > > > Peter Eisentraut writes

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-02-22 Thread Bruce Momjian
Simon Riggs wrote: > On Mon, 2009-10-19 at 12:56 -0300, Alvaro Herrera wrote: > > Simon Riggs wrote: > > > > > > On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: > > > > Tom Lane wrote: > > > > > Peter Eisentraut writes: > > > > > > Is there a good reason for $subject, other than that the

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-10-19 Thread Simon Riggs
On Mon, 2009-10-19 at 12:56 -0300, Alvaro Herrera wrote: > Simon Riggs wrote: > > > > On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: > > > Tom Lane wrote: > > > > Peter Eisentraut writes: > > > > > Is there a good reason for $subject, other than that the code is > > > > > entangled >

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-10-19 Thread Alvaro Herrera
Simon Riggs wrote: > > On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: > > Tom Lane wrote: > > > Peter Eisentraut writes: > > > > Is there a good reason for $subject, other than that the code is > > > > entangled > > > > with other ALTER TABLE code? > > > > > > I think it could be low

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-08-08 Thread Simon Riggs
On Fri, 2009-08-07 at 15:58 -0400, Alvaro Herrera wrote: > Tom Lane wrote: > > Peter Eisentraut writes: > > > Is there a good reason for $subject, other than that the code is > > > entangled > > > with other ALTER TABLE code? > > > > I think it could be lower, but it would take nontrivial rest

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-08-07 Thread Alvaro Herrera
Tom Lane wrote: > Peter Eisentraut writes: > > Is there a good reason for $subject, other than that the code is entangled > > with other ALTER TABLE code? > > I think it could be lower, but it would take nontrivial restructuring of > the ALTER TABLE support. In particular, consider what happens

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-08-07 Thread Tom Lane
Peter Eisentraut writes: > Is there a good reason for $subject, other than that the code is entangled > with other ALTER TABLE code? I think it could be lower, but it would take nontrivial restructuring of the ALTER TABLE support. In particular, consider what happens when you have a list of sub

[HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2009-08-07 Thread Peter Eisentraut
Is there a good reason for $subject, other than that the code is entangled with other ALTER TABLE code? The new SET DISTINCT might be equally affected. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/