Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Eduardo Morras
At 19:32 03/08/2011, you wrote: On 08/03/11 10:21 AM, Eduardo Morras wrote: One question, while you run your tests, does "IDLE IN TRANSACTION" messages happen? If you run your tests with a permanent connection to database, the tables are locked and autovacuum cannot work. its not that tables

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread John R Pierce
On 08/03/11 10:21 AM, Eduardo Morras wrote: One question, while you run your tests, does "IDLE IN TRANSACTION" messages happen? If you run your tests with a permanent connection to database, the tables are locked and autovacuum cannot work. its not that tables are locked, its that vacuum (aut

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Eduardo Morras
At 16:35 03/08/2011, Michael Graham wrote: Yeah it said it last ran yesterday (and is currently running now), but I did I notice in the log: 2011-08-02 19:43:35 BST ERROR: canceling autovacuum task 2011-08-02 19:43:35 BST CONTEXT: automatic vacuum of table "traffic.public.logdata5queue" Which

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Jerry Sievers
Michael Graham writes: > On Wed, 2011-08-03 at 10:17 -0400, Tom Lane wrote: > >> Michael Graham writes: >> > Would my applications >> > constant polling of the queue mean that the lock could not be easily >> > obtained? >> >> Very possible, depending on what duty cycle is involved there. > > Hm

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Tom Lane
Michael Graham writes: > Ah! This looks like it is very much the issue. Since I've got around > 150GB of data that should be truncatable and a select every ~2s. > Just to confirm would postgres write: > 2011-08-03 16:09:55 BST ERROR: canceling autovacuum task > 2011-08-03 16:09:55 BST CONTEXT

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Michael Graham
On Wed, 2011-08-03 at 11:40 -0400, Tom Lane wrote: > The other problem is that once autovacuum has gotten the lock, it has > to keep it for long enough to re-scan the truncatable pages (to make > sure they're still empty). And it is set up so that any access to the > table will kick autovacuum off

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Tom Lane
Pavan Deolasee writes: > On Wed, Aug 3, 2011 at 10:57 AM, Michael Graham wrote: >> On Wed, 2011-08-03 at 10:17 -0400, Tom Lane wrote: > Michael Graham writes: Would my applications constant polling of the queue mean that the lock could not be easily obtained? >>> Very possible, d

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Bill Moran
In response to Michael Graham : > On Wed, 2011-08-03 at 10:17 -0400, Tom Lane wrote: > > Michael Graham writes: > > > Would my applications > > > constant polling of the queue mean that the lock could not be easily > > > obtained? > > > > Very possible, depending on what duty cycle is involved t

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Scott Marlowe
On Wed, Aug 3, 2011 at 8:57 AM, Michael Graham wrote: > On Wed, 2011-08-03 at 10:17 -0400, Tom Lane wrote: >> Michael Graham writes: >> > Would my applications >> > constant polling of the queue mean that the lock could not be easily >> > obtained? >> >> Very possible, depending on what duty cycl

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Pavan Deolasee
On Wed, Aug 3, 2011 at 10:57 AM, Michael Graham wrote: > On Wed, 2011-08-03 at 10:17 -0400, Tom Lane wrote: >> Michael Graham writes: >> > Would my applications >> > constant polling of the queue mean that the lock could not be easily >> > obtained? >> >> Very possible, depending on what duty cyc

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Michael Graham
On Wed, 2011-08-03 at 10:17 -0400, Tom Lane wrote: > Michael Graham writes: > > Would my applications > > constant polling of the queue mean that the lock could not be easily > > obtained? > > Very possible, depending on what duty cycle is involved there. Hmm. The clients aren't that aggressive

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Michael Graham
On Wed, 2011-08-03 at 09:03 -0500, Andy Colson wrote: > Depending on how long you ran your test, and the conf settings, and > the size of your database, autovacuum may never have even tried. I know that the vacuum is definitely running (in fact isn't it the vacuum that set the reltuples to 0?), th

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Sergey Konoplev
On 3 August 2011 18:17, Tom Lane wrote: >> Would my applications >> constant polling of the queue mean that the lock could not be easily >> obtained? > > Very possible, depending on what duty cycle is involved there. Is there any ways of guaranteed concurrent obtaining it? > >                  

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Tom Lane
Michael Graham writes: > From reading the documentation I see that postgres would return this > space to that system after a normal vacuum if "one or more pages at the > end of a table become entirely free and an exclusive table lock can be > easily obtained". > What does "easily obtained" mean i

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Scott Marlowe
On Wed, Aug 3, 2011 at 8:03 AM, Andy Colson wrote: > If you have lots and lots of tables, autovacuum only checks one at a time, > then wait's a bit.  Did you run your test for several days? Not true. autovac naps by default 1 minute between each db. i.e. if you have 5 dbs it takes 5 minutes by d

Re: [GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Andy Colson
On 8/3/2011 4:47 AM, Michael Graham wrote: Hi all, I have an application that is reading from a queue table, as part of my testing I stressed the table to check performance, but after the test was completed I have the a very large empty table: SELECT relname, pg_size_pretty(pg_relation_size(oid

[GENERAL] Vacuum as "easily obtained" locks

2011-08-03 Thread Michael Graham
Hi all, I have an application that is reading from a queue table, as part of my testing I stressed the table to check performance, but after the test was completed I have the a very large empty table: SELECT relname, pg_size_pretty(pg_relation_size(oid)) AS size, reltuples::bigint FROM pg_class;