Re: [PERFORM] Problems with autovacuum

2009-06-09 Thread Tom Lane
Alvaro Herrera writes: > Glitches fixed in this version; will apply shortly to 8.3 and HEAD. Looks sane; one trivial grammar correction: > + /* the minimum allowed time between two awakening of the launcher */ Should read "two awakenings". regards, tom lane -- Sent vi

Re: [PERFORM] Problems with autovacuum

2009-06-09 Thread Alvaro Herrera
Alvaro Herrera escribió: > Tom Lane escribió: > > > Well, that code isn't even correct I think; you're not supposed to > > modify a GUC variable directly. I think you should just silently > > use a naptime of at least X without changing the nominal GUC variable. > > And definitely without the WAR

Re: [PERFORM] Problems with autovacuum

2009-06-09 Thread Alvaro Herrera
Tom Lane escribió: > Well, that code isn't even correct I think; you're not supposed to > modify a GUC variable directly. I think you should just silently > use a naptime of at least X without changing the nominal GUC variable. > And definitely without the WARNING --- that's nothing but log spam.

Re: [PERFORM] Problems with autovacuum

2009-06-08 Thread Tom Lane
Alvaro Herrera writes: > I've been giving this some thought and tried several approaches. In the > end the one that I like the most is raising autovacuum_naptime to a > reasonable value for the exiting number of databases. The only problem > I have with it is that it's trivial to change it in th

Re: [PERFORM] Problems with autovacuum

2009-06-08 Thread Alvaro Herrera
Tom Lane escribió: > Bingo, that's surely exactly what was happening to the OP. He had 2000 > databases and naptime at (I assume) the default; so he was rerunning > rebuild_database_list every 100ms. > > So that recovery code path needs some more thought. Maybe a lower bound > on how often to d

Re: [PERFORM] Problems with autovacuum

2009-05-26 Thread Scott Marlowe
2009/5/26 Tom Lane : > Alvaro Herrera writes: >> However I think there's a bigger problem here, which is that if the user >> has set naptime too low, i.e. to a value lower than >> number-of-databases * 100ms, we'll be running the (expensive) >> rebuild_database_list function on each iteration ...

Re: [PERFORM] Problems with autovacuum

2009-05-26 Thread Tom Lane
Alvaro Herrera writes: > However I think there's a bigger problem here, which is that if the user > has set naptime too low, i.e. to a value lower than > number-of-databases * 100ms, we'll be running the (expensive) > rebuild_database_list function on each iteration ... maybe we oughta put > a low

Re: [PERFORM] Problems with autovacuum

2009-05-26 Thread Alvaro Herrera
Tom Lane escribió: > Alvaro Herrera writes: > > Tom Lane escribi�: > >> Hmm, maybe we need to improve the code too. This example suggests that > >> there needs to be some limit on the worker launch rate, even if there > >> are so many databases that that means we don't meet naptime exactly. > >

Re: [PERFORM] Problems with autovacuum

2009-05-26 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane escribió: >> Hmm, maybe we need to improve the code too. This example suggests that >> there needs to be some limit on the worker launch rate, even if there >> are so many databases that that means we don't meet naptime exactly. > We already have a 100ms lower b

Re: [PERFORM] Problems with autovacuum

2009-05-26 Thread Alvaro Herrera
Tom Lane escribió: > Hmm, maybe we need to improve the code too. This example suggests that > there needs to be some limit on the worker launch rate, even if there > are so many databases that that means we don't meet naptime exactly. We already have a 100ms lower bound on the sleep time (see la

Re: [PERFORM] Problems with autovacuum

2009-05-26 Thread Tom Lane
Alvaro Herrera writes: > Sorry, it's the other way around actually -- correct for 8.3 onwards, > wrong for 8.1 and 8.2. In the earlier versions, it would do one run in > a chosen database, sleep during "naptime", then do another run. > Tom Lane escribió: >> I suppose the use of "minimum" means

Re: [PERFORM] Problems with autovacuum

2009-05-26 Thread Łukasz Jagiełło
W dniu 26 maja 2009 20:28 użytkownik Tom Lane napisał: >>> I believe the interpretation of autovacuum_naptime is that it should >>> examine each database that often, ie once a minute by default.  So >>> it's got more than 30 databases per second to look through. > >> Note that this is correct in 8

Re: [PERFORM] Problems with autovacuum

2009-05-26 Thread Alvaro Herrera
Tom Lane escribió: > Alvaro Herrera writes: > > Tom Lane escribi�: > >> I believe the interpretation of autovacuum_naptime is that it should > >> examine each database that often, ie once a minute by default. So > >> it's got more than 30 databases per second to look through. > > > Note that thi

Re: [PERFORM] Problems with autovacuum

2009-05-26 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane escribió: >> I believe the interpretation of autovacuum_naptime is that it should >> examine each database that often, ie once a minute by default. So >> it's got more than 30 databases per second to look through. > Note that this is correct in 8.1 and 8.2 but n

Re: [PERFORM] Problems with autovacuum

2009-05-26 Thread Alvaro Herrera
Tom Lane escribió: > =?UTF-8?B?xYF1a2FzeiBKYWdpZcWCxYJv?= writes: > > That autovacuum working hole time, shoudn't be run only when db needs ? > > With 2000 databases to cycle through, autovac is going to be spending > quite a lot of time just finding out whether it needs to do anything. > I belie

Re: [PERFORM] Problems with autovacuum

2009-05-25 Thread Grzegorz Jaśkiewicz
2009/5/25 Łukasz Jagiełło : > W dniu 25 maja 2009 17:32 użytkownik Scott Marlowe > napisał: >>> Recent change postgresql server from Amazon EC2 small into large one. >>> That gives me x86_64 arch, two core cpu and 7.5GB ram. Atm got almost >>> ~2000 small databases at that server and autovacuum wo

Re: [PERFORM] Problems with autovacuum

2009-05-25 Thread Łukasz Jagiełło
2009/5/25 Tom Lane : > With 2000 databases to cycle through, autovac is going to be spending > quite a lot of time just finding out whether it needs to do anything. > I believe the interpretation of autovacuum_naptime is that it should > examine each database that often, ie once a minute by default

Re: [PERFORM] Problems with autovacuum

2009-05-25 Thread Tom Lane
=?UTF-8?B?xYF1a2FzeiBKYWdpZcWCxYJv?= writes: > That autovacuum working hole time, shoudn't be run only when db needs ? With 2000 databases to cycle through, autovac is going to be spending quite a lot of time just finding out whether it needs to do anything. I believe the interpretation of autova

Re: [PERFORM] Problems with autovacuum

2009-05-25 Thread Łukasz Jagiełło
W dniu 25 maja 2009 17:50 użytkownik Grzegorz Jaśkiewicz napisał: >> So, in 2000 databases, there's only an average of 2 relations per db >> and 102 dead rows?  Cause that's all you got room for with those >> settings. >> >> Whats the last 20 or so lines of vacuum verbose as run by a superuser say

Re: [PERFORM] Problems with autovacuum

2009-05-25 Thread Łukasz Jagiełło
W dniu 25 maja 2009 17:32 użytkownik Scott Marlowe napisał: >> Recent change postgresql server from Amazon EC2 small into large one. >> That gives me x86_64 arch, two core cpu and 7.5GB ram. Atm got almost >> ~2000 small databases at that server and autovacuum working hole time > >> postgresql.con

Re: [PERFORM] Problems with autovacuum

2009-05-25 Thread Scott Marlowe
2009/5/25 Grzegorz Jaśkiewicz : > 2009/5/25 Scott Marlowe : > >> >> So, in 2000 databases, there's only an average of 2 relations per db >> and 102 dead rows?  Cause that's all you got room for with those >> settings. >> >> Whats the last 20 or so lines of vacuum verbose as run by a superuser say?

Re: [PERFORM] Problems with autovacuum

2009-05-25 Thread Grzegorz Jaśkiewicz
2009/5/25 Scott Marlowe : > > So, in 2000 databases, there's only an average of 2 relations per db > and 102 dead rows?  Cause that's all you got room for with those > settings. > > Whats the last 20 or so lines of vacuum verbose as run by a superuser say? according to http://www.postgresql.org/

Re: [PERFORM] Problems with autovacuum

2009-05-25 Thread Scott Marlowe
2009/5/24 Łukasz Jagiełło : > Hi, > > Recent change postgresql server from Amazon EC2 small into large one. > That gives me x86_64 arch, two core cpu and 7.5GB ram. Atm got almost > ~2000 small databases at that server and autovacuum working hole time > postgresql.conf: > max_fsm_pages = 204800 >

[PERFORM] Problems with autovacuum

2009-05-24 Thread Łukasz Jagiełło
Hi, Recent change postgresql server from Amazon EC2 small into large one. That gives me x86_64 arch, two core cpu and 7.5GB ram. Atm got almost ~2000 small databases at that server and autovacuum working hole time (witch isn't good for performance as I notice at cacti, one core is busy in 60% hole