Re: [GENERAL] Unexplained Major Vacuum Archive Activity During Vacuum

2012-11-03 Thread Andres Freund
On Thursday, November 01, 2012 05:40:23 PM Alban Hertroys wrote: > On 1 November 2012 17:19, Shaun Thomas wrote: > > On 11/01/2012 10:28 AM, Kevin Grittner wrote: > > Based on my past experience with 8.2, and my understanding of 9.1, I > > moved autovacuum_freeze_max_age up to 650M so we'd never g

Re: [GENERAL] Unexplained Major Vacuum Archive Activity During Vacuum

2012-11-02 Thread Shaun Thomas
On 11/02/2012 03:08 AM, Alban Hertroys wrote: 150M database transactions a day sounds excessive, is there no way to reduce that number? I wish. 150M is actually a conservative estimate. In fact, we average 141M, but have been as high as 270M. It's all market dependent. Here's a quick look a

Re: [GENERAL] Unexplained Major Vacuum Archive Activity During Vacuum

2012-11-02 Thread Alban Hertroys
On 1 Nov 2012, at 17:44, Shaun Thomas wrote: > On 11/01/2012 11:40 AM, Alban Hertroys wrote: > >> Instead of attempting to postpone freeze until beyond the life >> expectancy of our universe, what you probably should have done is >> vacuum more often so that vacuum has less work to do. > > More

Re: [GENERAL] Unexplained Major Vacuum Archive Activity During Vacuum

2012-11-01 Thread Shaun Thomas
On 11/01/2012 11:40 AM, Alban Hertroys wrote: Instead of attempting to postpone freeze until beyond the life expectancy of our universe, what you probably should have done is vacuum more often so that vacuum has less work to do. More often than every night, with autovacuum running in the backg

Re: [GENERAL] Unexplained Major Vacuum Archive Activity During Vacuum

2012-11-01 Thread Alban Hertroys
On 1 November 2012 17:19, Shaun Thomas wrote: > On 11/01/2012 10:28 AM, Kevin Grittner wrote: > Based on my past experience with 8.2, and my understanding of 9.1, I > moved autovacuum_freeze_max_age up to 650M so we'd never get a mid-day > freeze. And the default for vacuum_freeze_table_age is 150

Re: [GENERAL] Unexplained Major Vacuum Archive Activity During Vacuum

2012-11-01 Thread Shaun Thomas
On 11/01/2012 10:28 AM, Kevin Grittner wrote: http://www.postgresql.org/docs/9.2/interactive/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND I read that several times, and I still don't get how it applies to this case. Based on my past experience with 8.2, and my understanding of 9.1, I moved a

Re: [GENERAL] Unexplained Major Vacuum Archive Activity During Vacuum

2012-11-01 Thread Kevin Grittner
Shaun Thomas wrote: > Ok, that might explain it, then. We did in fact just upgrade from 8.2 to > 9.1 about 2 weeks ago. And no, I didn't do a VACUUM FREEZE, just a > VACUUM ANALYZE to make sure stats were ready. I'm still a little > uncertain what the tangible difference is between a FREEZE and

Re: [GENERAL] Unexplained Major Vacuum Archive Activity During Vacuum

2012-11-01 Thread Shaun Thomas
On 11/01/2012 09:18 AM, Kevin Grittner wrote: Did you bulk load this data (possibly through restoring pg_dump output)? If so, and you have not explicitly run VACUUM FREEZE afterward, the vacuum noticed that it was time to freeze all of these tuples. Ok, that might explain it, then. We did in f

Re: [GENERAL] Unexplained Major Vacuum Archive Activity During Vacuum

2012-11-01 Thread Kevin Grittner
Shaun Thomas wrote: > I don't notice any errors, which just makes this even more strange. > But after weeks of operating normally, our 10pm manual vacuum job > generated transaction logs basically equivalent to 3/4 of our > database, and I can't find any explanation. This amount is about 6x > high

[GENERAL] Unexplained Major Vacuum Archive Activity During Vacuum

2012-11-01 Thread Shaun Thomas
Hey guys, I don't notice any errors, which just makes this even more strange. But after weeks of operating normally, our 10pm manual vacuum job generated transaction logs basically equivalent to 3/4 of our database, and I can't find any explanation. This amount is about 6x higher than usual.