Re: [PERFORM] Database-wide vacuum can take a long time, during which

2006-11-01 Thread Matthew O'Connor
Steven Flatt wrote: Here is a potential problem with the auto-vacuum daemon, and I'm wondering if anyone has considered this. To avoid transaction ID wraparound, the auto-vacuum daemon will periodically determine that it needs to do a DB-wide vacuum, which takes a long time. On our system, i

[PERFORM] Database-wide vacuum can take a long time, during which tables are not being analyzed

2006-11-01 Thread Steven Flatt
Here is a potential problem with the auto-vacuum daemon, and I'm wondering if anyone has considered this.  To avoid transaction ID wraparound, the auto-vacuum daemon will periodically determine that it needs to do a DB-wide vacuum, which takes a long time.  On our system, it is on the order of a co

Re: [PERFORM] big transaction slows down over time - but disk seems

2006-11-01 Thread Richard Huxton
Ben wrote: On Wed, 1 Nov 2006, Richard Huxton wrote: 1. Avoid updating the same timestamp more than once (if that's happening) Each row is updated at most once, and not all rows are updated. 2. Update timestamps in one go at the end of the transaction (perhaps by loading updates into a te

Re: [PERFORM] big transaction slows down over time - but disk seems

2006-11-01 Thread Ben
On Wed, 1 Nov 2006, Richard Huxton wrote: 1. Avoid updating the same timestamp more than once (if that's happening) Each row is updated at most once, and not all rows are updated. 2. Update timestamps in one go at the end of the transaction (perhaps by loading updates into a temp table).

Re: [PERFORM] big transaction slows down over time - but disk seems almost unused

2006-11-01 Thread Ben
My transaction calls the same stored procedure many times over. Over the lifetime of the transaction, that stored procedure slows down by roughly 2 orders of magnitude. The procedure itself tries to look up several strings in dictionary tables, and if the strings aren't there (most of them

Re: [PERFORM] big transaction slows down over time - but disk seems

2006-11-01 Thread Richard Huxton
Ben wrote: My transaction calls the same stored procedure many times over. Over the lifetime of the transaction, that stored procedure slows down by roughly 2 orders of magnitude. The procedure itself tries to look up several strings in dictionary tables, and if the strings aren't there (most o

Re: [PERFORM] MVCC & indexes?

2006-11-01 Thread Ivan Voras
Alvaro Herrera wrote: > Well, there certainly is locking involved in inserting index entries, > but it's more fine-grained than you seem to think. Only one page of the > index is locked at any time, resulting in that typically there's very > little blocking involved. Two processes can be inserti

Re: [PERFORM] big transaction slows down over time - but disk

2006-11-01 Thread Andreas Kostyrka
Am Dienstag, den 31.10.2006, 21:58 -0800 schrieb Ben: > I've got a long-running, update-heavy transaction that increasingly slows > down the longer it runs. I would expect that behavior, if there was some > temp file creation going on. But monitoring vmstat over the life of the > transaction sho

Re: [PERFORM] big transaction slows down over time - but disk seems almost unused

2006-11-01 Thread Ben
Memory usage remains consistent, which is to say that postgres is using most available system memory all the time, as I configured it to. There is no swapping going on. It's not clear to me why forcing a WAL checkpoint would help anything but it doesn't matter, as only superusers can do

Re: [PERFORM] big transaction slows down over time - but disk seems

2006-11-01 Thread Heikki Linnakangas
Ben wrote: I've got a long-running, update-heavy transaction that increasingly slows down the longer it runs. I would expect that behavior, if there was some temp file creation going on. But monitoring vmstat over the life of the transaction shows virtually zero disk activity. Instead, the sys