Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13)

2007-05-27 Thread Jim C. Nasby
On Fri, May 25, 2007 at 10:29:30AM +0200, Bastian Voigt wrote: > Hi *, > for caching large autogenerated XML files, I have created a bytea table > in my database so that the cached files can be used by multiple servers. > There are about 500 rows and 10-20 Updates per minute on the table. The >

Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13)

2007-05-25 Thread Bastian Voigt
OK, I'll give that a try. What about pg_autovacuum then? Is it a problem when two processes try to vacuum the same table in parallel? Or do I need to deactivate autovacuum altogether? Try vacuuming every 3 minutes and see what happens. (Sorry Richard, forgot to reply to the list!) -- Bastia

Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13)

2007-05-25 Thread Tom Lane
Bastian Voigt <[EMAIL PROTECTED]> writes: > Now my big big problem is that the database gets really really slow > during these 20 minutes and after the vacuum process is running for a > short time, many transactions show state "UPDATE waiting" in the process > list. In my Java application server

Re: My quick and dirty "solution" (Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13))

2007-05-25 Thread Bastian Voigt
Richard Huxton wrote: Could you check the output of vacuum verbose on that table and see how much work it's doing? I'd have thought the actual bytea data would be TOASTed away to a separate table for storage, leaving the vacuum with very little work to do. I'm quite new to postgres (actually I

Re: My quick and dirty "solution" (Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13))

2007-05-25 Thread Bastian Voigt
Kristo Kaiv wrote: you should first cluster the table on primary key. The table is probably already bloated from the 3 hr delay it had before. First CLUSTER "primary key index name" ON group_fin_account_tst; Then vacuum it every 3 minutes. NB! clustering takes an access exclusive lock on table K

Re: My quick and dirty "solution" (Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13))

2007-05-25 Thread Alvaro Herrera
Bastian Voigt wrote: > No, this did not help. The vacuum process is still running far too long > and makes everything slow. It is even worse than before, cause now the > system is slow almost all the time while when vacuuming only every 3 > hours it is only slow once every three hours. > >

Re: My quick and dirty "solution" (Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13))

2007-05-25 Thread Kristo Kaiv
you should first cluster the table on primary key. The table is probably already bloated from the 3 hr delay it had before. First CLUSTER "primary key index name" ON group_fin_account_tst; Then vacuum it every 3 minutes. NB! clustering takes an access exclusive lock on table Kristo On 25.05.2007

Re: My quick and dirty "solution" (Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13))

2007-05-25 Thread Richard Huxton
Bastian Voigt wrote: No, this did not help. The vacuum process is still running far too long and makes everything slow. It is even worse than before, cause now the system is slow almost all the time while when vacuuming only every 3 hours it is only slow once every three hours. Could you

My quick and dirty "solution" (Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13))

2007-05-25 Thread Bastian Voigt
No, this did not help. The vacuum process is still running far too long and makes everything slow. It is even worse than before, cause now the system is slow almost all the time while when vacuuming only every 3 hours it is only slow once every three hours. I now did the following. Well,

Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13)

2007-05-25 Thread Richard Huxton
Bastian Voigt wrote: Richard Huxton wrote: I was about to say that you can tune pg_autovacuum, but I just checked your original post and you're running 8.0.x - not sure about that one. The system catalog pg_autovacuum which allows finetuning autovacuum at table level was introduced in 8.1 :-(

Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13)

2007-05-25 Thread Bastian Voigt
Richard Huxton wrote: I was about to say that you can tune pg_autovacuum, but I just checked your original post and you're running 8.0.x - not sure about that one. The system catalog pg_autovacuum which allows finetuning autovacuum at table level was introduced in 8.1 :-( You'll have to chec

Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13)

2007-05-25 Thread Richard Huxton
Bastian Voigt wrote: OK, I'll give that a try. What about pg_autovacuum then? Is it a problem when two processes try to vacuum the same table in parallel? Or do I need to deactivate autovacuum altogether? I was about to say that you can tune pg_autovacuum, but I just checked your original pos

Re: [PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13)

2007-05-25 Thread Richard Huxton
Bastian Voigt wrote: Hi *, for caching large autogenerated XML files, I have created a bytea table in my database so that the cached files can be used by multiple servers. There are about 500 rows and 10-20 Updates per minute on the table. The files stored in the bytea are anything from 10kB t

[PERFORM] Performance Problem with Vacuum of bytea table (PG 8.0.13)

2007-05-25 Thread Bastian Voigt
Hi *, for caching large autogenerated XML files, I have created a bytea table in my database so that the cached files can be used by multiple servers. There are about 500 rows and 10-20 Updates per minute on the table. The files stored in the bytea are anything from 10kB to 10MB. My PostgreSQL