Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-04 Thread Kevin Grittner
"Scott Otis" wrote: > So is there anything I can do in 8.3 to help this? I have tried > setting 'track_activities', 'track_counts' and 'autovacuum' to 'off' > (which has reduced CPU and I/O a bit) You're going to regret that very soon, unless you are *very* sure you have adequate manual vacuu

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-04 Thread Scott Otis
nce@postgresql.org Subject: Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3 On Fri, Sep 4, 2009 at 08:54, Scott Otis wrote: > Robert Haas : > >> Do you by any chance have a bazillion databases in this cluster?  Can >> you do these? > >> select sum

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-04 Thread Magnus Hagander
On Fri, Sep 4, 2009 at 08:54, Scott Otis wrote: > Robert Haas : > >> Do you by any chance have a bazillion databases in this cluster?  Can >> you do these? > >> select sum(1) from pg_database; > > 1555 Note that there are two features in 8.4 specifically designed to deal with the situation where y

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Claus Guttesen
>> > max_fsm_pages = 280 >> > max_fsm_relations = 16 > >> What does the last couple of lines from a 'vacuum analyze verbose' >> say? I have max_fsm_pages = 400 and max_fsm_relations = 1500. > >> You can also try to lower random_page_cost to a lower value like 1.2 >> but I doubt this wil

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Scott Otis
Robert Haas : > Do you by any chance have a bazillion databases in this cluster? Can > you do these? > select sum(1) from pg_database; 1555 > select pg_relation_size('pg_database'); 221184 > select sum(pg_column_size(d.*)) from pg_database; That gave me: ERROR: missing FROM-clause e

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Scott Otis
Claus Guttesen kome...@gmail.com: > > Would love to get some advice on how to change my conf settings / setup to > > get better I/O performance. > > > > Server Specs: > > > > 2x Intel Xeon Quad Core (@2 Ghz - Clovertown,L5335) > > 4GB RAM > > 4x Seagate 73GB SAS HDD 10k RPM - in RAID ( stripped an

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Robert Haas
On Thu, Sep 3, 2009 at 7:11 PM, Scott Otis wrote: >> Can you post to the list all the uncommented lines from your > postgresql.conf file and attach the results of "select * from > pg_stat_all_tables" as an attachment? > > I attached a CSV of "select * from pg_stat_all_tables" from one of our > more

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Scott Otis
> Can you post to the list all the uncommented lines from your postgresql.conf file and attach the results of "select * from pg_stat_all_tables" as an attachment? I attached a CSV of "select * from pg_stat_all_tables" from one of our more heavily used databases. Note: I turned off stats collectio

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Scott Otis
> Simply do "iostat mfid0 1" and post 10 lines of its output. tty mfid0 cpu tin tout KB/t tps MB/s us ni sy in id 0 152 108.54 335 35.51 43 0 30 1 27 0 525 85.73 759 63.55 14 0 12 0 74 0 86 67.72 520 34.39 13 0 12 0 75 0 86 86.89 746 63.26

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Ivan Voras
2009/9/3 Scott Otis : > Sorry about not responding to the whole list earlier - this is my first time > posting to a mailing list. > > Would providing more information about the size and complexities of the > databases help? > > I measure I/O stats with iostat - here is the command I use: > > iost

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Robert Haas
On Thu, Sep 3, 2009 at 5:27 PM, Andy Colson wrote: > Robert Haas wrote: >> >> On Thu, Sep 3, 2009 at 4:16 PM, Scott Otis wrote: >>> >>> Sorry about not responding to the whole list earlier - this is my first >>> time posting to a mailing list. >>> >>> Would providing more information about the size

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Andy Colson
Robert Haas wrote: On Thu, Sep 3, 2009 at 4:16 PM, Scott Otis wrote: Sorry about not responding to the whole list earlier - this is my first time posting to a mailing list. Would providing more information about the size and complexities of the databases help? I measure I/O stats with iostat

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Robert Haas
On Thu, Sep 3, 2009 at 4:16 PM, Scott Otis wrote: > Sorry about not responding to the whole list earlier - this is my first time > posting to a mailing list. > > Would providing more information about the size and complexities of the > databases help? > > I measure I/O stats with iostat - here is

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Andy Colson
Scott Otis wrote: Sorry about not responding to the whole list earlier - this is my first time posting to a mailing list. Would providing more information about the size and complexities of the databases help? I measure I/O stats with iostat - here is the command I use: iostat -d -x mfid0 -t

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Scott Otis
Sorry about not responding to the whole list earlier - this is my first time posting to a mailing list. Would providing more information about the size and complexities of the databases help? I measure I/O stats with iostat - here is the command I use: iostat -d -x mfid0 -t 290 2 I tried look

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Andy Colson
Scott Otis wrote: I agree that they don't make sense - part of the reason I am looking for help :) I am using iostat to get those numbers ( which I specify to average over 5 min then collect to display in Cacti ). 2 processes are taking up a good deal of CPU - the postgres stats collector a

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Andy Colson
Scott Otis wrote: Would love to get some advice on how to change my conf settings / setup to get better I/O performance. Total I/O (these number are pretty constant throughout the day): Reads: ~ 100 / sec for about 2.6 Mb/sec Writes: ~ 400 /sec for about 46.1Mb/sec Most of the SQL happening i

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Ivan Voras
Scott Otis wrote: > 2x Intel Xeon Quad Core (@2 Ghz - Clovertown,L5335) > 4GB RAM > 4x Seagate 73GB SAS HDD 10k RPM – in RAID ( stripped and mirrored ) Would love to get some advice on how to change my conf settings / setup to get better I/O performance. ~1500 databases w/ ~60 tables each

[PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-02 Thread Scott Otis
Would love to get some advice on how to change my conf settings / setup to get better I/O performance. Server Specs: 2x Intel Xeon Quad Core (@2 Ghz - Clovertown,L5335) 4GB RAM 4x Seagate 73GB SAS HDD 10k RPM - in RAID ( stripped and mirrored ) FreeBSD 6.4 Apache 2.2 PostgreSQL 8.3.