Re: [PERFORM] query memory consumption

2009-09-25 Thread Jeff Janes
2009/9/22 Grzegorz Jaśkiewicz : > On Tue, Sep 22, 2009 at 1:46 PM, Alan McKay wrote: >>> Best practice to avoid that, is to bump the work_mem temporarily >>> before the query, and than lower it again, lowers the chance of memory >>> exhaustion. >> >> Interesting - I can do that dynamically? > > yo

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-25 Thread Jeff Janes
On Fri, Sep 25, 2009 at 8:53 AM, Scott Carey wrote: > That won't work well anyway because the postgres shared_buffers dos not cache > things that are sequentially scanned (it uses a ring buffer for each scan).   > So, for > any data that is only accessed by sequential scan, you're relying on the O

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-25 Thread Aidan Van Dyk
* Scott Carey [090925 11:57]: > That won't work well anyway because the postgres shared_buffers dos not cache > things that are sequentially scanned (it uses a ring buffer for each scan). > So, for any data that is only accessed by sequential scan, you're relying on > the OS and the disks. If

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-25 Thread Scott Carey
That won't work well anyway because the postgres shared_buffers dos not cache things that are sequentially scanned (it uses a ring buffer for each scan). So, for any data that is only accessed by sequential scan, you're relying on the OS and the disks. If you access a table via index scan thou

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Craig James
Gerhard Wiesinger wrote: Hello Craig, Are you sure this is correct? The test program (see below) with autocommit=0 counts up when an insert is done in another session and there is no commit done. I think with each new select a new implicit transaction is done when no explicit "BEGIN" has be

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-25 Thread Aidan Van Dyk
* Dan Sugalski [090925 06:06]: > I'll have to go check, but I think it does. This box hasn't actually hit > swap since it started -- a good chunk of that RAM is used as > semi-permanent disk cache but unfortunately the regular day-to-day use of > this box (they won't let me have it as a dedi

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Gerhard Wiesinger
Hello Craig, Are you sure this is correct? The test program (see below) with autocommit=0 counts up when an insert is done in another session and there is no commit done. I think with each new select a new implicit transaction is done when no explicit "BEGIN" has been established. Can one

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Shiva Raman
As suggested, i had changed the log_statement='ddl' and now it is logging only the ddl statements . thanks for the tip. Can i delete the old log files in pg_log after backing up as zip archive ? is it neccesary to keep those log files ? Regards Shiva Raman > > 2009/9/25 Grzegorz Jaśkiewicz > >

Re: [PERFORM] PG 8.3 and large shared buffer settings

2009-09-25 Thread Dan Sugalski
At 12:36 AM -0400 9/25/09, Tom Lane wrote: Dan Sugalski writes: Is there any practical limit to the number of shared buffers PG 8.3.7 can handle before more becomes counter-productive? Probably, but I've not heard any definitive measurements showing an upper limit. The traditional wisdom o

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Grzegorz Jaśkiewicz
2009/9/25 Shiva Raman > As suggested, i had changed the log_statement='ddl' and now it is logging > only > the ddl statements . thanks for the tip. > Can i delete the old log files in pg_log after backing up as zip archive ? > is it neccesary to keep those log files ? > they're yours, you can d

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Grzegorz Jaśkiewicz
On Fri, Sep 25, 2009 at 9:06 AM, Shiva Raman wrote: > Hi Gerhard > I also found the pg_log has 73 G of data . > > clusternode2:/var/lib/pgsql/data # du -sh pg_log/ > 73G pg_log/ > > Is it necessary to keep this Log files? Can i backup the logs and delete it > from the original directory ? Is

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Shiva Raman
Hi Gerhard I also found the pg_log has 73 G of data . clusternode2:/var/lib/pgsql/data # du -sh pg_log/ 73G pg_log/ Is it necessary to keep this Log files? Can i backup the logs and delete it from the original directory ? Is this logs files necessary in case any data recovery to be done ? I

Re: [PERFORM] High CPU load on Postgres Server during Peak times!!!!

2009-09-25 Thread Shiva Raman
Hi Gerhard Thanks for the mail On Thu, Sep 24, 2009 at 7:19 PM, Gerhard Wiesinger wrote: > Hello Shiva, > > What I see from top (0.0%wa) you don't have any I/O problem but a major CPU > problem. But this is contrast to iostat where up to 50% of iowait is there > (sometimes). > > I think you ha