Re: [PERFORM] help tuning queries on large database

2006-01-10 Thread Ron
At 07:28 PM 1/10/2006, Mark Lewis wrote: Ron, A few days back you mentioned: > Upgrade your kernel to at least 2.6.12 > There's a known issue with earlier versions of the 2.6.x kernel and > 64b CPUs like the Opteron. See kernel.org for details. > I did some searching and couldn't find any ob

Re: [PERFORM] help tuning queries on large database

2006-01-10 Thread Mark Lewis
Ron, A few days back you mentioned: > Upgrade your kernel to at least 2.6.12 > There's a known issue with earlier versions of the 2.6.x kernel and > 64b CPUs like the Opteron. See kernel.org for details. > I did some searching and couldn't find any obvious mention of this issue (I gave up af

Re: [PERFORM] help tuning queries on large database

2006-01-10 Thread Ron
At 12:23 PM 1/9/2006, peter royal wrote: On Jan 8, 2006, at 4:35 PM, Ron wrote: Areca ARC-1220 8-port PCI-E controller Make sure you have 1GB or 2GB of cache. Get the battery backup and set the cache for write back rather than write through. The card we've got doesn't have a SODIMM socket,

Re: [PERFORM] help tuning queries on large database

2006-01-09 Thread Luke Lonergan
Peter, On 1/9/06 12:59 PM, "peter royal" <[EMAIL PROTECTED]> wrote: > Overall, I got a 50% boost in the overall speed of my test suite by > using XFS and the 16k read-ahead. Yes, it all looks pretty good for your config, though it looks like you might be adapter limited with the Areca - you sho

Re: [PERFORM] help tuning queries on large database

2006-01-09 Thread peter royal
On Jan 9, 2006, at 2:01 PM, Luke Lonergan wrote: Peter, On 1/9/06 9:23 AM, "peter royal" <[EMAIL PROTECTED]> wrote: This is a 2-disk RAID0 Your 2-disk results look fine - what about your 8-disk results? after some further research the 2-disk RAID0 numbers are not bad. I have a single driv

Re: [PERFORM] help tuning queries on large database

2006-01-09 Thread Luke Lonergan
Peter, On 1/9/06 9:23 AM, "peter royal" <[EMAIL PROTECTED]> wrote: > This is a 2-disk RAID0 Your 2-disk results look fine - what about your 8-disk results? Given that you want to run in production with RAID10, the most you should expect is 2x the 2-disk results using all 8 of your disks. If yo

Re: [PERFORM] help tuning queries on large database

2006-01-09 Thread peter royal
On Jan 8, 2006, at 1:42 PM, Luke Lonergan wrote: Have you tested the underlying filesystem for it's performance? Run this: time bash -c 'dd if=/dev/zero of=/my_file_system/bigfile bs=8k count= && sync' This is a 2-disk RAID0 [EMAIL PROTECTED] /opt/alt-2]# time bash -c 'dd if=/dev/zero of=

Re: [PERFORM] help tuning queries on large database

2006-01-09 Thread Harry Jackson
On 1/9/06, Kelly Burkhart <[EMAIL PROTECTED]> wrote: > On 1/8/06, Ron <[EMAIL PROTECTED]> wrote: > > > > Among the other tricks having lots of RAM allows: > > If some of your tables are Read Only or VERY rarely written to, you > > can preload them at boot time and make them RAM resident using the

Re: [PERFORM] help tuning queries on large database

2006-01-09 Thread Kelly Burkhart
On 1/8/06, Ron <[EMAIL PROTECTED]> wrote: Among the other tricks having lots of RAM allows:If some of your tables are Read Only or VERY rarely written to, youcan preload them at boot time and make them RAM resident using the/etc/tmpfs trick. What is the /etc/tmpfs trick? -K

Re: [PERFORM] help tuning queries on large database

2006-01-08 Thread Ron
I'll second all of Luke Lonergan's comments and add these. You should be able to increase both "cold" and "warm" performance (as well as data integrity. read below.) considerably. Ron At 05:59 PM 1/6/2006, peter royal wrote: Howdy. I'm running into scaling problems when testing with a 16gb

Re: [PERFORM] help tuning queries on large database

2006-01-08 Thread Luke Lonergan
Peter, On 1/6/06 2:59 PM, "peter royal" <[EMAIL PROTECTED]> wrote: > I have experimented with having all 8 disks in a single RAID0 set, a > single RAID10 set, and currently 4 RAID0 sets of 2 disks each. There > hasn't been an appreciable difference in the overall performance of > my test suite (w

Re: [PERFORM] help tuning queries on large database

2006-01-06 Thread David Lang
On Fri, 6 Jan 2006, Tom Lane wrote: Date: Fri, 06 Jan 2006 18:47:55 -0500 From: Tom Lane <[EMAIL PROTECTED]> To: peter royal <[EMAIL PROTECTED]> Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] help tuning queries on large database peter royal <[EMAIL PROTECTED]&g

Re: [PERFORM] help tuning queries on large database

2006-01-06 Thread Harry Jackson
On 1/6/06, peter royal <[EMAIL PROTECTED]> wrote: > PostgreSQL 8.1.1 > > shared_buffers = 1 # (It was higher, 50k, but didn't help any, > so brought down to free ram for disk cache) > work_mem = 8196 > random_page_cost = 3 > effective_cache_size = 25 I have played with both disk cache set

Re: [PERFORM] help tuning queries on large database

2006-01-06 Thread Tom Lane
peter royal <[EMAIL PROTECTED]> writes: > So, my question is, is there anything I can do to boost performance > with what I've got, or am I in a position where the only 'fix' is > more faster disks? I can't think of any schema/index changes that > would help, since everything looks pretty opt

[PERFORM] help tuning queries on large database

2006-01-06 Thread peter royal
Howdy. I'm running into scaling problems when testing with a 16gb (data +indexes) database. I can run a query, and it returns in a few seconds. If I run it again, it returns in a few milliseconds. I realize this is because during subsequent runs, the necessary disk pages have been cached b