Re: [PERFORM] Tuning for mid-size server

2003-12-14 Thread Andrew Sullivan
On Sun, Dec 14, 2003 at 12:42:21AM -0500, Bruce Momjian wrote: > > I know this is an old email, but have you tested larger shared buffers > in CVS HEAD with Jan's new cache replacement policy? Not yet. It's on our TODO list, for sure, because the consequences of relying too much on the filesyste

Re: [PERFORM] Tuning for mid-size server

2003-12-13 Thread Bruce Momjian
Andrew Sullivan wrote: > On Tue, Oct 21, 2003 at 03:11:17PM -0600, scott.marlowe wrote: > > I think where it makes sense is when you have something like a report > > server where the result sets may be huge, but the parellel load is load, > > i.e. 5 or 10 users tossing around 100 Meg or more at t

Re: [PERFORM] Tuning for mid-size server

2003-10-24 Thread William Yu
Anjan Dave wrote: Shared_buffers (25% of RAM / 8KB)) = 8589934592 * .25 / 8192 = 262144 250,000 is probably the max you can use due to the 2GB process limit unless you recompile the Linux Kernel to use 3GB process/1GB kernel. Yes, I've got 8GB also and I started at 262144 and kept working my way

Re: [PERFORM] Tuning for mid-size server

2003-10-24 Thread William Yu
So what is the ceiling on 32-bit processors for RAM? Most of the 64-bit vendors are pushing Athalon64 and G5 as "breaking the 4GB barrier", and even I can do the math on 2^32. All these 64-bit vendors, then, are talking about the limit on ram *per application* and not per machine? 64-bit CPU o

Re: [PERFORM] Tuning for mid-size server

2003-10-22 Thread Ron Johnson
On Tue, 2003-10-21 at 14:27, Christopher Browne wrote: > In the last exciting episode, [EMAIL PROTECTED] (Josh Berkus) wrote: > > So what is the ceiling on 32-bit processors for RAM? Most of the > > 64-bit vendors are pushing Athalon64 and G5 as "breaking the 4GB > > barrier", and even I can do the

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Josh Berkus
Scott, > I think where it makes sense is when you have something like a report > server where the result sets may be huge, but the parellel load is load, > i.e. 5 or 10 users tossing around 100 Meg or more at time. I've found that that question makes the difference between using 6% & 12% ... p

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Andrew Sullivan
On Tue, Oct 21, 2003 at 03:11:17PM -0600, scott.marlowe wrote: > I think where it makes sense is when you have something like a report > server where the result sets may be huge, but the parellel load is load, > i.e. 5 or 10 users tossing around 100 Meg or more at time. In our case, we were noti

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread scott.marlowe
On Tue, 21 Oct 2003, Andrew Sullivan wrote: > On Tue, Oct 21, 2003 at 11:51:02AM -0700, Josh Berkus wrote: > > > Of course, if you have anecdotal evidence to the contrary, then the > > only way to work this would be to have OSDL help us sort it out. > > Nope. I too have such anecdotal evidence

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Andrew Sullivan
On Tue, Oct 21, 2003 at 11:51:02AM -0700, Josh Berkus wrote: > Of course, if you have anecdotal evidence to the contrary, then the > only way to work this would be to have OSDL help us sort it out. Nope. I too have such anecdotal evidence that 25% is way too high. It also seems to depend pretty

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Anjan Dave
] Sent: Tue 10/21/2003 1:33 PM To: Josh Berkus Cc: Anjan Dave; Richard Huxton; [EMAIL PROTECTED] Subject: Re: [PERFORM] Tuning for mid-size server On Tue, 21 Oct 2003, Josh Berkus wrote: > An

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Anjan Dave
] Sent: Tue 10/21/2003 1:22 PM To: Anjan Dave; Richard Huxton; [EMAIL PROTECTED] Cc: Subject: Re: [PERFORM] Tuning for mid-size server Anjan, > From what I know, there is a cache-row-set functionality that doesn't

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Christopher Browne
In the last exciting episode, [EMAIL PROTECTED] (Josh Berkus) wrote: > So what is the ceiling on 32-bit processors for RAM? Most of the > 64-bit vendors are pushing Athalon64 and G5 as "breaking the 4GB > barrier", and even I can do the math on 2^32. All these 64-bit > vendors, then, are talking a

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Tom Lane
Andrew Sullivan <[EMAIL PROTECTED]> writes: > On Tue, Oct 21, 2003 at 10:12:15AM -0700, Josh Berkus wrote: >> So what is the ceiling on 32-bit processors for RAM? > ... Remember that, back in the old days on the > pre-386s, accessing the extended or expanded memory (anyone remember > which was whi

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Josh Berkus
Andrew, > I think the "25%" rule of thumb is slightly stale: above some > threshold, it just falls apart, and lots of people now have machines > well within that threshold. Heck, I'll bet Bruce's 2-way machine is > within that threshold. Sure. But we had a few people on this list do tests (incl

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Tom Lane
Andrew Sullivan <[EMAIL PROTECTED]> writes: > I think the "25%" rule of thumb is slightly stale: above some > threshold, it just falls apart, and lots of people now have machines > well within that threshold. Heck, I'll bet Bruce's 2-way machine is > within that threshold. IIRC, we've not seen mu

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Jeff
On Tue, 21 Oct 2003 10:12:15 -0700 Josh Berkus <[EMAIL PROTECTED]> wrote: > So what is the ceiling on 32-bit processors for RAM? Most of the > 64-bit vendors are pushing Athalon64 and G5 as "breaking the 4GB > barrier", and even I can do the math on 2^32. All these 64-bit > vendors, then, are ta

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Andrew Sullivan
On Tue, Oct 21, 2003 at 10:15:57AM -0700, Josh Berkus wrote: > Anjan, > > > I read an article that suggests you 'start' with 25% of memory for > > shared_buffers. Sort memory was suggested to be at 2-4%. Here's the > > link: > > http://www.ca.postgresql.org/docs/momjian/hw_performance/node8.html >

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Andrew Sullivan
On Tue, Oct 21, 2003 at 10:12:15AM -0700, Josh Berkus wrote: > > So what is the ceiling on 32-bit processors for RAM? Most of the 64-bit > vendors are pushing Athalon64 and G5 as "breaking the 4GB barrier", and even > I can do the math on 2^32. All these 64-bit vendors, then, are talking > ab

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread scott.marlowe
On Tue, 21 Oct 2003, Josh Berkus wrote: > Anjan, > > > From what I know, there is a cache-row-set functionality that doesn't > > exist with the newer postgres... > > What? PostgreSQL has always used the kernel cache for queries. > > > Concurrent users will start from 1 to a high of 5000 or mor

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread scott.marlowe
On Tue, 21 Oct 2003, Josh Berkus wrote: > Scott, > > > Also, if it's a read only environment, RAID5 with n drives equals the > > performance of RAID0 with n-1 drives. > > True. > > > Josh, you gotta get out more. IA32 has supported >4 gig ram for a long > > time now, and so has the linux kerne

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Josh Berkus
Anjan, > From what I know, there is a cache-row-set functionality that doesn't > exist with the newer postgres... What? PostgreSQL has always used the kernel cache for queries. > Concurrent users will start from 1 to a high of 5000 or more, and could > ramp up rapidly. So far, with increased us

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Josh Berkus
Anjan, > I read an article that suggests you 'start' with 25% of memory for > shared_buffers. Sort memory was suggested to be at 2-4%. Here's the > link: > http://www.ca.postgresql.org/docs/momjian/hw_performance/node8.html > Maybe, I misinterpreted it. No, I can see how you arrived at that concl

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Josh Berkus
Scott, > Also, if it's a read only environment, RAID5 with n drives equals the > performance of RAID0 with n-1 drives. True. > Josh, you gotta get out more. IA32 has supported >4 gig ram for a long > time now, and so has the linux kernel. It uses a paging method to do it. > Individual processe

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Anjan Dave
se lateron if needed. Thanks, Anjan -Original Message- From: Josh Berkus [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 12:21 PM To: Anjan Dave; [EMAIL PROTECTED] Subject: Re: [PERFORM] Tuning for mid-size server Anjan, > Pretty soon, a PowerEdge 6650 with 4 x 2Gh

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Anjan Dave
n and max (recommended by Sun) - on the app side. Thanks, Anjan -Original Message- From: Richard Huxton [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2003 11:57 AM To: Anjan Dave; [EMAIL PROTECTED] Subject: Re: [PERFORM] Tuning for mid-size server On Tuesday 21 October 2003

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread scott.marlowe
On Tue, 21 Oct 2003, Josh Berkus wrote: > Anjan, > > > Pretty soon, a PowerEdge 6650 with 4 x 2Ghz XEONs, and 8GB Memory, with > > internal drives on RAID5 will be delivered. Postgres will be from RH8.0. > > How many drives? RAID5 sucks for heavy read-write databases, unless you have > 5+ dri

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Josh Berkus
Anjan, > Pretty soon, a PowerEdge 6650 with 4 x 2Ghz XEONs, and 8GB Memory, with > internal drives on RAID5 will be delivered. Postgres will be from RH8.0. How many drives? RAID5 sucks for heavy read-write databases, unless you have 5+ drives. Or a large battery-backed cache. Also, last I ch

Re: [PERFORM] Tuning for mid-size server

2003-10-21 Thread Richard Huxton
On Tuesday 21 October 2003 15:28, Anjan Dave wrote: > Hi, > > Pretty soon, a PowerEdge 6650 with 4 x 2Ghz XEONs, and 8GB Memory, with > internal drives on RAID5 will be delivered. Postgres will be from RH8.0. You'll want to upgrade PG to v7.3.4 > I am planning for these values for the postgres co

[PERFORM] Tuning for mid-size server

2003-10-21 Thread Anjan Dave
Title: Tuning for mid-size server Hi, Pretty soon, a PowerEdge 6650 with 4 x 2Ghz XEONs, and 8GB Memory, with internal drives on RAID5 will be delivered. Postgres will be from RH8.0. I am planning for these values for the postgres configuration - to begin with: Shared_buffers (25% of RAM