Re: [PERFORM] Background writer underemphasized ...

2008-04-22 Thread Greg Smith
On Sun, 20 Apr 2008, James Mansion wrote: Are you suggesting that the disk subsystem has already decided on its strategy for a set of seeks and writes and will not insert new instructions into an existing elevator plan until it is completed and it looks at the new requests? No, just that eac

Re: [PERFORM] Background writer underemphasized ...

2008-04-20 Thread James Mansion
Greg Smith wrote: If you write a giant block of writes, those tend to be sorted by the OS and possibly the controller to reduce total seeks. That's a pretty efficient write and it can clear relatively fast. But if you're been trickling writes in an unstructured form and in low volume, there

Re: [PERFORM] Background writer underemphasized ...

2008-04-19 Thread Greg Smith
On Sat, 19 Apr 2008, James Mansion wrote: But isn't it the case that while using background writer might result in *slightly* more data to write (since data that is updated several times might actually be sent several times), the total amount of data in both cases is much the same? Really de

Re: [PERFORM] Background writer underemphasized ...

2008-04-19 Thread James Mansion
Greg Smith wrote: Using the background writer more assures that the cache on the controller is going to be written to aggressively, so it may be somewhat filled already come checkpoint time. If you leave the writer off, when the checkpoint comes you're much more likely to have the full 2GB av

Re: [PERFORM] Background writer underemphasized ...

2008-04-19 Thread Greg Smith
On Thu, 17 Apr 2008, Marinos Yannikos wrote: Controller is http://www.infortrend.com/main/2_product/es_a08(12)f-g2422.asp with 2GB cache (writeback was enabled). Ah. Sometimes these fiber channel controllers can get a little weird (compared with more direct storage) when the cache gets comp

Re: [PERFORM] Background writer underemphasized ...

2008-04-17 Thread Marinos Yannikos
Greg Smith schrieb: You also didn't mention what disk controller you have, or how much write cache it has (if any). 8.3.1, Controller is http://www.infortrend.com/main/2_product/es_a08(12)f-g2422.asp with 2GB cache (writeback was enabled). That's almost turning the background writer off. I

Re: [PERFORM] Background writer underemphasized ...

2008-04-16 Thread Bill Moran
In response to Greg Smith <[EMAIL PROTECTED]>: > On Wed, 16 Apr 2008, Bill Moran wrote: > > >> bgwriter_delay = 1ms # 10-1ms between rounds > >> bgwriter_lru_maxpages = 1000 # 0-1000 max buffers written/round > > Have you watched closely under load to ensure that you're not se

Re: [PERFORM] Background writer underemphasized ...

2008-04-16 Thread Greg Smith
On Wed, 16 Apr 2008, Bill Moran wrote: bgwriter_delay = 1ms # 10-1ms between rounds bgwriter_lru_maxpages = 1000 # 0-1000 max buffers written/round Have you watched closely under load to ensure that you're not seeing a huge performance hit every 10s when the bgwriter kicks o

Re: [PERFORM] Background writer underemphasized ...

2008-04-16 Thread Greg Smith
On Wed, 16 Apr 2008, Marinos Yannikos wrote: to save some people a headache or two: I believe we just solved our performance problem in the following scenario: I was about to ask your PostgreSQL version but since I see you mention wal_writer_delay it must be 8.3. Knowing your settings for sh

Re: [PERFORM] Background writer underemphasized ...

2008-04-16 Thread Bill Moran
In response to Marinos Yannikos <[EMAIL PROTECTED]>: > Hi, > > to save some people a headache or two: I believe we just solved our > performance problem in the following scenario: > > - Linux 2.6.24.4 > - lots of RAM (32GB) > - enough CPU power (4 cores) > - disks with relatively slow random wr

Re: [PERFORM] Background writer underemphasized ...

2008-04-16 Thread Chris Browne
[EMAIL PROTECTED] (Marinos Yannikos) writes: > This helped with our configuration: > bgwriter_delay = 1ms # 10-1ms between rounds > bgwriter_lru_maxpages = 1000 # 0-1000 max buffers written/round FYI, I'd be inclined to reduce both of those numbers, as it should reduce the vari

[PERFORM] Background writer underemphasized ...

2008-04-16 Thread Marinos Yannikos
Hi, to save some people a headache or two: I believe we just solved our performance problem in the following scenario: - Linux 2.6.24.4 - lots of RAM (32GB) - enough CPU power (4 cores) - disks with relatively slow random writes (SATA RAID-5 / 7 disks, 128K stripe, ext2) Our database is aro