Scott Carey wrote:
As long as fsync() works _properly_ which is true for any file system + disk combination
with a damn (not HFS+ on OSX, not FAT, not a few other things), then it will tell the
drive to flush its cache _before_ fsync() returns. There is NO REASON for a raid card to
turn off a
On 16/07/10 09:22, Scott Marlowe wrote:
> On Thu, Jul 15, 2010 at 10:30 AM, Scott Carey wrote:
>>
>> On Jul 14, 2010, at 7:50 PM, Ben Chobot wrote:
>>
>>> On Jul 14, 2010, at 6:57 PM, Scott Carey wrote:
>>>
But none of this explains why a 4-disk raid 10 is slower than a 1 disk
system.
On 16/07/10 06:18, Ben Chobot wrote:
> There are also caches on all your disk drives. Write caching there is always
> dangerous, which is why almost all raid cards always disable the hard drive
> write caching, with or without a BBU. I'm not even sure how many raid cards
> let you enable the wr
On Jul 15, 2010, at 8:16 PM, Scott Carey wrote:
> On Jul 15, 2010, at 12:35 PM, Ben Chobot wrote:
>
>> On Jul 15, 2010, at 9:30 AM, Scott Carey wrote:
>>
Many raid controllers are smart enough to always turn off write caching on
the drives, and also disable the feature on their own bu
On Jul 15, 2010, at 6:22 PM, Scott Marlowe wrote:
> On Thu, Jul 15, 2010 at 10:30 AM, Scott Carey wrote:
>>
>> On Jul 14, 2010, at 7:50 PM, Ben Chobot wrote:
>>
>>> On Jul 14, 2010, at 6:57 PM, Scott Carey wrote:
>>>
But none of this explains why a 4-disk raid 10 is slower than a 1 disk
On Jul 15, 2010, at 12:35 PM, Ben Chobot wrote:
> On Jul 15, 2010, at 9:30 AM, Scott Carey wrote:
>
>>> Many raid controllers are smart enough to always turn off write caching on
>>> the drives, and also disable the feature on their own buffer without a BBU.
>>> Add a BBU, and the cache on the
On Thu, Jul 15, 2010 at 10:30 AM, Scott Carey wrote:
>
> On Jul 14, 2010, at 7:50 PM, Ben Chobot wrote:
>
>> On Jul 14, 2010, at 6:57 PM, Scott Carey wrote:
>>
>>> But none of this explains why a 4-disk raid 10 is slower than a 1 disk
>>> system. If there is no write-back caching on the RAID, it
Most (all?) hard drives have cache built into them. Many raid cards have
cache built into them. When the power dies, all the data in any cache is
lost, which is why it's dangerous to use it for write caching. For that
reason, you can attach a BBU to a raid card which keeps the cache alive
On Thu, Jul 15, 2010 at 12:35 PM, Ben Chobot wrote:
> On Jul 15, 2010, at 9:30 AM, Scott Carey wrote:
>
> >> Many raid controllers are smart enough to always turn off write caching
> on the drives, and also disable the feature on their own buffer without a
> BBU. Add a BBU, and the cache on the c
On Jul 15, 2010, at 2:40 PM, Ryan Wexler wrote:
> On Thu, Jul 15, 2010 at 12:35 PM, Ben Chobot wrote:
> On Jul 15, 2010, at 9:30 AM, Scott Carey wrote:
>
> >> Many raid controllers are smart enough to always turn off write caching on
> >> the drives, and also disable the feature on their own bu
On Jul 14, 2010, at 7:50 PM, Ben Chobot wrote:
> On Jul 14, 2010, at 6:57 PM, Scott Carey wrote:
>
>> But none of this explains why a 4-disk raid 10 is slower than a 1 disk
>> system. If there is no write-back caching on the RAID, it should still be
>> similar to the one disk setup.
>
> Many
On Jul 15, 2010, at 9:30 AM, Scott Carey wrote:
>> Many raid controllers are smart enough to always turn off write caching on
>> the drives, and also disable the feature on their own buffer without a BBU.
>> Add a BBU, and the cache on the controller starts getting used, but *not*
>> the cache
On Jul 15, 2010, at 12:40 PM, Ryan Wexler wrote:
> On Wed, Jul 14, 2010 at 7:50 PM, Ben Chobot wrote:
> On Jul 14, 2010, at 6:57 PM, Scott Carey wrote:
>
> > But none of this explains why a 4-disk raid 10 is slower than a 1 disk
> > system. If there is no write-back caching on the RAID, it sh
On Jul 14, 2010, at 6:57 PM, Scott Carey wrote:
> But none of this explains why a 4-disk raid 10 is slower than a 1 disk
> system. If there is no write-back caching on the RAID, it should still be
> similar to the one disk setup.
Many raid controllers are smart enough to always turn off write
lushing its buffers.
> >
> > Craig
> >
> >>
> >> -Original Message-
> >> From: pgsql-performance-ow...@postgresql.org
> >> [mailto:pgsql-performance-ow...@postgresql.org] On Behalf Of Craig
> James
> >> Sent: Thursday, July
lto:pgsql-performance-ow...@postgresql.org] On Behalf Of Craig James
>> Sent: Thursday, July 08, 2010 4:02 PM
>> To: pgsql-performance@postgresql.org
>> Subject: Re: [PERFORM] performance on new linux box
>>
>> On 7/8/10 12:47 PM, Ryan Wexler wrote:
>>>
>&g
On 07/11/2010 03:02 PM, Ryan Wexler wrote:
Well I got me a new raid card, MegaRAID 8708EM2, fully equipped with
BBU and read and write caching are enabled. It completely solved my
performance problems. Now everything is way faster than the previous
server. Thanks for all the help everyone.
Ryan Wexler wrote:
One question I do have is this card has a setting called Read Policy
which apparently helps with sequentially reads. Do you think that is
something I should enable?
Linux will do some amount of read-ahead in a similar way on its own.
You run "blockdev --getra" and "blockd
On Fri, Jul 9, 2010 at 2:38 AM, Samuel Gendler wrote:
> On Fri, Jul 9, 2010 at 2:08 AM, Russell Smith wrote:
> > On 09/07/10 02:31, Ryan Wexler wrote:
> >
> >
> > The only other difference between the boxes is the postgresql version.
> The
> > new one has 8.4-2 from the yum install instructions o
On Fri, Jul 9, 2010 at 2:08 AM, Russell Smith wrote:
> On 09/07/10 02:31, Ryan Wexler wrote:
>
>
> The only other difference between the boxes is the postgresql version. The
> new one has 8.4-2 from the yum install instructions on the site:
> http://yum.pgrpms.org/reporpms/repoview/pgdg-centos.ht
On 09/07/10 02:31, Ryan Wexler wrote:
> Thanks a lot for all the comments. The fact that both my windows box
> and the old linux box both show a massive performance improvement over
> the new linux box seems to point to hardware to me. I am not sure how
> to test the fsync issue, but i don't see
On 7/8/2010 3:18 PM, timothy.noo...@emc.com wrote:
How does the linux machine know that there is a BBU installed and to
change its behavior or change the behavior of Postgres? I am
experiencing performance issues, not with searching but more with IO.
It doesn't change its behavior at all. It'
a disk that's exceptionally fast at flushing
its buffers.
Craig
-Original Message-
From: pgsql-performance-ow...@postgresql.org
[mailto:pgsql-performance-ow...@postgresql.org] On Behalf Of Craig James
Sent: Thursday, July 08, 2010 4:02 PM
To: pgsql-performance@postgresql.org
Subj
Thursday, July 8, 2010, 11:02:50 PM you wrote:
> Here is what I got:
> # ./tw_cli /c0 show
If that's all you get, than there's no BBU installed, or not correctly
connected to the controller.
You could try 'tw_cli /c0/bbu show all' to be sure, but I doubt your output
will change-
--
Jochen Erw
-ow...@postgresql.org] On Behalf Of Craig James
Sent: Thursday, July 08, 2010 4:02 PM
To: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] performance on new linux box
On 7/8/10 12:47 PM, Ryan Wexler wrote:
>
>
> On Thu, Jul 8, 2010 at 12:46 PM, Kevin Grittner
> mailto
On Thu, Jul 8, 2010 at 12:13 PM, Jochen Erwied <
joc...@pgsql-performance.erwied.eu> wrote:
> Thursday, July 8, 2010, 7:16:47 PM you wrote:
>
> > Thanks. The server is hosted, so it is a bit of a hassle to figure this
> > stuff out, but I am having someone check.
>
> If you have root access to th
On 7/8/10 12:47 PM, Ryan Wexler wrote:
On Thu, Jul 8, 2010 at 12:46 PM, Kevin Grittner
mailto:kevin.gritt...@wicourts.gov>> wrote:
Ryan Wexler mailto:r...@iridiumsuite.com>>
wrote:
> One thing I don't understand is why BBU will result in a huge
> performance gain. I thought
Ryan Wexler wrote:
> It still amazes me that it would account for a 5x change in IO.
If you were doing one INSERT per database transaction, for instance,
that would not be at all surprising. If you were doing one COPY in
of a million rows, it would be a bit more surprising.
Each COMMIT of a
On 7/8/2010 1:47 PM, Ryan Wexler wrote:
Thanks for the explanations that makes things clearer. It still
amazes me that it would account for a 5x change in IO.
The buffering allows decoupling of the write rate from the disk rotation
speed.
Disks don't spin that fast, at least not relative to t
On Thu, Jul 8, 2010 at 12:46 PM, Kevin Grittner wrote:
> Ryan Wexler wrote:
>
> > One thing I don't understand is why BBU will result in a huge
> > performance gain. I thought BBU was all about power failures?
>
> Well, it makes it safe for the controller to consider the write
> complete as soo
Ryan Wexler wrote:
> One thing I don't understand is why BBU will result in a huge
> performance gain. I thought BBU was all about power failures?
Well, it makes it safe for the controller to consider the write
complete as soon as it hits the RAM cache, rather than waiting for
persistence to
On Jul 8, 2010, at 12:37 PM, Ryan Wexler wrote:
> One thing I don't understand is why BBU will result in a huge performance
> gain. I thought BBU was all about power failures?
When you have a working BBU, the raid card can safely do write caching. Without
it, many raid cards are good about tur
On Thu, Jul 8, 2010 at 12:32 PM, Jochen Erwied <
joc...@pgsql-performance.erwied.eu> wrote:
> Thursday, July 8, 2010, 9:18:20 PM you wrote:
>
> > However, I just verified with the hosting company that BBU is off on the
> > raid controller. I am trying to find out my options, turn it on,
> differe
Thursday, July 8, 2010, 9:18:20 PM you wrote:
> However, I just verified with the hosting company that BBU is off on the
> raid controller. I am trying to find out my options, turn it on, different
> card, etc...
Turning it on requires the external BBU to be installed, so even if a 9650
has BBU
Ryan Wexler wrote:
> I just verified with the hosting company that BBU is off on the
> raid controller. I am trying to find out my options, turn it on,
> different card, etc...
In the "etc." category, make sure that when you get it turned on,
the cache is configured for "write back" mode, not
On Thu, Jul 8, 2010 at 12:13 PM, Jochen Erwied <
joc...@pgsql-performance.erwied.eu> wrote:
> Thursday, July 8, 2010, 7:16:47 PM you wrote:
>
> > Thanks. The server is hosted, so it is a bit of a hassle to figure this
> > stuff out, but I am having someone check.
>
> If you have root access to th
Thursday, July 8, 2010, 7:16:47 PM you wrote:
> Thanks. The server is hosted, so it is a bit of a hassle to figure this
> stuff out, but I am having someone check.
If you have root access to the machine, you should try 'tw_cli /cx show',
where the x in /cx is the controller number. If not presen
-- Forwarded message --
From: Ryan Wexler
Date: Thu, Jul 8, 2010 at 10:12 AM
Subject: Re: [PERFORM] performance on new linux box
To: Craig James
On Thu, Jul 8, 2010 at 10:10 AM, Craig James wrote:
> On 7/8/10 9:31 AM, Ryan Wexler wrote:
>
>> Thanks a lot for all
On 7/8/10 9:31 AM, Ryan Wexler wrote:
Thanks a lot for all the comments. The fact that both my windows box
and the old linux box both show a massive performance improvement over
the new linux box seems to point to hardware to me. I am not sure how
to test the fsync issue, but i don't see how th
On Thu, 2010-07-08 at 09:31 -0700, Ryan Wexler wrote:
> The raid card the server has in it is:
> 3Ware 4 Port 9650SE-4LPML RAID Card
>
> Looking it up, it seems to indicate that it has BBU
No. It supports a BBU. It doesn't have one necessarily.
You need to go into your RAID BIOS. It will tell y
Thanks a lot for all the comments. The fact that both my windows box and
the old linux box both show a massive performance improvement over the new
linux box seems to point to hardware to me. I am not sure how to test the
fsync issue, but i don't see how that could be it.
The raid card the serve
Eliot Gable wrote:
> If you can post some of your queries, there are a lot of bright
> people on this discussion list that can probably help you solve
> your bottleneck
Sure, but the original post was because the brand new server class
machine was performing much worse than the single-drive de
On Thu, Jul 8, 2010 at 9:53 AM, Kevin Grittner
wrote:
> Eliot Gable
> >
> wrote:
>
> > For about $2k - $3k, you can get a server that will do upwards of
> > 300 MB/sec, assuming the bulk of that cost goes to a good
> > hardware-based RAID controller with a battery backed-up cache and
> > some goo
Eliot Gable wrote:
> For about $2k - $3k, you can get a server that will do upwards of
> 300 MB/sec, assuming the bulk of that cost goes to a good
> hardware-based RAID controller with a battery backed-up cache and
> some good 15k RPM SAS drives.
FWIW, I concur that the description so far sugg
On Wed, Jul 7, 2010 at 10:07 PM, Andy Colson wrote:
> On 07/07/2010 06:06 PM, Ryan Wexler wrote:
>
>> Postgresql was previously running on a single cpu linux machine with 2
>> gigs of memory and a single sata drive (v8.3). Basically a desktop with
>> linux on it. I experienced slow performance.
On the new system the bulk loads are extremely slower than on the
previous
machine and so are the more complex queries. The smaller transactional
queries seem comparable but i had expected an improvement. Performing a
db
import via psql -d databas -f dbfile illustrates this problem.
If
On 07/07/2010 06:06 PM, Ryan Wexler wrote:
Postgresql was previously running on a single cpu linux machine with 2 gigs of
memory and a single sata drive (v8.3). Basically a desktop with linux on it.
I experienced slow performance.
So, I finally moved it to a real server. A dually zeon cento
On Wed, Jul 7, 2010 at 4:06 PM, Ryan Wexler wrote:
> Postgresql was previously running on a single cpu linux machine with 2 gigs
> of memory and a single sata drive (v8.3). Basically a desktop with linux on
> it. I experienced slow performance.
>
> So, I finally moved it to a real server. A dua
Ryan Wexler writes:
> Postgresql was previously running on a single cpu linux machine with 2 gigs
> of memory and a single sata drive (v8.3). Basically a desktop with linux on
> it. I experienced slow performance.
> So, I finally moved it to a real server. A dually zeon centos machine with
> 6
Postgresql was previously running on a single cpu linux machine with 2 gigs
of memory and a single sata drive (v8.3). Basically a desktop with linux on
it. I experienced slow performance.
So, I finally moved it to a real server. A dually zeon centos machine with
6 gigs of memory and raid 10, po
50 matches
Mail list logo
