Re: [PERFORM] postgresql 8.3 tps rate

2009-01-23 Thread Scott Marlowe
On Thu, Jan 22, 2009 at 10:52 PM, Ibrahim Harrani
 wrote:
> Hi Craig,
>
> Here is the result. It seems that disk write is terrible!.
>
> r...@myserver /usr]#  time (dd if=/dev/zero of=bigfile bs=8192
> count=100; sync)
>
>
> 100+0 records in
> 100+0 records out
> 819200 bytes transferred in 945.343806 secs (8665630 bytes/sec)

That's pretty horrific.  Can you try your disks without the
motherboard RAID controller in the way either singly or using software
RAID?

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


[PERFORM] SSD performance

2009-01-23 Thread david
I spotted a new interesting SSD review. it's a $379 5.25" drive bay device 
that holds up to 8 DDR2 DIMMS (up to 8G per DIMM) and appears to the 
system as a SATA drive (or a pair of SATA drives that you can RAID-0 to 
get past the 300MB/s SATA bottleneck)


the best review I've seen only ran it on windows (and a relativly old 
hardware platform at that), I suspect it's performance would be even 
better under linux and with a top-notch controller card (especially with 
the RAID option)


it has a battery backup (good for 4 hours or so) and a CF cardslot that it 
can back the ram up to (~20 min to save 32G and 15 min to restore, so not 
something you really want to make use of, but a good safety net)


the review also includes the Intel X-25E and X-25M drives (along with a 
variety of SCSI and SATA drives)


http://techreport.com/articles.x/16255/1

equipped with 16G the street price should be ~$550, with 32G it should be 
~$1200 with 64G even more expensive, but the performance is very good. 
there are times when the X-25E matches it or edges it out in these tests, 
so there is room for additional improvement, but as I noted above it may 
do better with a better controller and non-windows OS.


power consumption is slightly higher than normal hard drives at about 12w 
(_much_ higher than the X-25)


they also have a review of the X-25E vs the X-25M

http://techreport.com/articles.x/15931/1

one thing that both of these reviews show is that if you are doing a 
significant amount of writing the X-25M is no better than a normal hard 
drive (and much of the time in the middle to bottom of the pack compared 
to normal hard drives)


David Lang

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread Glyn Astill
> I spotted a new interesting SSD review. it's a $379
> 5.25" drive bay device that holds up to 8 DDR2 DIMMS
> (up to 8G per DIMM) and appears to the system as a SATA
> drive (or a pair of SATA drives that you can RAID-0 to get
> past the 300MB/s SATA bottleneck)
> 

Sounds very similar to the Gigabyte iRam drives of a few years ago

http://en.wikipedia.org/wiki/I-RAM






-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread david

On Fri, 23 Jan 2009, Glyn Astill wrote:


I spotted a new interesting SSD review. it's a $379
5.25" drive bay device that holds up to 8 DDR2 DIMMS
(up to 8G per DIMM) and appears to the system as a SATA
drive (or a pair of SATA drives that you can RAID-0 to get
past the 300MB/s SATA bottleneck)



Sounds very similar to the Gigabyte iRam drives of a few years ago

http://en.wikipedia.org/wiki/I-RAM


similar concept, but there are some significant differences

the iRam was limited to 4G, used DDR ram, and used a PCI slot for power 
(which can be in 
short supply nowdays)


this new drive can go to 64G, uses DDR2 ram (cheaper than DDR nowdays), 
gets powered like a normal SATA drive, can use two SATA channels (to be 
able to get past the throughput limits of a single SATA interface), and 
has a CF card slot to backup the data to if the system powers down.


plus the performance appears to be significantly better (even without 
using the second SATA interface)


David Lang


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread Luke Lonergan
Why not simply plug your server into a UPS and get 10-20x the performance using 
the same approach (with OS IO cache)?

In fact, with the server it's more robust, as you don't have to transit several 
intervening physical devices to get to the RAM.

If you want a file interface, declare a RAMDISK.

Cheaper/faster/improved reliability.

- Luke

- Original Message -
From: pgsql-performance-ow...@postgresql.org 

To: Glyn Astill 
Cc: pgsql-performance@postgresql.org 
Sent: Fri Jan 23 04:39:07 2009
Subject: Re: [PERFORM] SSD performance

On Fri, 23 Jan 2009, Glyn Astill wrote:

>> I spotted a new interesting SSD review. it's a $379
>> 5.25" drive bay device that holds up to 8 DDR2 DIMMS
>> (up to 8G per DIMM) and appears to the system as a SATA
>> drive (or a pair of SATA drives that you can RAID-0 to get
>> past the 300MB/s SATA bottleneck)
>>
>
> Sounds very similar to the Gigabyte iRam drives of a few years ago
>
> http://en.wikipedia.org/wiki/I-RAM

similar concept, but there are some significant differences

the iRam was limited to 4G, used DDR ram, and used a PCI slot for power
(which can be in
short supply nowdays)

this new drive can go to 64G, uses DDR2 ram (cheaper than DDR nowdays),
gets powered like a normal SATA drive, can use two SATA channels (to be
able to get past the throughput limits of a single SATA interface), and
has a CF card slot to backup the data to if the system powers down.

plus the performance appears to be significantly better (even without
using the second SATA interface)

David Lang


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread david

On Fri, 23 Jan 2009, Luke Lonergan wrote:

Why not simply plug your server into a UPS and get 10-20x the 
performance using the same approach (with OS IO cache)?


In fact, with the server it's more robust, as you don't have to transit 
several intervening physical devices to get to the RAM.


If you want a file interface, declare a RAMDISK.

Cheaper/faster/improved reliability.


you can also disable fsync to not wait for your disks if you trust your 
system to never go down. personally I don't trust any system to not go 
down.


if you have a system crash or reboot your RAMDISK will loose it's content, 
this device won't.


also you are limited to how many DIMMS you can put on your motherboard 
(for the dual-socket systems I am buying nowdays, I'm limited to 32G of 
ram) going to a different motherboard that can support additional ram can 
be quite expensive.


this isn't for everyone, but for people who need the performance, data 
reliability, this looks like a very interesting option.


David Lang


- Luke

- Original Message -
From: pgsql-performance-ow...@postgresql.org 

To: Glyn Astill 
Cc: pgsql-performance@postgresql.org 
Sent: Fri Jan 23 04:39:07 2009
Subject: Re: [PERFORM] SSD performance

On Fri, 23 Jan 2009, Glyn Astill wrote:


I spotted a new interesting SSD review. it's a $379
5.25" drive bay device that holds up to 8 DDR2 DIMMS
(up to 8G per DIMM) and appears to the system as a SATA
drive (or a pair of SATA drives that you can RAID-0 to get
past the 300MB/s SATA bottleneck)



Sounds very similar to the Gigabyte iRam drives of a few years ago

http://en.wikipedia.org/wiki/I-RAM


similar concept, but there are some significant differences

the iRam was limited to 4G, used DDR ram, and used a PCI slot for power
(which can be in
short supply nowdays)

this new drive can go to 64G, uses DDR2 ram (cheaper than DDR nowdays),
gets powered like a normal SATA drive, can use two SATA channels (to be
able to get past the throughput limits of a single SATA interface), and
has a CF card slot to backup the data to if the system powers down.

plus the performance appears to be significantly better (even without
using the second SATA interface)

David Lang


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance



--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread Matthew Wakeling

On Fri, 23 Jan 2009, Luke Lonergan wrote:
Why not simply plug your server into a UPS and get 10-20x the 
performance using the same approach (with OS IO cache)?


In fact, with the server it's more robust, as you don't have to transit 
several intervening physical devices to get to the RAM.


If you want a file interface, declare a RAMDISK.

Cheaper/faster/improved reliability.


I'm sure we have gone over that one before. With that method, your data is 
at the mercy of the *entire system*. Any fault in any part of the computer 
(hardware or software) will result in the loss of all your data. In 
contrast, a RAM-based SSD is isolated from such failures, especially if it 
backs up to another device on power fail. You can completely trash the 
computer, remove the SSD and put it into another machine, and boot it up 
as normal.


Computers break. Nothing is going to stop that from happening. Except VMS 
maybe.


Not arguing that your method is faster though.

Matthew

--
"Finger to spiritual emptiness underlying everything."
   -- How a foreign C manual referred to a "pointer to void."

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread Luke Lonergan
Hmm - I wonder what OS it runs ;-)

- Luke

- Original Message -
From: da...@lang.hm 
To: Luke Lonergan
Cc: glynast...@yahoo.co.uk ; 
pgsql-performance@postgresql.org 
Sent: Fri Jan 23 04:52:27 2009
Subject: Re: [PERFORM] SSD performance

On Fri, 23 Jan 2009, Luke Lonergan wrote:

> Why not simply plug your server into a UPS and get 10-20x the
> performance using the same approach (with OS IO cache)?
>
> In fact, with the server it's more robust, as you don't have to transit
> several intervening physical devices to get to the RAM.
>
> If you want a file interface, declare a RAMDISK.
>
> Cheaper/faster/improved reliability.

you can also disable fsync to not wait for your disks if you trust your
system to never go down. personally I don't trust any system to not go
down.

if you have a system crash or reboot your RAMDISK will loose it's content,
this device won't.

also you are limited to how many DIMMS you can put on your motherboard
(for the dual-socket systems I am buying nowdays, I'm limited to 32G of
ram) going to a different motherboard that can support additional ram can
be quite expensive.

this isn't for everyone, but for people who need the performance, data
reliability, this looks like a very interesting option.

David Lang

> - Luke
>
> - Original Message -
> From: pgsql-performance-ow...@postgresql.org 
> 
> To: Glyn Astill 
> Cc: pgsql-performance@postgresql.org 
> Sent: Fri Jan 23 04:39:07 2009
> Subject: Re: [PERFORM] SSD performance
>
> On Fri, 23 Jan 2009, Glyn Astill wrote:
>
>>> I spotted a new interesting SSD review. it's a $379
>>> 5.25" drive bay device that holds up to 8 DDR2 DIMMS
>>> (up to 8G per DIMM) and appears to the system as a SATA
>>> drive (or a pair of SATA drives that you can RAID-0 to get
>>> past the 300MB/s SATA bottleneck)
>>>
>>
>> Sounds very similar to the Gigabyte iRam drives of a few years ago
>>
>> http://en.wikipedia.org/wiki/I-RAM
>
> similar concept, but there are some significant differences
>
> the iRam was limited to 4G, used DDR ram, and used a PCI slot for power
> (which can be in
> short supply nowdays)
>
> this new drive can go to 64G, uses DDR2 ram (cheaper than DDR nowdays),
> gets powered like a normal SATA drive, can use two SATA channels (to be
> able to get past the throughput limits of a single SATA interface), and
> has a CF card slot to backup the data to if the system powers down.
>
> plus the performance appears to be significantly better (even without
> using the second SATA interface)
>
> David Lang
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread Craig Ringer
Luke Lonergan wrote:
> Why not simply plug your server into a UPS and get 10-20x the performance 
> using the same approach (with OS IO cache)?

A big reason is that your machine may already have as much RAM as is
currently economical to install. Hardware with LOTS of RAM slots can
cost quite a bit.

Another reason is that these devices won't lose data because of an
unexpected OS reboot. If they're fitted with a battery backup and CF
media for emergency write-out, they won't lose data if your UPS runs out
of juice either.

I'd be much more confident with something like those devices than I
would with an OS ramdisk plus startup/shutdown scripts to initialize it
from a file and write it out to a file. Wouldn't it be a pain if the UPS
didn't give the OS enough warning to write the RAM disk out before
losing power...

In any case, you're very rarely better off dedicating host memory to a
ramdisk rather than using the normal file system and letting the host
cache it. A ramdisk really only seems to help when you're really using
it to bypass safeties like the effects of fsync() and ordered
journaling. There are other ways to avoid those if you really don't care
about your data.

These devices would be interesting for a few uses, IMO. One is temp
table space and sort space in Pg. Another is scratch space for apps
(like Photoshop) that do their own VM management. There's also potential
for use as 1st priority OS swap space, though at least on Linux I think
the CPU overhead involved in swapping is so awful you wouldn't benefit
from it much.

I've been hoping this sort of thing would turn up again in a new
incarnation with battery backup and CF/SD for BBU-flat safety.

--
Craig Ringer

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread Florian Weimer
* Craig Ringer:

> I'd be much more confident with something like those devices than I
> would with an OS ramdisk plus startup/shutdown scripts to initialize it
> from a file and write it out to a file. Wouldn't it be a pain if the UPS
> didn't give the OS enough warning to write the RAM disk out before
> losing power...

The cache warm-up time can also be quite annoying.  Of course, with
flash-backed DRAM, this is a concern as long as you use the cheaper,
slower variants for the backing storage.

-- 
Florian Weimer
BFK edv-consulting GmbH   http://www.bfk.de/
Kriegsstraße 100  tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] postgresql 8.3 tps rate

2009-01-23 Thread Merlin Moncure
On 1/23/09, Ibrahim Harrani  wrote:
> Hi Craig,
>
>  Here is the result. It seems that disk write is terrible!.
>
>  r...@myserver /usr]#  time (dd if=/dev/zero of=bigfile bs=8192
>  count=100; sync)

Note, while sequential write speeds are a good indication of general
raid crappyness, they are not the main driver of your low pgbench
results (buy they may be involved with poor insert performance)

That is coming from your seek performance, which is also lousy at ~130
according to bonnie.

merlin

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread Merlin Moncure
On 1/23/09, da...@lang.hm  wrote:
>  the review also includes the Intel X-25E and X-25M drives (along with a
> variety of SCSI and SATA drives)
>

The x25-e is a game changer for database storage.  It's still a little
pricey for what it does but who can argue with these numbers?
http://techreport.com/articles.x/15931/9

merlin

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


[PERFORM] More Troubles Dumping a Very Large Table

2009-01-23 Thread Ted Allen

Hey All,

I previously post about the troubles I was having dumping a >1Tb (size 
with indexes) table.  The rows in the table could be very large.  Using 
perl's DBD::Pg we were some how able to add these very large rows 
without running in to the >1Gb row bug.  With everyones help I 
determined I needed to get move the largest rows elsewhere.   This 
seemed to solve that problem but a new problem has cropped up.


When I ran pg_dump again, it ran successfully without error.  Although 
there were no errors, pg_dump dumped less then half of the rows that 
actually exist in the table.   When examining the dump file (I did not 
dump in -F c format)  the copy statement created by the dump is 
terminated correctly (with a \.) and there are indeed 300+ million rows 
in the file as opposed to the 700+ million I was expecting.  I don't 
believe I specified anything that would have caused pg_dump to dump a 
truncated version of the table.  The last row successfully dumped 
contains only normal ascii characters and is not particularly big in 
size.   The row immediately after the last row successfully dumped 
contains an installer file (.bin) stored in a bytea cell.  It is about 
138 Mb in size. 


I've also been having troubles recreating this situation on a smaller DB.

We are dumping the table using this command.

/var/lib/pgsql-8.3.5/bin/pg_dump  -O -x -t large_table mydb | gzip -c  
>  large_table.pgsql.gz



The stats of the db server is as follows,

Processors:  4x Opteron 2.4 Ghz cores
Memory: 16 GB 
Disks: 42x 15K SCSI 146 GB disks.




Thanks again for or your help,
Ted

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread david

On Fri, 23 Jan 2009, Merlin Moncure wrote:


On 1/23/09, da...@lang.hm  wrote:

 the review also includes the Intel X-25E and X-25M drives (along with a
variety of SCSI and SATA drives)



The x25-e is a game changer for database storage.  It's still a little
pricey for what it does but who can argue with these numbers?
http://techreport.com/articles.x/15931/9


take a look at this ram based drive, specificly look at the numbers here
http://techreport.com/articles.x/16255/9

they are about as much above the X25-e as the X25-e is above normal 
drives.


David Lang


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread M. Edward (Ed) Borasky
da...@lang.hm wrote:
> On Fri, 23 Jan 2009, Luke Lonergan wrote:
> 
>> Why not simply plug your server into a UPS and get 10-20x the
>> performance using the same approach (with OS IO cache)?
>>
>> In fact, with the server it's more robust, as you don't have to
>> transit several intervening physical devices to get to the RAM.
>>
>> If you want a file interface, declare a RAMDISK.
>>
>> Cheaper/faster/improved reliability.
> 
> you can also disable fsync to not wait for your disks if you trust your
> system to never go down. personally I don't trust any system to not go
> down.
> 
> if you have a system crash or reboot your RAMDISK will loose it's
> content, this device won't.
> 
> also you are limited to how many DIMMS you can put on your motherboard
> (for the dual-socket systems I am buying nowdays, I'm limited to 32G of
> ram) going to a different motherboard that can support additional ram
> can be quite expensive.
> 
> this isn't for everyone, but for people who need the performance, data
> reliability, this looks like a very interesting option.
> 
> David Lang
> 
>> - Luke
>>
>> - Original Message -
>> From: pgsql-performance-ow...@postgresql.org
>> 
>> To: Glyn Astill 
>> Cc: pgsql-performance@postgresql.org 
>> Sent: Fri Jan 23 04:39:07 2009
>> Subject: Re: [PERFORM] SSD performance
>>
>> On Fri, 23 Jan 2009, Glyn Astill wrote:
>>
 I spotted a new interesting SSD review. it's a $379
 5.25" drive bay device that holds up to 8 DDR2 DIMMS
 (up to 8G per DIMM) and appears to the system as a SATA
 drive (or a pair of SATA drives that you can RAID-0 to get
 past the 300MB/s SATA bottleneck)

>>>
>>> Sounds very similar to the Gigabyte iRam drives of a few years ago
>>>
>>> http://en.wikipedia.org/wiki/I-RAM
>>
>> similar concept, but there are some significant differences
>>
>> the iRam was limited to 4G, used DDR ram, and used a PCI slot for power
>> (which can be in
>> short supply nowdays)
>>
>> this new drive can go to 64G, uses DDR2 ram (cheaper than DDR nowdays),
>> gets powered like a normal SATA drive, can use two SATA channels (to be
>> able to get past the throughput limits of a single SATA interface), and
>> has a CF card slot to backup the data to if the system powers down.
>>
>> plus the performance appears to be significantly better (even without
>> using the second SATA interface)
>>
>> David Lang
>>
>>
>> -- 
>> Sent via pgsql-performance mailing list
>> (pgsql-performance@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-performance
>>
> 

Can I call a time out here? :) There are "always" going to be memory
hierarchies -- registers on the processors, multiple levels of caches,
RAM used for programs / data / I-O caches, and non-volatile rotating
magnetic storage. And there are "always" going to be new hardware
technologies cropping up at various levels in the hierarchy.

There are always going to be cost / reliability / performance
trade-offs, leading to "interesting" though perhaps not really
business-relevant "optimizations". The equations are there for anyone to
use should they want to optimize for a given workload at a given point
in time with given business / service level constraints. See

http://www.amazon.com/Storage-Network-Performance-Analysis-Huseyin/dp/076451685X

for all the details.

I question, however, whether there's much point in seeking an optimum.
As was noted long ago by Nobel laureate Herbert Simon, in actual fact
managers / businesses rarely optimize. Instead, they satisfice. They do
what is "good enough", not what is best. And my own personal opinion in
the current context -- PostgreSQL running on an open-source operating
system -- is that

* large-capacity inexpensive rotating disks,
* a hardware RAID controller containing a battery-backed cache,
* as much RAM as one can afford and the chassis will hold, and
* enough cores to keep the workload from becoming processor-bound

are good enough. And given that, a moderate amount of software tweaking
and balancing will get you close to a local optimum.

-- 
M. Edward (Ed) Borasky

I've never met a happy clam. In fact, most of them were pretty steamed.

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread Joshua D. Drake
On Fri, 2009-01-23 at 09:22 -0800, M. Edward (Ed) Borasky wrote:

> I question, however, whether there's much point in seeking an optimum.
> As was noted long ago by Nobel laureate Herbert Simon, in actual fact
> managers / businesses rarely optimize. Instead, they satisfice. They do
> what is "good enough", not what is best. And my own personal opinion in
> the current context -- PostgreSQL running on an open-source operating
> system -- is that

This community is notorious for "optimum". MySQL is notorious for "satisfy".

Which one would you rather store your financial information in?

I actually agree with you to a degree. A loud faction of this community
spends a little too much time mentally masturbating but without that we
wouldn't have a lot of the very interesting features we have now.


There is no correct in left.
There is no correct in right.
Correctness is the result of friction caused by the mingling of the two.

Sincerely,

Joshua D. Drake


-- 
PostgreSQL - XMPP: jdr...@jabber.postgresql.org
   Consulting, Development, Support, Training
   503-667-4564 - http://www.commandprompt.com/
   The PostgreSQL Company, serving since 1997


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread Matthew Wakeling

On Fri, 23 Jan 2009, M. Edward (Ed) Borasky wrote:

* large-capacity inexpensive rotating disks,
* a hardware RAID controller containing a battery-backed cache,
* as much RAM as one can afford and the chassis will hold, and
* enough cores to keep the workload from becoming processor-bound

are good enough. And given that, a moderate amount of software tweaking
and balancing will get you close to a local optimum.


That's certainly the case for very large-scale (in terms of data quantity) 
databases. However, these solid state devices do have quite an advantage 
when what you want to scale is the performance, rather than the data 
quantity.


The thing is, it isn't just a matter of storage heirarchy. There's the 
volatility matter there as well. What you have in these SSDs is a device 
which is non-volatile, like a disc, but fast, like RAM.


Matthew

--
Anyone who goes to a psychiatrist ought to have his head examined.

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] postgresql 8.3 tps rate

2009-01-23 Thread Greg Smith

On Fri, 23 Jan 2009, Merlin Moncure wrote:

Note, while sequential write speeds are a good indication of general 
raid crappyness, they are not the main driver of your low pgbench 
results (buy they may be involved with poor insert performance) That is 
coming from your seek performance, which is also lousy at ~130 according 
to bonnie.


The bonnie seek test includes a write component to it:  "In 10% of cases, 
it is dirtied and written back with write(2)."  Since ~130 isn't that much 
worse than normal for a cheap 7200 RPM drive (near 200), it could easily 
be the case that the awful performance on those writes is pulling down the 
seek score.  Sequential writes are low on the scale of things that matter 
in a database context, but if they're screwed up bad enough it can ripple 
into more important areas.


--
* Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] SSD performance

2009-01-23 Thread M. Edward (Ed) Borasky
Joshua D. Drake wrote:
> This community is notorious for "optimum". MySQL is notorious for "satisfy".

Within *this* community, MySQL is just plain notorious. Let's face it --
we are *not* dolphin-safe.



> 
> Which one would you rather store your financial information in?

The one that had the best data integrity, taking into account the RDBMS
*and* the hardware and other software.

> I actually agree with you to a degree. A loud faction of this community
> spends a little too much time mentally masturbating but without that we
> wouldn't have a lot of the very interesting features we have now.

Yes -- you will never hear *me* say "Premature optimization is the root
of all evil." I don't know why Hoare or Dijkstra or Knuth or Wirth or
whoever coined that phrase, but it's been used too many times as an
excuse for not doing any performance engineering, forcing the deployed
"solution" to throw hardware at performance issues.


> 
> 
> There is no correct in left.
> There is no correct in right.
> Correctness is the result of friction caused by the mingling of the two.

"The only good I/O is a dead I/O" -- Mark Friedman

-- 
M. Edward (Ed) Borasky

I've never met a happy clam. In fact, most of them were pretty steamed.

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


[PERFORM] "tablespace" for tranaction log?

2009-01-23 Thread Thomas Finneid

Hi

I have noticed that my transaction log has quite large activity volume 
(up to 15MB per transaction), so with the amount of data I am using I 
have manually moved the pg_xlog directory to a different disk. This 
allows me to have both the table space and transaction log on two 
different high performance disks.
But my question is, since I can do this with tablespace for tables, 
indexes and so on, is there any possibilites to do a similar thing for 
the transaction log from inside postgres? as in


create indexspace = 

regards

thomas

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance