Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 2:45 PM, Shaun Thomas wrote: That read rate and that throughput suggest 8k reads. The queue size is 270+, which is pretty high for a single device, even when it's an SSD. Some SSDs seem to break down on queue sizes over 4, and 15 sectors spread across a read queue of 270 is pretty hash

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 05/23/2013 03:47 PM, Merlin Moncure wrote: [cut] [..] There are two components to the Swingbench test we're running here: the database itself, and the redo log. The redo log stores all changes that are made to the database, which allows the database to be reconstructed in the event of a fa

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Thu, May 23, 2013 at 1:56 AM, Andrea Suisani wrote: > On 05/22/2013 03:30 PM, Merlin Moncure wrote: >> >> On Tue, May 21, 2013 at 7:19 PM, Greg Smith wrote: >>> >>> On 5/20/13 6:32 PM, Merlin Moncure wrote: > > > [cut] > > >>> The only really huge gain to be had using SSD is commit rate at a l

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 05/22/2013 03:30 PM, Merlin Moncure wrote: On Tue, May 21, 2013 at 7:19 PM, Greg Smith wrote: On 5/20/13 6:32 PM, Merlin Moncure wrote: [cut] The only really huge gain to be had using SSD is commit rate at a low client count. There you can easily do 5,000/second instead of a spinning di

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 05/22/2013 07:17 PM, Merlin Moncure wrote: > According the the data sheet it is power safe. > > http://investors.micron.com/releasedetail.cfm?ReleaseID=732650 > http://www.micron.com/products/solid-state-storage/client-ssd/m500-ssd Wow, that seems like a pretty good deal then assuming i

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 23/05/13 14:26, Mark Kirkwood wrote: On 23/05/13 14:22, Greg Smith wrote: On 5/22/13 10:04 PM, Mark Kirkwood wrote: Make that quite a few capacitors (top right corner): http://regmedia.co.uk/2013/05/07/m500_4.jpg There are some more shots and descriptions of the internals in the excellent

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 23/05/13 14:22, Greg Smith wrote: On 5/22/13 10:04 PM, Mark Kirkwood wrote: Make that quite a few capacitors (top right corner): http://regmedia.co.uk/2013/05/07/m500_4.jpg There are some more shots and descriptions of the internals in the excellent review at http://techreport.com/review/

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 10:04 PM, Mark Kirkwood wrote: Make that quite a few capacitors (top right corner): http://regmedia.co.uk/2013/05/07/m500_4.jpg There are some more shots and descriptions of the internals in the excellent review at http://techreport.com/review/24666/crucial-m500-ssd-reviewed That

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Wednesday, May 22, 2013, Joshua D. Drake wrote: > > On 05/22/2013 04:37 PM, Merlin Moncure wrote: >> >> On Wed, May 22, 2013 at 5:42 PM, Joshua D. Drake wrote: >>> >>> I am curious how the 710 or S3700 stacks up against the new M500 from >>> Crucial? I know Intel is kind of the goto for these

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 23/05/13 13:32, Mark Kirkwood wrote: On 23/05/13 13:01, Joshua D. Drake wrote: On 05/22/2013 04:37 PM, Merlin Moncure wrote: On Wed, May 22, 2013 at 5:42 PM, Joshua D. Drake wrote: I am curious how the 710 or S3700 stacks up against the new M500 from Crucial? I know Intel is kind of the

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 4:57 PM, Merlin Moncure wrote: Oh, the major vendors will still keep their rip-off going on a little longer selling their storage trays, raid controllers, entry/mid level SANS, SAS HBAs etc at huge markup to customers who don't need them (some will still need them, but the bar suddenly

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 6:42 PM, Joshua D. Drake wrote: I am curious how the 710 or S3700 stacks up against the new M500 from Crucial? I know Intel is kind of the goto for these things but the m500 is power off protected and rated at: Endurance: 72TB total bytes written (TBW), equal to 40GB per day for 5 year

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 23/05/13 13:01, Joshua D. Drake wrote: On 05/22/2013 04:37 PM, Merlin Moncure wrote: On Wed, May 22, 2013 at 5:42 PM, Joshua D. Drake wrote: I am curious how the 710 or S3700 stacks up against the new M500 from Crucial? I know Intel is kind of the goto for these things but the m500 is p

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 05/22/2013 04:37 PM, Merlin Moncure wrote: On Wed, May 22, 2013 at 5:42 PM, Joshua D. Drake wrote: I am curious how the 710 or S3700 stacks up against the new M500 from Crucial? I know Intel is kind of the goto for these things but the m500 is power off protected and rated at: Endurance: 7

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Wed, May 22, 2013 at 5:42 PM, Joshua D. Drake wrote: > I am curious how the 710 or S3700 stacks up against the new M500 from > Crucial? I know Intel is kind of the goto for these things but the m500 is > power off protected and rated at: Endurance: 72TB total bytes written (TBW), > equal to 40G

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 05/22/2013 01:57 PM, Merlin Moncure wrote: On Wed, May 22, 2013 at 3:06 PM, Greg Smith wrote: You bet, and I haven't recommended anyone buy a 710 since the announcement. However, "hit the street" is still an issue. No one has been able to keep DC S3700 drives in stock very well yet. It t

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On May 22, 2013, at 4:06 PM, Greg Smith wrote: > And there are some other products with interesting price/performance/capacity > combinations that are also sensitive to wearout. Seagate's hybrid drives > have turned interesting now that they cache writes safely for example. > There's no cheap

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Wed, May 22, 2013 at 3:06 PM, Greg Smith wrote: > You bet, and I haven't recommended anyone buy a 710 since the announcement. > However, "hit the street" is still an issue. No one has been able to keep > DC S3700 drives in stock very well yet. It took me three tries through > Newegg before my

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 3:51 PM, Merlin Moncure wrote: s3700 is rated for 10 drive writes/day for 5 years. so, for 200gb drive, that's 200gb * 10/day * 365 days * 5, that's 3.65 million gigabytes or ~ 3.5 petabytes. Yes, they've improved on the 1.5PB that the 710 drives topped out at. For that particular d

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 05/22/2013 02:51 PM, Merlin Moncure wrote: s3700 is rated for 10 drive writes/day for 5 years. so, for 200gb drive, that's 200gb * 10/day * 365 days * 5, that's 3.65 million gigabytes or ~ 3.5 petabytes. Nice. And on that note: http://www.tomshardware.com/reviews/ssd-dc-s3700-raid-0-benchm

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Wed, May 22, 2013 at 2:30 PM, Greg Smith wrote: > On 5/22/13 3:06 PM, Joshua D. Drake wrote: >> >> Greg, can you elaborate on the SSD + Xlog issue? What type of burn >> through are we talking about? > > > You're burning through flash cells at a multiple of the total WAL write > volume. The sys

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 3:06 PM, Joshua D. Drake wrote: Greg, can you elaborate on the SSD + Xlog issue? What type of burn through are we talking about? You're burning through flash cells at a multiple of the total WAL write volume. The system I gave iostat snapshots from upthread (with the Intel 710 hit

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 05/22/2013 11:06 AM, Greg Smith wrote: I have some moderately fast SSD based transactional systems that are still using traditional drives with battery-backed cache for the sequential writes of the WAL volume, where the data volume is on Intel 710 disks. WAL writes really burn through flash

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 05/22/2013 12:31 PM, David Boreham wrote: Device: r/sw/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util sdd 2702.80 19.40 19.67 0.1614.91 273.68 71.74 0.37 100.00 sdd 2707.60 13.00 19.53 0.1014.78 276.61 90.34 0.37 100.00 That's an Intel 710 being cr

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 05/22/2013 01:06 PM, Greg Smith wrote: There's a corresponding price hit though, and having to provision PCI-E cards is a pain in some systems. Oh, totally. Specialist devices like RAMSAN, FusionIO, Virident, or Whiptail are hideously expensive, even compared to high-end SSDs. I was just

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 12:56 PM, Shaun Thomas wrote: Well, you may not be able to make that claim, but I can. While we don't use Intel SSDs, our first-gen FusinoIO cards can deliver about 20k PostgreSQL TPS of our real-world data right off the device before caching effects start boosting the numbers. I've

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/22/2013 8:18 AM, Greg Smith wrote: They can easily hit that number. Or they can do this: Device: r/sw/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util sdd 2702.80 19.40 19.67 0.1614.91 273.68 71.74 0.37 100.00 sdd 2707.60 13.00 19.53 0.1014.78 2

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 05/22/2013 08:30 AM, Merlin Moncure wrote: I'm not claiming to work with extremely high transaction rate systems but then again neither are most of the people reading this list. Disk drives are obsolete for database installations. Well, you may not be able to make that claim, but I can. Whi

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 11:05 AM, Merlin Moncure wrote: unfortunately, I don't have a s3700 to test with, but based on everything i've seen it looks like it's a mostly solved problem. (for example, see here: http://www.storagereview.com/intel_ssd_dc_s3700_series_enterprise_ssd_review). Tests that drive the

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Wed, May 22, 2013 at 9:18 AM, Greg Smith wrote: > On 5/22/13 9:30 AM, Merlin Moncure wrote: >> >> That's most certainly *not* the only gain to be had: random read rates >> of large databases (a very important metric for data analysis) can >> easily hit 20k tps. So I'll stand by the figure. > >

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 9:30 AM, Merlin Moncure wrote: That's most certainly *not* the only gain to be had: random read rates of large databases (a very important metric for data analysis) can easily hit 20k tps. So I'll stand by the figure. They can easily hit that number. Or they can do this: Device:

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Tue, May 21, 2013 at 7:19 PM, Greg Smith wrote: > On 5/20/13 6:32 PM, Merlin Moncure wrote: > >> When it comes to databases, particularly in the open source postgres >> world, hard drives are completely obsolete. SSD are a couple of >> orders of magnitude faster and this (while still slow in c

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/20/13 6:32 PM, Merlin Moncure wrote: When it comes to databases, particularly in the open source postgres world, hard drives are completely obsolete. SSD are a couple of orders of magnitude faster and this (while still slow in computer terms) is fast enough to put storage into the modern a

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Mon, May 20, 2013 at 3:57 PM, Tomas Vondra wrote: > But yes, I do agree that the provider should be ashamed for not > providing reliable SSDs in the first place. Getting reliable SSDs should > be the first option - all these suggestions are really just workarounds > of this rather simple issue.

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 20.5.2013 05:00, Greg Smith wrote: > On 5/16/13 8:06 PM, Tomas Vondra wrote: >> Have you considered using a UPS? That would make the SSDs about as >> reliable as SATA/SAS drives - the UPS may fail, but so may a BBU unit on >> the SAS controller. > > That's not true at all. Any decent RAID cont

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/16/13 7:52 PM, Cuong Hoang wrote: The standby host will be disk-based so it will be less vulnerable to power loss. If it can keep up with replay from the faster master, that sounds like a decent backup. Make sure you setup all write caches very carefully on that system, because it's goi

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 5/16/13 8:06 PM, Tomas Vondra wrote: Have you considered using a UPS? That would make the SSDs about as reliable as SATA/SAS drives - the UPS may fail, but so may a BBU unit on the SAS controller. That's not true at all. Any decent RAID controller will have an option to stop write-back cac

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

Thanks for suggestion Tomas. We're about to set up WAL backup to Amazon S3. I think this should cover all of our bases. At least for the moment, SAS-based standby seems to keep up with the master because that's its sole purpose. We're not sending queries to the hot standby. We also consider switchi

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

Do you really need a running standby for fast failover? What about doing plain WAL archiging? I'd definitely consider that, because even if you setup a SAS-based replica, you can't use it for production as it does no handle the load. I think you could setup WAL archiving and in case of crash just

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 17.5.2013 03:34, Mark Kirkwood wrote: > On 17/05/13 12:06, Tomas Vondra wrote: >> Hi, >> >> On 16.5.2013 16:46, Cuong Hoang wrote: > >>> Pro for the master server. I'm aware of write cache issue on SSDs in >>> case of power loss. However, our hosting provider doesn't offer any >>> other choices

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Fri, May 17, 2013 at 8:17 AM, Merlin Moncure wrote: > On Fri, May 17, 2013 at 1:34 AM, David Rees wrote: >> On Thu, May 16, 2013 at 7:46 AM, Cuong Hoang wrote: >>> For our application, a few seconds of data loss is acceptable. >> >> If a few seconds of data loss is acceptable, I would serious

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Fri, May 17, 2013 at 1:34 AM, David Rees wrote: > On Thu, May 16, 2013 at 7:46 AM, Cuong Hoang wrote: >> For our application, a few seconds of data loss is acceptable. > > If a few seconds of data loss is acceptable, I would seriously look at > the synchronous_commit setting and think about tu

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Thu, May 16, 2013 at 7:46 AM, Cuong Hoang wrote: > For our application, a few seconds of data loss is acceptable. If a few seconds of data loss is acceptable, I would seriously look at the synchronous_commit setting and think about turning that off rather than risk silent corruption with non-e

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On 17/05/13 12:06, Tomas Vondra wrote: Hi, On 16.5.2013 16:46, Cuong Hoang wrote: Pro for the master server. I'm aware of write cache issue on SSDs in case of power loss. However, our hosting provider doesn't offer any other choices of SSD drives with supercapacitor. To minimise risk, we will

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

Hi Tomas, We have a lot of small updates and some inserts. The database size is at 35GB including indexes and TOAST. We think it will keep growing to about 200GB. We usually have a burst of about 500k writes in about 5-10 minutes which basically cripples IO on the current servers. I've tried to in

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

Hi, On 16.5.2013 16:46, Cuong Hoang wrote: > Hi all, > > Our application is heavy write and IO utilisation has been the problem > for us for a while. We've decided to use RAID 10 of 4x500GB Samsung 840 What does "heavy write" mean in your case? Does that mean a lot of small transactions or few l

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

Thank you for your advice guys. We'll definitely turn off init.d script for PostgreSQL on the master. The standby host will be disk-based so it will be less vulnerable to power loss. I forgot to mention that we'll set up Wal-e to ship base backups and WALs to Amazo

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Thu, May 16, 2013 at 11:46 AM, Merlin Moncure wrote: > On Thu, May 16, 2013 at 1:34 PM, Jeff Janes wrote: > > On Thu, May 16, 2013 at 7:46 AM, Cuong Hoang > wrote: > >> > >> Hi all, > >> > >> Our application is heavy write and IO utilisation has been the problem > for > >> us for a while. We

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Thu, May 16, 2013 at 1:34 PM, Jeff Janes wrote: > On Thu, May 16, 2013 at 7:46 AM, Cuong Hoang wrote: >> >> Hi all, >> >> Our application is heavy write and IO utilisation has been the problem for >> us for a while. We've decided to use RAID 10 of 4x500GB Samsung 840 Pro for >> the master serv

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Thu, May 16, 2013 at 7:46 AM, Cuong Hoang wrote: > Hi all, > > Our application is heavy write and IO utilisation has been the problem for > us for a while. We've decided to use RAID 10 of 4x500GB Samsung 840 Pro for > the master server. I'm aware of write cache issue on SSDs in case of power >

Re: [PERFORM] Reliability with RAID 10 SSD and Streaming Replication

On Thu, May 16, 2013 at 9:46 AM, Cuong Hoang wrote: > Hi all, > > Our application is heavy write and IO utilisation has been the problem for > us for a while. We've decided to use RAID 10 of 4x500GB Samsung 840 Pro for > the master server. I'm aware of write cache issue on SSDs in case of power >

[PERFORM] Reliability with RAID 10 SSD and Streaming Replication

Hi all, Our application is heavy write and IO utilisation has been the problem for us for a while. We've decided to use RAID 10 of 4x500GB Samsung 840 Pro for the master server. I'm aware of write cache issue on SSDs in case of power loss. However, our hosting provider doesn't offer any other choi