Re: [ceph-users] Sharing SSD journals and SSD drive choice

On 02-05-17 23:53, David Turner wrote: > I was only interjecting on the comment "So that is 5 . Which is real > easy to obtain" and commenting on what the sustained writes into a > cluster of 2,000 OSDs would require to actually sustain that 5 MBps on > each SSD journal. Reading your calculation b

Re: [ceph-users] Sharing SSD journals and SSD drive choice

I was only interjecting on the comment "So that is 5 . Which is real easy to obtain" and commenting on what the sustained writes into a cluster of 2,000 OSDs would require to actually sustain that 5 MBps on each SSD journal. My calculation was off because I forgot replica size, but my corrected ma

Re: [ceph-users] Sharing SSD journals and SSD drive choice

On 02-05-17 19:54, David Turner wrote: > Are you guys talking about 5Mbytes/sec to each journal device? Even if > you had 8 OSDs per journal and had 2000 osds... you would need a > sustained 1.25 Gbytes/sec to average 5Mbytes/sec per journal device. I'm not sure I'm following this... But I'm rath

Re: [ceph-users] Sharing SSD journals and SSD drive choice

Are you guys talking about 5Mbytes/sec to each journal device? Even if you had 8 OSDs per journal and had 2000 osds... you would need a sustained 1.25 Gbytes/sec to average 5Mbytes/sec per journal device. On Tue, May 2, 2017 at 1:47 PM Willem Jan Withagen wrote: > On 02-05-17 19:16, Дробышевски

Re: [ceph-users] Sharing SSD journals and SSD drive choice

On 02-05-17 19:16, Дробышевский, Владимир wrote: > Willem, > > please note that you use 1.6TB Intel S3520 endurance rating in your > calculations but then compare prices with 480GB model, which has only > 945TBW or 1.1DWPD ( > https://ark.intel.com/products/93026/Intel-SSD-DC-S3520-Series-480GB-

Re: [ceph-users] Sharing SSD journals and SSD drive choice

Willem, please note that you use 1.6TB Intel S3520 endurance rating in your calculations but then compare prices with 480GB model, which has only 945TBW or 1.1DWPD ( https://ark.intel.com/products/93026/Intel-SSD-DC-S3520-Series-480GB-2_5in-SATA-6Gbs-3D1-MLC ). It also worth to notice that S3710

Re: [ceph-users] Sharing SSD journals and SSD drive choice

On 27-4-2017 20:46, Alexandre DERUMIER wrote: > Hi, > >>> What I'm trying to get from the list is /why/ the "enterprise" drives >>> are important. Performance? Reliability? Something else? > > performance, for sure (for SYNC write, > https://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-tes

Re: [ceph-users] Sharing SSD journals and SSD drive choice

il 26, 2017 5:54 PM To: ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com> Subject: Re: [ceph-users] Sharing SSD journals and SSD drive choice Thanks everyone for the replies. I will be avoiding TLC drives, it was just something easy to

Re: [ceph-users] Sharing SSD journals and SSD drive choice

15-April/000610.html >> - >> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-April/000611.html >> - >> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-April/000798.html >> >> Regards, >> Jens Dueholm Christensen >> Rambøl

Re: [ceph-users] Sharing SSD journals and SSD drive choice

> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-April/000611.html > - > http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-April/000798.html > > Regards, > Jens Dueholm Christensen > Rambøll Survey IT > > -----Original Message- > From: ceph-us

Re: [ceph-users] Sharing SSD journals and SSD drive choice

.html > > Regards, > Jens Dueholm Christensen > Rambøll Survey IT > > -Original Message- > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Adam > Carheden > Sent: Wednesday, April 26, 2017 5:54 PM > To: ceph-users@lists.ceph.com &g

Re: [ceph-users] Sharing SSD journals and SSD drive choice

al Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Adam Carheden Sent: Wednesday, April 26, 2017 5:54 PM To: ceph-users@lists.ceph.com Subject: Re: [ceph-users] Sharing SSD journals and SSD drive choice Thanks everyone for the replies. I will be avoiding TLC driv

Re: [ceph-users] Sharing SSD journals and SSD drive choice

On 04/27/2017 12:46 PM, Alexandre DERUMIER wrote: > >>> Also, 4 x Intel DC S3520 costs as much as 1 x Intel DC S3610. Obviously >>> the single drive leaves more bays free for OSD disks, but is there any >>> other reason a single S3610 is preferable to 4 S3520s? Wouldn't 4xS3520s >>> mean: > > wh

Re: [ceph-users] Sharing SSD journals and SSD drive choice

re any >>other reason a single S3610 is preferable to 4 S3520s? Wouldn't 4xS3520s >>mean: where do you see this price difference ? for me , S3520 are around 25-30% cheaper than S3610 - Mail original - De: "Adam Carheden" À: "ceph-users" Envoyé: Merc

Re: [ceph-users] Sharing SSD journals and SSD drive choice

Hi Adam, What Greg and Chris are referring to is the SSD write cliff aka write amplification: - https://flashstorageguy.wordpress.com/tag/write-cliff/ - https://en.wikipedia.org/wiki/Write_amplification This and the MTBF are the main reasons to choose enterprise grade SSDs over consumer grad

Re: [ceph-users] Sharing SSD journals and SSD drive choice

Adam, 2017-04-26 20:54 GMT+05:00 Adam Carheden : > Thanks everyone for the replies. > > Any thoughts on multiple Intel 35XX vs a single 36XX/37XX? All have "DC" > prefixes and are listed in the Data Center section of their marketing > pages, so I assume they'll all have the same quality underlyi

Re: [ceph-users] Sharing SSD journals and SSD drive choice

At a meeting with Intel folks a while back, they discussed the idea that future large devices — which we’re starting to now see — would achieve greater *effective* durability via a lower cost/GB that encourages the use of larger than needed devices. Which is a sort of overprovisioning, just mor

Re: [ceph-users] Sharing SSD journals and SSD drive choice

Thanks everyone for the replies. I will be avoiding TLC drives, it was just something easy to benchmark with existing equipment. I hadn't though of unscrupulous data durability lies or performance suddenly tanking in unpredictable ways. I guess it all comes down to trusting the vendor since it wou

Re: [ceph-users] Sharing SSD journals and SSD drive choice

Hi Adam, How did you settle on the P3608 vs say the P3600 or P3700 for journals? And also the 1.6T size? Seems overkill, unless its pulling double duty beyond OSD journals. Only improvement over the P3x00 is the move from x4 lanes to x8 lanes on the PCIe bus, but the P3600/P3700 offer much mor

Re: [ceph-users] Sharing SSD journals and SSD drive choice

Adam, Before we deployed our cluster, we did extensive testing on all kinds of SSDs, from consumer-grade TLC SATA all the way to Enterprise PCI-E NVME Drives. We ended up going with a ratio of 1x Intel P3608 PCI-E 1.6 TB to 12x HGST 10TB SAS3 HDDs. It provided the best price/performance/den

Re: [ceph-users] Sharing SSD journals and SSD drive choice

On Wed, Apr 26, 2017 at 10:53 AM, Adam Carheden wrote: > What I'm trying to get from the list is /why/ the "enterprise" drives > are important. Performance? Reliability? Something else? Generically, enterprise drives 1) have higher endurance ratings 2) are significantly more capable of fast synch

Re: [ceph-users] Sharing SSD journals and SSD drive choice

What I'm trying to get from the list is /why/ the "enterprise" drives are important. Performance? Reliability? Something else? The Intel was the only one I was seriously considering. The others were just ones I had for other purposes, so I thought I'd see how they fared in benchmarks. The Intel w

Re: [ceph-users] Sharing SSD journals and SSD drive choice

Adam, What David said before about SSD drives is very important. I will tell you another way: use enterprise grade SSD drives, not consumer grade. Also, pay attention to endurance. The only suitable drive for Ceph I see in your tests is SSDSC2BB150G7, and probably it isn't even the most suit

Re: [ceph-users] Sharing SSD journals and SSD drive choice

On 04/25/2017 11:57 AM, David wrote: > On 19 Apr 2017 18:01, "Adam Carheden" > wrote: > > Does anyone know if XFS uses a single thread to write to it's journal? > > > You probably know this but just to avoid any confusion, the journal in > this context isn't the me

Re: [ceph-users] Sharing SSD journals and SSD drive choice

On 19 Apr 2017 18:01, "Adam Carheden" wrote: Does anyone know if XFS uses a single thread to write to it's journal? You probably know this but just to avoid any confusion, the journal in this context isn't the metadata journaling in XFS, it's a separate journal written to by the OSD daemons I