On Monday, May 4, 2015, Christian Balzer <ch...@gol.com> wrote: > On Mon, 13 Apr 2015 10:39:57 +0530 Sanjoy Dasgupta wrote: > > > Hi! > > > > This is an often discussed and clarified topic, but Reason why I am > > asking is because > > > > If We use a RAID controller with Lot of Cache (FBWC) and Configure each > > Drive as Single Drive RAID0, then Write to disks will benefit by using > > FBWC and accelerate I/O performance. Is this correct assumption ? > > > In the case of Ceph, the journal writes (assuming journals on the HDDs, > not separate SSDs) will benefit from this indeed.
Each of my OSD nodes has 7 2TB disks on one RAID card with 1GB of FBWC with BBU. I get pretty good performance. With 6 of the above osd nodes and no replication, using rados bench, I can get around 7500 4k write iops and it can peak to over 10k iops. The write throughput at 4MB is around 1000 MB/s, It will peak to around 1309 MB/s. > > Also, it indeed helps, what are the downside of using RAID Controller > > with Cache in above manner (Apart from Cost of RAID controller) ? > > > If you have too much money or existing HW, knock yourself out. ^.^ > > Aside from the cost, the cache should be battery backed, additional costs > and maintenance issues. > Setting up RAID0 drives can be painful (megacli must die), and adds > another step to OSD replacement/deployment. > The resulting drive may or may not have all SMART features exposed and > (not applicable in your case) won't support TRIM. > > Lastly the cache tends to be small when shared with many HDDs and there is > also competition over it by reads and writes, but that's something to keep > in mind, not a disadvantage per se. Yes, all of the above. I think it is worth the money. I thought I wouldn't need SSD journals, and the raid cards were cheaper than a few good SSDs. Due to my odd workload, I end up with all of my writes being below 256k in size. I have to use iscsi proxy nodes to connect to my VMware hosts. This limits my VM's throughput to around 200MB/s which isn't fast enough for my application. I'm planning on building new osd nodes with the same raid cards and SSD journals to help further coalesce the small writes. > Christian > -- > Christian Balzer Network/Systems Engineer > ch...@gol.com <javascript:;> Global OnLine Japan/Fusion > Communications > http://www.gol.com/ > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com <javascript:;> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
