To expand upon this, the very nature and existence of Ceph is to replace RAID. The FS itself replicates data and handles the HA functionality that you're looking for. If you're going to build a single server with all those disks, backed by a ZFS RAID setup, you're going to be much better suited with an iSCSI setup. The idea of ceph is that it takes the place of all the ZFS bells and whistles. A CEPH cluster that only has one OSD backed by that huge ZFS setup becomes just a wire-protocol to speak to the server. The magic in ceph comes from the replication and distribution of the data across many OSDs, hopefully living in many hosts. My own setup for instance uses 96 OSDs that are spread across 4 hosts (I know I know guys - CPU is a big deal with SSDs so 24 per host is a tall order - didn't know that when we built it - been working ok so far) that is then distributed between 2 cabinets on 2 separate cooling/power/data zones in our datacenter. My CRUSH map is currently setup for 3 copies of all data, and laid out so that at least one copy is located in each cabinet, and then the cab that gets the 2 copies also makes sure that each copy is on a different host. No RAID needed because ceph makes sure that I have a "safe" amount of copies of the data, in a distribution layout that allows us to sleep at night. In my opinion, ceph is much more pleasant, powerful, and versatile to deal with than both hardware RAID and ZFS (Both of which we have instances of deployed as well from previous iterations of infrastructure deployments). Now, you could always create small little zRAID clusters using ZFS, and then give an OSD to each of those, if you wanted even an additional layer of safety. Heck, you could even have hardware RAID behind the zRAID, for even another layer. Where YOU need to make the decision is the trade-off between HA functionality/peace of mind, performance, and useability/maintainability.
Would me happy to answer any questions you still have... Cheers, -- Stephen Mercier Senior Systems Architect Attainia, Inc. Phone: 866-288-2464 ext. 727 Email: stephen.merc...@attainia.com Web: www.attainia.com Capital equipment lifecycle planning & budgeting solutions for healthcare On Mar 4, 2015, at 10:42 AM, Alexandre DERUMIER wrote: > Hi for hardware, inktank have good guides here: > > http://www.inktank.com/resource/inktank-hardware-selection-guide/ > http://www.inktank.com/resource/inktank-hardware-configuration-guide/ > > ceph works well with multiple osd daemon (1 osd by disk), > so you should not use raid. > > (xfs is the recommended fs for osd daemons). > > you don't need disk spare too, juste enough disk space to handle a disk > failure. > (datas are replicated-rebalanced on other disks/osd in case of disk failure) > > > ----- Mail original ----- > De: "Adrian Sevcenco" <adrian.sevce...@cern.ch> > À: "ceph-users" <ceph-users@lists.ceph.com> > Envoyé: Mercredi 4 Mars 2015 18:30:31 > Objet: [ceph-users] CEPH hardware recommendations and cluster design > questions > > Hi! I seen the documentation > http://ceph.com/docs/master/start/hardware-recommendations/ but those > minimum requirements without some recommendations don't tell me much ... > > So, from what i seen for mon and mds any cheap 6 core 16+ gb ram amd > would do ... what puzzles me is that "per daemon" construct ... > Why would i need/require to have multiple daemons? with separate servers > (3 mon + 1 mds - i understood that this is the requirement) i imagine > that each will run a single type of daemon.. did i miss something? > (beside that maybe is a relation between daemons and block devices and > for each block device should be a daemon?) > > for mon and mds : would help the clients if these are on 10 GbE? > > for osd : i plan to use a 36 disk server as osd server (ZFS RAIDZ3 all > disks + 2 ssds mirror for ZIL and L2ARC) - that would give me ~ 132 TB > how much ram i would really need? (128 gb would be way to much i think) > (that RAIDZ3 for 36 disks is just a thought - i have also choices like: > 2 X 18 RAIDZ2 ; 34 disks RAIDZ3 + 2 hot spare) > > Regarding journal and scrubbing : by using ZFS i would think that i can > safely not use the CEPH ones ... is this ok? > > Do you have some other advises and recommendations for me? (the > read:writes ratios will be 10:1) > > Thank you!! > Adrian > > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
