I've been running a cluster like this for several years serving VM block devices and CephFS with no difficulty. As previously mentioned it's a question of resources and the corresponding expectations. I use a set of original HP Microservers each with 16GB RAM which is more than enough, although it certainly won't break any speed records.
One aspect to consider is the CRUSH map. Due to the small number of nodes having one go down takes a large portion of the OSDs out. Depending on the layout this could generate a lot of rebalancing load. To avoid this I use three replicas but require them to be on separate nodes. Thus one node down only causes degraded pgs, but with a pool min_size of 2 things are still functional. I wouldn't build a large production cluster in this manner, but it does work and the cluster can grow out of it later. Some process information for reference: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND ceph 13006 1.7 0.5 486588 89180 ? Ssl Apr29 179:39 /usr/bin/ceph-mon ceph 13532 13.9 2.9 1645784 478672 ? Ssl Apr29 1398:26 /usr/bin/ceph-osd ceph 13607 21.5 3.7 1785448 616292 ? Ssl Apr29 2155:32 /usr/bin/ceph-osd ceph 14074 0.1 1.4 631988 239140 ? Ssl Apr29 13:55 /usr/bin/ceph-mds Steve Stock On Fri, May 06, 2016 at 11:07:51AM +0000, Dacre, Alex wrote: > That's good to know. My plan is to have three nodes initially; each with > every role. > > Thanks, > Alex > > -----Original Message----- > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of > Oliver Dzombic > Sent: 06 May 2016 12:06 > To: ceph-users@lists.ceph.com > Subject: Re: [ceph-users] Dual Role Nodes > > Hi Alex, > > thats no problem. > > It might get harder to troubleshoot problems, because the serverressourced > will be used for everything. Also you will have on the same node logs for > every different service. > > But in general, its no problem technically to have all ( Mon+Osd+MDS ) on one > node. > > -- > Mit freundlichen Gruessen / Best regards > > Oliver Dzombic > IP-Interactive > > mailto:i...@ip-interactive.de > > Anschrift: > > IP Interactive UG ( haftungsbeschraenkt ) Zum Sonnenberg 1-3 > 63571 Gelnhausen > > HRB 93402 beim Amtsgericht Hanau > Geschäftsführung: Oliver Dzombic > > Steuer Nr.: 35 236 3622 1 > UST ID: DE274086107 > > > Am 06.05.2016 um 13:00 schrieb Dacre, Alex: > > Hi, > > > > > > > > I'm looking at standing up a small Ceph cluster, but I'm curious as to > > what the general opinion is around having dual role nodes. E.g. each > > node is both an OSD and MON. > > > > > > > > Regards, > > > > Alex > > > > > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ceph.com_lis > > tinfo.cgi_ceph-2Dusers-2Dceph.com&d=DQIF-g&c=O3LcjD-V2Iepl5V0N1424A&r= > > MW8D_KLIjbkgPV4I3ExMPW-6rQA8xlUsAXjzeSmCE80&m=Zrz-dViIT_3QUcQUXdQGZEsH > > av9rT228pj_fFI0a8oQ&s=NZ4hiYs315UX7MIGhTyabHpGqrFXEju-WBZjIVFeGUU&e= > > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ceph.com_listinfo.cgi_ceph-2Dusers-2Dceph.com&d=DQIF-g&c=O3LcjD-V2Iepl5V0N1424A&r=MW8D_KLIjbkgPV4I3ExMPW-6rQA8xlUsAXjzeSmCE80&m=Zrz-dViIT_3QUcQUXdQGZEsHav9rT228pj_fFI0a8oQ&s=NZ4hiYs315UX7MIGhTyabHpGqrFXEju-WBZjIVFeGUU&e= > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
