Besides the recoverry scenario , in a write only scenario the cluster network will use the almost the same bandwith as public network.
linyunfan Anthony D'Atri <anthony.da...@gmail.com> 于2020年5月9日周六 下午4:32写道： > > > > Hi, > > > > I deployed few clusters with two networks as well as only one network. > > There has little impact between them for my experience. > > > > I did a performance test on nautilus cluster with two networks last week. > > What I found is that the cluster network has low bandwidth usage > > During steady-state, sure. Heartbeats go over that, as do replication ops > when clients write data. > > During heavy recovery or backfill, including healing from failures, > balancing, adding/removing drives, much more will be used. > > Convention wisdom has been to not let that traffic DoS clients, or clients to > DoS heartbeats. > > But this I think dates to a time when 1Gb/s networks were common. If one’s > using modern multiple/bonded 25Gb/s or 40Gb/s links …. > > > while public network bandwidth is nearly full. > > If your public network is saturated, that actually is a problem, last thing > you want is to add recovery traffic, or to slow down heartbeats. For most > people, it isn’t saturated. > > How do you define “full” ? TOR uplinks? TORs to individual nodes? Switch > backplanes? Are you using bonding with the wrong hash policy? > > > As a result, I don't think the cluster network is necessary. > > For an increasing percentage of folks deploying production-quality clusters, > agreed. > > > > > > > Willi Schiegel <willi.schie...@technologit.de> 于2020年5月8日周五 下午6:14写道： > > > >> Hello Nghia, > >> > >> I once asked a similar question about network architecture and got the > >> same answer as Martin wrote from Wido den Hollander: > >> > >> There is no need to have a public and cluster network with Ceph. Working > >> as a Ceph consultant I've deployed multi-PB Ceph clusters with a single > >> public network without any problems. Each node has a single IP-address, > >> nothing more, nothing less. > >> > >> In the current Ceph manual you can read > >> > >> It is possible to run a Ceph Storage Cluster with two networks: a public > >> (front-side) network and a cluster (back-side) network. However, this > >> approach complicates network configuration (both hardware and software) > >> and does not usually have a significant impact on overall performance. > >> For this reason, we generally recommend that dual-NIC systems either be > >> configured with two IPs on the same network, or bonded. > >> > >> I followed the advice from Wido "One system, one IP address" and > >> everything works fine. So, you should be fine with one interface for > >> MONs, MGRs, and OSDs. > >> > >> Best > >> Willi > >> > >> On 5/8/20 11:57 AM, Nghia Viet Tran wrote: > >>> Hi Martin, > >>> > >>> Thanks for your response. You mean one network interface for only MON > >>> hosts or for the whole cluster including OSD hosts? I’m confusing now > >>> because there are some projects that only useone public network for the > >>> whole cluster. That means the rebalancing, replicating objects and > >>> heartbeats from OSD hostswould affects the performance of Ceph client. > >>> > >>> *From: *Martin Verges <martin.ver...@croit.io> > >>> *Date: *Friday, May 8, 2020 at 16:20 > >>> *To: *Nghia Viet Tran <nghia.viet.t...@mgm-tp.com> > >>> *Cc: *"ceph-users@ceph.io" <ceph-users@ceph.io> > >>> *Subject: *Re: [ceph-users] Cluster network and public network > >>> > >>> Hello Nghia, > >>> > >>> just use one network interface card and use frontend and backend traffic > >>> on the same. No problem with that. > >>> > >>> If you have a dual port card, use both ports as an LACP channel and > >>> maybe separate it using VLANs if you want to, but not required as well. > >>> > >>> > >>> -- > >>> > >>> Martin Verges > >>> Managing director > >>> > >>> Mobile: +49 174 9335695 > >>> E-Mail: martin.ver...@croit.io <mailto:martin.ver...@croit.io> > >>> Chat: https://t.me/MartinVerges > >>> > >>> croit GmbH, Freseniusstr. 31h, 81247 Munich > >>> CEO: Martin Verges - VAT-ID: DE310638492 > >>> Com. register: Amtsgericht Munich HRB 231263 > >>> > >>> Web: https://croit.io > >>> YouTube: https://goo.gl/PGE1Bx > >>> > >>> Am Fr., 8. Mai 2020 um 09:29 Uhr schrieb Nghia Viet Tran > >>> <nghia.viet.t...@mgm-tp.com <mailto:nghia.viet.t...@mgm-tp.com>>: > >>> > >>> Hi everyone, > >>> > >>> I have a question about the network setup. From the document, It’s > >>> recommended to have 2 NICs per hosts as described in below picture > >>> > >>> Diagram > >>> > >>> In the picture, OSD hosts will connect to the Cluster network for > >>> replicate and heartbeat between OSDs, therefore, we definitely need > >>> 2 NICs for it. But seems there are no connections between Ceph MON > >>> and Cluster network. Can we install 1 NIC on Ceph MON then? > >>> > >>> I appreciated any comments! > >>> > >>> Thank you! > >>> > >>> -- > >>> > >>> Nghia Viet Tran (Mr) > >>> > >>> _______________________________________________ > >>> ceph-users mailing list -- ceph-users@ceph.io > >>> <mailto:ceph-users@ceph.io> > >>> To unsubscribe send an email to ceph-users-le...@ceph.io > >>> <mailto:ceph-users-le...@ceph.io> > >>> > >>> > >>> _______________________________________________ > >>> ceph-users mailing list -- ceph-users@ceph.io > >>> To unsubscribe send an email to ceph-users-le...@ceph.io > >>> > >> _______________________________________________ > >> ceph-users mailing list -- ceph-users@ceph.io > >> To unsubscribe send an email to ceph-users-le...@ceph.io > >> > > _______________________________________________ > > ceph-users mailing list -- ceph-users@ceph.io > > To unsubscribe send an email to ceph-users-le...@ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
