This page explains what happens quite well: http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-osd/#flapping-osds
"We recommend using both a public (front-end) network and a cluster (back-end) network so that you can better meet the capacity requirements of object replication. Another advantage is that you can run a cluster network such that it isn’t connected to the internet, thereby preventing some denial of service attacks. When OSDs peer and check heartbeats, they use the cluster (back-end) network when it’s available. See Monitor/OSD Interaction for details. However, if the cluster (back-end) network fails or develops significant latency while the public (front-end) network operates optimally, OSDs currently do not handle this situation well. What happens is that OSDs mark each other down on the monitor, while marking themselves up. We call this scenario ‘flapping`." -----Original Message----- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Alexandre DERUMIER Sent: 07 May 2015 12:43 To: MEGATEL / Rafał Gawron Cc: ceph-users Subject: Re: [ceph-users] Networking question Hi, >>If I have two networks (public and cluster network) and one link in public >>network is broken ( cluster network is fine) what I will see in my cluster ? See http://ceph.com/docs/master/rados/configuration/network-config-ref/ only osd between them use private network. so if public network not work (on osd), mon will not see osd, so osd will be out. >>Or how works ceph if link to cluster network was broken and public network is >>fine ? I'm not sure here, osd will be enable to replicate, so maybe it going out itself ? ----- Mail original ----- De: "MEGATEL / Rafał Gawron" <rafal.gaw...@megatel.com.pl> À: "ceph-users" <ceph-users@lists.ceph.com> Envoyé: Jeudi 7 Mai 2015 10:11:20 Objet: [ceph-users] Networking question Hi I have theoretical question about network in ceph. If I have two networks (public and cluster network) and one link in public network is broken ( cluster network is fine) what I will see in my cluster ? How work ceph in this situation ? Or how works ceph if link to cluster network was broken and public network is fine ? CEPH node will available ? --- rgawron _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com Please visit our new website at www.pml.ac.uk and follow us on Twitter @PlymouthMarine Winner of the Environment & Conservation category, the Charity Awards 2014. Plymouth Marine Laboratory (PML) is a company limited by guarantee registered in England & Wales, company number 4178503. Registered Charity No. 1091222. Registered Office: Prospect Place, The Hoe, Plymouth PL1 3DH, UK. This message is private and confidential. If you have received this message in error, please notify the sender and remove it from your system. You are reminded that e-mail communications are not secure and may contain viruses; PML accepts no liability for any loss or damage which may be caused by viruses. _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
