Hi all, I've been playing with Ceph across high latency high speed links, with a range of results.
In general, Ceph MDS, monitors, and OSDs are solid across thousand kilometre network links. Jitter is low, latency is predictable, and capacity of the network is well beyond what the servers can push. I get the obvious expected slowdowns to do with quorum and other communications, the cluster remaining reliable and in sync. What is the generally accepted maximum latency for a usable Ceph cluster between nodes? I realise it's quite an open ended question ,with quite a number of ifs and buts about it. I am however interested to hear what people have done in production and accepted, and what tweaks have been done. For reference, I have an experimental cluster running across 12ms between the nodes, and while the IOPS itself is down, I'm able to write in at a few hundred Mbps which would cover many use cases. The latency was the obvious issue. David Jericho Senior Systems Administrator AARNet Pty Ltd t. +61 (0) 7 3317 9576 m. +61 423 027 185 e. david.jeri...@aarnet.edu.au w. www.aarnet.edu.au street address: Ground Floor, 143 Coronation Drive, Milton, QLD, 4064, Australia Please consider the environment before printing this email. important This email and any files transmitted with it are confidential, and the rights of confidentiality in such information are not waived or lost by its mistaken delivery to you. Any dissemination, copying, use or disclosure of the email and/or such files without the permission of AARNet, or the sender, is strictly prohibited. If you have received this email in error, please contact the sender immediately and delete all copies of this transmission.
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
