> @Janne: i will checkout/implement the peer config per your suggestion. > However what confuses us is that chrony thinks the clocks match, and > only ceph feels it doesn't. So we are not sure if the peer config will > actually help in this situation. But time will tell.
Ar ar. Chrony thinks that the clocks match *what*, though? That each system matches the public pool against which it’s synced? Something I’ve noticed, especially when using the public pool in Asia, is that DNS rotation results in the pool FQDNs to resolve differently multiple times a day. And that the quality of those servers varies considerably. Naturally the ones in that pool that I set up a few years ago are spot-on, but I digress ;) Consider this scenario: Ceph mon A resolves the pool FQDN to serverX, which is 10ms slow Ceph mon B resolves the pool FQDN to serverY, which is 20ms fast with lots of jitter That can get you a 30ms spread right there. This is the benefit of having the mons peer with each other as well as with upstream servers of varying stratum/quality — worst case, they will select one of their own to sync with. With iburst and modern client polling backoff, there usually isn’t much reason to not configure a bunch of peers. Multiple Public/vendor pool FQDNs are reasonable to include, but I also like to hardcode in a few known-good public peers as well, even one in a different region if necessary. Have your systems peer against each other too. Depending on the size of your operation, consider your own timeserver to deploy on-prem, though antenna placement can be a hassle. This is horribly non-enterprise, but I also suggest picking up one of these: https://www.netburner.com/products/network-time-server/pk70-ex-ntp-network-time-server/ It’s cheap and it can’t handle tens of thousands of clients, but it doesn’t have to. Stick it in an office window and add it to your peer lists. If you have a larger number of clients, have your internal NTP servers configure it (as well as each other, K-complete). If you don’t, include it in their local peer constellation. Best case you have an excellent low-stratum source for your systems for cheap. Worst case you are no worse off than you were before. Now, whether this situation is what you’re seeing I can’t say without more info, but it is at least plausible. _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
