Re: [ceph-users] Ceph with Clos IP fabric

2017-04-23 Thread Christian Balzer
On Sun, 23 Apr 2017 21:59:17 -0700 Richard Hesse wrote: > It's not a requirement to build out homogeneous racks of ceph gear. Most > larger places don't do that (it creates weird hot spots). If you have 5 > racks of gear, you're better off spreading out servers in those 5 than just > a pair of ra

Re: [ceph-users] Ceph with Clos IP fabric

2017-04-23 Thread Richard Hesse
It's not a requirement to build out homogeneous racks of ceph gear. Most larger places don't do that (it creates weird hot spots). If you have 5 racks of gear, you're better off spreading out servers in those 5 than just a pair of racks that are really built up. In Aaron's case, he can easily do t

Re: [ceph-users] Ceph with Clos IP fabric

2017-04-23 Thread Christian Balzer
Hello, Aaron pretty much stated most of what I was going to write, but to generalize things and make some points more obvious, I shall pipe up as well. On Sat, 22 Apr 2017 21:45:58 -0700 Richard Hesse wrote: > Out of curiosity, why are you taking a scale-up approach to building your > ceph clus

Re: [ceph-users] Ceph with Clos IP fabric

2017-04-23 Thread Aaron Bassett
We have space limitations in our DCs and so have to build as densely as possibly. These clusters are two racks of 500 osds each, though there is more hardware en route to start scaling them out. With just two racks, the risk of losing a ToR and taking down the cluster was enough to justify the s