Re: [ceph-users] Planning all flash cluster

2018-06-20 Thread Luis Periquito
Adding more nodes from the beginning would probably be a good idea. On Wed, Jun 20, 2018 at 12:58 PM Nick A wrote: > > Hello Everyone, > > We're planning a small cluster on a budget, and I'd like to request any > feedback or tips. > > 3x Dell R720XD with: > 2x Xeon E5-2680v2 or very similar The

Re: [ceph-users] Planning all flash cluster

2018-06-20 Thread Paul Emmerich
Another great thing about lots of small servers vs. few big servers is that you can use erasure coding. You can save a lot of money by using erasure coding, but performance will have to be evaluated for your use case. I'm working with several clusters that are 8-12 servers with 6-10 SSDs each runn

Re: [ceph-users] Planning all flash cluster

2018-06-20 Thread Blair Bethwaite
This is true, but misses the point that the OP is talking about old hardware already - you're not going to save much money on removing a 2nd hand CPU from a system. On Wed, 20 Jun 2018 at 22:10, Wido den Hollander wrote: > > > On 06/20/2018 02:00 PM, Robert Sander wrote: > > On 20.06.2018 13:58,

Re: [ceph-users] Planning all flash cluster

2018-06-20 Thread Wido den Hollander
On 06/20/2018 02:00 PM, Robert Sander wrote: > On 20.06.2018 13:58, Nick A wrote: > >> We'll probably add another 2 OSD drives per month per node until full >> (24 SSD's per node), at which point, more nodes. > > I would add more nodes earlier to achieve better overall performance. Exactly. No

Re: [ceph-users] Planning all flash cluster

2018-06-20 Thread Paul Emmerich
* More small servers give better performance then few big servers, maybe twice the number of servers with half the disks, cpus and RAM * 2x 10 gbit is usually enough, especially with more servers. that will rarely be the bottleneck (unless you have extreme bandwidth requirements) * maybe save money

Re: [ceph-users] Planning all flash cluster

2018-06-20 Thread Robert Sander
On 20.06.2018 13:58, Nick A wrote: > We'll probably add another 2 OSD drives per month per node until full > (24 SSD's per node), at which point, more nodes. I would add more nodes earlier to achieve better overall performance. Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b,

[ceph-users] Planning all flash cluster

2018-06-20 Thread Nick A
Hello Everyone, We're planning a small cluster on a budget, and I'd like to request any feedback or tips. 3x Dell R720XD with: 2x Xeon E5-2680v2 or very similar 96GB RAM 2x Samsung SM863 240GB boot/OS drives 4x Samsung SM863 960GB OSD drives Dual 40/56Gbit Infiniband using IPoIB. 3 replica, MON