Hi,
Has anyone here investigated what level of bisection bandwidth is needed
for a Hadoop cluster which spans more than one rack?
I'm currently sizing and planning a new Hadoop cluster and I'm wondering
what the performance implications will be if we end up with a cluster
spread across two racks. I'd expect we'll have one 48-port gigabit
switch in each 42u rack. If we end up with 60 systems spread across
these two switches - how much bandwidth should I have between the racks?
I'll have 6 gigabit ports available for links between racks - i.e. up to
6 Gbps. Would this be sufficient bisection bandwidth for Hadoop or
should I be considering increased bandwidth between racks (maybe using
fibre links between the switches or introducing another switch)?
Thanks for any thoughts on this.
-stephen
--
Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie http://webstar.deri.ie http://sindice.com
