Re: Understanding Network Utilization of TeraSort

2015-01-12 Thread Andrew Wang
Hi Eitan, Is it possible you have speculative execution enabled? Check to make sure the # of tasks being run matches up with your expectations. You could also try running the same measurements for TeraGen with different replication factors, for another comparison point. Best, Andrew On Fri, Jan

Understanding Network Utilization of TeraSort

2015-01-09 Thread Eitan Rosenfeld
My goal is to see how the performance and network utilization of TeraSort is affected by varying the replication factor from 1-3 on my 16-node cluster. (I have modified TeraSort such that it uses my system's replication factor.) I am sorting 100GB. In particular, I am confused by the network utili