Re: [OMPI users] very low performance over infiniband

2013-01-28 Thread John Hearns
Have you run ibstat on every single node and made sure all links are up at the correct speed? Have you checkef the output to make sure that you are not domehow running over ethernet?

Re: [OMPI users] very low performance over infiniband

2013-01-28 Thread Shamis, Pavel
Also make sure that processes were not swapped out to a hard drive. Pavel (Pasha) Shamis --- Computer Science Research Group Computer Science and Math Division Oak Ridge National Laboratory On Jan 27, 2013, at 6:39 AM, John Hearns mailto:hear...@googlemail.com>> wrote: 2 percent? Have yo

Re: [OMPI users] very low performance over infiniband

2013-01-27 Thread John Hearns
2 percent? Have you logged into a compute node and run a simple top when the job is running? Are all the processes distributed across the CPU cores? Are the processes being pinned properly to a core? Or are they hopping from core to core? Also make SURE all nodes havenooted with all cores online

[OMPI users] very low performance over infiniband

2013-01-27 Thread Bob Lin
Dear developers, I have met a rather weird problem . Our newly assembled cluster is using openmpi-1.6.3. When doing hpcc test, no err is reported , the network latency ,bandwith , are quite normal. but the GFlops which using HPL and MPIFFT is too low , just 2% of the theoretical value. The sys