Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem

2013-06-12 Thread Blosch, Edwin L
>>> >>> From: users-boun...@open-mpi.org<mailto:users-boun...@open-mpi.org> >>> [mailto:users-boun...@open-mpi.org<mailto:users-boun...@open-mpi.org>] On >>> Behalf Of Ralph Castain >>> Sent: Sunday, June 09, 2013 6:48 PM >>> To: Open MPI

Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem

2013-06-12 Thread Mike Dubman
6 and found that performance is already falling off. Perhaps > >>> I did not build OpenMPI properly to support the Mellanox adapters that > >>> are used in the backplane, or I need some configuration setting similar > >>> to FAQ #19 in the Tuning/Openfabrics section.

Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem

2013-06-11 Thread Ralph Castain
ion. >>> >>> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On >>> Behalf Of Ralph Castain >>> Sent: Sunday, June 09, 2013 6:48 PM >>> To: Open MPI Users >>> Subject: Re: [OMPI users] EXTERNAL: Re: Need advice on performance

Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem

2013-06-11 Thread eblosch
the Tuning/Openfabrics section. >> >> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On >> Behalf Of Ralph Castain >> Sent: Sunday, June 09, 2013 6:48 PM >> To: Open MPI Users >> Subject: Re: [OMPI users] EXTERNAL: Re: Need advice on performan

Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem

2013-06-11 Thread Ralph Castain
pi.org [mailto:users-boun...@open-mpi.org] On > Behalf Of Ralph Castain > Sent: Sunday, June 09, 2013 6:48 PM > To: Open MPI Users > Subject: Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem > > Strange - it looks like a classic oversubscription behavior. Anoth

Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem

2013-06-11 Thread Blosch, Edwin L
g [mailto:users-boun...@open-mpi.org] On Behalf Of Ralph Castain Sent: Sunday, June 09, 2013 6:48 PM To: Open MPI Users Subject: Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem Strange - it looks like a classic oversubscription behavior. Another possibility is that it isn'

Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem

2013-06-09 Thread Ralph Castain
13 6:18 PM > To: Open MPI Users > Subject: Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem > > So, just to be sure - when you run 320 "cores", you are running across 20 > nodes? > > Just want to ensure we are using "core" the same way - som

Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem

2013-06-09 Thread Blosch, Edwin L
Correct. 20 nodes, 8 cores per dual-socket on each node = 360. From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Ralph Castain Sent: Sunday, June 09, 2013 6:18 PM To: Open MPI Users Subject: Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem So

Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem

2013-06-09 Thread Ralph Castain
So, just to be sure - when you run 320 "cores", you are running across 20 nodes? Just want to ensure we are using "core" the same way - some people confuse cores with hyperthreads. On Jun 9, 2013, at 3:50 PM, "Blosch, Edwin L" wrote: > 16. dual-socket Xeon, E5-2670. > > I am trying a larger

Re: [OMPI users] EXTERNAL: Re: Need advice on performance problem

2013-06-09 Thread Blosch, Edwin L
16. dual-socket Xeon, E5-2670. I am trying a larger model to see if the performance drop-off happens at a different number of cores. Also I'm running some intermediate core-count sizes to refine the curve a bit. I also added mpi_show_mca_params all, and at the same time, btl_openib_use_eager_rd