[OMPI users] RE : RE : RE : Latency of 250 microseconds with Open-MPI 1.4.3, Mellanox Infiniband and 256 MPI ranks

2011-09-21 Thread Sébastien Boisvert
> I would still be suspicious -- ofud is not well tested, and it can definitely > hang if there are network drops. It hanged. > > De : users-boun...@open-mpi.org [users-boun...@open-mpi.org] de la part de > Jeff Squyres [jsquy...@cisco.com] > Date d'env

Re: [OMPI users] Building with thread support on Windows?

2011-09-21 Thread Shiqing Fan
Hi Bjorn, Unfortunately, the currently version of Open MPI for Windows doesn't support Posix nor Solaris threads. However, the work of supporting MinGW is proceeding, which will support GNU compilers for building Open MPI on Windows, and it may partly support pthread, but still needs a lot o

Re: [OMPI users] RE : RE : Latency of 250 microseconds with Open-MPI 1.4.3, Mellanox Infiniband and 256 MPI ranks

2011-09-21 Thread Jeff Squyres
On Sep 21, 2011, at 4:24 PM, Sébastien Boisvert wrote: >> What happens if you run 2 ibv_rc_pingpong's on each node? Or N >> ibv_rc_pingpongs? > > With 11 ibv_rc_pingpong's > > http://pastebin.com/85sPcA47 > > Code to do that => https://gist.github.com/1233173 > > Latencies are around 20 micr

[OMPI users] RE : RE : Latency of 250 microseconds with Open-MPI 1.4.3, Mellanox Infiniband and 256 MPI ranks

2011-09-21 Thread Sébastien Boisvert
> What happens if you run 2 ibv_rc_pingpong's on each node? Or N > ibv_rc_pingpongs? With 11 ibv_rc_pingpong's http://pastebin.com/85sPcA47 Code to do that => https://gist.github.com/1233173 Latencies are around 20 microseconds. My job seems to do well so far with ofud ! [sboisver12@col

Re: [OMPI users] Typo in MPI_Cart_coords man page

2011-09-21 Thread Jeff Squyres
Fixed in the trunk; thanks! On Sep 19, 2011, at 3:14 PM, Jeremiah Willcock wrote: > The bottom of the MPI_Cart_coords man page (in SVN trunk as well as some > releases) states: > > The inverse mapping, rank-to-coordinates translation is provided by > MPI_Cart_coords. > > Although that is true

Re: [OMPI users] RE : Latency of 250 microseconds with Open-MPI 1.4.3, Mellanox Infiniband and 256 MPI ranks

2011-09-21 Thread Jeff Squyres
On Sep 21, 2011, at 3:17 PM, Sébastien Boisvert wrote: > Meanwhile, I contacted some people at SciNet, which is also part of Compute > Canada. > > They told me to try Open-MPI 1.4.3 with the Intel compiler with --mca btl > self,ofud to use the ofud BTL instead of openib for OpenFabrics transpo

[OMPI users] RE : Latency of 250 microseconds with Open-MPI 1.4.3, Mellanox Infiniband and 256 MPI ranks

2011-09-21 Thread Sébastien Boisvert
Hi Yevgeny, You are right on comparing apples with apples. But MVAPICH2 is not installed on colosse, which is in the CLUMEQ consortium, a part of Compute Canada. Meanwhile, I contacted some people at SciNet, which is also part of Compute Canada. They told me to try Open-MPI 1.4.3 with the

Re: [OMPI users] Building with thread support on Windows?

2011-09-21 Thread Björn Regnström
I am building with VS 2008 and the compiler (cl) and the standard libraries that goes with  it, including the windows thread library. I have noted that ompi_info requires either Posix  or Solaris threads to report that open-mpi has thread support. Do I  need to change the  thread library and/or do

Re: [OMPI users] EXTERNAL: Re: Question about compilng with fPIC

2011-09-21 Thread Tim Prince
On 9/21/2011 12:22 PM, Blosch, Edwin L wrote: Thanks Tim. I'm compiling source units and linking them into an executable. Or perhaps you are talking about how OpenMPI itself is built? Excuse my ignorance... The source code units are compiled like this: /usr/mpi/intel/openmpi-1.4.3/bin/mpif90

Re: [OMPI users] Question about compilng with fPIC

2011-09-21 Thread Blosch, Edwin L
Follow-up: I misread the coding, so now I think mpi_iprobe is probably not being used for this case. I'll have to pin the blame somewhere else. -fPIC definitely fixes the problem, as I tried removing -mcmodel=medium and it still worked. Our usual communication pattern is mpi_irecv, mpi_isen

Re: [OMPI users] EXTERNAL: Re: Question about compilng with fPIC

2011-09-21 Thread Blosch, Edwin L
Thanks Tim. I'm compiling source units and linking them into an executable. Or perhaps you are talking about how OpenMPI itself is built? Excuse my ignorance... The source code units are compiled like this: /usr/mpi/intel/openmpi-1.4.3/bin/mpif90 -D_GNU_SOURCE -traceback -align -pad -xHost -f

Re: [OMPI users] Large TCP cluster timeout issue

2011-09-21 Thread Henderson, Brent
The data there would not have helped me too much I'm afraid. I'm used to working with big IB clusters, but needed help with the TCP side of the house. I needed things like the 'mpi_preconnect_all' flag suggestion, sysctl settings for the TCP stack, file descriptor limits for the user and system

Re: [OMPI users] Question about compilng with fPIC

2011-09-21 Thread Tim Prince
On 9/21/2011 11:44 AM, Blosch, Edwin L wrote: Follow-up to a mislabeled thread: "How could OpenMPI (or MVAPICH) affect floating-point results?" I have found a solution to my problem, but I would like to understand the underlying issue better. To rehash: An Intel-compiled executable linked wi

[OMPI users] Question about compilng with fPIC

2011-09-21 Thread Blosch, Edwin L
Follow-up to a mislabeled thread: "How could OpenMPI (or MVAPICH) affect floating-point results?" I have found a solution to my problem, but I would like to understand the underlying issue better. To rehash: An Intel-compiled executable linked with MVAPICH runs fine; linked with OpenMPI fails

Re: [OMPI users] Building with thread support on Windows?

2011-09-21 Thread Tim Prince
On 9/21/2011 11:18 AM, Björn Regnström wrote: Hi, I am trying to build Open MPI 1.4.3 with thread support on Windows. A trivial test program runs if it calls MPI_Init or MP_Init_thread(int *argc, char ***argv, int required, int *provide) with reguired=0 but hangs if required>0. ompi_info for my

[OMPI users] Building with thread support on Windows?

2011-09-21 Thread Björn Regnström
Hi, I am trying to build Open MPI 1.4.3 with thread support on Windows. A trivial test program runs if it calls MPI_Init or MP_Init_thread(int *argc, char ***argv, int required, int *provide) with reguired=0 but hangs if required>0. ompi_info for my build reports that there is no thread support b