Re: [OMPI users] OpenMPI locking up only on IB

2008-07-03 Thread Pavel Shamis (Pasha)
Brock Palen wrote: Ok it looks like a bigger problem. The segfault is not related to OMPI because when I go and rebuild 1.2 or another version we use with IB all the time, it will now fail with a segfault when forcing IB. The old libs of the same version still work. They of-course do not ha

Re: [OMPI users] OpenMPI locking up only on IB

2008-07-03 Thread Brock Palen
Ok it looks like a bigger problem. The segfault is not related to OMPI because when I go and rebuild 1.2 or another version we use with IB all the time, it will now fail with a segfault when forcing IB. The old libs of the same version still work. They of-course do not have the flag to t

Re: [OMPI users] OpenMPI locking up only on IB

2008-07-03 Thread Jeff Squyres
On Jul 2, 2008, at 11:51 PM, Pavel Shamis (Pasha) wrote: In trying to build 1.2.6 with the pgi compilers it makes an MPI library that works with tcp, sm. But it segfaults on openib. Both our intel compiler version and pgi version of 1.2.6 blow up like this when we force IB. So this is a n

Re: [OMPI users] OpenMPI locking up only on IB

2008-07-03 Thread Pavel Shamis (Pasha)
In trying to build 1.2.6 with the pgi compilers it makes an MPI library that works with tcp, sm. But it segfaults on openib. Both our intel compiler version and pgi version of 1.2.6 blow up like this when we force IB. So this is a new issue. I have ompi 1.2.6 installed on my machines with I

Re: [OMPI users] OpenMPI locking up only on IB

2008-07-02 Thread Brock Palen
In trying to build 1.2.6 with the pgi compilers it makes an MPI library that works with tcp, sm. But it segfaults on openib. Both our intel compiler version and pgi version of 1.2.6 blow up like this when we force IB. So this is a new issue. Is there a way to shut off early completion in

Re: [OMPI users] OpenMPI locking up only on IB

2008-07-02 Thread Pavel Shamis (Pasha)
May be this FAQ will help : http://www.open-mpi.org/faq/?category=openfabrics#v1.2-use-early-completion Brock Palen wrote: We have a code (arts) that locks up only when running on IB. Works fine on tcp and sm. When we ran it in a debugger. It locked up on a MPI_Comm_split() That as far a

[OMPI users] OpenMPI locking up only on IB

2008-07-02 Thread Brock Palen
We have a code (arts) that locks up only when running on IB. Works fine on tcp and sm. When we ran it in a debugger. It locked up on a MPI_Comm_split() That as far as I could tell was valid. Because the split was a hack they did to use MPI_File_open() on a single cpu, we reworked it to