Re: [OMPI users] gfortran bindings apparently not built on mac os leopard

2008-06-18 Thread Jeff Squyres
FWIW, I added this issue to the FAQ under the "OS X" section to make  
it a little easier to find.


On Jun 17, 2008, at 5:17 PM, Jeff Squyres wrote:


Sorry for not replying earlier -- glad you figured it out.

Yes, if you didn't also reset the LIBRARY_PATH, the "wrong" libmpi  
(and friends) will get picked up, and it will therefore use the hard- 
coded directory path in those "wrong" libraries to find the wrapper  
data text files (the ones without Fortran support).



On Jun 17, 2008, at 4:15 PM, Weirs, V Gregory wrote:



Alright, I’m an idiot. I didn’t adjust my DYLD_LIBRARY_PATH.
Sorry about that-
Greg

On 6/17/08 7:08 AM, "Jeff Squyres"  wrote:

This is pretty odd.

What files do you see in $prefix/share/openmpi/mpi*txt?

What do you see from:

   grep required_file $prefix/share/openmpi/mpi*txt



On Jun 16, 2008, at 2:12 PM, Weirs, V Gregory wrote:

>
> Dave Gunter:
>
> According to ./configure —help, by default  —enable-mpi-f77 and —
> enable-mpi-f90 are set. Also, ompi_info (see output in run_output  
in

> the tarball) seems to indicate they are there. I rebuilt with them
> just to be sure, and got the same results.
>
>
> Doug Reeder:
> In my run_output file, you can see results of ‘which mpif77’ that
> shows I am using the version I built. (current is a symlink to  
1.2.6-
> gcc4.0). In any case, I get the same results with the absolute  
path /

> usr/local/openmpi/1.2.6-4.0/bin/mpif77.
>
>
> Thanks,
> Greg
>
>
> On 6/16/08 11:04 AM, "users-requ...@open-mpi.org"  > wrote:
>
> Send users mailing list submissions to
> us...@open-mpi.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://www.open-mpi.org/mailman/listinfo.cgi/users
> or, via email, send a message with subject or body 'help' to
> users-requ...@open-mpi.org
>
> You can reach the person managing the list at
> users-ow...@open-mpi.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of users digest..."
>
>
> Today's Topics:
>
>1. Re: gfortran bindings apparently not built on mac os
> leopard
>   (David Gunter)
>2. Re: gfortran bindings apparently not built on mac os
> leopard
>   (Doug Reeder)
>3. Re: gfortran bindings apparently not built on mac os
> leopard
>   (Doug Reeder)
>4. help me please, about Open MPI (Tony Smith)
>5. Re: help me please, about Open MPI (Andreas Sch?fer)
>
>
>  
--

>
> Message: 1
> Date: Mon, 16 Jun 2008 10:37:22 -0600
> From: David Gunter 
> Subject: Re: [OMPI users] gfortran bindings apparently not built on
> mac os  leopard
> To: Open MPI Users 
> Message-ID: <4683c284-4fef-45ca-a2bc-8be91a439...@lanl.gov>
> Content-Type: text/plain; charset="windows-1252"; Format="flowed";
> DelSp="yes"
>
> I believe you still must add "--enable-f77" and "--enable-f90" to  
the

> OMPI configure line in addition to setting the FC and F77 env
> variables.
>
> -david
> --
> David Gunter
> HPC-3: Parallel Tools Team
> Los Alamos National Laboratory
>
> On Jun 16, 2008, at 10:25 AM, Weirs, V Gregory wrote:
>
> >
> >
> > I am having trouble building mpif77/mpif90 with gfortran on Mac  
OS

> > 10.5. Or maybe just running. The configure, make all, and make
> > install seemed to go just fine, finding my gfortran and  
apparently
> > using it, but the scripts mpif77 and mpif90 give the error that  
my
> > openmpi was not built with fortran bindings. Mpicc and mpicxx  
don?t

> > give this error.  Ompi_info says the f77 and f90 bindings were
> built.
> >
> > I know that OS X 10.5 comes with openmpi mpicc and mpicxx  
installed,

> > but not fortran bindings, and I was careful to put the openmpi I
> > built first in the path.
> >
> > Some run output (mpif77 ?version, ompi_info), config.log,
> > configure.log, make.out, make-install.out are in the attached
> tarball.
> >
> > Any clues?
> >
> > Thanks,
> > Greg
> >
> >
> > --
> > V. Gregory Weirs
> > Sandia National Laboratoriesvgwe...@sandia.gov
> > P.O.Box 5800, MS 0378phone: 505 845 2032
> > Albuquerque, NM 87185-0378  fax: 505 284 0154
> > ___
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> -- next part --
> HTML attachment scrubbed and removed
>
> --
>
> Message: 2
> Date: Mon, 16 Jun 2008 09:32:21 -0700
> From: Doug Reeder 
> Subject: Re: [OMPI users] gfortran bindings apparently not built on
> mac os  leopard
> To: Open MPI Users 
> Message-ID: <5047581c-f2e2-473b-8576-3e11f0d54...@rain.org>
> Content-Type: text/plain; charset="windows-1252"; Format="flowed";
> DelSp="yes"
>
> Greg,
>
> If you use the absolute path names to run your mpif77 and mpif90  
what

> output do you get. In spite of the results from which mpif77, the
> outputs from mpif77 an

Re: [OMPI users] gfortran bindings apparently not built on mac os leopard

2008-06-18 Thread Brian W. Barrett
Thanks for adding the section, but the information contained within is 
wrong (as was the information in this e-mail thread).  OS X essentially 
*always* adds a -rpath to both executables depending on libraries and 
libraries depending on other libraries.  There is no need to set 
LD_LIBRARY_PATH, which on OS X is only used by the linker at link time and 
by dlopen, but not the linker at run time.  Setting DYLD_LIBRARY_PATH is 
not only unnecessary, but considered harmful.  There are very few 
situations where one would need to set DYLD_LIBRARY_PATH, and they should 
never be part of a production environment.


If you built Open MPI as a set of static libraries, there is a library 
search order problem (which neither of the two LIBRARY_PATH variables will 
solve) in that OS X searches the entire library path for a shared library 
of the specified name, then searches the entire library path again for a 
static library of the specified name.  Since /usr/lib/libmpi.dylib would 
then always be found before problem.  An easy workaround is to add -Wl,-search_paths_first to the link 
line when linking MPI applications.  I'd accomplish this by adding the 
following to OMPI's configure:


  --with-wrapper-ldflags="-Wl,-search_paths_first"

Getting back to the original problem, unless the user already has set 
DYLD_LIBRARY_PATH (which they shouldn't do), there is no need to modify 
DYLD_LIBRARY_PATH for OMPI to work properly (even with the system 
installed version in /usr/lib).  We should not mention DYLD_LIBRARY_PATH 
in the FAQ.  If we do mention it, it should only be to say that it should 
not be set.


Brian


On Wed, 18 Jun 2008, Jeff Squyres wrote:

FWIW, I added this issue to the FAQ under the "OS X" section to make it a 
little easier to find.


On Jun 17, 2008, at 5:17 PM, Jeff Squyres wrote:


Sorry for not replying earlier -- glad you figured it out.

Yes, if you didn't also reset the LIBRARY_PATH, the "wrong" libmpi (and 
friends) will get picked up, and it will therefore use the hard-coded 
directory path in those "wrong" libraries to find the wrapper data text 
files (the ones without Fortran support).



On Jun 17, 2008, at 4:15 PM, Weirs, V Gregory wrote:



Alright, I’m an idiot. I didn’t adjust my DYLD_LIBRARY_PATH.
Sorry about that-
Greg

On 6/17/08 7:08 AM, "Jeff Squyres"  wrote:

This is pretty odd.

What files do you see in $prefix/share/openmpi/mpi*txt?

What do you see from:

  grep required_file $prefix/share/openmpi/mpi*txt



On Jun 16, 2008, at 2:12 PM, Weirs, V Gregory wrote:



Dave Gunter:

According to ./configure —help, by default  —enable-mpi-f77 and —
enable-mpi-f90 are set. Also, ompi_info (see output in run_output in
the tarball) seems to indicate they are there. I rebuilt with them
just to be sure, and got the same results.


Doug Reeder:
In my run_output file, you can see results of ‘which mpif77’ that
shows I am using the version I built. (current is a symlink to 1.2.6-
gcc4.0). In any case, I get the same results with the absolute path /
usr/local/openmpi/1.2.6-4.0/bin/mpif77.


Thanks,
Greg


On 6/16/08 11:04 AM, "users-requ...@open-mpi.org" 

> wrote:

Send users mailing list submissions to
us...@open-mpi.org

To subscribe or unsubscribe via the World Wide Web, visit
http://www.open-mpi.org/mailman/listinfo.cgi/users
or, via email, send a message with subject or body 'help' to
users-requ...@open-mpi.org

You can reach the person managing the list at
users-ow...@open-mpi.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of users digest..."


Today's Topics:

   1. Re: gfortran bindings apparently not built on mac os
leopard
  (David Gunter)
   2. Re: gfortran bindings apparently not built on mac os
leopard
  (Doug Reeder)
   3. Re: gfortran bindings apparently not built on mac os
leopard
  (Doug Reeder)
   4. help me please, about Open MPI (Tony Smith)
   5. Re: help me please, about Open MPI (Andreas Sch?fer)


--

Message: 1
Date: Mon, 16 Jun 2008 10:37:22 -0600
From: David Gunter 
Subject: Re: [OMPI users] gfortran bindings apparently not built on
mac os  leopard
To: Open MPI Users 
Message-ID: <4683c284-4fef-45ca-a2bc-8be91a439...@lanl.gov>
Content-Type: text/plain; charset="windows-1252"; Format="flowed";
DelSp="yes"

I believe you still must add "--enable-f77" and "--enable-f90" to the
OMPI configure line in addition to setting the FC and F77 env
variables.

-david
--
David Gunter
HPC-3: Parallel Tools Team
Los Alamos National Laboratory

On Jun 16, 2008, at 10:25 AM, Weirs, V Gregory wrote:

>
>
> I am having trouble building mpif77/mpif90 with gfortran on Mac OS
> 10.5. Or maybe just running. The configure, make all, and make
> install seemed to go just fine, finding my gfortran and apparently
> using it, but the scripts mpif77 and mpif90 give the error that my
> openmpi was not built with fortran b

Re: [OMPI users] gfortran bindings apparently not built on mac os leopard

2008-06-18 Thread Jeff Squyres

Thanks Brian -- I've updated the FAQ.


On Jun 18, 2008, at 9:56 AM, Brian W. Barrett wrote:

Thanks for adding the section, but the information contained within  
is wrong (as was the information in this e-mail thread).  OS X  
essentially *always* adds a -rpath to both executables depending on  
libraries and libraries depending on other libraries.  There is no  
need to set LD_LIBRARY_PATH, which on OS X is only used by the  
linker at link time and by dlopen, but not the linker at run time.   
Setting DYLD_LIBRARY_PATH is not only unnecessary, but considered  
harmful.  There are very few situations where one would need to set  
DYLD_LIBRARY_PATH, and they should never be part of a production  
environment.


If you built Open MPI as a set of static libraries, there is a  
library search order problem (which neither of the two LIBRARY_PATH  
variables will solve) in that OS X searches the entire library path  
for a shared library of the specified name, then searches the entire  
library path again for a static library of the specified name.   
Since /usr/lib/libmpi.dylib would then always be found before install/lib/libmpi.a, you'd have a problem.  An easy workaround is  
to add -Wl,-search_paths_first to the link line when linking MPI  
applications.  I'd accomplish this by adding the following to OMPI's  
configure:


 --with-wrapper-ldflags="-Wl,-search_paths_first"

Getting back to the original problem, unless the user already has  
set DYLD_LIBRARY_PATH (which they shouldn't do), there is no need to  
modify DYLD_LIBRARY_PATH for OMPI to work properly (even with the  
system installed version in /usr/lib).  We should not mention  
DYLD_LIBRARY_PATH in the FAQ.  If we do mention it, it should only  
be to say that it should not be set.


Brian


On Wed, 18 Jun 2008, Jeff Squyres wrote:

FWIW, I added this issue to the FAQ under the "OS X" section to  
make it a little easier to find.


On Jun 17, 2008, at 5:17 PM, Jeff Squyres wrote:


Sorry for not replying earlier -- glad you figured it out.
Yes, if you didn't also reset the LIBRARY_PATH, the "wrong" libmpi  
(and friends) will get picked up, and it will therefore use the  
hard-coded directory path in those "wrong" libraries to find the  
wrapper data text files (the ones without Fortran support).

On Jun 17, 2008, at 4:15 PM, Weirs, V Gregory wrote:

Alright, I’m an idiot. I didn’t adjust my DYLD_LIBRARY_PATH.
Sorry about that-
Greg
On 6/17/08 7:08 AM, "Jeff Squyres"  wrote:
This is pretty odd.
What files do you see in $prefix/share/openmpi/mpi*txt?
What do you see from:

 grep required_file $prefix/share/openmpi/mpi*txt
On Jun 16, 2008, at 2:12 PM, Weirs, V Gregory wrote:

Dave Gunter:
According to ./configure —help, by default  —enable-mpi-f77 and —
enable-mpi-f90 are set. Also, ompi_info (see output in  
run_output in

the tarball) seems to indicate they are there. I rebuilt with them
just to be sure, and got the same results.
Doug Reeder:
In my run_output file, you can see results of ‘which mpif77’ that
shows I am using the version I built. (current is a symlink to  
1.2.6-
gcc4.0). In any case, I get the same results with the absolute  
path /

usr/local/openmpi/1.2.6-4.0/bin/mpif77.
Thanks,
Greg
On 6/16/08 11:04 AM, "users-requ...@open-mpi.org"  wrote:
Send users mailing list submissions to
   us...@open-mpi.org
To subscribe or unsubscribe via the World Wide Web, visit
   http://www.open-mpi.org/mailman/listinfo.cgi/users
or, via email, send a message with subject or body 'help' to
   users-requ...@open-mpi.org
You can reach the person managing the list at
   users-ow...@open-mpi.org
When replying, please edit your Subject line so it is more  
specific

than "Re: Contents of users digest..."
Today's Topics:

  1. Re: gfortran bindings apparently not built on mac os
leopard
 (David Gunter)
  2. Re: gfortran bindings apparently not built on mac os
leopard
 (Doug Reeder)
  3. Re: gfortran bindings apparently not built on mac os
leopard
 (Doug Reeder)
  4. help me please, about Open MPI (Tony Smith)
  5. Re: help me please, about Open MPI (Andreas Sch?fer)
--
Message: 1
Date: Mon, 16 Jun 2008 10:37:22 -0600
From: David Gunter 
Subject: Re: [OMPI users] gfortran bindings apparently not built  
on

   mac os  leopard
To: Open MPI Users 
Message-ID: <4683c284-4fef-45ca-a2bc-8be91a439...@lanl.gov>
Content-Type: text/plain; charset="windows-1252"; Format="flowed";
   DelSp="yes"
I believe you still must add "--enable-f77" and "--enable-f90"  
to the

OMPI configure line in addition to setting the FC and F77 env
variables.
-david
--
David Gunter
HPC-3: Parallel Tools Team
Los Alamos National Laboratory
On Jun 16, 2008, at 10:25 AM, Weirs, V Gregory wrote:
>
>
> I am having trouble building mpif77/mpif90 with gfortran on  
Mac OS

> 10.5. Or maybe just running. The configure, make all, and make
> install seemed to go just fine, finding my gfortr

[OMPI users] Error in mx_init message

2008-06-18 Thread SLIM H.A.

I have OpenMPI-1.2.5 configured with myrinet and infiniband:

OMPI_MCA_btl=openib,self,sm

The job runs with the "error" message 

"Error in mx_init (error MX driver not loaded.)"

which makes sense in itself as there is no myrinet card on the node.

Is it correct to assume that the ib interconnect will be used anyway? 

I saw an old email with a similar query and a comment that this message
would be removed in a future release, has that happened yet?

Thanks

Henk



Re: [OMPI users] Error in mx_init message

2008-06-18 Thread George Bosilca
This is a different issue. On top of Myrinet, Open MPI support two  
nodes: cm and the others. Basically, the difference is how the card  
will be used. If you only specify the btls then Open MPI will try to  
initialize the CM PML and that's how this error message appears. If  
you add OMPI_MCA_pml=^cm to your environment, then this warning will  
go away.


  george.

On Jun 18, 2008, at 4:22 PM, SLIM H.A. wrote:



I have OpenMPI-1.2.5 configured with myrinet and infiniband:

OMPI_MCA_btl=openib,self,sm

The job runs with the "error" message

"Error in mx_init (error MX driver not loaded.)"

which makes sense in itself as there is no myrinet card on the node.

Is it correct to assume that the ib interconnect will be used anyway?

I saw an old email with a similar query and a comment that this  
message

would be removed in a future release, has that happened yet?

Thanks

Henk

___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users




smime.p7s
Description: S/MIME cryptographic signature


[OMPI users] Open MPI timeout problems.

2008-06-18 Thread Peter Diamessis
Dear folks,

I would appreciate your help on the following:

I'm running a parallel CFD code on the Army Research Lab's MJM Linux
cluster, which uses Open-MPI. I've run the same code on other Linux
clusters that use MPICH2 and had never run into this problem.

I'm quite convinced that the bottleneck for my code is this data
transposition routine, although I have not done any rigorous profiling
to check on it. This is where 90% of the parallel communication takes
place. I'm running a CFD code that uses a 3-D rectangular domain which
is partitioned across processors in such a way that each processor
stores vertical slabs that are contiguous in the x-direction but shared
across processors in the y-dir. . When a 2-D Fast Fourier Transform
(FFT) needs to be done, data is transposed such that the vertical slabs
are now contiguous in the y-dir. in each processor. 

The code would normally be run for about 10,000 timesteps. In the
specific case which blocks, the job crashes after ~200 timesteps and at
each timestep a large number of 2-D FFTs are performed. For a domain
with resolution of Nx * Ny * Nz points and P processors, during one FFT,
each processor performs P Sends and P Receives of a message of size
(Nx*Ny*Nz)/P, i.e. there are a total of 2*P^2 such Sends/Receives. 

I've focused on a case using P=32 procs with Nx=256, Ny=128, Nz=175. You
can see that each FFT involves 2048 communications. I totally rewrote my
data transposition routine to no longer use specific blocking/non-
blocking Sends/Receives but to use MPI_ALLTOALL which I would hope is
optimized for the specific MPI Implementation to do data transpositions.
Unfortunately, my code still crashes with time-out problems like before.

This happens for P=4, 8, 16 & 32 processors. The same MPI_ALLTOALL code
worked fine on a smaller cluster here. Note that in the future I would
like to work with resolutions of (Nx,Ny,Nz)=(512,256,533) and P=128 or
256 procs. which will involve an order of magnitude more communication.

Note that I ran the job by submitting it to an LSF queue system. I've
attached the script file used for that. I basically enter bsub -x <
script_openmpi at the command line. 

When I communicated with a consultant at ARL, he recommended I use
3 specific script files which I've attached. I believe these enable
control over some of the MCA parameters. I've experimented with values
of  btl_mvapi_ib_timeout = 14, 18, 20, 24 and 30 and I still have this
problem. I am still in contact with this consultant but thought it would
be good to contact you folks directly.

Note:
a) echo $PATH returns: 

/opt/mpi/x86_64/pgi/6.2/openmpi-1.2/bin:
/opt/compiler/pgi/linux86-64/6.2/bin:/usr/lsf/6.2/linux2.6-glibc2.3-
ia32e/bin:/usr/lsf/6.2/linux2.6-glibc2.3-
ia32e/etc:/usr/cta/modules/3.1.6/bin:
/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/games:/opt/gnome/bin:
.:/usr/lib/java/bin:/opt/gm/bin:/opt/mx/bin:/opt/PST/bin

b) echo $LD_LIBRARY_PATH returns:
/opt/mpi/x86_64/pgi/6.2/openmpi-1.2/lib:
/opt/compiler/pgi/linux86-64/6.2/lib:
/opt/compiler/pgi/linux86-64/6.2/libso:/usr/lsf/6.2/linux2.6-glibc2.3-
ia32e/lib

I've attached the following files:
1) Gzipped versions of the .out & .err files of the failed job.
2) ompi_info.log: The output of ompi_info -all
3) mpirun, mpirun.lsf, openmpi_wrapper: the three script files provided
to me by the ARL consultant. I store these in my home directory and
experimented with the MCA parameter btl_mvapi_ib_timeout in mpirun.
4) The script file script_openmpi that I use to submit the job.

I am unable to provide you with the config.log file as I cannot find it
in the top level Open MPI directory.

I am also unable to provide you with details on the specific cluster
that I'm running in terms of the network. I know they use Infiniband and
some more detail may be found on:

http://www.arl.hpc.mil/Systems/mjm.html

Some other info:
a) uname -a returns: 
Linux l1 2.6.5-7.308-smp.arl-msrc #2 SMP Thu Jan 10 09:18:41 EST 2008
x86_64 x86_64 x86_64 GNU/Linux

b) ulimit -l returns: unlimited

I cannot see a pattern as to which nodes are bad and which are good ...


Note that I found in the mail archives that someone had a similar
problem in transposing a matrix with 16 million elements. The only
answer I found in the thread was to increase the value of
btl_mvapi_ib_timeout to 14 or 16, something I've done already.

I'm hoping that there must be a way out of this problem. I need to
get my code running as I'm under pressure to produce results for a
grant that's paying me.

If you have any feedback I would be hugely grateful.

Sincerely,

Peter Diamessis
Cornell University




308964.err.gz
Description: GNU Zip compressed data


308964.out.gz
Description: GNU Zip compressed data


mpirun
Description: application/shellscript


mpirun.lsf
Description: application/shellscript
Open MPI: 1.2.3
   Open MPI SVN revision: r15136
Open RTE: 1.2.3
   Open RTE SVN revision: r15136
OPAL: 1.2.3
   

Re: [OMPI users] Error in mx_init message

2008-06-18 Thread SLIM H.A.


> -Original Message-
> From: users-boun...@open-mpi.org 
> [mailto:users-boun...@open-mpi.org] On Behalf Of George Bosilca
> Sent: 18 June 2008 15:27
> To: Open MPI Users
> Subject: Re: [OMPI users] Error in mx_init message
> 
> This is a different issue. On top of Myrinet, Open MPI support two
> nodes: cm and the others. Basically, the difference is how 
> the card will be used. If you only specify the btls then Open 
> MPI will try to initialize the CM PML and that's how this 
> error message appears. If you add OMPI_MCA_pml=^cm to your 
> environment, then this warning will go away.
> 
>george.
> 
> On Jun 18, 2008, at 4:22 PM, SLIM H.A. wrote:
> 
> >
> > I have OpenMPI-1.2.5 configured with myrinet and infiniband:
> >
> > OMPI_MCA_btl=openib,self,sm
> >
> > The job runs with the "error" message
> >
> > "Error in mx_init (error MX driver not loaded.)"
> >
> > which makes sense in itself as there is no myrinet card on the node.
> >
> > Is it correct to assume that the ib interconnect will be 
> used anyway?
> >
> > I saw an old email with a similar query and a comment that this 
> > message would be removed in a future release, has that happened yet?
> >
> > Thanks
> >
> > Henk
> >
> > ___
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> 
> 



Re: [OMPI users] Error in mx_init message

2008-06-18 Thread SLIM H.A.
Hi George

I just tried that and got a different error message:


--
No available pml components were found!

This means that there are no components of this type installed on your
system or all the components reported that they could not be used.

This is a fatal error; your MPI process is likely to abort.  Check the
output of the "ompi_info" command and ensure that components of this
type are available on your system.  You may also wish to check the
value of the "component_path" MCA parameter and ensure that it has at
least one directory that contains valid MCA components.


--
[node258:13595] PML ^cm cannot be selected

--

These are the lines in my GridEngine script:

setenv OMPI_MCA_btl "openib,self,sm"
setenv OMPI_MCA_pml "^cm"
mpirun -np $NSLOTS $exe 

The run appears to fail.

Am I doing something wrong here?

Thanks

Henk

> -Original Message-
> From: users-boun...@open-mpi.org 
> [mailto:users-boun...@open-mpi.org] On Behalf Of George Bosilca
> Sent: 18 June 2008 15:27
> To: Open MPI Users
> Subject: Re: [OMPI users] Error in mx_init message
> 
> This is a different issue. On top of Myrinet, Open MPI support two
> nodes: cm and the others. Basically, the difference is how 
> the card will be used. If you only specify the btls then Open 
> MPI will try to initialize the CM PML and that's how this 
> error message appears. If you add OMPI_MCA_pml=^cm to your 
> environment, then this warning will go away.
> 
>george.
> 
> On Jun 18, 2008, at 4:22 PM, SLIM H.A. wrote:
> 
> >
> > I have OpenMPI-1.2.5 configured with myrinet and infiniband:
> >
> > OMPI_MCA_btl=openib,self,sm
> >
> > The job runs with the "error" message
> >
> > "Error in mx_init (error MX driver not loaded.)"
> >
> > which makes sense in itself as there is no myrinet card on the node.
> >
> > Is it correct to assume that the ib interconnect will be 
> used anyway?
> >
> > I saw an old email with a similar query and a comment that this 
> > message would be removed in a future release, has that happened yet?
> >
> > Thanks
> >
> > Henk
> >
> > ___
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> 
> 



Re: [OMPI users] Error in mx_init message

2008-06-18 Thread Rainer Keller
Dear Henk,
could You please try the reverse and specifically select the pml, that would 
adress the openib BTLs, by using:

mpirun -np $NSLOTS --mca pml ob1  $exe

Hope, this helps.

CU,
Rainer

On Mittwoch, 18. Juni 2008, SLIM H.A. wrote:
> Hi George
>
> I just tried that and got a different error message:
>
> 
> --
> No available pml components were found!
>
> This means that there are no components of this type installed on your
> system or all the components reported that they could not be used.
>
> This is a fatal error; your MPI process is likely to abort.  Check the
> output of the "ompi_info" command and ensure that components of this
> type are available on your system.  You may also wish to check the
> value of the "component_path" MCA parameter and ensure that it has at
> least one directory that contains valid MCA components.
>
> 
> --
> [node258:13595] PML ^cm cannot be selected
> 
> --
>
> These are the lines in my GridEngine script:
>
> setenv OMPI_MCA_btl "openib,self,sm"
> setenv OMPI_MCA_pml "^cm"
> mpirun -np $NSLOTS $exe
>
> The run appears to fail.
>
> Am I doing something wrong here?
>
> Thanks
>
> Henk
>
> > -Original Message-
> > From: users-boun...@open-mpi.org
> > [mailto:users-boun...@open-mpi.org] On Behalf Of George Bosilca
> > Sent: 18 June 2008 15:27
> > To: Open MPI Users
> > Subject: Re: [OMPI users] Error in mx_init message
> >
> > This is a different issue. On top of Myrinet, Open MPI support two
> > nodes: cm and the others. Basically, the difference is how
> > the card will be used. If you only specify the btls then Open
> > MPI will try to initialize the CM PML and that's how this
> > error message appears. If you add OMPI_MCA_pml=^cm to your
> > environment, then this warning will go away.
> >
> >george.
> >
> > On Jun 18, 2008, at 4:22 PM, SLIM H.A. wrote:
> > > I have OpenMPI-1.2.5 configured with myrinet and infiniband:
> > >
> > > OMPI_MCA_btl=openib,self,sm
> > >
> > > The job runs with the "error" message
> > >
> > > "Error in mx_init (error MX driver not loaded.)"
> > >
> > > which makes sense in itself as there is no myrinet card on the node.
> > >
> > > Is it correct to assume that the ib interconnect will be
> >
> > used anyway?
> >
> > > I saw an old email with a similar query and a comment that this
> > > message would be removed in a future release, has that happened yet?
> > >
> > > Thanks
> > >
> > > Henk
> > >
> > > ___
> > > users mailing list
> > > us...@open-mpi.org
> > > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users



-- 

Dipl.-Inf. Rainer Keller   http://www.hlrs.de/people/keller
 HLRS  Tel: ++49 (0)711-685 6 5858
 Nobelstrasse 19  Fax: ++49 (0)711-685 6 5832
 70550 Stuttgartemail: kel...@hlrs.de 
 Germany AIM/Skype:rusraink