Re: [OMPI users] Open-MPI and TCP port range
Greetings. Apologies it's taken us so long to reply -- we're all at an Open MPI workshop this week and it's consuming just about all of our time. Right now, there is no way to restrict the port range that Open MPI will use. We simply ask the operating system for available ports and it gives us a random one that is not being used. So if you're setting up servers that used fixed TCP/UDP ports, there shouldn't be a problem -- when MPI jobs run, those servers should be occupying the TCP/UDP ports in question, and therefore the MPI jobs won't use them. If you've got dynamic applications that aren't "always running" (or nearly so) and therefore the TCP/UDP ports in question may not always be reserved, you *could* clash with MPI jobs (e.g., an MPI job could randomly take the port that you're expecting to be able to use). BTW, I'm *assuming* that your jobs will use non-privileged ports. Is this right? If they use privileged ports, then there's no conflict -- Open MPI doesn't use privileged ports. However, in practice, there's little chance of a conflict. The port range is so large that it's unlikely to happen (of course, saying that pretty much guarantees that it *will* happen ;-) ). Right now, there's no way in Open MPI to restrict which TCP ports it uses. If this becomes a problem for you (i.e., the random MPI-chose-the-same-port-as-your-app events happen a lot), let us know and we can probably put in some controls to work around this. > -Original Message- > From: users-boun...@open-mpi.org > [mailto:users-boun...@open-mpi.org] On Behalf Of > laurent.po...@fr.thalesgroup.com > Sent: Tuesday, April 18, 2006 10:09 AM > To: us...@open-mpi.org > Subject: [OMPI users] Open-MPI and TCP port range > > Hi, > > I am a new user of Open-MPI, and I need to use 2 kinds of > programs on an unique cluster : > 1) MPI based programs > 2) Others, using TCP and UDP > > In order to get my non-MPI programs run, I need to know which > ports may be used by MPI programs. > Is there a way to know/set the range of the ports used by MPI > programs ? > > Thanks > Laurent. > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users >
[OMPI users] Configuration error
Folks I am trying to compile openmpi v1.0.2 on MAC OSX 10.4.6 It gives me ac_nonexistent.h error I am enclosing my config.log here config.out Description: Binary data
[OMPI users] Configuration error
Folks I am trying to compile openmpi v1.0.2 on MAC OSX 10.4.6 It gives me ac_nonexistent.h error I am enclosing my config.log here config.out Description: Binary data
Re: [OMPI users] Configuration error
On Apr 20, 2006, at 8:23 AM, sdamjad wrote: Folks I am trying to compile openmpi v1.0.2 on MAC OSX 10.4.6 It gives me ac_nonexistent.h error I am enclosing my config.log here The config.out file you attached points to a problem with your Fortran compiler. However, without seeing the config.log file that configure generated, I can't tell exactly what the issue was. It would be useful if you could include all the information we need for debugging, which is specified at our "Getting Help" web page: http://www.open-mpi.org/community/help/ Thanks, Brian -- Brian Barrett Open MPI developer http://www.open-mpi.org/
[OMPI users] OMPI-F90-CHECK macro needs to be updated?
Getting warnings like: WARNING: *** Fortran 77 alignment for INTEGER (1) does not match WARNING: *** Fortran 90 alignment for INTEGER (4) WARNING: *** OMPI-F90-CHECK macro needs to be updated! same for LOGICAL, REAL, COMPLEX, INTEGER*2, INTEGER*4, INTEGER*8, etc. I believe these are new within the last week. The configure script is generating these warnings when building openmpi-1.1a3r9663 using g95 from April 19 on OS X 10.4.6. Configured with "./configure F77=g95 FC=g95 LDFLAGS=-lSystemStubs" config.log.gz Description: GNU Zip compressed data config-output.txt.gz Description: GNU Zip compressed data
[OMPI users] OpenMPI and SLURM configuration ?
Hi,all£¬ recently I've installed OpenMPI on our cluster (4 nodes, each node got one installation), but finally I found it could not work well with the underlying resource management system SLURM. That is,after I typed the following, it seemed mpirun hang there: [semper@IA64_node2] srun ¨CN 2 ¨CA [semper@IA64_node2] mpirun -np 2 /tmp/cpi I installed OpenMPI and SLURM all under HOME directory of user semper , and each works well separately. I can test with such commands: "srun ¨CN 4 /bin/hostname ¨Cs", there goes correct results( 2 from node2, and 2 from node4). And "mpirun ¨Cnp 4 cpi" works well on the local node.btw ,following is some information. NodeUserHOMEOpenMPI(SLURM) IA64_node0 semper /home/disk0/semper $HOME/openmpi(slurm) IA64_node1 semper /home/disk1/semper $HOME/openmpi(slurm) IA64_node2 semper /home/disk2/semper $HOME/openmpi(slurm) IA64_node3 semper /home/disk3/semper $HOME/openmpi(slurm) PATH=$HOME/openmpi/bin:$HOME/slurm/sbin:$HOME/slurm/bin:$PATH LD_LIBRARY_PATH=$HOME/openmpi/lib:$LD_LIBRARY_PATH Does that different HOME path matter? Can anybody detail me how to configure these two to work together?[ It nearly drives me crazy 8-( ] Thank you.
Re: [OMPI users] OpenMPI and SLURM configuration ?
On Apr 20, 2006, at 9:57 AM, 杨科 wrote: [semper@IA64_node2] srun ¨CN 2 ¨CA [semper@IA64_node2] mpirun -np 2 /tmp/cpi Do you really have a shared /tmp on your cluster ? We do not copy the file on the nodes, they have to be on a shared file system, or at least they have to exist on the same place on all the nodes. george.
Re: [OMPI users] OpenMPI and SLURM configuration ?
>Do you really have a shared /tmp on your cluster ? We do not copy the >file on the nodes, they have to be on a shared file system, or at >least they have to exist on the same place on all the nodes. > > george. > No,but I put a copy of cpi in the /tmp directory on each node. Because I wonder that put the executables under HOME direcotry(which differs) might have some trouble.
Re: [OMPI users] Open-MPI and TCP port range
On Thu, 20 Apr 2006, Jeff Squyres (jsquyres) wrote: > Right now, there is no way to restrict the port range that Open MPI > will use. ... If this becomes a problem for you (i.e., the random > MPI-chose-the-same-port-as-your-app events happen a lot), let us > know and we can probably put in some controls to work around this. I would welcome a discussion about this; on the LAM/MPI lists several people asked for a limited port range to allow them to pass through firewalls or to do tunelling. -- Bogdan Costescu IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868 E-mail: bogdan.coste...@iwr.uni-heidelberg.de
[OMPI users] f90 interface error?: MPI_Comm_get_attr
Error in: openmpi-1.1a3r9663/ompi/mpi/f90/mpi-f90-interfaces.h subroutine MPI_Comm_get_attr(comm, comm_keyval, attribute_val, flag, ierr) include 'mpif.h' integer, intent(in) :: comm integer, intent(in) :: comm_keyval integer(kind=MPI_ADDRESS_KIND), intent(out) :: attribute_val integer, intent(out) :: flag integer, intent(out) :: ierr end subroutine MPI_Comm_get_attr flag should be a logical. Only in C is it an integer. Compare with page 294 of MPI--The Complete Reference, Volume 1 and page 176 of Using MPI, 2nd ed. In my test case I'm using: call MPI_Comm_get_attr(MPI_COMM_WORLD, MPI_UNIVERSE_SIZE, & universe_size, flag, ierr) integer :: ierr integer (kind=MPI_ADDRESS_KIND) :: universe_size logical :: flag This compiled and worked as of version 9427. Michael
Re: [OMPI users] f90 interface error?: MPI_Comm_get_attr
The file 'ompi/mpi/f90/mpi-f90-interfaces.h' is automatically
generated by ompi/mpi/f90/scripts/mpi-f90-interfaces.h.sh? I
couldn't get my temp fix to stick so I modified the latter.
Should be?
subroutine ${procedure}(comm, comm_keyval, attribute_val, flag, ierr)
include 'mpif.h'
integer, intent(in) :: comm
integer, intent(in) :: comm_keyval
integer(kind=MPI_ADDRESS_KIND), intent(out) :: attribute_val
logical, intent(out) :: flag
integer, intent(out) :: ierr
end subroutine ${procedure}
EOF
}
start MPI_Comm_get_attr small
output MPI_Comm_get_attr
end MPI_Comm_get_attr
I notice that f77 is correct in: ompi/mpi/f77/prototypes_mpi.h
PN(void, mpi_comm_get_attr, MPI_COMM_GET_ATTR, (MPI_Fint *comm,
MPI_Fint *comm_keyval, MPI_Aint *attribute_val, MPI_Flogical *flag,
MPI_Fint *ierr));
On Apr 20, 2006, at 2:24 PM, Michael Kluskens wrote:
Error in:
openmpi-1.1a3r9663/ompi/mpi/f90/mpi-f90-interfaces.h
subroutine MPI_Comm_get_attr(comm, comm_keyval, attribute_val, flag,
ierr)
include 'mpif.h'
integer, intent(in) :: comm
integer, intent(in) :: comm_keyval
integer(kind=MPI_ADDRESS_KIND), intent(out) :: attribute_val
integer, intent(out) :: flag
integer, intent(out) :: ierr
end subroutine MPI_Comm_get_attr
flag should be a logical. Only in C is it an integer.
Compare with page 294 of MPI--The Complete Reference, Volume 1 and
page 176 of Using MPI, 2nd ed.
In my test case I'm using:
call MPI_Comm_get_attr(MPI_COMM_WORLD, MPI_UNIVERSE_SIZE, &
universe_size, flag, ierr)
integer :: ierr
integer (kind=MPI_ADDRESS_KIND) :: universe_size
logical :: flag
This compiled and worked as of version 9427.
Michael
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] Open-MPI and TCP port range
> -Original Message- > From: users-boun...@open-mpi.org > [mailto:users-boun...@open-mpi.org] On Behalf Of Bogdan Costescu > Sent: Thursday, April 20, 2006 10:32 AM > To: Open MPI Users > Subject: Re: [OMPI users] Open-MPI and TCP port range > > On Thu, 20 Apr 2006, Jeff Squyres (jsquyres) wrote: > > > Right now, there is no way to restrict the port range that Open MPI > > will use. ... If this becomes a problem for you (i.e., the random > > MPI-chose-the-same-port-as-your-app events happen a lot), let us > > know and we can probably put in some controls to work around this. > > I would welcome a discussion about this; on the LAM/MPI lists several > people asked for a limited port range to allow them to pass through > firewalls or to do tunneling. Recall that we didn't end up doing this in LAM because limiting the port ranges is not necessarily sufficient to allow you to run parallel computing spanning firewalls. The easiest solution is to have a single routing entity that can be exposed publicly (in front of the firewall, either virtually or physically) that understands MPI -- so that MPI processes outside the firewall can send to this entity and it routes the messages to the appropriate back-end MPI process. This routable entity does not exist for LAM (*), and does not yet exist for Open MPI (there have been discussions about creating it, but nothing has been done about it). (*) Disclaimer: the run-time environment for LAM actually does support this kind of routing, but we stopped actively maintaining it years ago -- it may or may not work at the MPI layer. Other scenarios are also possible, two of which include: 1. make a virtual public IP address in front of the firewall for each back-end node. MPI processes who send data to the public IP address will be routed [by the firewall] back to the back-end node. 2. use a single virtual public IP address in front of the firewall with N ports open. MPI processes who send data to the public IP address dispatch to the back-end node [by the firewall] based on the port number. Both of these require opening a bunch of holes in the firewall which is at least somewhat unattractive. So probably the best solution is to have an MPI-level routable entity that can do this stuff. Then you only need one public IP address and potentially a small number of ports opened. That being said, we are not opposed to putting port number controls in Open MPI. Especially if it really is a problem for someone, not just a hypothetical problem ;-). But such controls should not be added to support firewalled operations, because -- at a minimum -- unless you do a bunch of other firewall configuration, it will not be enough. -- Jeff Squyres Server Virtualization Business Unit Cisco Systems
Re: [OMPI users] OpenMPI and SLURM configuration ?
No, the location of $HOME should not matter. What happens if you "mpirun -np 2 uptime"? (i.e., use mpirun to launch a non-MPI application) > -Original Message- > From: users-boun...@open-mpi.org > [mailto:users-boun...@open-mpi.org] On Behalf Of ?? > Sent: Thursday, April 20, 2006 10:17 AM > To: us...@open-mpi.org > Subject: Re: [OMPI users] OpenMPI and SLURM configuration ? > > >Do you really have a shared /tmp on your cluster ? We do not > copy the > >file on the nodes, they have to be on a shared file system, or at > >least they have to exist on the same place on all the nodes. > > > > george. > > > > No,but I put a copy of cpi in the /tmp directory on each node. > Because I wonder that put the executables under HOME > direcotry(which differs) > might have some trouble. > > > > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users >
Re: [OMPI users] Open-MPI and TCP port range
Just as a further point here - the biggest issue with making a routable public IP address is deciding what that address should be. This is not a simple problem as (a) we operate exclusively at the user level, and so (b) we can't define a single address that we can reliably know from a remote location (we would need at least one for every user, which isn't a feasible solution). Our solution so far is to launch a "probe" process on the publicly-accessible node and have it check a known location on the file system for the IP address of the accessible "router" for this user. Not a perfect solution by any means - but that's where we are for now. Ralph Jeff Squyres (jsquyres) wrote: -Original Message- From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Bogdan Costescu Sent: Thursday, April 20, 2006 10:32 AM To: Open MPI Users Subject: Re: [OMPI users] Open-MPI and TCP port range On Thu, 20 Apr 2006, Jeff Squyres (jsquyres) wrote: Right now, there is no way to restrict the port range that Open MPI will use. ... If this becomes a problem for you (i.e., the random MPI-chose-the-same-port-as-your-app events happen a lot), let us know and we can probably put in some controls to work around this. I would welcome a discussion about this; on the LAM/MPI lists several people asked for a limited port range to allow them to pass through firewalls or to do tunneling. Recall that we didn't end up doing this in LAM because limiting the port ranges is not necessarily sufficient to allow you to run parallel computing spanning firewalls. The easiest solution is to have a single routing entity that can be exposed publicly (in front of the firewall, either virtually or physically) that understands MPI -- so that MPI processes outside the firewall can send to this entity and it routes the messages to the appropriate back-end MPI process. This routable entity does not exist for LAM (*), and does not yet exist for Open MPI (there have been discussions about creating it, but nothing has been done about it). (*) Disclaimer: the run-time environment for LAM actually does support this kind of routing, but we stopped actively maintaining it years ago -- it may or may not work at the MPI layer. Other scenarios are also possible, two of which include: 1. make a virtual public IP address in front of the firewall for each back-end node. MPI processes who send data to the public IP address will be routed [by the firewall] back to the back-end node. 2. use a single virtual public IP address in front of the firewall with N ports open. MPI processes who send data to the public IP address dispatch to the back-end node [by the firewall] based on the port number. Both of these require opening a bunch of holes in the firewall which is at least somewhat unattractive. So probably the best solution is to have an MPI-level routable entity that can do this stuff. Then you only need one public IP address and potentially a small number of ports opened. That being said, we are not opposed to putting port number controls in Open MPI. Especially if it really is a problem for someone, not just a hypothetical problem ;-). But such controls should not be added to support firewalled operations, because -- at a minimum -- unless you do a bunch of other firewall configuration, it will not be enough.
Re: [OMPI users] (no subject)
> No, the location of $HOME should not matter. > > What happens if you "mpirun -np 2 uptime"? (i.e., use mpirun to launch > a non-MPI application) > Thanks. it returns right result! but still only 2 local processes. I tried again to add a hostfile option "--hostfile $HOME/openmpi/bin/hostfile" to mpirun, with the hostfile containning two items: IA64_node0 and IA64_node1 ,but this time it complained: [time returned by IA64_node0] bash line 1: orted : command not found oops,did I do a misconfig for openmpi? btw,openmpi was installed under HOME directory of user semper on each node of the cluster. The 4 nodes can "ssh" or "rsh" login in each other without password prompt.
Re: [OMPI users] OpenMPI and SLURM configuration ??
> No, the location of $HOME should not matter. > > What happens if you "mpirun -np 2 uptime"? (i.e., use mpirun to launch > a non-MPI application) > Thanks. it returns right result! but still only 2 local processes. I tried again to add a hostfile option "--hostfile $HOME/openmpi/bin/hostfile" to mpirun, with the hostfile containning two items: IA64_node0 and IA64_node1 ,but this time it complained: [time returned by IA64_node0] bash line 1: orted : command not found oops! Should I have done a misconfig for openmpi? btw,openmpi was installed under HOME directory of user semper on each node of the cluster. The 4 nodes can "ssh" or "rsh" login in each other without password prompt.
Re: [OMPI users] (no subject)
I think that you need to install Open MPI into other machine as well. You might want to setup NSF (network file system) for the master (you are saying your local machine) so that your slave nodes could see your mpi executable. > bash line 1: orted : command not found This error might go away if you install Open MPI into all machines. Thank you, Sang Chul On 4/20/06, semper wrote: > > > No, the location of $HOME should not matter. > > > > What happens if you "mpirun -np 2 uptime"? (i.e., use mpirun to launch > > a non-MPI application) > > > Thanks. > it returns right result! but still only 2 local processes. > > I tried again to add a hostfile option "--hostfile > $HOME/openmpi/bin/hostfile" to > mpirun, with the hostfile containning two items: IA64_node0 and IA64_node1 > ,but > this time it complained: > [time returned by IA64_node0] > bash line 1: orted : command not found > > oops,did I do a misconfig for openmpi? > btw,openmpi was installed under HOME directory of user semper on each node > of the > cluster. The 4 nodes can "ssh" or "rsh" login in each other without > password > prompt. > > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users >
