[OMPI users] Jumbo frames
Hi, I am trying to use Jumbo frames but mpirun will not start the job. I am using OpenMPI v1.1 shipped with the latest Rocks (4.2). Ifconfig is reporting that all the NIC on the cluster are using an MTU of 9000 and the switch (HP Procurve) should be able to use Jumbo frames. Is there any special flag I need to pass to mpirun or a configuration file I need to edit? Thanks Massimiliano
[OMPI users] problem with ompi_info
Hi, I tried to install openmpi-1.1 on a AMD 64 OPTERON dual core (RHEL 4). I got no error message from ./configure and make all install. Not all tests passed (oob_test oob_test_self oob_test_packed and test_schema were skipped)so I entered ompi_info in the bin directory and got the following error message: [root@dhcp76-200 openmpi-1.1]# ompi_info ompi_info: error while loading shared libraries: libmpi.so.0: cannot open shared object file: Nosuch file or directory The library libmpi.so.0 exists in the lib directory and is linked to libmpi.so.0.0.0. Thanks for any help, Christine
Re: [OMPI users] Jumbo frames
Massimiliano, It should work automatically, but I have seen instances where switches or Ethernet cards can't support the full 9000 bytes per frame, and we've had to go as low as 6000 bytes to get consistent performance. It seems like everyone's interpretation of what the 9000 bytes is for is a little different. Does it work with the defaults 1500byte setting? You might try increasing in smaller steps to see where it stops working. Good luck. --andrew > -Original Message- > From: users-boun...@open-mpi.org > [mailto:users-boun...@open-mpi.org] On Behalf Of Massimiliano Fatica > Sent: Friday, August 25, 2006 1:30 AM > To: us...@open-mpi.org > Subject: [OMPI users] Jumbo frames > > Hi, > I am trying to use Jumbo frames but mpirun will not start the job. > I am using OpenMPI v1.1 shipped with the latest Rocks (4.2). > Ifconfig is reporting that all the NIC on the cluster are > using an MTU of 9000 and the switch (HP Procurve) should be > able to use Jumbo frames. > > Is there any special flag I need to pass to mpirun or a > configuration file I need to edit? > > Thanks > Massimiliano > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users >
[OMPI users] OpenMPI-1.1 virtual memory overhead
Hi there, I'm using openmpi-1.1 on a linux-amd64 machine and also a linux-32bit x86 chroot environment on that same machine. (distro is gentoo, compilers: gcc-4.1.1 and gcc-3.4.6) In both cases openmpi-1.1 shows a +/-400MB overhead in virtual memory usage (virtual address space usage) for each MPI process. In my case this is quite troublesome because my application in 32bit mode is counting on using the whole 4GB address space for the problem set size and associated data. This means that I have a reduction in the size of the problems which it can solve. (my aplication isn't 64bit safe yet, so I need to run in 32bit mode, and use effectively the 4GB address space) Is there a way to tweak this overhead, by configuring openmpi to use smaller buffers, or anything else ? I do not see this with mpich2. Best regards, -- Miguel Sousa Filipe
Re: [OMPI users] problem with ompi_info
The directory when the libmpi.so is have to be added to the LD_LIBRARY_PATH, and of course the bin directory have to be added to the PATH. For more information about how and why please read http://www.open-mpi.org/faq/?category=running#adding-ompi-to-path Thanks, george. On Fri, 25 Aug 2006, Christine Kreuzer wrote: Hi, I tried to install openmpi-1.1 on a AMD 64 OPTERON dual core (RHEL 4). I got no error message from ./configure and make all install. Not all tests passed (oob_test oob_test_self oob_test_packed and test_schema were skipped)so I entered ompi_info in the bin directory and got the following error message: [root@dhcp76-200 openmpi-1.1]# ompi_info ompi_info: error while loading shared libraries: libmpi.so.0: cannot open shared object file: Nosuch file or directory The library libmpi.so.0 exists in the lib directory and is linked to libmpi.so.0.0.0. Thanks for any help, Christine ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users "We must accept finite disappointment, but we must never lose infinite hope." Martin Luther King
Re: [OMPI users] OpenMPI-1.1 virtual memory overhead
Hello Miguel, this is caused by the shared memory mempool. Per default this shared memory mapping has a size of 512 MB. You can use the "mpool_sm_size" parameter to reduce size e.g. mpirun -mca mpool_sm_size ... Thanks, Sven On Friday 25 August 2006 15:04, Miguel Figueiredo Mascarenhas Sousa Filipe wrote: > Hi there, > I'm using openmpi-1.1 on a linux-amd64 machine and also a linux-32bit x86 > chroot environment on that same machine. > (distro is gentoo, compilers: gcc-4.1.1 and gcc-3.4.6) > > In both cases openmpi-1.1 shows a +/-400MB overhead in virtual memory usage > (virtual address space usage) for each MPI process. > > In my case this is quite troublesome because my application in 32bit mode is > counting on using the whole 4GB address space for the problem set size and > associated data. > This means that I have a reduction in the size of the problems which it can > solve. > (my aplication isn't 64bit safe yet, so I need to run in 32bit mode, and use > effectively the 4GB address space) > > > Is there a way to tweak this overhead, by configuring openmpi to use smaller > buffers, or anything else ? > > I do not see this with mpich2. > > Best regards, > > -- > Miguel Sousa Filipe >
Re: [OMPI users] OpenMPI-1.1 virtual memory overhead
I suspect this is the shared memory used to communicate between processes. Please run your application adding the flag "--mca btl tcp,self" to the mpirun command line (*before the application name). If the virtual memory usage goes down then the 400MB are definitively comming from the shared memory and there are ways to limit this amount (http://www.open-mpi.org/faq/?category=tuning provide a full range of options). Otherwise ... we will have to find out where they come from differently. Thanks, george. On Fri, 25 Aug 2006, Miguel Figueiredo Mascarenhas Sousa Filipe wrote: Hi there, I'm using openmpi-1.1 on a linux-amd64 machine and also a linux-32bit x86 chroot environment on that same machine. (distro is gentoo, compilers: gcc-4.1.1 and gcc-3.4.6) In both cases openmpi-1.1 shows a +/-400MB overhead in virtual memory usage (virtual address space usage) for each MPI process. In my case this is quite troublesome because my application in 32bit mode is counting on using the whole 4GB address space for the problem set size and associated data. This means that I have a reduction in the size of the problems which it can solve. (my aplication isn't 64bit safe yet, so I need to run in 32bit mode, and use effectively the 4GB address space) Is there a way to tweak this overhead, by configuring openmpi to use smaller buffers, or anything else ? I do not see this with mpich2. Best regards, "We must accept finite disappointment, but we must never lose infinite hope." Martin Luther King
Re: [OMPI users] OpenMPI-1.1 virtual memory overhead
Hi, On 8/25/06, Sven Stork wrote: Hello Miguel, this is caused by the shared memory mempool. Per default this shared memory mapping has a size of 512 MB. You can use the "mpool_sm_size" parameter to reduce size e.g. mpirun -mca mpool_sm_size ... using mpirun -mca mpool_sm_size 0 is acceptable ? to what will it fallback ? sockets? pipes? tcp? smoke signals? thankyou very much by the fast answer. Thanks, Sven On Friday 25 August 2006 15:04, Miguel Figueiredo Mascarenhas Sousa Filipe wrote: > Hi there, > I'm using openmpi-1.1 on a linux-amd64 machine and also a linux-32bit x86 > chroot environment on that same machine. > (distro is gentoo, compilers: gcc-4.1.1 and gcc-3.4.6) > > In both cases openmpi-1.1 shows a +/-400MB overhead in virtual memory usage > (virtual address space usage) for each MPI process. > > In my case this is quite troublesome because my application in 32bit mode is > counting on using the whole 4GB address space for the problem set size and > associated data. > This means that I have a reduction in the size of the problems which it can > solve. > (my aplication isn't 64bit safe yet, so I need to run in 32bit mode, and use > effectively the 4GB address space) > > > Is there a way to tweak this overhead, by configuring openmpi to use smaller > buffers, or anything else ? > > I do not see this with mpich2. > > Best regards, > > -- > Miguel Sousa Filipe > ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users -- Miguel Sousa Filipe
Re: [OMPI users] OpenMPI-1.1 virtual memory overhead
Hello Miguel, On Friday 25 August 2006 15:40, Miguel Figueiredo Mascarenhas Sousa Filipe wrote: > Hi, > > On 8/25/06, Sven Stork wrote: > > > > Hello Miguel, > > > > this is caused by the shared memory mempool. Per default this shared > > memory > > mapping has a size of 512 MB. You can use the "mpool_sm_size" parameter to > > reduce size e.g. > > > > mpirun -mca mpool_sm_size ... > > > > using > mpirun -mca mpool_sm_size 0 > is acceptable ? > to what will it fallback ? sockets? pipes? tcp? smoke signals? 0 will not work. But if you don't need shared memory communication you can disable the sm btl like: mpirun -mca btl ^sm Thanks, Sven > thankyou very much by the fast answer. > > Thanks, > > Sven > > > > On Friday 25 August 2006 15:04, Miguel Figueiredo Mascarenhas Sousa Filipe > > wrote: > > > Hi there, > > > I'm using openmpi-1.1 on a linux-amd64 machine and also a linux-32bit > > x86 > > > chroot environment on that same machine. > > > (distro is gentoo, compilers: gcc-4.1.1 and gcc-3.4.6) > > > > > > In both cases openmpi-1.1 shows a +/-400MB overhead in virtual memory > > usage > > > (virtual address space usage) for each MPI process. > > > > > > In my case this is quite troublesome because my application in 32bit > > mode is > > > counting on using the whole 4GB address space for the problem set size > > and > > > associated data. > > > This means that I have a reduction in the size of the problems which it > > can > > > solve. > > > (my aplication isn't 64bit safe yet, so I need to run in 32bit mode, and > > use > > > effectively the 4GB address space) > > > > > > > > > Is there a way to tweak this overhead, by configuring openmpi to use > > smaller > > > buffers, or anything else ? > > > > > > I do not see this with mpich2. > > > > > > Best regards, > > > > > > -- > > > Miguel Sousa Filipe > > > > > ___ > > users mailing list > > us...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/users > > > > > > -- > Miguel Sousa Filipe >
[OMPI users] LSF with OpenMPI
Is there anyone running OpenMPI on a machine with LSF batch queueing system. Last time I attempted this I discovered that PATH and LD_LIBRARY_PATH were not making it to the client nodes. I could force PATH to work using an OpenMPI option but I could not even force LD_LIBRARY_PATH over to the client nodes. I'd rather fix both and all other environmental variables with one fix so my test case is simply to use openmpi to run hostname. Before I started on this again I'd like to know if anyone has made more progress than I have. Michael
