Re: [OMPI users] Binding to Core Warning

I did one "chapter" of it on Jeff's blog and probably should complete it. Definitely need to update the FAQ for the new options. Sadly, outside of that and the mpirun man page, there isn't much available yet. I'm woefully far behind on it. On Feb 26, 2014, at 4:47 PM, Saliya Ekanayake wrote:

Re: [OMPI users] Binding to Core Warning

Thank you Ralph, this is very insightful and I think I can better understand performance of our application. If I may ask, is there a document describing this affinity options? I've been looking at tuning FAQ and Jeff's blog posts. Thank you, Saliya On Wed, Feb 26, 2014 at 7:34 PM, Ralph Castai

Re: [OMPI users] Binding to Core Warning

On Feb 26, 2014, at 4:29 PM, Saliya Ekanayake wrote: > I see, so if I understand correctly, the best scenario for threads would be > to bind 2 procs to sockets as --map-by socket:pe=4 and use 4 threads in each > proc. Yes, that would be the best solution. If you have 4 cores in each socket,

Re: [OMPI users] Binding to Core Warning

I see, so if I understand correctly, the best scenario for threads would be to bind 2 procs to sockets as --map-by socket:pe=4 and use 4 threads in each proc. Also, as you've mentioned binding threads to get memory locality, I guess this has to be done at application level and not an option in OMP

Re: [OMPI users] OrangeFS ROMIO support

that was my fault, I did not follow up the time, got probably side tracked by something. Anyway, I suspect that you actually have the patch, otherwise the current Open MPI trunk and the 1.7 release series would not have the patch after the last ROMIO update - at least I did not reapply it, not sur

Re: [OMPI users] OrangeFS ROMIO support

On Tue, 2014-02-25 at 07:26 -0600, Edgar Gabriel wrote: > this was/is a bug in ROMIO, in which they assume a datatype is an int. I > fixed it originally in a previous version of Open MPI on the trunk, but > it did not get ported upstream, so we might have to do the same fix again. > Sorry about t

Re: [OMPI users] Connection timed out with multiple nodes

o I should report there has been an important developement in this problem, before anyone spends time on my previous post. We have got the original test program to run without hanging by directly connecting the two test compute nodes together (thus bypassing the switch) as shown here, where eth2

Re: [OMPI users] OpenMPI-ROMIO-OrangeFS

On Wed, 2014-02-26 at 15:27 -0600, Edgar Gabriel wrote: > ok, then this must be a difference between OrangeFS and PVFs2. It turns > out that trunk and 1.7 does actually have the patch, but 1.6 series does > not have it. The actual commit was done in > > https://svn.open-mpi.org/trac/ompi/changese

Re: [OMPI users] Binding to Core Warning

Sorry, had to run some errands. On Feb 26, 2014, at 1:03 PM, Saliya Ekanayake wrote: > Is it possible to bind to cores of multiple sockets? Say I have a machine > with 2 sockets each with 4 cores and if I run 8 threads with 1 proc can I > utilize all 8 cores for 8 threads? In that scenario, y

Re: [OMPI users] OpenMPI-ROMIO-OrangeFS

ok, then this must be a difference between OrangeFS and PVFs2. It turns out that trunk and 1.7 does actually have the patch, but 1.6 series does not have it. The actual commit was done in https://svn.open-mpi.org/trac/ompi/changeset/24768 and based on the line numbers, I think it should apply cl

Re: [OMPI users] OpenMPI-ROMIO-OrangeFS

At first Thank you very much for your time. "--with-file-system=pvfs2+ufs+nfs" didnt help. But if find (by google) some part of orangefs test. I dont know what is this exactly doing, but when I edited source code of OpenMPI like doing this line, all seems that it is working now. (changing ADIOI_

Re: [OMPI users] Binding to Core Warning

Is it possible to bind to cores of multiple sockets? Say I have a machine with 2 sockets each with 4 cores and if I run 8 threads with 1 proc can I utilize all 8 cores for 8 threads? Thank you for speedy replies Saliya On Wed, Feb 26, 2014 at 3:21 PM, Ralph Castain wrote: > > On Feb 26, 2014,

Re: [OMPI users] Binding to Core Warning

On Feb 26, 2014, at 12:17 PM, Saliya Ekanayake wrote: > I have a followup question on this. In our application we have parallel for > loops similar to OMP parallel for. I noticed that in order to gain speedup > with threads I've to set --bind-to none, otherwise multiple threads will bind > to

Re: [OMPI users] Binding to Core Warning

I have a followup question on this. In our application we have parallel for loops similar to OMP parallel for. I noticed that in order to gain speedup with threads I've to set --bind-to none, otherwise multiple threads will bind to same core giving no increase in performance. For example, I get fol

Re: [OMPI users] OpenMPI-ROMIO-OrangeFS

not sure whether its the problem or not, but usually have an additional flag set : --with-io-romio-flags="--with-file-system=pvfs2+ufs+nfs --with-pvfs2=/opt/pvfs-2.8.2" compilation is a bit slow for me today... Edgar On 2/26/2014 2:05 PM, vithanousek wrote: > Now I compiled by doing this: > O

Re: [OMPI users] OpenMPI-ROMIO-OrangeFS

Now I compiled by doing this: OrangeFS (original, withou editing): ./configure --prefix=/usr/local/orangefs --with-kernel=/usr/src/kernels/2.6. 32-431.5.1.el6.x86_64 --with-openib=/usr --without-bmi-tcp --enable-shared make make kmod make install make kmod_install Without error. OpenMPI (with ed

Re: [OMPI users] OpenMPI-ROMIO-OrangeFS

Thanks for your Time, I'm little bit confused, what is diferent between pvfs2 and orangefs. I was thinking, that only project changes name. I get hint from OrangeFS maillist, to compile OrangeFs with --enable-shared. This produce a some shared library (.so) in /usr/local/orangefs/lib and I can

Re: [OMPI users] OpenMPI-ROMIO-OrangeFS

so we had ROMIO working with PVFS2 (not OrangeFS, which is however registered as PVFS2 internally). We have one cluster which uses OrangeFS, on that machine however we used OMPIO, not ROMIO. I am currently compiling the 1.6 version of Open MPI to see whether I can reproduce your problem. Thanks Ed

Re: [OMPI users] OpenMPI-ROMIO-OrangeFS

Thanks for reply, Is it possible that the patch solvs all this problems, not only "switch => ifs" problem? I realy dont know, wher the problem is now (OpenMPI, ROMIO, OrangeFS). Thanks Hanousek Vít -- Původní zpráva -- Od: Ralph Castain Komu: Open MPI Users Datum: 26

Re: [OMPI users] Compiling Open MPI 1.7.4 using PGI 14.2 and Mellanox HCOLL enabled

Perhaps you could try the nightly 1.7.5 tarball? I believe some PGI fixes may have gone in there On Feb 25, 2014, at 3:22 PM, Filippo Spiga wrote: > Dear all, > > I came across another small issue while I was compiling Open MPI 1.7.4 using > PGI 14.2 and building the support for Mellanox Hie

Re: [OMPI users] OpenMPI-ROMIO-OrangeFS

Edgar hasn't had a chance to find the necessary patch - he was on travel, returning soon. On Feb 26, 2014, at 9:27 AM, vithanousek wrote: > Hello, > > I have still problems with compiling OpenMPI 1.6.5 with OrangeFS 2.8.7 > support. > > I compiled OrangeFS by this: > > ./configure --prefi

[OMPI users] OpenMPI-ROMIO-OrangeFS

Hello, I have still problems with compiling OpenMPI 1.6.5 with OrangeFS 2.8.7 support. I compiled OrangeFS by this:  ./configure --prefix=/usr/local/orangefs2 --with-kernel=/usr/src/kernels/ 2.6.32-431.5.1.el6.x86_64 --with-openib=/usr --without-bmi-tcp make -j 32 make -j 32 kmod make install m

Re: [OMPI users] Binding to Core Warning

Thank you Ralph, I'll check this. On Wed, Feb 26, 2014 at 10:04 AM, Ralph Castain wrote: > It means that OMPI didn't get built against libnuma, and so we can't > ensure that memory is being bound local to the proc binding. Check to see > if numactl and numactl-devel are installed, or you can tu

Re: [OMPI users] slowdown with infiniband and latest CentOS kernel

Hi, I found this thread from before Christmas, and I wondered what the status of this problem is. We experience the same problems since our upgrade to Scientific Linux 6.4, kernel 2.6.32-431.1.2.el6.x86_64, and OpenMPI 1.6.5. Users have reported severe slowdowns in all kinds of applications

Re: [OMPI users] OpenMPI 1.7.5 and "--map-by" new syntax

My bad - I'll fix the help message. Thanks! On Feb 26, 2014, at 6:42 AM, Filippo Spiga wrote: > Yes it works. Information provided by mpirun is confusing but I get the right > syntax now. Thank you! > > F > > > > On Feb 26, 2014, at 12:34 PM, tmish...@jcity.maeda.co.jp wrote: >> Hi, this he

Re: [OMPI users] Binding to Core Warning

It means that OMPI didn't get built against libnuma, and so we can't ensure that memory is being bound local to the proc binding. Check to see if numactl and numactl-devel are installed, or you can turn off the warning using "-mca hwloc_base_mem_bind_failure_action silent" On Feb 25, 2014, at

Re: [OMPI users] OpenMPI 1.7.5 and "--map-by" new syntax

Yes it works. Information provided by mpirun is confusing but I get the right syntax now. Thank you! F On Feb 26, 2014, at 12:34 PM, tmish...@jcity.maeda.co.jp wrote: > Hi, this help message might be just a simple mistake. > > Please try: mpirun -np 20 --map-by ppr:5:socket -bind-to core osu

Re: [OMPI users] OpenMPI 1.7.5 and "--map-by" new syntax

Hi, this help message might be just a simple mistake. Please try: mpirun -np 20 --map-by ppr:5:socket -bind-to core osu_alltoall There's no available explanation yet as far as I know, because it's still alfa version. Tetsuya Mishima > Dear all, > > I am playing with Open MPI 1.7.5 and with th

[OMPI users] OpenMPI 1.7.5 and "--map-by" new syntax

Dear all, I am playing with Open MPI 1.7.5 and with the "--map-by" option but I am not sure I am doing thing correctly despite I am following the instruction. Here what I got $mpirun -np 20 --npersocket 5 -bind-to core osu_alltoall --

Re: [OMPI users] run a program

Hi Raha Yes, that is correct. You have to make sure that max-slots is less or equal to the number of cpus in the node to avoid oversubscribing it. Have a look at the other entries in the FAQ, they give information on many other options you can use. http://www.open-mpi.org/faq/?category=running

Re: [OMPI users] run a program

Dear Jody Thank you for your reply. Based on hostfile examples you show me, I understand 'slots' is number of cpus of each node I mentioned in the file, am I true? Wishes On Wed, Feb 26, 2014 at 1:02 PM, jody wrote: > Hi > I think you should use the "--host" or "--hostfile" options: > http:

Re: [OMPI users] run a program

Dear John Hearns Thank you for your prompt reply. Could you send me a hostfile sample, please? and a sample command that I must use for my program based on my last post? Wishes On Wed, Feb 26, 2014 at 12:49 PM, John Hearns wrote: > Khadije - you need to give a list of compute hosts to mpirun.

Re: [OMPI users] run a program

Hi I think you should use the "--host" or "--hostfile" options: http://www.open-mpi.org/faq/?category=running#simple-spmd-run http://www.open-mpi.org/faq/?category=running#mpirun-host Hope this helps Jody On Wed, Feb 26, 2014 at 8:31 AM, raha khalili wrote: > Dear Users > > This is my firs

Re: [OMPI users] run a program

Khadije - you need to give a list of compute hosts to mpirun. And probably have to set up passwordless ssh to each host.

[OMPI users] run a program

Dear Users This is my first post in open-mpi forum and I am beginner in using mpi. I want to run a program which does between 4 systems consist of one server and three nodes with 20 cpus. When I run: *mpirun -np 20 /home/khalili/espresso-5.0.2/bin/pw.x -in si.in | tee si.out*, after

[OMPI users] Binding to Core Warning

Hi, I tried to run an MPI Java program with --bind-to core. I receive the following warning and wonder how to fix this. WARNING: a request was made to bind a process. While the system supports binding the process itself, at least one node does NOT support binding memory to the process location.