Re: [OMPI users] Performance degradation of OpenMPI 1.10.2 when oversubscribed?

2017-03-24 Thread Jeff Squyres (jsquyres)
On Mar 24, 2017, at 6:10 PM, Reuti wrote: > >> - Disabling HT in the BIOS means that the one hardware thread left in each >> core will get all the cores resources (buffers, queues, processor units, >> etc.). >> - Enabling HT in the BIOS means that each of the 2 hardware threads will >> statica

Re: [OMPI users] Performance degradation of OpenMPI 1.10.2 when oversubscribed?

2017-03-24 Thread Tim Prince via users
On 3/24/2017 6:10 PM, Reuti wrote: Hi, Am 24.03.2017 um 20:39 schrieb Jeff Squyres (jsquyres): Limiting MPI processes to hyperthreads *helps*, but current generation Intel hyperthreads are not as powerful as cores (they have roughly half the resources of a core), so -- depending on your appl

Re: [OMPI users] Performance degradation of OpenMPI 1.10.2 when oversubscribed?

2017-03-24 Thread Reuti
Hi, Am 24.03.2017 um 20:39 schrieb Jeff Squyres (jsquyres): > Limiting MPI processes to hyperthreads *helps*, but current generation Intel > hyperthreads are not as powerful as cores (they have roughly half the > resources of a core), so -- depending on your application and your exact > system

Re: [OMPI users] Suppressing Nvidia warnings

2017-03-24 Thread Sylvain Jeaugey
I'm still working to get a clear confirmation of what is printing this error message and since when. However, running strings, I could only find this string in /usr/lib/libnvidia-ml.so, which comes with the CUDA driver, so it should not be related to the CUDA runtime version ... but again, unt

Re: [OMPI users] Suppressing Nvidia warnings

2017-03-24 Thread Roland Fehrenbacher
> "SJ" == Sylvain Jeaugey writes: Hi Sylvain, SJ> Hi Roland, I can't find this message in the Open MPI source SJ> code. Could it be hwloc ? Some other library you are using ? after a longer detour about the suspicion it might have something to do with nvml support of hwloc, I now fo

Re: [OMPI users] Performance degradation of OpenMPI 1.10.2 when oversubscribed?

2017-03-24 Thread Jeff Squyres (jsquyres)
Performance goes out the window if you oversubscribe your machines (i.e., run more MPI processes than cores). The effect of oversubscription is non-deterministic. (for the next few paragraphs, assume that HT is disabled in the BIOS -- i.e., that there's only 1 hardware thread on each core) Op

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread Jeff Squyres (jsquyres)
On Mar 24, 2017, at 6:41 AM, Jordi Guitart wrote: > > Docker containers have different IP addresses, indeed, so now we know why it > does not work. I think that this could be a nice feature for OpenMPI, so I'll > probably issue a request for it ;-) Cool. I don't think any of the current devel

Re: [OMPI users] openmpi installation error

2017-03-24 Thread Vinay Mittal
Thanks Jeff and Renato for your help. Renato, You are right. Perl alias was pointing to somewhere else. I was able to install openmpi using previous released version of it. Thanks. Vinay K. Mittal On Thu, Mar 23, 2017 at 2:04 PM, Renato Golin wrote: > On 23 March 2017 at 17:39, Vinay Mittal

[OMPI users] Issues with PGI 16.10, OpenMPI 2.1.0 on macOS: Fortran issues with hello world (running and dylib)

2017-03-24 Thread Matt Thompson
All, Aside from the issues I'm having with pgc++ on macOS, I'm also having an issue not related to that configure issue. Namely, if I build Open MPI using the siterc figured out by PGI Help: http://www.pgroup.com/userforum/viewtopic.php?p=21105#21105 note I used the siterc, but I did not enab

Re: [OMPI users] Help with Open MPI 2.1.0 and PGI 16.10: Configure and C++

2017-03-24 Thread Matt Thompson
Gilles, The library I have having issues linking is ESMF and it is a C++/Fortran application. From http://www.earthsystemmodeling.org/esmf_releases/non_public/ESMF_7_0_0/ESMF_usrdoc/node9.html#SECTION00092000 : The following compilers and utilities *are required* for compiling, linkin

[OMPI users] Performance degradation of OpenMPI 1.10.2 when oversubscribed?

2017-03-24 Thread Jordi Guitart
Hello, I'm running experiments with BT NAS benchmark on OpenMPI. I've identified a very weird performance degradation of OpenMPI v1.10.2 (and later versions) when the system is oversubscribed. In particular, note the performance difference between 1.10.2 and 1.10.1 when running 36 MPI process

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread Jordi Guitart
Hello Jeff, Docker containers have different IP addresses, indeed, so now we know why it does not work. I think that this could be a nice feature for OpenMPI, so I'll probably issue a request for it ;-) Thanks for your help. On 24/03/2017 11:27, Jeff Squyres (jsquyres) wrote: If the Docker

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread Jeff Squyres (jsquyres)
If the Docker containers have different IP addresses, Open MPI will think that they are different "nodes" (or "hosts" or "servers" or whatever your favorite word is), and therefore will assume that they processes in these different containers are unable to share memory. Meaning: no work has bee

Re: [OMPI users] a question about MPI dynamic process manage

2017-03-24 Thread Jeff Squyres (jsquyres)
(keeping the user's list in the CC) > On Mar 24, 2017, at 4:05 AM, gzzh...@buaa.edu.cn wrote: > > hi jeff: > I tried to call MPI_Comm_spawn("./child", MPI_ARGV_NULL, 1, > MPI_INFO_NULL, root, MPI_COMM_WORLD, &newcomm, &errs) > in order every MPI process in MPI_COMM_WORLD can spawn one child

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread Jordi Guitart
Hello John, Yes, in fact, I'm comparing Docker with Singularity regarding the execution of MPI applications :-) I'd like to make the comparison fairer by allowing Docker containers to share memory. Thanks On 24/03/2017 10:00, John Hearns via users wrote: Jordi, this is not an answer to

Re: [OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread John Hearns via users
Jordi, this is not an answer to your question. However have you looked at Singularity: http://singularity.lbl.gov/ On 24 March 2017 at 08:54, Jordi Guitart wrote: > Hello, > > Docker allows several containers running in the same host to share the > same IPC namespace, thus they can share mem

[OMPI users] Communicating MPI processes running in Docker containers in the same host by means of shared memory?

2017-03-24 Thread Jordi Guitart
Hello, Docker allows several containers running in the same host to share the same IPC namespace, thus they can share memory (see example here: https://github.com/docker/docker/pull/8211#issuecomment-56873448). I assume this could be used by OpenMPI to communicate MPI processes running in dif