Re: [OMPI users] Multiple windows for the same communicator at thesame time

Yeah; the cid needs to be unique from the communicator passed to win_create because the win functions will call collectives on their communicator. It would be unfortunate to have user collectives and window collectives colliding (unfortunate meaning violating the standard here). It just makes

Re: [OMPI users] Do MPI calls ever sleep?

No, we really shouldn't. Having just fought with a program using usleep(1) which was behaving even worse, working around this particular inability of the Linux kernel development team to do something sane will only lead to more pain. There are no good options, so the best option is to not try

Re: [OMPI users] Granular locks?

On Oct 1, 2010, at 3:24 PM, Gijsbert Wiesenekker wrote: > I have a large array that is shared between two processes. One process > updates array elements randomly, the other process reads array elements > randomly. Most of the time these writes and reads do not overlap. > The current version of

Re: [OMPI users] Granular locks?

On Oct 2, 2010, at 2:54 AM, Gijsbert Wiesenekker wrote: > On Oct 1, 2010, at 23:24 , Gijsbert Wiesenekker wrote: > >> I have a large array that is shared between two processes. One process >> updates array elements randomly, the other process reads array elements >> randomly. Most of the time t

Re: [OMPI users] location of ompi libraries

David - You're correct - adding --enable-static (or it's file equivalent enable_static) causes components to be linked into libmpi instead of left as individual components. This is probably a bug, but it's what Open MPI's done for it's entire life, so it's unlikely to change. Removing the ena

Re: [OMPI users] Using MPI_Put/Get correctly?

Matt - There's a known bug in the datatype engine of Open MPI 1.5 that breaks MPI One-sided when used with user-defined datatypes. Unfortunately, I don't have a timetable as to when it will be fixed. Brian On Jan 3, 2011, at 9:18 AM, Grismer,Matthew J Civ USAF AFMC AFRL/RBAT wrote: > Unf

Re: [OMPI users] MPI_ERR_INTERN with MPI_Accumulate using MPI_REPLACE

Jeff, I don't believe it is. I'm still waiting for a compile to finish to test, but there shouldn't be a problem with predefined datatypes. It's just user-defined that the ddt->opal move screwed up. Brian On Jan 6, 2011, at 2:19 PM, Jeff Squyres wrote: > Jeremiah -- > > Is this the same as:

Re: [OMPI users] MPI_Win_create with size=0 expose memory anyway

Patrick - Your program is erroneous, so the behavior of the MPI is not defined. The default implementation of RMA with Open MPI uses active-message like semantics to locally deliver the message, and does not do bounds checking, so the error was not caught. Brian On Feb 11, 2011, at 5:41 AM,

Re: [OMPI users] nonblock alternative to MPI_Win_complete

On Feb 18, 2011, at 8:59 AM, Toon Knapen wrote: > (Probably this issue has been discussed at length before but unfortunately I > did not find any threads (on this site or anywhere else) on this topic, if > you are able to provide me with links to earlier discussions on this topic, > please do n

Re: [OMPI users] MPI one-sided passive synchronization.

This is mostly an issue of how MPICH2 and Open MPI implement lock/unlock. Some might call what I'm about to describe erroneous. I wrote the one-sided code in Open MPI and may be among those people. In both implementations, one-sided communication is not necessarily truly asynchronous. That is, t

Re: [OMPI users] RES: RES: Error with ARM target

--host is the correct option, but the host string "arm" is not valid; it needs to be a proper triple, something like "x86_64-unknown-linux-gnu". Either way, ARM was not a supported platform in the 1.4.x release; the earliest version of Open MPI to support the ARM platform was 1.5.2. Brian On 4/25

Re: [OMPI users] Deadlock with barrier und RMA

There are no missing calls to MPI_WIN_FENCE as the code is using passive synchronization (lock/unlock). The test code looks correct, I think this is a bug in Open MPI. The code also fails on the development trunk, so upgrading will not fix the bug. I've filed a bug (#2809). Unfortunately, I'm n

Re: [OMPI users] Passive target performance (was: Deadlock with barrier und RMA)

On 6/15/11 5:45 AM, "Ole Kliemann" wrote: >On Mon, Jun 13, 2011 at 04:11:44PM +0000, Barrett, Brian W wrote: >> There are no missing calls to MPI_WIN_FENCE as the code is using passive >> synchronization (lock/unlock). The test code looks correct, I think >>thi

Re: [OMPI users] OpenMPI on Tile architectures (no atomic primitives)

All - Tile is based (loosely) on MIPS, but it's not quite the same. The TilePro only has a simple test-and-set instruction. I have getting Open MPI compiling on the Tilera on my todo list for another project, so I may be able to make progress on that front over the next couple of weeks. We're n

Re: [OMPI users] CMAQ crashes with OpenMPI

The error message looks like it's no where near an MPI function; I would guess that this is not an Open MPI problem but, particularly given your statements about Snow Leopard) a CMAQ problem. The easiest way to debug on OS X is to launch the application code in a debugger, something like: mpiru

Re: [OMPI users] Compiling both 32-bit and 64-bit?

On 8/24/11 11:29 AM, "Dmitry N. Mikushin" wrote: >Quick question: is there an easy switch to compile and install both >32-bit and 64-bit OpenMPI libraries into a single tree? E.g. 64-bit in >/prefix/lib64 and 32-bit in /prefix/lib. Quick answer: not easily. Long answer: There's not an easy way,

Re: [OMPI users] Compiling both 32-bit and 64-bit?

help' for more information. >link: invalid option -- 'd' >Try `link --help' for more information. >link: invalid option -- 'd' >Try `link --help' for more information. >link: invalid option -- 'd' >Try `link --help' for more inf

Re: [OMPI users] Open MPI and Objective C

The problem you're running into is not due to Open MPI. The Objective C and C compilers on OS X (and most platforms) are the same binary, so you should be able to use mpicc without any problems. It will see the .m extension and switch to Objective C mode. However, NSLog is in the Foundation f

Re: [OMPI users] MPI on MacOS Lion help

I think you have something wrong with your Xcode install; on my Lion machine, gcc is installed in /usr/bin as always. Also, on OS X, you should never have to set LD_LIBRARY_PATH. Brian On 11/4/11 3:36 PM, "Ralph Castain" wrote: >Just glancing at the output, it appears to be finding a different

Re: [OMPI users] [EXTERNAL] Re: Question regarding osu-benchamarks 3.1.1

I'm pretty sure that they are correct. Our one-sided implementation is buggier than I'd like (indeed, I'm in the process of rewriting most of it as part of Open MPI's support for MPI-3's revised RDMA), so it's likely that the bugs are in Open MPI's onesided support. Can you try a more recent rele

Re: [OMPI users] [EXTERNAL] Possible to build ompi-1.4.3 or 1.4.5 without a C++ compiler?

On 3/20/12 10:06 AM, "Gunter, David O" wrote: >I need to build ompi-1.4.3 (or the newer 1.4.5) with an older Intel 10.0 >compiler, but on a newer system in which the default g++ headers are >incompatible with Intel. Thus the C and Fortran compilers function >normally but the Intel C++ compiler fa

Re: [OMPI users] [EXTERNAL] Possible to build ompi-1.4.3 or 1.4.5 without a C++ compiler?

to `__intel_sse2_strcpy' > >and other messages for every single Intel-implemented standard C-function. > >-david >-- >David Gunter >HPC-3: Infrastructure Team >Los Alamos National Laboratory > > > > >On Mar 20, 2012, at 8:18 AM, Barrett, Brian W wrote: > >

Re: [OMPI users] [EXTERNAL] Possible to build ompi-1.4.3 or 1.4.5 without a C++ compiler?

it along to the user to try >out. > >Thanks! > >-david >-- >David Gunter >HPC-3: Infrastructure Team >Los Alamos National Laboratory > > > > >On Mar 20, 2012, at 9:52 AM, Barrett, Brian W wrote: > >> That doesn't make a whole lot of sense; what comp

Re: [OMPI users] [EXTERNAL] Using One-sided communication with lock/unlock

On 4/3/12 12:01 PM, "Sebastian Rettenberger" wrote: >I posted the bug report a week ago, but unfortunately I didn't get any >response: >https://svn.open-mpi.org/trac/ompi/ticket/3067 > >The example (see bug report) is very simple, however it still fails. >Other MPI >versions work fine (e.g. Inte

Re: [OMPI users] [EXTERNAL] Re: mpicc link shouldn't add -ldl and -lhwloc

On 5/22/12 10:36 PM, "Orion Poplawski" wrote: >On 05/22/2012 10:34 PM, Orion Poplawski wrote: >> On 05/21/2012 06:15 PM, Jeff Squyres wrote: >>> On May 15, 2012, at 10:37 AM, Orion Poplawski wrote: >>> $ mpicc -showme:link -pthread -m64 -L/usr/lib64/openmpi/lib -lmpi -ldl -lhwloc >

Re: [OMPI users] MPI One-Sided Communication, indexed datatype and segmentation fault.

Ziaul - Your program is erroneous; you can not modify the local communication buffer of an MPI_ACCUMULATE call until after the next synchronization call (Section 11.3 of MPI 2.2). In your example, that would be after the MPI_FENCE call following the call to MPI_ACCUMULATE. Brian On 6/6/12 9:44

Re: [OMPI users] [EXTERNAL] Re: MPI One-Sided Communication, indexed datatype and segmentation fault.

m not modifying the local communication buffer that contains >the data. I am modifying the the buffer that contains the indices of the >data buffer(source_disp and target_disp). > >in MPICH2 this is not a problem. I am not sure about Open MPI. > >Thanks, >Ziaul > >On Wed, Ju

Re: [OMPI users] [EXTERNAL] Possible memory leak(s) in OpenMPI 1.6.3?

Thanks for the bug report. I've fixed the leak in our development branch and it should make its way to both the 1.6 and 1.7 release series. Brian On 1/21/13 6:53 AM, "Victor Vysotskiy" mailto:victor.vysots...@teokem.lu.se>> wrote: Since my question unanswered for 4 days, I repeat the origina

Re: [OMPI users] [EXTERNAL] Re: problem building 32-bit openmpi-1.9a1r27979 with Sun C

Actually, the errors are in code that has changed. A couple weeks ago, we removed the inline assembly support for C++, but I'm guessing not entirely correctly. That is, I believe something is wrong in the #defines such that opal_atomic_add_32 is being declared inline, but no inline body will be p

Re: [OMPI users] [EXTERNAL] Re: problem building 32-bit openmpi-1.9a1r27979 with Sun C

On 1/30/13 12:01 PM, "Barrett, Brian W" wrote: >Actually, the errors are in code that has changed. A couple weeks ago, we >removed the inline assembly support for C++, but I'm guessing not entirely >correctly. That is, I believe something is wrong in the #defines such

Re: [OMPI users] [EXTERNAL] MPI based HLA/RTI ?

On 4/15/13 8:22 AM, "John Chludzinski" wrote: >Is anyone aware of an MPI based HLA/RTI (DoD High Level Architecture >(HLA) / Runtime Infrastructure)? Information Sciences Institute wrote an MPI transport for RTI-s 8 years or so ago. I'm not sure what happened to that code, but it was useful for

Re: [OMPI users] [EXTERNAL] Re: Hints for running OpenMPI on Intel/Phi (MIC) enabled hosts

Although this particular bug should be fixed in 1.6.5 and 1.7.2; which version of Open MPI are you using? Brian On 7/10/13 10:29 AM, "Ralph Castain" wrote: >Yeah, we discussed taking things from your thread, plus the wiki page on >cross-compiling OMPI, and creating a new FAQ area. I'll do so -

Re: [OMPI users] [EXTERNAL] MPI_THREAD_SINGLE vs. MPI_THREAD_FUNNELED

On 10/22/13 10:23 AM, "Jai Dayal" mailto:dayals...@gmail.com>> wrote: I, for the life of me, can't understand the difference between these two init_thread modes. MPI_THREAD_SINGLE states that "only one thread will execute", but MPI_THREAD_FUNNELED states "The process may be multi-threaded, but

Re: [OMPI users] [EXTERNAL] RDMA question

On 10/28/13 5:22 AM, "Luis Kornblueh" wrote: >My question would be to all openmpi power users and developers, what >would be required to get this properly running. > >In case it is required to give more information, please come back to me. >Maybe the explanation what we do is insufficient. Open

Re: [OMPI users] [EXTERNAL] Re: Configuration for rendezvous and eager protocols: two-sided comm

Siddhartha - Christoph mentioned how to change the cross-over for shared memory, but it's really per-transport (so you'd have to change it for your off-node transport as well). That's all in the FAQ you mentioned, so hopefully you can take it from there. Note that, in general, moving the eage

Re: [OMPI users] [EXTERNAL] Re: What's the status of OpenMPI and thread safety?

Pablo - As Ralph mentioned, it will be different, possibly not for the better, in 1.7. This is an area of active work, so any help would be appreciated. However, the one issue you brought up is going to be problematic, even with threads. Our design essentially makes it such that blocking MPI ca

Re: [OMPI users] [EXTERNAL] Re: Segmentation fault on OMPI 1.6.5 built with gcc 4.4.7 and PGI pgfortran 11.10

Open MPI unfortunately has to play some tricks with the malloc system when using InfiniBand or the Cray interconnects. One other option is to set the environment variable OMPI_MCA_memory_linux_disable to some non-zero value. That will disable the evil memory hooks, which might help if PGI is

Re: [OMPI users] [EXTERNAL] OpenMPI with dual port Myrinet cards

Victor - I don't think our multi-port support with MX is particularly well tested (I know I don't test that path). It looks like you might be able to work around the problem by setting -mca mtl_mx_endpoint_num 1 on the mpirun command line, which will only use the first port found. But I could

Re: [OMPI users] Compilation Failure on Franklin with OpenMPI

Hi - I haven't built on Franklin, but generally you don't want to use cc - you want to use the gcc which targets the backend nodes (I'm not sure what it's called on CNL machines, as I work mostly with Catamount machines). That should fix the error you are seeing, as part of the problem is Libtool

Re: [OMPI users] nonblocking MPI_File_iwrite() does block?

On 11/23/09 8:42 AM, "Rob Latham" wrote: > Is it OK to mention MPICH2 on this list? I did prototype some MPI > extensions that allowed ROMIO to do true async I/O (at least as far > as the underlying operating system supports it). If you really need > to experiment with async I/O, I'd love to

Re: [OMPI users] flex.exe

I have to agree with the two requests here. Having either a windows tarball or a windows build tools tarball doesn't seem too burdensom, and could even be done automatically at make dist time. Brian - Original Message - From: users-boun...@open-mpi.org To: us...@open-mpi.org Sent: Th

Re: [OMPI users] Progress in MPI_Win_unlock

Dorian - Your observation is correct - Open MPI will only make progress on a passive target communication if the target enters the MPI library in some meaningful way (tests on a request which hasn't completed, makes a one-sided call, starts communication, etc.). I'm the author of the onesided

Re: [OMPI users] Problems building Open MPI 1.4.1 with Pathscale

Adding the memory and cc will certainly do no harm, and someone tried to remove them as an optimization. I wouldn't change the input and output lines - the differences are mainly syntactic sugar. Brian On Feb 10, 2010, at 7:04 AM, Ake Sandgren wrote: > On Wed, 2010-02-10 at 08:21 -0500, Jeff

Re: [OMPI users] gadget-3 locks up using openmpi and infiniband (or myrinet)

I don't know if it's the same problem or not (and we haven't tested on Myrinet), but we have one code which frequently hangs on smallish (64 node) runs. I unfortunately haven't been able to deep dive into the problem, but the hang is in a bcast call, where peers are doing sendrecv calls. All b

Re: [OMPI users] SM failure with mixed 32/64-bit procs on the samemachine

Jeff - Is indicating we don't support it really the right thing to do? Given that SM should already have the proc data, it seems that setting the reachable bit to zero for the other process of different "architecture" is all that is required. Brian On Jun 4, 2010, at 8:26 AM, Jeff Squyres wro

Re: [OMPI users] Xgrid an openmpi 1.2 and 1.5rc1

You have to set two environment variables (XGRID_CONTROLLER_HOSTNAME and XGRID_CONTROLLER_PASSWORD) with the correct information in order for the XGrid starter to work. Due to the way XGrid works, the nolocal option will not work properly when launching with XGrid. Brian On Jun 21, 2010, at 1

Re: [OMPI users] [OMPI USERS] Jumbo frames

But in many ways, it’s also not helpful to change the MTU from Open MPI. It sounds like you made a bunch of changes all at once; I’d break them down and build up. MTU is a very system-level configuration. Use a tcp transmission test (iperf, etc.) to make sure TCP connections work between the

Re: [OMPI users] Network performance over TCP

Adam - The btl_tcp_links flag does not currently work (for various reasons) in the 2.x and 3.x series. It’s on my todo list to fix, but I’m not sure it will get done before the 3.0.0 release. Part of the reason that it hasn’t been a priority is that most applications (outside of benchmarks) d

Re: [OMPI users] --enable-builtin-atomics

Well, if you’re trying to get Open MPI running on a platform for which we don’t have atomics support, built-in atomics solves a problem for you… Brian > On Aug 1, 2017, at 9:42 AM, Nathan Hjelm wrote: > > So far only cons. The gcc and sync builtin atomic provide slower performance > on x86-64

[OMPI users] Open MPI v3.0.0 released

The Open MPI Team, representing a consortium of research, academic, and industry partners, is pleased to announce the release of Open MPI version 3.0.0. v3.0.0 is the start of a new release series for Open MPI. Open MPI 3.0.0 enables MPI_THREAD_MULTIPLE by default, so a build option to Open MPI

Re: [OMPI users] How can I measure synchronization time of MPI_Bcast()

Gilles suggested your best next course of action; time the MPI_Bcast and MPI_Barrier calls and see if there’s a non-linear scaling effect as you increase group size. You mention that you’re using m3.large instances; while this isn’t the list for in-depth discussion about EC2 instances (the AWS

Re: [OMPI users] Q: Binding to cores on AWS?

Jumping in a little late… Today, EC2 instances don’t expose all the required information for memory pinning to work, which is why you see the warning. The action-less error message is obviously a bit annoying (although it makes sense in the general case), but we haven’t had the time to work ou

[OMPI users] Open MPI v3.1.0 Released

The Open MPI Team, representing a consortium of research, academic, and industry partners, is pleased to announce the release of Open MPI version 3.1.0. v3.1.0 is the start of a new release series for Open MPI. New Features include a monitoring framework to track data movement in MPI operatio

Re: [OMPI users] RDMA over Ethernet in Open MPI - RoCE on AWS?

It sounds like what you’re asking is “how do I get the best performance from Open MPI in AWS?”. The TCP BTL is your best option for performance in AWS. RoCE is going to be a bunch of work to get setup, and you’ll still end up with host processing of every packet. There are a couple simple ins

Re: [OMPI users] How do I build 3.1.0 (or later) with mellanox's libraries

Yeah, there’s no good answer here from an “automatically do the right thing” point of view. The reachable:netlink component (which is used for the TCP BTL) only works with libnl-3 because libnl-1 is a real pain to deal with if you’re trying to parse route behaviors. It will do the right thing

Re: [OMPI users] RDMA over Ethernet in Open MPI - RoCE on AWS?

On Sep 11, 2018, at 10:46 AM, Benjamin Brock mailto:br...@cs.berkeley.edu>> wrote: Thanks for your response. One question: why would RoCE still requiring host processing of every packet? I thought the point was that some nice server Ethernet NICs can handle RDMA requests directly? Or am I mis

Re: [OMPI users] Limit to number of asynchronous sends/receives?

Adam - There are a couple of theoretical limits on how many requests you can have outstanding (at some point, you will run the host out of memory). However, those issues should be a problem when posting the MPI_Isend or MPI_Irecv, not during MPI_Waitall. 2.1.0 is pretty old; the first step in

Re: [OMPI users] disabling ucx over omnipath

What you're asking for is an ugly path of interconnected dependencies between products owned by different companies. It also completely blows any object model we can think of out of the water. It's all bad in the general case. The best we've come up with for the Libfabric MTL is to disable it

[OMPI users] Open MPI release update

Greetings - As you may know, Open MPI 5.0 is going to include an ambitious improvement in Open MPI's runtime system along with a number of performance improvements, and was targeted to ship this summer. While we are still going to make those improvements to our runtime system, it is taking us

Re: [OMPI users] How to set parameters to utilize multiple network interfaces?

John - Open MPI's OFI implementation does not stripe messages across processes. Instead, an Open MPI process will choose the "closest" NIC on the system (based on PCI hops and PCI topology, using hwloc). If there is ore than one "closest" NIC, as is the case on P4, where each Intel socket has

Re: [OMPI users] MPI_THREAD_MULTIPLE question

Yes, this is the case for Open MPI 4.x and earlier, due to various bugs. When Open MPI 5.0 ships, we will resolve this issue. Brian On 9/9/22, 9:58 PM, "users on behalf of mrlong336 via users" mailto:users-boun...@lists.open-mpi.org> on behalf of users@lists.open-mpi.org

Re: [OMPI users] Beginner Troubleshooting OpenMPI Installation - pmi.h Error

Can you include the configure command you used for Open MPI, as well as the output of “make all V=1” (it’s ok if that’s from a tree you’ve already tried to build, the full compile command for the file that is failing to compile is the part of interest. Thanks, Brian On 10/4/22, 9:06 AM, "user

Re: [OMPI users] question about the Open-MPI ABI

Because we’ve screwed up in the past? I think the ompi_message_null was me, and I was in a hurry to prototype for the MPI Forum. And then it stuck. Brian On 2/1/23, 3:16 AM, "users on behalf of Jeff Hammond via users" mailto:users-boun...@lists.open-mpi.org> on behalf of users@lists.open-mpi