Hi all,
I have a MPI_Isend/MPI_Recv problem in a multi-thread program.
In the program:
The first machine has one thread does some computation and call MPI_Isend
to send buffer to the second machine, and another thread is always trying to
MPI_Recv data from the second machine. And the first thread will MPI_Wait its
last MPI_Isend to complete before call MPI_Isend again.
The second machine does the exact same thing.
Then I got the result that :
The first machine:
Thread 0 : MPI_Isend data to the second machine successfully. but blocked
in MPI_Wait because last MPI_Isend did not complete.
Thread 1 : try to MPI_Recv data from the second machine, but no data and it
blocked.
The second machine:
Thread 0 : MPI_Isend data to the first machine successfully. but blocked in
MPI_Wait because last MPI_Isend did not complete.
Thread 1 : try to MPI_Recv data from the first machine, but no data and it
blocked.
Does anyone have any ideas? I appreciated it very much, because I have
tracked the problem for two days but no progress.
Eason Zhao
At 2012-12-05 01:00:02,[email protected] wrote:
>Send users mailing list submissions to
> [email protected]
>
>To subscribe or unsubscribe via the World Wide Web, visit
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>or, via email, send a message with subject or body 'help' to
> [email protected]
>
>You can reach the person managing the list at
> [email protected]
>
>When replying, please edit your Subject line so it is more specific
>than "Re: Contents of users digest..."
>
>
>Today's Topics:
>
> 1. Re: 0xc000007b error exit on 64-bit Windows 7
> ([email protected])
> 2. Re: 0xc000007b error exit on 64-bit Windows 7 (Damien)
> 3. Windows support for OpenMPI (Damien)
> 4. Re: Windows support for OpenMPI (John R. Cary)
> 5. Re: Windows support for OpenMPI (Damien)
> 6. Romio and OpenMPI builds (Brock Palen)
> 7. Re: Romio and OpenMPI builds (Eric Chamberland)
> 8. Re: CentOS 6.3 & OpenMPI 1.6.3 (Joseph Farran)
> 9. Re: BLCR + Qlogic infiniband (William Hay)
>
>
>----------------------------------------------------------------------
>
>Message: 1
>Date: Mon, 3 Dec 2012 10:03:41 -0700
>From: [email protected]
>Subject: Re: [OMPI users] 0xc000007b error exit on 64-bit Windows 7
>To: Open MPI Users <[email protected]>
>Message-ID: <233b9754-b3a9-47d7-9dd1-3e6680ba4318.maildroid@localhost>
>Content-Type: text/plain; charset="utf-8"
>
>Ignore what I posted, Hristo is right. On my phone screen the C looked like
>a 0. Sorry.
>
>Damien
>
>Sent from my android device.
>
>
>
>-----Original Message-----
>From: "Jeff Squyres (jsquyres)" <[email protected]>
>To: Open MPI Users <[email protected]>
>Cc: "[email protected]" <[email protected]>
>Sent: Mon, 03 Dec 2012 9:01 AM
>Subject: Re: [OMPI users] 0xc000007b error exit on 64-bit Windows 7
>
>I'm afraid we've lost the open MPI community windows developer. So I don't
>know if you'll get a good answer to this question.
>
>Sorry! :(
>
>Sent from my phone. No type good.
>
>On Dec 3, 2012, at 6:40 AM, "Ilias Miroslav" <[email protected]> wrote:
>
>> Dear experts,
>>
>> I just installed
>> http://www.open-mpi.org/software/ompi/v1.6/downloads/OpenMPI_v1.6.1-1_win64.exe
>> on our Intel i7 64-bit Windows 7 system.
>>
>>
>> When I try to run some executable, I am getting error "Application Error
>> The application was unable to start correctly (0xc000007b)..."
>>
>> Any help please ? The "C:\Program Files (x86)\OpenMPI_v1.6.1-x64\bin" string
>> is in my %Path% variable.
>>
>> Yours, Miro
>>
>> PS: On 32-bit Windows 7 the 32-bit OpenMPI application works fine.
>> _______________________________________________
>> users mailing list
>> [email protected]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>_______________________________________________
>users mailing list
>[email protected]
>http://www.open-mpi.org/mailman/listinfo.cgi/users
>-------------- next part --------------
>HTML attachment scrubbed and removed
>
>------------------------------
>
>Message: 2
>Date: Mon, 03 Dec 2012 10:19:39 -0700
>From: Damien <[email protected]>
>Subject: Re: [OMPI users] 0xc000007b error exit on 64-bit Windows 7
>To: Open MPI Users <[email protected]>
>Message-ID: <[email protected]>
>Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"
>
>I just tried it on a clean VM, the 64-bit OpenMPI installer does install
>to Program Files (x86). That's not the end of the world, but you have
>to watch your paths.
>
>Miroslav, when you ran the installer did you say yes to adding OpenMPI
>to the system path? If you installed both 32 and 64-bit binaries, and
>added both to the system path, it will typically just append the paths.
>So if you installed 32-bit first, then 64-bit, whenever you run
>something it will load the 32-bit OpenMPI runtime first, even running
>64-bit, which will cause that bad image error. I think that's why your
>32-bit run works and 64-bit doesn't.
>
>I suggest uninstalling both 32 and 64-bit OpenMPIs, make sure they're
>removed from the path, then reinstall them *without* putting them into
>the system path, and try again from there. You'll have to set your
>paths manually, but you'll be running with the right binaries each time.
>
>Damien
>
>On 03/12/2012 9:55 AM, Iliev, Hristo wrote:
>> Hi,
>>
>> 0xC000007B is STATUS_INVALID_IMAGE_FORMAT. It mostly means that some of the
>> dynamic link libraries (DLLs) that the executable is linked against are of
>> different "bitness", e.g. 32-bit. It could be a packaging error in Open MPI,
>> or it could be messed up installation. You could use the Dependency Walker
>> tool to examine the list of DLLs that the executable depends upon and see
>> which one is the culprit. Dependency Walker is available here:
>>
>> http://www.dependencywalker.com/
>>
>> Which brings me to the question: why the win64 version of Open MPI is
>> installed in "Program Files (x86)", where 32-bit things go?!
>>
>> Hope that helps.
>>
>> Kind regards,
>> Hristo
>>
>> --
>> Hristo Iliev, Ph.D. -- High Performance Computing
>> RWTH Aachen University, Center for Computing and Communication
>> Rechen- und Kommunikationszentrum der RWTH Aachen
>> Seffenter Weg 23, D 52074 Aachen (Germany)
>>
>>
>>> -----Original Message-----
>>> From: [email protected] [mailto:[email protected]]
>>> On Behalf Of Ilias Miroslav
>>> Sent: Monday, December 03, 2012 3:40 PM
>>> To: [email protected]
>>> Subject: [OMPI users] 0xc000007b error exit on 64-bit Windows 7
>>>
>>> Dear experts,
>>>
>>> I just installed http://www.open-
>>> mpi.org/software/ompi/v1.6/downloads/OpenMPI_v1.6.1-1_win64.exe on
>>> our Intel i7 64-bit Windows 7 system.
>>>
>>>
>>> When I try to run some executable, I am getting error "Application Error
>> The
>>> application was unable to start correctly (0xc000007b)..."
>>>
>>> Any help please ? The "C:\Program Files (x86)\OpenMPI_v1.6.1-x64\bin"
>>> string is in my %Path% variable.
>>>
>>> Yours, Miro
>>>
>>> PS: On 32-bit Windows 7 the 32-bit OpenMPI application works fine.
>>> _______________________________________________
>>> users mailing list
>>> [email protected]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> [email protected]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>-------------- next part --------------
>HTML attachment scrubbed and removed
>
>------------------------------
>
>Message: 3
>Date: Mon, 03 Dec 2012 10:32:57 -0700
>From: Damien <[email protected]>
>Subject: [OMPI users] Windows support for OpenMPI
>To: Open MPI Users <[email protected]>
>Message-ID: <[email protected]>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>All,
>
>I completely missed the message about Shiqing departing as the OpenMPI
>Windows maintainer. I'll try and keep Windows builds going for 1.6 at
>least, I have 2011 and 2013 Intel licenses and VS2008 and 2012, but not
>2010. I see that the 1.6.3 code base already doesn't build on Windows
>in VS2012 :-(.
>
>While I can try and keep builds going, I don't have access to a Windows
>cluster right now, and I'm flat out on two other projects. I can test on
>my workstation, but that will only go so far. Longer-term, there needs
>to be a decision made on whether Windows gets to be a first-class
>citizen in OpenMPI or not. Jeff's already told me that 1.7 is lagging
>behind on Windows. It would be a shame to have all the work Shiqing put
>in gradually decay because it can't be supported enough. If there's any
>Microsoft/HPC/Azure folks observing this list, or any other vendors who
>run on Windows with OpenMPI, maybe we can see what can be done if you're
>interested.
>
>Damien
>
>
>------------------------------
>
>Message: 4
>Date: Mon, 03 Dec 2012 11:59:13 -0700
>From: "John R. Cary" <[email protected]>
>Subject: Re: [OMPI users] Windows support for OpenMPI
>To: [email protected]
>Message-ID: <[email protected]>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>Dear OpenMPI community,
>
>This email is about whether a commercial version of OpenMPI for Windows
>could be successful. I hesitated before sending this, but upon asking
>some others (notably Jeff) on this list, it seemed appropriate.
>
>We at Tech-X have been asking whether a commercial/freemium support
>model for a Windows
>version of OpenMPI would work. We are currently working on this for some
>other products, notably PETSc, which is discussed at
>http://www.txcorp.com/home/cosml.
>
>We see some downsides - in particular, with Microsoft's HPC Pack,
>Windows users
>have free access to an MPI solution. This has to be balanced by some
>particular advantages of OpenMPI such that there would be a group of
>users who would pay for it for anyone to make this work.
>
>We would be very interested in hearing from folks on this list who either
>(1) help define the competitive advantage of having OpenMPI on Windows or
>(2) would be interested in a commercial solution, were it available.
>
>Naturally, any solution should benefit the OpenMPI community as well to
>be a success.
>
>I would be glad to hear from folks on list or off.
>
>Thx....John Cary
>
>
>
>
>
>
>
>On 12/3/2012 10:32 AM, Damien wrote:
>> All,
>>
>> I completely missed the message about Shiqing departing as the OpenMPI
>> Windows maintainer. I'll try and keep Windows builds going for 1.6 at
>> least, I have 2011 and 2013 Intel licenses and VS2008 and 2012, but
>> not 2010. I see that the 1.6.3 code base already doesn't build on
>> Windows in VS2012 :-(.
>>
>> While I can try and keep builds going, I don't have access to a
>> Windows cluster right now, and I'm flat out on two other projects. I
>> can test on my workstation, but that will only go so far. Longer-term,
>> there needs to be a decision made on whether Windows gets to be a
>> first-class citizen in OpenMPI or not. Jeff's already told me that
>> 1.7 is lagging behind on Windows. It would be a shame to have all the
>> work Shiqing put in gradually decay because it can't be supported
>> enough. If there's any Microsoft/HPC/Azure folks observing this list,
>> or any other vendors who run on Windows with OpenMPI, maybe we can see
>> what can be done if you're interested.
>>
>> Damien
>> _______________________________________________
>> users mailing list
>> [email protected]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>
>
>
>------------------------------
>
>Message: 5
>Date: Mon, 03 Dec 2012 12:42:23 -0700
>From: Damien <[email protected]>
>Subject: Re: [OMPI users] Windows support for OpenMPI
>To: Open MPI Users <[email protected]>
>Message-ID: <[email protected]>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>This is a good start. Stepping up a level and without wanting to start
>a bunfight with other MPI implementations, what are the advantages of
>OpenMPI over those other various MPI implementations, irrespective of
>platform? There must be some advantages, or OpenMPI wouldn't exist. Do
>those advantages apply on Windows and would they justify ongoing Windows
>support?
>
>Damien
>
>
>On 03/12/2012 11:59 AM, John R. Cary wrote:
>> Dear OpenMPI community,
>>
>> This email is about whether a commercial version of OpenMPI for Windows
>> could be successful. I hesitated before sending this, but upon asking
>> some others (notably Jeff) on this list, it seemed appropriate.
>>
>> We at Tech-X have been asking whether a commercial/freemium support
>> model for a Windows
>> version of OpenMPI would work. We are currently working on this for some
>> other products, notably PETSc, which is discussed at
>> http://www.txcorp.com/home/cosml.
>>
>> We see some downsides - in particular, with Microsoft's HPC Pack,
>> Windows users
>> have free access to an MPI solution. This has to be balanced by some
>> particular advantages of OpenMPI such that there would be a group of
>> users who would pay for it for anyone to make this work.
>>
>> We would be very interested in hearing from folks on this list who either
>> (1) help define the competitive advantage of having OpenMPI on Windows or
>> (2) would be interested in a commercial solution, were it available.
>>
>> Naturally, any solution should benefit the OpenMPI community as well to
>> be a success.
>>
>> I would be glad to hear from folks on list or off.
>>
>> Thx....John Cary
>>
>>
>>
>>
>>
>>
>>
>> On 12/3/2012 10:32 AM, Damien wrote:
>>> All,
>>>
>>> I completely missed the message about Shiqing departing as the
>>> OpenMPI Windows maintainer. I'll try and keep Windows builds going
>>> for 1.6 at least, I have 2011 and 2013 Intel licenses and VS2008 and
>>> 2012, but not 2010. I see that the 1.6.3 code base already doesn't
>>> build on Windows in VS2012 :-(.
>>>
>>> While I can try and keep builds going, I don't have access to a
>>> Windows cluster right now, and I'm flat out on two other projects. I
>>> can test on my workstation, but that will only go so far.
>>> Longer-term, there needs to be a decision made on whether Windows
>>> gets to be a first-class citizen in OpenMPI or not. Jeff's already
>>> told me that 1.7 is lagging behind on Windows. It would be a shame to
>>> have all the work Shiqing put in gradually decay because it can't be
>>> supported enough. If there's any Microsoft/HPC/Azure folks observing
>>> this list, or any other vendors who run on Windows with OpenMPI,
>>> maybe we can see what can be done if you're interested.
>>>
>>> Damien
>>> _______________________________________________
>>> users mailing list
>>> [email protected]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>
>> _______________________________________________
>> users mailing list
>> [email protected]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
>
>------------------------------
>
>Message: 6
>Date: Mon, 3 Dec 2012 17:37:36 -0500
>From: Brock Palen <[email protected]>
>Subject: [OMPI users] Romio and OpenMPI builds
>To: Open MPI Users <[email protected]>
>Message-ID: <[email protected]>
>Content-Type: text/plain; charset=us-ascii
>
>I was trying to use hints with ROMIO and lustre prompted by another post on
>this list.
>
>I have a simple MPI-IO code and I cannot using the notes I find set the lustre
>striping using the config file and setting ROMIO_HINTS.
>
>Question:
>
>How can I check which ADIO drivers ROMIO in OPenMPI was built with when I
>built it?
>Can I make ROMIO go into 'verbose' mode and have it print what it is setting
>all its values to?
>
>Thanks!
>
>Brock Palen
>www.umich.edu/~brockp
>CAEN Advanced Computing
>[email protected]
>(734)936-1985
>
>
>
>
>
>
>------------------------------
>
>Message: 7
>Date: Mon, 03 Dec 2012 19:12:27 -0500
>From: Eric Chamberland <[email protected]>
>Subject: Re: [OMPI users] Romio and OpenMPI builds
>To: [email protected]
>Message-ID: <[email protected]>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>Le 12/03/2012 05:37 PM, Brock Palen a ?crit :
>> I was trying to use hints with ROMIO and lustre prompted by another post on
>> this list.
>>
>> I have a simple MPI-IO code and I cannot using the notes I find set the
>> lustre striping using the config file and setting ROMIO_HINTS.
>>
>> Question:
>>
>> How can I check which ADIO drivers ROMIO in OPenMPI was built with when I
>> built it?
>> Can I make ROMIO go into 'verbose' mode and have it print what it is setting
>> all its values to?
>Try "ompi_info -a" and check for lustre in the output:
>
>ompi_info -a | grep -i romio
>...
> MCA io: information "io_romio_user_configure_params"
>(value: <--with-file-system=testfs+ufs+nfs+lustre>, data source: default
>value)
> User-specified command line parameters passed
>to ROMIO's configure script
> MCA io: information
>"io_romio_complete_configure_params" (value:
><--with-file-system=testfs+ufs+nfs+lustre CFLAGS='-DNDEBUG -O3 -xHOST
>-Wall -finline-functions -fno-strict-aliasing -restrict -pthread'
>CPPFLAGS='
>-I/clumeq/src/Open-MPI/1.6.3/intel/openmpi-1.6.3/opal/mca/hwloc/hwloc132/hwloc/include
>
>-I/clumeq/src/Open-MPI/1.6.3/intel/Build/opal/mca/hwloc/hwloc132/hwloc/include
>
>-I/usr/include/infiniband -I/usr/include/infiniband' FFLAGS='' LDFLAGS='
>' --enable-shared --enable-static
>--with-file-system=testfs+ufs+nfs+lustre
>--prefix=/software/MPI/openmpi/1.6.3_intel --with-mpi=open_mpi
>--disable-aio>, data source: default value)
> Complete set of command line parameters
>passed to ROMIO's configure script
>
>Eric
>
>
>
>------------------------------
>
>Message: 8
>Date: Mon, 03 Dec 2012 23:20:49 -0800
>From: Joseph Farran <[email protected]>
>Subject: Re: [OMPI users] CentOS 6.3 & OpenMPI 1.6.3
>To: Mike Dubman <[email protected]>
>Cc: Open MPI Users <[email protected]>
>Message-ID: <[email protected]>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>Hi Mike.
>
>Removed the old mxm, downloaded and installed:
>
>/tmp/mxm/v1.1/per-ofed/1.5.4.1/mxm-1.1.3a5e745-1.x86_64-rhel6u3.rpm
>
>I am suing OFED 1.5.4.1 and it still fails at the same spot:
>
>make[2]: Entering directory `/data/apps/sources/openmpi-1.6.3/ompi/mca/mtl/mxm'
> CC mtl_mxm.lo
> CC mtl_mxm_cancel.lo
> CC mtl_mxm_component.lo
> CC mtl_mxm_endpoint.lo
> CC mtl_mxm_probe.lo
> CC mtl_mxm_recv.lo
> CC mtl_mxm_send.lo
> CCLD mca_mtl_mxm.la
>/bin/grep: /usr/local/mofed-inst/1.5.4.1/lib/librdmacm.la: No such file or
>directory
>/bin/sed: can't read /usr/local/mofed-inst/1.5.4.1/lib/librdmacm.la: No such
>file or directory
>libtool: link: `/usr/local/mofed-inst/1.5.4.1/lib/librdmacm.la' is not a valid
>libtool archive
>make[2]: *** [mca_mtl_mxm.la] Error 1
>make[2]: Leaving directory `/data/apps/sources/openmpi-1.6.3/ompi/mca/mtl/mxm'
>make[1]: *** [all-recursive] Error 1
>make[1]: Leaving directory `/data/apps/sources/openmpi-1.6.3/ompi'
>make: *** [all-recursive] Error 1
>
>
>On 12/2/2012 10:18 PM, Mike Dubman wrote:
>> ohh.. you have MOFED 1.5.4.1, thought it was 1.5.3-3.1.0
>> will provide you a link to mxm package compiled with this MOFED version
>> (thanks to no ABI in OFED).
>>
>> On Sun, Dec 2, 2012 at 10:04 PM, Joseph Farran <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> 1.5.4.1
>>
>>
>
>
>
>------------------------------
>
>Message: 9
>Date: Tue, 4 Dec 2012 10:43:05 +0000
>From: William Hay <[email protected]>
>Subject: Re: [OMPI users] BLCR + Qlogic infiniband
>To: [email protected]
>Message-ID:
> <caoeauj71xf06ow6oo3tw+_sxm1zqjxnydqil2wg9nqd7zrk...@mail.gmail.com>
>Content-Type: text/plain; charset="iso-8859-1"
>
>On 28 November 2012 11:14, William Hay <[email protected]> wrote:
>
>> I'm trying to build openmpi with support for BLCR plus qlogic infiniband
>> (plus grid engine). Everything seems to compile OK and checkpoints are
>> taken but whenever I try to restore a checkpoint I get the following error:
>> - do_mmap(<file>, 00002aaab18c7000, 0000000000001000, ...) failed:
>> ffffffffffffffea
>> - mmap failed: /dev/ipath
>> - thaw_threads returned error, aborting. -22
>> - thaw_threads returned error, aborting. -22
>> Restart failed: Invalid argument
>>
>> This occurs whether I specify psm or openib as the btl.
>>
>> This looks like the sort of thing I would expect to be handled by the blcr
>> supporting code in openmpi. So I guess I have a couple ofquestions.
>> 1)Are Infiniband and BLCR support in openmpi compatible?
>> 2)Are there any special tricks necessary to get them working together.
>>
>> A third question occurred to me that may be relevant. How do I verify
>that my openmpi install has blcr support built in? I would have thought
>this would mean that either mpiexec or binaries built with mpicc would have
>libcr linked in. However running ldd doesn't report this in either case.
> I'm setting LD_PRELOAD to point to it but I would have thought openmpi
>would need to register a callback with blcr and it would be easier to do
>this if the library were linked in rather than trying to detect whether it
>has been LD_PRELOADed. I'm building with the following options:
>./configure --prefix=/home/ccaawih/openmpi-blcr --with-openib --without-psm
>--with-blcr=/usr --with-blcr-libdir=/usr/lib64 --with-ft=cr
>--enable-ft-thread --enable-mpi-threads --with-sge
>-------------- next part --------------
>HTML attachment scrubbed and removed
>
>------------------------------
>
>_______________________________________________
>users mailing list
>[email protected]
>http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>End of users Digest, Vol 2428, Issue 1
>**************************************
