I thought Ralph answered this one already: http://www.open-mpi.org/community/lists/users/2012/01/18102.php
On Jan 5, 2012, at 1:25 PM, devendra rai wrote: > Hello Ralph, > > Here is the command that I use to run my application: > > /home/Morpheus/CustomInstalledPrograms/OpenMPI-1.4.2/bin/mpirun -np 2 -v > --timestamp-output --tag-output --d kdbg ./MPIDerivedDataTypes > > where MPIDerivedDataTypes is the sample application. > > Best, > > Devendra > > From: Ralph Castain <r...@open-mpi.org> > To: devendra rai <rai.deven...@yahoo.co.uk> > Sent: Thursday, 5 January 2012, 19:14 > Subject: Re: [OMPI users] Strange Segfault in mpirun: orterun.c:486 > > That is indeed rather strange. The failure occurs at a rather trivial point > in the code, well before anything useful can happen. What does your cmd line > look like? > > > On Jan 5, 2012, at 11:07 AM, devendra rai wrote: > >> Hello Ralph, >> >> This time, rebuilding with the configure line that I showed did not fix the >> problem. And, I am mighty clueless (with some desperation thrown in). >> >> Any ideas would be awesome. >> >> Best, >> >> Devendra >> >> From: Ralph Castain <r...@open-mpi.org> >> To: devendra rai <rai.deven...@yahoo.co.uk>; Open MPI Users >> <us...@open-mpi.org> >> Sent: Thursday, 5 January 2012, 18:49 >> Subject: Re: [OMPI users] Strange Segfault in mpirun: orterun.c:486 >> >> Guess I'm still confused a bit - are you saying that the rebuild no longer >> fixes the problem? >> >> Otherwise, I don't see a ready solution. If your system undergoes a >> significant change, then rebuilding is not an unexpected requirement. >> >> >> On Jan 5, 2012, at 10:39 AM, devendra rai wrote: >> >>> Hello Ralph, >>> >>> Yes, historically, the rebuild fixed the problem. But, now, how do I >>> rebuild against a new libc? And doesn't mpi build process automatically >>> pick up whatever libc version I have? >>> >>> I cant seem to google this solution. I already included my configure line >>> for building open-mpi. >>> >>> Thanks a lot. >>> >>> Devendra Rai >>> >>> From: Ralph Castain <r...@open-mpi.org> >>> To: devendra rai <rai.deven...@yahoo.co.uk>; Open MPI Users >>> <us...@open-mpi.org> >>> Sent: Thursday, 5 January 2012, 18:32 >>> Subject: Re: [OMPI users] Strange Segfault in mpirun: orterun.c:486 >>> >>> Let me see if I understand correctly. You performed some upgrades on Linux, >>> and then orterun fails. Rebuilding OMPI resolved the problem. >>> >>> Is that correct? If so, then I'm not terribly surprised - it depends on >>> what got upgraded. For example, if the default libc changed due to a >>> revision upgrade, then you probably do need to rebuild against it. >>> >>> >>> On Jan 5, 2012, at 10:28 AM, devendra rai wrote: >>> >>>> Hello All, >>>> >>>> This has happened to me many times, but a simple rebuild of the Open-MPI >>>> generally fixed it. >>>> >>>> I had a perfectly working (and tested) MPI application till a few days >>>> ago. Since then, there has been some upgrades on my Linux OS (basically, >>>> the ones which the system picks up automatically). >>>> >>>> Now, I rebuild and rerun the same application, I get this: >>>> >>>> mpirun: orterun.c:486: orterun: Assertion `((0xdeafbeedULL << 32) + >>>> 0xdeafbeedULL) == ((opal_object_t *) (&cmd_line))->obj_magic_id' failed. >>>> [Morpheus:24572] *** Process received signal *** >>>> [Morpheus:24572] Signal: Aborted (6) >>>> [Morpheus:24572] Signal code: (-6) >>>> [Morpheus:24572] [ 0] /lib64/libpthread.so.0(+0xf2d0) [0x7fd23c9312d0] >>>> [Morpheus:24572] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x7fd23c5e7ab5] >>>> [Morpheus:24572] [ 2] /lib64/libc.so.6(abort+0x186) [0x7fd23c5e8fb6] >>>> [Morpheus:24572] [ 3] /lib64/libc.so.6(__assert_fail+0xf5) [0x7fd23c5e0385] >>>> [Morpheus:24572] [ 4] >>>> /home/Morpheus/CustomInstalledPrograms/OpenMPI-1.4.2/bin/mpirun() >>>> [0x4037ed] >>>> [Morpheus:24572] [ 5] >>>> /home/Morpheus/CustomInstalledPrograms/OpenMPI-1.4.2/bin/mpirun() >>>> [0x4032a4] >>>> [Morpheus:24572] [ 6] /lib64/libc.so.6(__libc_start_main+0xfd) >>>> [0x7fd23c5d3bfd] >>>> [Morpheus:24572] [ 7] >>>> /home/Morpheus/CustomInstalledPrograms/OpenMPI-1.4.2/bin/mpirun() >>>> [0x4031c9] >>>> [Morpheus:24572] *** End of error message *** >>>> >>>> >>>> I have a custom installation of open-mpi, with the configure script >>>> looking like this: >>>> >>>> ./configure --enable-mpi-threads=yes >>>> -prefix=/home/Morpheus/CustomInstalledPrograms/OpenMPI-1.4.2 >>>> --enable-mem-profile --enable-mem-debug --enable-debug --enable-trace >>>> --enable-memchecker --with-valgrind=/usr --with-threads=posix >>>> >>>> The rebuild of my application was successful. >>>> >>>> Where should I fix it? and How? >>>> >>>> Thanks for ideas. >>>> >>>> Best >>>> >>>> Devendra >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>> >>> >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/users >> >> >> > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/