[OMPI users] Strange Segmentation Fault inside MPI_Init
Hello OMPI Users,
I'm using OpenMPI 1.4.1 with gcc 4.4.3 on my x86_64 linux system running the
latest Ubuntu 10.04 distro. I don't seem to be able to run any OpenMPI
application. I try running the simplest application, which goes like this
#include
int main(int argc, char * argv[])
{
MPI_Init(NULL, NULL);
MPI_Finalize();
}
Compiling it with "mpicc -g test.c"
Running with "mpirun -n 2 -hostfile hosts a.out"
hosts file contains "localhost slots=2"
On run, I get this
[starbuck:18829] *** Process received signal ***
[starbuck:18830] *** Process received signal ***
[starbuck:18830] Signal: Segmentation fault (11)
[starbuck:18830] Signal code: Address not mapped (1)
[starbuck:18830] Failing at address: 0x3c
[starbuck:18829] Signal: Segmentation fault (11)
[starbuck:18829] Signal code: Address not mapped (1)
[starbuck:18829] Failing at address: 0x3c
[starbuck:18830] [ 0] /lib/libpthread.so.0(+0xf8f0) [0x7f3b0aae08f0]
[starbuck:18830] [ 1] /usr/local/lib/libmca_common_sm.so.1(+0x1561)
[0x7f3b082e8561]
[starbuck:18830] [ 2]
/usr/local/lib/libmca_common_sm.so.1(mca_common_sm_mmap_init+0x6c1)
[0x7f3b082e9137]
[starbuck:18830] [ 3] /usr/lib/openmpi/lib/openmpi/mca_mpool_sm.so(+0x137b)
[0x7f3b084ed37b]
[starbuck:18830] [ 4]
/usr/lib/libmpi.so.0(mca_mpool_base_module_create+0x7d) [0x7f3b0bacc38d]
[starbuck:18830] [ 5] /usr/lib/openmpi/lib/openmpi/mca_btl_sm.so(+0x2a38)
[0x7f3b06c52a38]
[starbuck:18830] [ 6] /usr/lib/openmpi/lib/openmpi/mca_bml_r2.so(+0x18e7)
[0x7f3b076a48e7]
[starbuck:18830] [ 7] /usr/lib/openmpi/lib/openmpi/mca_pml_ob1.so(+0x258c)
[0x7f3b07aae58c]
[starbuck:18830] [ 8] /usr/lib/libmpi.so.0(+0x392bf) [0x7f3b0ba8b2bf]
[starbuck:18830] [ 9] /usr/lib/libmpi.so.0(MPI_Init+0x170) [0x7f3b0baac330]
[starbuck:18830] [10] a.out(main+0x22) [0x400866]
[starbuck:18830] [11] /lib/libc.so.6(__libc_start_main+0xfd)
[0x7f3b0a76cc4d]
[starbuck:18830] [12] a.out() [0x400789]
[starbuck:18830] *** End of error message ***
[starbuck:18829] [ 0] /lib/libpthread.so.0(+0xf8f0) [0x7fb6efefe8f0]
[starbuck:18829] [ 1] /usr/local/lib/libmca_common_sm.so.1(+0x1561)
[0x7fb6ed706561]
[starbuck:18829] [ 2]
/usr/local/lib/libmca_common_sm.so.1(mca_common_sm_mmap_init+0x6c1)
[0x7fb6ed707137]
[starbuck:18829] [ 3] /usr/lib/openmpi/lib/openmpi/mca_mpool_sm.so(+0x137b)
[0x7fb6ed90b37b]
[starbuck:18829] [ 4]
/usr/lib/libmpi.so.0(mca_mpool_base_module_create+0x7d) [0x7fb6f0eea38d]
[starbuck:18829] [ 5] /usr/lib/openmpi/lib/openmpi/mca_btl_sm.so(+0x2a38)
[0x7fb6ec070a38]
[starbuck:18829] [ 6] /usr/lib/openmpi/lib/openmpi/mca_bml_r2.so(+0x18e7)
[0x7fb6ecac28e7]
[starbuck:18829] [ 7] /usr/lib/openmpi/lib/openmpi/mca_pml_ob1.so(+0x258c)
[0x7fb6ececc58c]
[starbuck:18829] [ 8] /usr/lib/libmpi.so.0(+0x392bf) [0x7fb6f0ea92bf]
[starbuck:18829] [ 9] /usr/lib/libmpi.so.0(MPI_Init+0x170) [0x7fb6f0eca330]
[starbuck:18829] [10] a.out(main+0x22) [0x400866]
[starbuck:18829] [11] /lib/libc.so.6(__libc_start_main+0xfd)
[0x7fb6efb8ac4d]
[starbuck:18829] [12] a.out() [0x400789]
[starbuck:18829] *** End of error message ***
--
mpirun noticed that process rank 1 with PID 18830 on node starbuck exited on
signal 11 (Segmentation fault).
--
My stack trace from gdb is:
Program received signal SIGSEGV, Segmentation fault.
0x743c2561 in opal_list_get_first (list=0x745c5240)
at ../../../../../opal/class/opal_list.h:201
201 assert(1 == item->opal_list_item_refcount);
(gdb) bt
#0 0x743c2561 in opal_list_get_first (list=0x745c5240)
at ../../../../../opal/class/opal_list.h:201
#1 0x743c3137 in mca_common_sm_mmap_init (procs=0x673cb0,
num_procs=2, size=67113040,
file_name=0x673c40
"/tmp/openmpi-sessions-srikanth@starbuck_0/1510/1/shared_mem_pool.starbuck",
size_ctl_structure=4176, data_seg_alignment=8)
at ../../../../../ompi/mca/common/sm/common_sm_mmap.c:291
#2 0x745c737b in mca_mpool_sm_init (resources=)
at ../../../../../../ompi/mca/mpool/sm/mpool_sm_component.c:214
#3 0x77ba638d in mca_mpool_base_module_create ()
from /usr/lib/libmpi.so.0
#4 0x72d2ca38 in sm_btl_first_time_init (btl=,
nprocs=, procs=,
peers=, reachability=)
at ../../../../../../ompi/mca/btl/sm/btl_sm.c:228
#5 mca_btl_sm_add_procs (btl=,
nprocs=, procs=,
peers=, reachability=)
at ../../../../../../ompi/mca/btl/sm/btl_sm.c:500
#6 0x7377e8e7 in mca_bml_r2_add_procs (nprocs=,
procs=0x2, reachable=0x7fffdd00)
at ../../../../../../ompi/mca/bml/r2/bml_r2.c:206
#7 0x73b8858c in mca_pml_ob1_add_procs (procs=0x678ce0, nprocs=2)
---Type to continue, or q to quit---
at ../../../../../../ompi/mca/pml/ob1/pml_ob1.c:315
#8 0x77b652bf in ?? () from /usr/lib/libmpi.so.0
#9 0x77b86330 in PMPI_Init () from /usr/lib/libmpi.so.0
#10 0x00400866 in main (argc=1, argv=0x7fffe008)
at test.c:4
Re: [OMPI users] Strange Segmentation Fault inside MPI_Init
Is there any chance you can update to Open MPI 1.4.2?
On Sep 11, 2010, at 9:35 AM, Srikanth Raju wrote:
> Hello OMPI Users,
> I'm using OpenMPI 1.4.1 with gcc 4.4.3 on my x86_64 linux system running the
> latest Ubuntu 10.04 distro. I don't seem to be able to run any OpenMPI
> application. I try running the simplest application, which goes like this
>
> #include
> int main(int argc, char * argv[])
> {
> MPI_Init(NULL, NULL);
> MPI_Finalize();
> }
>
> Compiling it with "mpicc -g test.c"
> Running with "mpirun -n 2 -hostfile hosts a.out"
> hosts file contains "localhost slots=2"
> On run, I get this
>
>
> [starbuck:18829] *** Process received signal ***
> [starbuck:18830] *** Process received signal ***
> [starbuck:18830] Signal: Segmentation fault (11)
> [starbuck:18830] Signal code: Address not mapped (1)
> [starbuck:18830] Failing at address: 0x3c
> [starbuck:18829] Signal: Segmentation fault (11)
> [starbuck:18829] Signal code: Address not mapped (1)
> [starbuck:18829] Failing at address: 0x3c
> [starbuck:18830] [ 0] /lib/libpthread.so.0(+0xf8f0) [0x7f3b0aae08f0]
> [starbuck:18830] [ 1] /usr/local/lib/libmca_common_sm.so.1(+0x1561)
> [0x7f3b082e8561]
> [starbuck:18830] [ 2]
> /usr/local/lib/libmca_common_sm.so.1(mca_common_sm_mmap_init+0x6c1)
> [0x7f3b082e9137]
> [starbuck:18830] [ 3] /usr/lib/openmpi/lib/openmpi/mca_mpool_sm.so(+0x137b)
> [0x7f3b084ed37b]
> [starbuck:18830] [ 4] /usr/lib/libmpi.so.0(mca_mpool_base_module_create+0x7d)
> [0x7f3b0bacc38d]
> [starbuck:18830] [ 5] /usr/lib/openmpi/lib/openmpi/mca_btl_sm.so(+0x2a38)
> [0x7f3b06c52a38]
> [starbuck:18830] [ 6] /usr/lib/openmpi/lib/openmpi/mca_bml_r2.so(+0x18e7)
> [0x7f3b076a48e7]
> [starbuck:18830] [ 7] /usr/lib/openmpi/lib/openmpi/mca_pml_ob1.so(+0x258c)
> [0x7f3b07aae58c]
> [starbuck:18830] [ 8] /usr/lib/libmpi.so.0(+0x392bf) [0x7f3b0ba8b2bf]
> [starbuck:18830] [ 9] /usr/lib/libmpi.so.0(MPI_Init+0x170) [0x7f3b0baac330]
> [starbuck:18830] [10] a.out(main+0x22) [0x400866]
> [starbuck:18830] [11] /lib/libc.so.6(__libc_start_main+0xfd) [0x7f3b0a76cc4d]
> [starbuck:18830] [12] a.out() [0x400789]
> [starbuck:18830] *** End of error message ***
> [starbuck:18829] [ 0] /lib/libpthread.so.0(+0xf8f0) [0x7fb6efefe8f0]
> [starbuck:18829] [ 1] /usr/local/lib/libmca_common_sm.so.1(+0x1561)
> [0x7fb6ed706561]
> [starbuck:18829] [ 2]
> /usr/local/lib/libmca_common_sm.so.1(mca_common_sm_mmap_init+0x6c1)
> [0x7fb6ed707137]
> [starbuck:18829] [ 3] /usr/lib/openmpi/lib/openmpi/mca_mpool_sm.so(+0x137b)
> [0x7fb6ed90b37b]
> [starbuck:18829] [ 4] /usr/lib/libmpi.so.0(mca_mpool_base_module_create+0x7d)
> [0x7fb6f0eea38d]
> [starbuck:18829] [ 5] /usr/lib/openmpi/lib/openmpi/mca_btl_sm.so(+0x2a38)
> [0x7fb6ec070a38]
> [starbuck:18829] [ 6] /usr/lib/openmpi/lib/openmpi/mca_bml_r2.so(+0x18e7)
> [0x7fb6ecac28e7]
> [starbuck:18829] [ 7] /usr/lib/openmpi/lib/openmpi/mca_pml_ob1.so(+0x258c)
> [0x7fb6ececc58c]
> [starbuck:18829] [ 8] /usr/lib/libmpi.so.0(+0x392bf) [0x7fb6f0ea92bf]
> [starbuck:18829] [ 9] /usr/lib/libmpi.so.0(MPI_Init+0x170) [0x7fb6f0eca330]
> [starbuck:18829] [10] a.out(main+0x22) [0x400866]
> [starbuck:18829] [11] /lib/libc.so.6(__libc_start_main+0xfd) [0x7fb6efb8ac4d]
> [starbuck:18829] [12] a.out() [0x400789]
> [starbuck:18829] *** End of error message ***
> --
> mpirun noticed that process rank 1 with PID 18830 on node starbuck exited on
> signal 11 (Segmentation fault).
> --
>
> My stack trace from gdb is:
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x743c2561 in opal_list_get_first (list=0x745c5240)
> at ../../../../../opal/class/opal_list.h:201
> 201 assert(1 == item->opal_list_item_refcount);
> (gdb) bt
> #0 0x743c2561 in opal_list_get_first (list=0x745c5240)
> at ../../../../../opal/class/opal_list.h:201
> #1 0x743c3137 in mca_common_sm_mmap_init (procs=0x673cb0,
> num_procs=2, size=67113040,
> file_name=0x673c40
> "/tmp/openmpi-sessions-srikanth@starbuck_0/1510/1/shared_mem_pool.starbuck",
> size_ctl_structure=4176, data_seg_alignment=8)
> at ../../../../../ompi/mca/common/sm/common_sm_mmap.c:291
> #2 0x745c737b in mca_mpool_sm_init (resources=)
> at ../../../../../../ompi/mca/mpool/sm/mpool_sm_component.c:214
> #3 0x77ba638d in mca_mpool_base_module_create ()
>from /usr/lib/libmpi.so.0
> #4 0x72d2ca38 in sm_btl_first_time_init (btl=,
> nprocs=, procs=,
> peers=, reachability=)
> at ../../../../../../ompi/mca/btl/sm/btl_sm.c:228
> #5 mca_btl_sm_add_procs (btl=,
> nprocs=, procs=,
> peers=, reachability=)
> at ../../../../../../ompi/mca/btl/sm/btl_sm.c:500
> #6 0x7377e8e7 in mca_bml_r2_add_procs (nprocs=,
> procs=0x2, reachable=0x7fffdd00)
> at ../../../../../../ompi/mca/bml/r2/bml_r2.c:206
> #7 0x73b8858c
[OMPI users] Check out my photos on Facebook
Hi Users, I set up a Facebook profile where I can post my pictures, videos and events and I want to add you as a friend so you can see it. First, you need to join Facebook! Once you join, you can also create your own profile. Thanks, Arunkumar To sign up for Facebook, follow the link below: http://www.facebook.com/p.php?i=619869813&k=45LT6W6ZVT6F3FMCXJ222US2WTJE26X&r Already have an account? Add this email address to your account: http://www.facebook.com/n/?merge_accounts.php&e=users%40open-mpi.org&c=599df6e090fa49d94c9af9e8fd111ea3 === us...@open-mpi.org was invited to join Facebook by Arunkumar Cr. If you do not wish to receive this type of email from Facebook in the future, please follow the link below to unsubscribe. http://www.facebook.com/o.php?k=20c27c&u=1134939847&mid=2f54583G43a5cec7G0G8 Facebook, Inc. P.O. Box 10005, Palo Alto, CA 94303
Re: [OMPI users] Strange Segmentation Fault inside MPI_Init
How did you configure OMPI?
On Sep 11, 2010, at 1:35 AM, Srikanth Raju wrote:
> Hello OMPI Users,
> I'm using OpenMPI 1.4.1 with gcc 4.4.3 on my x86_64 linux system running the
> latest Ubuntu 10.04 distro. I don't seem to be able to run any OpenMPI
> application. I try running the simplest application, which goes like this
>
> #include
> int main(int argc, char * argv[])
> {
> MPI_Init(NULL, NULL);
> MPI_Finalize();
> }
>
> Compiling it with "mpicc -g test.c"
> Running with "mpirun -n 2 -hostfile hosts a.out"
> hosts file contains "localhost slots=2"
> On run, I get this
>
>
> [starbuck:18829] *** Process received signal ***
> [starbuck:18830] *** Process received signal ***
> [starbuck:18830] Signal: Segmentation fault (11)
> [starbuck:18830] Signal code: Address not mapped (1)
> [starbuck:18830] Failing at address: 0x3c
> [starbuck:18829] Signal: Segmentation fault (11)
> [starbuck:18829] Signal code: Address not mapped (1)
> [starbuck:18829] Failing at address: 0x3c
> [starbuck:18830] [ 0] /lib/libpthread.so.0(+0xf8f0) [0x7f3b0aae08f0]
> [starbuck:18830] [ 1] /usr/local/lib/libmca_common_sm.so.1(+0x1561)
> [0x7f3b082e8561]
> [starbuck:18830] [ 2]
> /usr/local/lib/libmca_common_sm.so.1(mca_common_sm_mmap_init+0x6c1)
> [0x7f3b082e9137]
> [starbuck:18830] [ 3] /usr/lib/openmpi/lib/openmpi/mca_mpool_sm.so(+0x137b)
> [0x7f3b084ed37b]
> [starbuck:18830] [ 4] /usr/lib/libmpi.so.0(mca_mpool_base_module_create+0x7d)
> [0x7f3b0bacc38d]
> [starbuck:18830] [ 5] /usr/lib/openmpi/lib/openmpi/mca_btl_sm.so(+0x2a38)
> [0x7f3b06c52a38]
> [starbuck:18830] [ 6] /usr/lib/openmpi/lib/openmpi/mca_bml_r2.so(+0x18e7)
> [0x7f3b076a48e7]
> [starbuck:18830] [ 7] /usr/lib/openmpi/lib/openmpi/mca_pml_ob1.so(+0x258c)
> [0x7f3b07aae58c]
> [starbuck:18830] [ 8] /usr/lib/libmpi.so.0(+0x392bf) [0x7f3b0ba8b2bf]
> [starbuck:18830] [ 9] /usr/lib/libmpi.so.0(MPI_Init+0x170) [0x7f3b0baac330]
> [starbuck:18830] [10] a.out(main+0x22) [0x400866]
> [starbuck:18830] [11] /lib/libc.so.6(__libc_start_main+0xfd) [0x7f3b0a76cc4d]
> [starbuck:18830] [12] a.out() [0x400789]
> [starbuck:18830] *** End of error message ***
> [starbuck:18829] [ 0] /lib/libpthread.so.0(+0xf8f0) [0x7fb6efefe8f0]
> [starbuck:18829] [ 1] /usr/local/lib/libmca_common_sm.so.1(+0x1561)
> [0x7fb6ed706561]
> [starbuck:18829] [ 2]
> /usr/local/lib/libmca_common_sm.so.1(mca_common_sm_mmap_init+0x6c1)
> [0x7fb6ed707137]
> [starbuck:18829] [ 3] /usr/lib/openmpi/lib/openmpi/mca_mpool_sm.so(+0x137b)
> [0x7fb6ed90b37b]
> [starbuck:18829] [ 4] /usr/lib/libmpi.so.0(mca_mpool_base_module_create+0x7d)
> [0x7fb6f0eea38d]
> [starbuck:18829] [ 5] /usr/lib/openmpi/lib/openmpi/mca_btl_sm.so(+0x2a38)
> [0x7fb6ec070a38]
> [starbuck:18829] [ 6] /usr/lib/openmpi/lib/openmpi/mca_bml_r2.so(+0x18e7)
> [0x7fb6ecac28e7]
> [starbuck:18829] [ 7] /usr/lib/openmpi/lib/openmpi/mca_pml_ob1.so(+0x258c)
> [0x7fb6ececc58c]
> [starbuck:18829] [ 8] /usr/lib/libmpi.so.0(+0x392bf) [0x7fb6f0ea92bf]
> [starbuck:18829] [ 9] /usr/lib/libmpi.so.0(MPI_Init+0x170) [0x7fb6f0eca330]
> [starbuck:18829] [10] a.out(main+0x22) [0x400866]
> [starbuck:18829] [11] /lib/libc.so.6(__libc_start_main+0xfd) [0x7fb6efb8ac4d]
> [starbuck:18829] [12] a.out() [0x400789]
> [starbuck:18829] *** End of error message ***
> --
> mpirun noticed that process rank 1 with PID 18830 on node starbuck exited on
> signal 11 (Segmentation fault).
> --
>
> My stack trace from gdb is:
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x743c2561 in opal_list_get_first (list=0x745c5240)
> at ../../../../../opal/class/opal_list.h:201
> 201 assert(1 == item->opal_list_item_refcount);
> (gdb) bt
> #0 0x743c2561 in opal_list_get_first (list=0x745c5240)
> at ../../../../../opal/class/opal_list.h:201
> #1 0x743c3137 in mca_common_sm_mmap_init (procs=0x673cb0,
> num_procs=2, size=67113040,
> file_name=0x673c40
> "/tmp/openmpi-sessions-srikanth@starbuck_0/1510/1/shared_mem_pool.starbuck",
> size_ctl_structure=4176, data_seg_alignment=8)
> at ../../../../../ompi/mca/common/sm/common_sm_mmap.c:291
> #2 0x745c737b in mca_mpool_sm_init (resources=)
> at ../../../../../../ompi/mca/mpool/sm/mpool_sm_component.c:214
> #3 0x77ba638d in mca_mpool_base_module_create ()
>from /usr/lib/libmpi.so.0
> #4 0x72d2ca38 in sm_btl_first_time_init (btl=,
> nprocs=, procs=,
> peers=, reachability=)
> at ../../../../../../ompi/mca/btl/sm/btl_sm.c:228
> #5 mca_btl_sm_add_procs (btl=,
> nprocs=, procs=,
> peers=, reachability=)
> at ../../../../../../ompi/mca/btl/sm/btl_sm.c:500
> #6 0x7377e8e7 in mca_bml_r2_add_procs (nprocs=,
> procs=0x2, reachable=0x7fffdd00)
> at ../../../../../../ompi/mca/bml/r2/bml_r2.c:206
> #7 0x73b8858c in mca_pml_ob1_add_procs (p
Re: [OMPI users] What will happen if a non-MPI executible loaded a MPI enabled shared library?
On 7 September 2010 17:08, Allen Zhao wrote: > Hi, All, > > A newbie question: > > Suppose that I have a main executable (a non-MPI code, which just > dynamically loaded a named share library), and I do compile the shared > library with mpi-gcc. This way, when the main executable loads the shared > library, what is going to happen? > Should work just fine. This is the way I use many MPI-based libs and MPI itself from Python. However, make sure you dlopen() libmpi.so with mode RTLD_GLOBAL, or things can co bad (this is the case with OpenMPI). > Can I parallel this application without calling MPI_Init() inside the main > code, instead, I call MPI_Init() when the shared library is loaded by the > main executable? Of course. > Will it then be able to run in parallel? Unlikely. But you could use MPI_Comm_spawn() to spawn a new application, then your main sequential code "chats" to the child parallel app using MPI calls. > If this scenario > is possible, when should I call MPI_Finalize()? > When you know you will not use MPI any more. Perhaps you could register a finalizer using atexit()... -- Lisandro Dalcin --- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169
