Does the difference persist if you run the single process using mpirun? In other words, does "mpirun -np 1 ./my_hybrid_app..." behave the same as "mpirun -np 2 ./..."?
There is a slight difference in the way procs start when run as singletons. It shouldn't make a difference here, but worth testing. On Oct 24, 2011, at 12:37 AM, 吕慧伟 wrote: > Dear List, > > I have a hybrid MPI/Pthreads program named "my_hybrid_app", this program is > memory-intensive and take advantage of multi-threading to improve memory > throughput. I run "my_hybrid_app" on two machines, which have same hardware > configuration but different OS and GCC. The problem is: when I run > "my_hybrid_app" with one process, two machines behaves the same: the more > number of threads, the better the performance; however, when I run > "my_hybrid_app" with two or more processes. The first machine still increase > performance with more threads, the second machine degrades in performance > with more threads. > > Since running "my_hybrid_app" with one process behaves correctly, I suspect > my linking to MPI library has some problem. Would somebody point me in the > right direction? Thanks in advance. > > Attached are the commandline used, my machine informantion and link > informantion. > p.s. 1: Commandline > single process: ./my_hybrid_app <number of threads> > multiple process: mpirun -np 2 ./my_hybrid_app <number of threads> > > p.s. 2: Machine Informantion > The first machine is CentOS 5.3 with GCC 4.1.2: > Target: x86_64-redhat-linux > Configured with: ../configure --prefix=/usr --mandir=/usr/share/man > --infodir=/usr/share/info --enable-shared --enable-threads=posix > --enable-checking=release --with-system-zlib --enable-__cxa_atexit > --disable-libunwind-exceptions --enable-libgcj-multifile > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk > --disable-dssi --enable-plugin > --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic > --host=x86_64-redhat-linux > Thread model: posix > gcc version 4.1.2 20080704 (Red Hat 4.1.2-44) > The second machine is SUSE Enterprise Server 11 with GCC 4.3.4: > Target: x86_64-suse-linux > Configured with: ../configure --prefix=/usr --infodir=/usr/share/info > --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 > --enable-languages=c,c++,objc,fortran,obj-c++,java,ada > --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3 > --enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/ > --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap > --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit > --enable-libstdcxx-allocator=new --disable-libstdcxx-pch > --enable-version-specific-runtime-libs --program-suffix=-4.3 > --enable-linux-futex --without-system-libunwind --with-cpu=generic > --build=x86_64-suse-linux > Thread model: posix > gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux) > > p.s. 3: ldd Informantion > The first machine: > $ ldd my_hybrid_app > libm.so.6 => /lib64/libm.so.6 (0x000000358d400000) > libmpi.so.0 => /usr/local/openmpi/lib/libmpi.so.0 (0x00002af0d53a7000) > libopen-rte.so.0 => /usr/local/openmpi/lib/libopen-rte.so.0 > (0x00002af0d564a000) > libopen-pal.so.0 => /usr/local/openmpi/lib/libopen-pal.so.0 > (0x00002af0d5895000) > libdl.so.2 => /lib64/libdl.so.2 (0x000000358d000000) > libnsl.so.1 => /lib64/libnsl.so.1 (0x000000358f000000) > libutil.so.1 => /lib64/libutil.so.1 (0x000000359a600000) > libgomp.so.1 => /usr/lib64/libgomp.so.1 (0x00002af0d5b07000) > libpthread.so.0 => /lib64/libpthread.so.0 (0x000000358d800000) > libc.so.6 => /lib64/libc.so.6 (0x000000358cc00000) > /lib64/ld-linux-x86-64.so.2 (0x000000358c800000) > librt.so.1 => /lib64/librt.so.1 (0x000000358dc00000) > The second machine: > $ ldd my_hybrid_app > linux-vdso.so.1 => (0x00007fff3eb5f000) > libmpi.so.0 => /root/opt/openmpi/lib/libmpi.so.0 (0x00007f68627a1000) > libm.so.6 => /lib64/libm.so.6 (0x00007f686254b000) > libopen-rte.so.0 => /root/opt/openmpi/lib/libopen-rte.so.0 > (0x00007f68622fc000) > libopen-pal.so.0 => /root/opt/openmpi/lib/libopen-pal.so.0 > (0x00007f68620a5000) > libdl.so.2 => /lib64/libdl.so.2 (0x00007f6861ea1000) > libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f6861c89000) > libutil.so.1 => /lib64/libutil.so.1 (0x00007f6861a86000) > libgomp.so.1 => /usr/lib64/libgomp.so.1 (0x00007f686187d000) > libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6861660000) > libc.so.6 => /lib64/libc.so.6 (0x00007f6861302000) > /lib64/ld-linux-x86-64.so.2 (0x00007f6862a58000) > librt.so.1 => /lib64/librt.so.1 (0x00007f68610f9000) > I installed openmpi-1.4.2 to a user directory /root/opt/openmpi and use > "-L/root/opt/openmpi -Wl,-rpath,/root/opt/openmpi" when linking. > -- > Huiwei Lv > PhD. student at Institute of Computing Technology, > Beijing, China > http://asg.ict.ac.cn/lhw > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
