MIke,
That may help. It depends on your initialization scripts. Some scripts could check items and skip sections ( like skipping based on stty). Anyway, I am glad you are working. Regards, Joe From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Michael Kuklik Sent: Wednesday, May 27, 2009 10:04 PM To: us...@open-mpi.org Subject: Re: [OMPI users] users Digest, Vol 1242, Issue 5 Joe, I don't know why I didn't think about it. It works with the whole path. I put the intel env script in user .bash_login file. Do you think I should put the intel env script in the global shell config file like /etc/profile in order for libtool to see icc? Thanks for help, mike ________________________________ From: "users-requ...@open-mpi.org" <users-requ...@open-mpi.org> To: us...@open-mpi.org Sent: Wednesday, May 27, 2009 9:24:00 PM Subject: users Digest, Vol 1242, Issue 5 Send users mailing list submissions to us...@open-mpi.org To subscribe or unsubscribe via the World Wide Web, visit http://www.open-mpi.org/mailman/listinfo.cgi/users or, via email, send a message with subject or body 'help' to users-requ...@open-mpi.org You can reach the person managing the list at users-ow...@open-mpi.org When replying, please edit your Subject line so it is more specific than "Re: Contents of users digest..." Today's Topics: 1. Re: problem with installing openmpi with intelcompiler onubuntu (Joe Griffin) ---------------------------------------------------------------------- Message: 1 List-Post: users@lists.open-mpi.org Date: Wed, 27 May 2009 18:23:10 -0700 From: "Joe Griffin" <joe.grif...@mscsoftware.com> Subject: Re: [OMPI users] problem with installing openmpi with intelcompiler onubuntu To: "Open MPI Users" <us...@open-mpi.org>, <us...@open-mpi.org> Message-ID: <1d367926756e9848babd800e249aa5e04bf...@nascmex01.na.mscsoftware.com> Content-Type: text/plain; charset="iso-8859-1" MK, Hmm.. What if you put CC=/usr/local/intel/Compiler/11.0/083/bin/intel64/icc on the build line. Joe ________________________________ From: users-boun...@open-mpi.org on behalf of Michael Kuklik Sent: Wed 5/27/2009 5:05 PM To: us...@open-mpi.org Subject: Re: [OMPI users] problem with installing openmpi with intelcompiler onubuntu Joe 'which icc' returns the path to icc /usr/local/intel/Compiler/11.0/083/bin/intel64/icc and I used the env variable script provided by intel. so my shell env is ok and I think libtool should inherit my shell environment just in case I send you the env printout MKLROOT=/usr/local/intel/Compiler/11.0/083/mkl MANPATH=/usr/local/intel/Compiler/11.0/083/man:/usr/local/intel/Compiler /11.0/083/mkl/man/en_US:/usr/local/intel/Compiler/11.0/083/man:/usr/loca l/intel/Compiler/11.0/083/mkl/man/en_US:/usr/local/man:/usr/local/share/ man:/usr/share/man INTEL_LICENSE_FILE=/usr/local/intel/Compiler/11.0/083/licenses:/opt/inte l/licenses:/home/mkuklik/intel/licenses:/usr/local/intel/Compiler/11.0/0 83/licenses:/opt/intel/licenses:/home/mkuklik/intel/licenses IPPROOT=/usr/local/intel/Compiler/11.0/083/ipp/em64t TERM=xterm-color SHELL=/bin/bash XDG_SESSION_COOKIE=d03e782e0b3c90f7ce8380174a15d9d2-1243468120.315267-10 57427925 SSH_CLIENT=128.151.210.198 54616 22 LIBRARY_PATH=/usr/local/intel/Compiler/11.0/083/ipp/em64t/lib:/usr/local /intel/Compiler/11.0/083/mkl/lib/em64t:/usr/local/intel/Compiler/11.0/08 3/tbb/em64t/cc4.1.0_libc2.4_kernel2.6.16.21/lib:/usr/local/intel/Compile r/11.0/083/ipp/em64t/lib:/usr/local/intel/Compiler/11.0/083/mkl/lib/em64 t:/usr/local/intel/Compiler/11.0/083/tbb/em64t/cc4.1.0_libc2.4_kernel2.6 .16.21/lib FPATH=/usr/local/intel/Compiler/11.0/083/mkl/include:/usr/local/intel/Co mpiler/11.0/083/mkl/include SSH_TTY=/dev/pts/4 LC_ALL=C USER=mkuklik LD_LIBRARY_PATH=/usr/local/intel/Compiler/11.0/083/lib/intel64:/usr/loca l/intel/Compiler/11.0/083/ipp/em64t/sharedlib:/usr/local/intel/Compiler/ 11.0/083/mkl/lib/em64t:/usr/local/intel/Compiler/11.0/083/tbb/em64t/cc4. 1.0_libc2.4_kernel2.6.16.21/lib:/usr/local/intel/Compiler/11.0/083/lib/i ntel64:/usr/local/intel/Compiler/11.0/083/ipp/em64t/sharedlib:/usr/local /intel/Compiler/11.0/083/mkl/lib/em64t:/usr/local/intel/Compiler/11.0/08 3 /tbb/em64t/cc4.1.0_libc2.4_kernel2.6.16.21/lib LIB=/usr/local/intel/Compiler/11.0/083/ipp/em64t/lib:/usr/local/intel/Co mpiler/11.0/083/ipp/em64t/lib: CPATH=/usr/local/intel/Compiler/11.0/083/ipp/em64t/include:/usr/local/in tel/Compiler/11.0/083/mkl/include:/usr/local/intel/Compiler/11.0/083/tbb /include:/usr/local/intel/Compiler/11.0/083/ipp/em64t/include:/usr/local /intel/Compiler/11.0/083/mkl/include:/usr/local/intel/Compiler/11.0/083/ tbb/include NLSPATH=/usr/local/intel/Compiler/11.0/083/lib/intel64/locale/%l_%t/%N:/ usr/local/intel/Compiler/11.0/083/ipp/em64t/lib/locale/%l_%t/%N:/usr/loc al/intel/Compiler/11.0/083/mkl/lib/em64t/locale/%l_%t/%N:/usr/local/inte l/Compiler/11.0/083/idb/intel64/locale/%l_%t/%N:/usr/local/intel/Compile r/11.0/083/lib/intel64/locale/%l_%t/%N:/usr/local/intel/Compiler/11.0/08 3/ipp/em64t/lib/locale/%l_%t/%N:/usr/local/intel/Compiler/11.0/083/mkl/l ib/em64t/locale/%l_%t/%N:/usr/local/intel/Compiler/11.0/083/idb/intel64/ loca le/%l_%t/%N MAIL=/var/mail/mkuklik PATH=/usr/local/intel/Compiler/11.0/083/bin/intel64:/usr/local/intel/Com piler/11.0/083/bin/intel64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr /bin:/sbin:/bin:/usr/games PWD=/home/mkuklik LANG=en_US SHLVL=1 HOME=/home/mkuklik DYLD_LIBRARY_PATH=/usr/local/intel/Compiler/11.0/083/tbb/em64t/cc4.1.0_l ibc2.4_kernel2.6.16.21/lib:/usr/local/intel/Compiler/11.0/083/tbb/em64t/ cc4.1.0_libc2.4_kernel2.6.16.21/lib LOGNAME=mkuklik SSH_CONNECTION=128.151.210.198 54616 128.151.210.190 22 INCLUDE=/usr/local/intel/Compiler/11.0/083/ipp/em64t/include:/usr/local/ intel/Compiler/11.0/083/mkl/include:/usr/local/intel/Compiler/11.0/083/i pp/em64t/include:/usr/local/intel/Compiler/11.0/083/mkl/include _=/usr/bin/env Thanks, mk ________________________________ ---------------------------------------------------------------------- Message: 1 List-Post: users@lists.open-mpi.org Date: Tue, 26 May 2009 19:51:48 -0700 From: "Joe Griffin" <joe.grif...@mscsoftware.com> Subject: Re: [OMPI users] problem with installing openmpi with intel compiler onubuntu To: "Open MPI Users" <us...@open-mpi.org> Message-ID: <1d367926756e9848babd800e249aa5e04bf...@nascmex01.na.mscsoftware.com> Content-Type: text/plain; charset="iso-8859-1" MK, Is "icc" in your path? What if you type "which icc"? Joe ________________________________ From: users-boun...@open-mpi.org on behalf of Michael Kuklik Sent: Tue 5/26/2009 7:05 PM To: us...@open-mpi.org Subject: [OMPI users] problem with installing openmpi with intel compiler onubuntu Hi everybody, I try to compile openmpi with intel compiler on ubuntu 9.04. I compiled openmpi on Redhat and os x many times and I could always find a problem. But the error that I'm getting now, gives me no clues where to even search for the problem. my config line is a follows: ./configure CC=icc CXX=icpc --prefix=/usr/local/intel/openmpi Everything configures and compiles OK. But then when I try to install I get this error Making install in etc make[2]: Entering directory `/tmp/openmpi-1.3.2/orte/etc' make[3]: Entering directory `/tmp/openmpi-1.3.2/orte/etc' make[3]: Nothing to be done for `install-exec-am'. /bin/mkdir -p /usr/local/intel/openmpi/etc ******************************* WARNING ************************************ *** Not installing new openmpi-default-hostfile over existing file in: *** /usr/local/intel/openmpi/etc/openmpi-default-hostfile ******************************* WARNING ************************************ make[3]: Leaving directory `/tmp/openmpi-1.3.2/orte/etc' make[2]: Leaving directory `/tmp/openmpi-1.3.2/orte/etc' Making install in . make[2]: Entering directory `/tmp/openmpi-1.3.2/orte' make[3]: Entering directory `/tmp/openmpi-1.3.2/orte' test -z "/usr/local/intel/openmpi/lib" || /bin/mkdir -p "/usr/local/intel/openmpi/lib" /bin/bash ../libtool --mode=install /usr/bin/install -c 'libopen-rte.la <http://libopen-rte.la/> ' '/usr/local/intel/openmpi/lib/libopen-rte.la' libtool: install: warning: relinking `libopen-rte.la' libtool: install: (cd /tmp/openmpi-1.3.2/orte; /bin/bash /tmp/openmpi-1.3.2/libtool --tag CC --mode=relink icc -O3 -DNDEBUG -finline-functions -fno-strict-aliasing ................ ) libtool: relink: icc -shared runtime/.libs/orte_finalize.o runtime/.libs/orte_init.o runtime/.libs/orte_locks.o runtime/.libs/orte_globals.o runtime/data_type_support/.libs/orte_dt_compare_fns.o runtime/data_type_support/.libs/orte_dt_copy_fns.o runtime/data_type_support/.libs/orte_dt_print_fns.o runtime/data_type_support/.libs/orte_dt_release_fns.o runtime/data_type_support/.libs/orte_dt_size_fns.o runtime/data_type_support/.libs/orte_dt_packing_fns.o runtime/data_type_support/.libs/orte_dt_unpacking_fns.o runtime/.libs/orte_mca_params.o runtime/.libs/orte_wait.o runtime/.libs/orte_cr.o runtime/.libs/..................................... -Wl,libopen-rte.so <http://libopen-rte.so/> .0 -o .libs/libopen-rte.so.0.0.0 /tmp/openmpi-1.3.2/libtool: line 7847: icc: command not found libtool: install: error: relink `libopen-rte.la' with the above command before installing it make[3]: *** [install-libLTLIBRARIES] Error 1 make[3]: Leaving directory `/tmp/openmpi-1.3.2/orte' make[2]: *** [install-am] Error 2 make[2]: Leaving directory `/tmp/openmpi-1.3.2/orte' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/tmp/openmpi-1.3.2/orte' make: *** [install-recursive] Error 1 libtool is the one from ubuntu repository i.e. 2.2.6a-1 icc and icpc are the newest ones i.e. 11.083 Ouputs of configure make and install are attached. Any clues what's wrong? Thanks for help mk -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 5414 bytes Desc: not available URL: <http://www.open-mpi.org/MailArchives/users/attachments/20090526/9737163 d/attachment.bin> ------------------------------ Message: 2 List-Post: users@lists.open-mpi.org Date: Wed, 27 May 2009 13:09:27 +0300 From: vasilis <gka...@ipta.demokritos.gr> Subject: Re: [OMPI users] "An error occurred in MPI_Recv" with more than 2 CPU To: Open MPI Users <us...@open-mpi.org> Message-ID: <200905271309.27914.gka...@ipta.demokritos.gr> Content-Type: Text/Plain; charset="iso-8859-1" Thank you Eugene for your suggestion. I used different tags for each variable, and now I do not get this error. The problem now is that I am getting a different solution when I use more than 2 CPUs. I checked the matrices and I found that they differ by a very small amount of the order 10^(-10). Actually, I am getting a different solution if I use 4CPUs or 16CPUs!!! Do you have any idea what could cause this behavior? Thank you, Vasilis On Tuesday 26 of May 2009 7:21:32 pm you wrote: > vasilis wrote: > >Dear openMpi users, > > > >I am trying to develop a code that runs in parallel mode with openMPI > > (1.3.2 version). The code is written in Fortran 90, and I am running on > > a cluster > > > >If I use 2 CPU the program runs fine, but for a larger number of CPUs I > > get the following error: > > > >[compute-2-6.local:18491] *** An error occurred in MPI_Recv > >[compute-2-6.local:18491] *** on communicator MPI_COMM_WORLD > >[compute-2-6.local:18491] *** MPI_ERR_TRUNCATE: message truncated > >[compute-2-6.local:18491] *** MPI_ERRORS_ARE_FATAL (your MPI job will now > >abort) > > > >Here is the part of the code that this error refers to: > >if( mumps_par%MYID .eq. 0 ) THEN > > res=res+res_cpu > > do iw=1,total_elem_cpu*unique > > jacob(iw)=jacob(iw)+jacob_cpu(iw) > > position_col(iw)=position_col(iw)+col_cpu(iw) > > position_row(iw)=position_row(iw)+row_cpu(iw) > > end do > > > > do jw=1,nsize-1 > > call > >MPI_recv(jacob_cpu,total_elem_cpu*unique,MPI_DOUBLE_PRECISION,MPI_ANY_S OUR > >CE,MPI_ANY_TAG,MPI_COMM_WORLD,status1,ierr) call > >MPI_recv(res_cpu,total_unknowns,MPI_DOUBLE_PRECISION,MPI_ANY_SOURCE,MPI _AN > >Y_TAG,MPI_COMM_WORLD,status2,ierr) call > >MPI_recv(row_cpu,total_elem_cpu*unique,MPI_INTEGER,MPI_ANY_SOURCE,MPI_A NY_ > >TAG,MPI_COMM_WORLD,status3,ierr) call > >MPI_recv(col_cpu,total_elem_cpu*unique,MPI_INTEGER,MPI_ANY_SOURCE,MPI_A NY_ > >TAG,MPI_COMM_WORLD,status4,ierr) > > > > res=res+res_cpu > > do iw=1,total_elem_cpu*unique > > > > jacob(status1(MPI_SOURCE)*total_elem_cpu*unique+iw)=& > > jacob(status1(MPI_SOURCE)*total_elem_cpu*unique+iw)+jacob_cpu(iw) > > position_col(status4(MPI_SOURCE)*total_elem_cpu*unique+iw)=& > > position_col(status4(MPI_SOURCE)*total_elem_cpu*unique+iw)+col_cpu(iw) > > position_row(status3(MPI_SOURCE)*total_elem_cpu*unique+iw)=& > > position_row(status3(MPI_SOURCE)*total_elem_cpu*unique+iw)+row_cpu(iw) > > end do > > end do > > else > > call > >MPI_Isend(jacob_cpu,total_elem_cpu*unique,MPI_DOUBLE_PRECISION,0,mumps_ par > >%MYID,MPI_COMM_WORLD,request1,ierr) call > >MPI_Isend(res_cpu,total_unknowns,MPI_DOUBLE_PRECISION,0,mumps_par%MYID, MPI > >_COMM_WORLD,request2,ierr) call > >MPI_Isend(row_cpu,total_elem_cpu*unique,MPI_INTEGER,0,mumps_par%MYID,MP I_C > >OMM_WORLD,request3,ierr) call > >MPI_Isend(col_cpu,total_elem_cpu*unique,MPI_INTEGER,0,mumps_par%MYID,MP I_C > >OMM_WORLD,request4,ierr) call MPI_Wait(request1, status1, ierr) > > call MPI_Wait(request2, status2, ierr) > > call MPI_Wait(request3, status3, ierr) > > call MPI_Wait(request4, status4, ierr) > > end if > > > > > >I am also using the MUMPS library > > > >Could someone help to track this error down. Is really annoying to use > > only two processors. > >The cluster has about 8 nodes and each has 4 dual core CPU. I tried to run > > the code on a single node with more than 2 CPU but I got the same error!! > > I think the error message means that the received message was longer > than the receive buffer that was specified. If I look at your code and > try to reason about its correctness, I think of the message-passing > portion as looking like this: > > if( mumps_par%MYID .eq. 0 ) THEN > do jw=1,nsize-1 > call > MPI_recv(jacob_cpu,total_elem_cpu*unique,MPI_DOUBLE_PRECISION,MPI_ANY_SO URC >E,MPI_ANY_TAG,MPI_COMM_WORLD,status1,ierr) call MPI_recv( > res_cpu,total_unknowns > ,MPI_DOUBLE_PRECISION,MPI_ANY_SOURCE,MPI_ANY_TAG,MPI_COMM_WORLD,status2, ier >r) call MPI_recv( > row_cpu,total_elem_cpu*unique,MPI_INTEGER > ,MPI_ANY_SOURCE,MPI_ANY_TAG,MPI_COMM_WORLD,status3,ierr) > call MPI_recv( > col_cpu,total_elem_cpu*unique,MPI_INTEGER > ,MPI_ANY_SOURCE,MPI_ANY_TAG,MPI_COMM_WORLD,status4,ierr) > end do > else > call > MPI_Send(jacob_cpu,total_elem_cpu*unique,MPI_DOUBLE_PRECISION,0,mumps_pa r%M >YID,MPI_COMM_WORLD,ierr) call MPI_Send( res_cpu,total_unknowns > ,MPI_DOUBLE_PRECISION,0,mumps_par%MYID,MPI_COMM_WORLD,ierr) > call MPI_Send( row_cpu,total_elem_cpu*unique,MPI_INTEGER > ,0,mumps_par%MYID,MPI_COMM_WORLD,ierr) > call MPI_Send( col_cpu,total_elem_cpu*unique,MPI_INTEGER > ,0,mumps_par%MYID,MPI_COMM_WORLD,ierr) > end if > > If you're running on two processes, then the messages you receive are in > the order you expect. If there are more than two processes, however, > certainly messages will start appearing "out of order" and your > indiscriminate use of MPI_ANY_SOURCE and MPI_ANY_TAG will start getting > them mixed up. You won't just get all messages from one rank and then > all from another and then all from another. Rather, the messages from > all these other processes will come interwoven, but you interpret them > in a fixed order. > > Here is what I mean. Let's say you have 3 processes. So, rank 0 will > receive 8 messages: 4 from rank 1and 4 from rank 2. Correspondingly, > rank 1 and rank 2 will each send 4 messages to rank 0. Here is a > possibility for the order in which messages are received: > > jacob_cpu from rank 1 > jacob_cpu from rank 2 > res_cpu from rank 1 > row_cpu from rank 1 > res_cpu from rank 2 > row_cpu from rank 2 > col_cpu from rank 2 > col_cpu from rank 1 > > Rank 0, however, is trying to unpack these in the order you prescribed > in your code. Data will get misinterpreted. More to the point here, > you will be trying to receive data into buffers of the wrong size (some > of the time). > > Maybe you should use tags to distinguish between the different types of > messages you're trying to send. > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users ------------------------------ Message: 3 List-Post: users@lists.open-mpi.org Date: Wed, 27 May 2009 07:41:08 -0700 From: Eugene Loh <eugene....@sun.com> Subject: Re: [OMPI users] "An error occurred in MPI_Recv" with more than 2 CPU To: Open MPI Users <us...@open-mpi.org> Message-ID: <4a1d5104.7090...@sun.com> Content-Type: text/plain; CHARSET=US-ASCII; format=flowed vasilis wrote: >Thank you Eugene for your suggestion. I used different tags for each variable, >and now I do not get this error. >The problem now is that I am getting a different solution when I use more than >2 CPUs. I checked the matrices and I found that they differ by a very small >amount of the order 10^(-10). Actually, I am getting a different solution if I >use 4CPUs or 16CPUs!!! >Do you have any idea what could cause this behavior? > > Sure. Rank 0 accumulates all the res_cpu values into a single array, res. It starts with its own res_cpu and then adds all other processes. When np=2, that means the order is prescribed. When np>2, the order is no longer prescribed and some floating-point rounding variations can start to occur. If you want results to be more deterministic, you need to fix the order in which res is aggregated. E.g., instead of using MPI_ANY_SOURCE, loop over the peer processes in a specific order. P.S. It seems to me that you could use MPI collective operations to implement what you're doing. E.g., something like: call MPI_Reduce(res_cpu, res, total_unknown, MPI_DOUBLE_PRECISION, MPI_SUM, 0, MPI_COMM_WORLD, ierr) call MPI_Gather(jacob_cpu, total_elem_cpu * unique, MPI_DOUBLE_PRECISION, & jacob , total_elem_cpu * unique, MPI_DOUBLE_PRECISION, 0, MPI_COMM_WORLD, ierr) call MPI_Gather( row_cpu, total_elem_cpu * unique, MPI_INTEGER , & row , total_elem_cpu * unique, MPI_INTEGER , 0, MPI_COMM_WORLD, ierr) call MPI_Gather( col_cpu, total_elem_cpu * unique, MPI_INTEGER , & col , total_elem_cpu * unique, MPI_INTEGER , 0, MPI_COMM_WORLD, ierr) I think the res part is right. The jacob/row/col parts are not quite right since you don't just want to gather the elements, but add them into particular arrays. Not sure if you really want to allocate a new scratch array to use for this purpose or what. Nor would this solve the res_cpu indeterministic problem you had. I just wanted to make sure you knew about the MPI collective operations as an alternative to your point-to-point implementation. ------------------------------ Message: 4 List-Post: users@lists.open-mpi.org Date: Wed, 27 May 2009 10:28:42 -0400 From: Jeff Squyres <jsquy...@cisco.com> Subject: Re: [OMPI users] How to use Multiple links with OpenMPI?????????????????? To: "Open MPI Users" <us...@open-mpi.org> Message-ID: <8864ed55-66a8-424e-b1b9-249f03381...@cisco.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Open MPI considers hosts differently than network links. So you should only list the actual hostname in the hostfile, with slots equal to the number of processors (4 in your case, I think?). Once the MPI processes are launched, they each look around on the host that they're running and find network paths to each of their peers. If they are multiple paths between pairs of peers, Open MPI will round- robin stripe messages across each of the links. We don't really have an easy setting for each peer pair only using 1 link. Indeed, since connectivity is bidirectional, the traffic patterns become less obvious if you want MPI_COMM_WORLD rank X to only use link Y -- what does that mean to the other 4 MPI processes on the other host (with whom you have assumedly assigned their own individual links as well)? On May 26, 2009, at 12:24 AM, shan axida wrote: > Hi everyone, > I want to ask how to use multiple links (multiple NICs) with OpenMPI. > For example, how can I assign a link to each process, if there are 4 > links > and 4 processors on each node in our cluster? > Is this a correct way? > hostfile: > ---------------------- > host1-eth0 slots=1 > host1-eth1 slots=1 > host1-eth2 slots=1 > host1-eth3 slots=1 > host2-eth0 slots=1 > host2-eth1 slots=1 > host2-eth2 slots=1 > host2-eth3 slots=1 > ... ... > ... ... > host16-eth0 slots=1 > host16-eth1 slots=1 > host16-eth2 slots=1 > host16-eth3 slots=1 > ------------------------ > > > > > > > > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- Jeff Squyres Cisco Systems ------------------------------ _______________________________________________ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users End of users Digest, Vol 1242, Issue 1 ************************************** -------------- next part -------------- HTML attachment scrubbed and removed ------------------------------ _______________________________________________ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users End of users Digest, Vol 1242, Issue 5 **************************************
