Re: [O-MPI users] Open-MPI all-to-all performance

2006-02-06 Thread Konstantin Kudin
Dear Galen, It actually turns out that there is a problem not only with MPI_Alltoall_Isend_Irecv, but also with another related operation insyncol_MPI_Alltoallv-nodes-long-SM.ski (this what seems to be holding down the FFTs, I checked the source code, and it uses alltoallv): #/*@insyncol_MPI_Al

Re: [O-MPI users] Open-MPI all-to-all performance

2006-02-05 Thread Galen M. Shipman
Hi Konstantin, MPI_Alltoall_Isend_Irecv This is a very unscalable algorithm in skampi as it simply posts N MPI_Irecv's and MPI_Isend's and then does a Waitall. We shouldn't have an issue though on 8 procs but in general I would expect the performance of this algorithm to degrade quite q

Re: [O-MPI users] Open-MPI all-to-all performance

2006-02-04 Thread Konstantin Kudin
Sorry, I forgot to specify everything properly in my previous e-mail: mpirun -np 8 -mca btl tcp -mca coll self,basic,tuned -mca \ mpi_paffinity_alone 1 skampi41 #/*@insyncol_MPI_Alltoall-nodes-long-SM.ski*/ 2 256.4 2.8 8 256.4 2.8 8 32153.0 13.0

Re: [O-MPI users] Open-MPI all-to-all performance

2006-02-04 Thread Konstantin Kudin
Dear Jeff and Galen, I have tried openmpi-1.1a1r8890. The good news is that it seems like the freaky long latencies for certain packet sizes went away with the options they showed up with before. Also, one version of all-to-all appears to behave nicer with a specified set of parameters. However,

Re: [O-MPI users] Open-MPI all-to-all performance

2006-02-03 Thread Jeff Squyres
Greetings Konstantin. Many thanks for this report. Another user submitted almost the same issue earlier today (poor performance of Open MPI 1.0.x collectives; see http://www.open-mpi.org/community/lists/users/2006/02/0558.php). Let me provide an additional clarification on Galen's reply:

Re: [O-MPI users] Open-MPI all-to-all performance

2006-02-03 Thread Galen M. Shipman
Hello Konstantin, By using coll_basic_crossover 8 you are forcing all of your benchmarks to use the basic collectives, which offer poor performance. I ran the skampi Alltoall benchmark with the tuned collectives I get the following results which seem to scale quite well, when I have a bit

Re: [O-MPI users] Open-MPI all-to-all performance

2006-02-02 Thread Konstantin Kudin
Hi all, There seem to have been problems with the attachement. Here is the report: I did some tests of Open-MPI version 1.0.2a4r8848. My motivation was an extreme degradation of all-to-all MPI performance on 8 cpus (ran like 1 cpu). At the same time, MPICH 1.2.7 on 8 cpus runs more like on 4 (

[O-MPI users] Open-MPI all-to-all performance

2006-02-02 Thread Konstantin Kudin
Hi all, Please see the attached file for a detailed report of Open-MPI performance. Any fixes in the pipeline for that? Konstantin __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com repor