Did you bind the processes? If not you may be seeing the impact of having
processes bouncing between cpus, and/or processes not being local to their
memory. Try adding -bind-to-core or -bind-to-socket to your cmd line and see if
things smooth out. I'm assuming, of course, that you are running on a system
that supports binding...
The time can also be impacted by other things running on your cpu - could be
context switching.
Final point: since both processes are running on the same node, IB will have no
involvement - the messages are going to flow over shared memory.
On Aug 4, 2010, at 6:51 AM, Tad Lake wrote:
> Hi,
>
> I have a little program for execution time.
> =================================================
> #include "mpi.h"
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> #include <math.h>
> int main (int argc, char *argv[]) {
> MPI_Status Stat;
> struct timeval tv0, tv1;
>
> long int totaltime = 0;
> int i, j;
> int buf[10240];
> int numtasks, rank;
>
> MPI_Init (&argc, &argv);
> MPI_Comm_size (MPI_COMM_WORLD, &numtasks);
> MPI_Comm_rank (MPI_COMM_WORLD, &rank);
>
>
> if (rank == 0) {
> gettimeofday("&tv0, NULL);
> for(i=0;i<1000;i++){
> MPI_Send (buf, 10240, MPI_INT, 1, tag, MPI_COMM_WORLD);
> MPI_Recv (buf, 10240, MPI_INT, 1, tag,MPI_COMM_WORLD, &Stat);
> }
> gettimeofday (&tv1, NULL);
> }else{
> gettimeofday(&tv0, NULL);
> for(i=0;i<1000;i++){
> MPI_Recv(buf, 10240,MPI_INT, 0, tag, MPI_COMM_WORLD, &Stat);
> MPI_Send(buf, 10240, MPI_INT, 1, tag, MPI_COMM_WORLD);
> }
> gettimeofday(&tv1, NULL);
> }
>
> totaltime = (tv1.tv_sec - tv0.tv_sec) * 1000000 + (tv1.tv_usec -
> tv0.tv_usec);
> fprintf (stdout, "rank %d with total time is %d",rank, totaltime);
> }
>
> MPI_Finalize ();
>
> return 0;
> }
> =======================================================
>
> I run it :
> mpirun -np 2 --host node2 ./a.out
>
> But the result of time is not stable with difference of 100 times. For
> example, the max value of time can be 3000, meanwhile the min is 100.
>
> Is there anything wrong ?
> I am using 1.4.2 and openib.
>
>
>
>
>
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
