Out of curiosity, can you try mpirun --mca btl self,sm ... and mpirun --mca btl self,vader ...
and see if one performs better than the other ? Cheers, Gilles On Thursday, May 5, 2016, Zhen Wang <tod...@gmail.com> wrote: > Gilles, > > Thanks for your reply. > > Best regards, > Zhen > > On Wed, May 4, 2016 at 8:43 PM, Gilles Gouaillardet < > gilles.gouaillar...@gmail.com > <javascript:_e(%7B%7D,'cvml','gilles.gouaillar...@gmail.com');>> wrote: > >> Note there is no progress thread in openmpi 1.10 >> from a pragmatic point of view, that means that for "large" messages, no >> data is sent in MPI_Isend, and the data is sent when MPI "progresses" e.g. >> call a MPI_Test, MPI_Probe, MPI_Recv or some similar subroutine. >> in your example, the data is transferred after the first usleep completes. >> > I agree. > >> >> that being said, it takes quite a while, and there could be an issue. >> what if you use MPI_Send instead () ? >> > Works as expected. > > MPI 1: Recv of 0 started at 08:37:10. > MPI 1: Recv of 0 finished at 08:37:10. > MPI 0: Send of 0 started at 08:37:10. > MPI 0: Send of 0 finished at 08:37:10. > > >> what if you send/Recv a large message first (to "warmup" connections), >> MPI_Barrier, and then start your MPI_Isend ? >> > Not working. For what I want to accomplish, is my code the right way to > go? Is there an altenative method? Thanks. > > MPI 1: Recv of 0 started at 08:38:46. > MPI 0: Isend of 0 started at 08:38:46. > MPI 0: Isend of 1 started at 08:38:46. > MPI 0: Isend of 2 started at 08:38:46. > MPI 0: Isend of 3 started at 08:38:46. > MPI 0: Isend of 4 started at 08:38:46. > MPI 0: MPI_Test of 0 at 08:38:46. > MPI 0: MPI_Test of 0 at 08:38:46. > MPI 0: MPI_Test of 0 at 08:38:46. > MPI 0: MPI_Test of 0 at 08:38:46. > MPI 0: MPI_Test of 0 at 08:38:46. > MPI 0: MPI_Test of 0 at 08:38:46. > MPI 0: MPI_Test of 0 at 08:38:46. > MPI 0: MPI_Test of 0 at 08:38:47. > MPI 0: MPI_Test of 0 at 08:38:47. > MPI 0: MPI_Test of 0 at 08:38:47. > MPI 0: MPI_Test of 0 at 08:38:47. > MPI 0: MPI_Test of 0 at 08:38:47. > MPI 0: MPI_Test of 0 at 08:38:47. > MPI 0: MPI_Test of 0 at 08:38:47. > MPI 0: MPI_Test of 0 at 08:38:47. > MPI 0: MPI_Test of 0 at 08:38:47. > MPI 0: MPI_Test of 0 at 08:38:47. > MPI 0: MPI_Test of 0 at 08:38:48. > MPI 0: MPI_Test of 0 at 08:38:48. > MPI 0: MPI_Test of 0 at 08:38:48. > MPI 0: MPI_Test of 0 at 08:38:48. > MPI 0: MPI_Test of 0 at 08:38:48. > MPI 0: MPI_Test of 0 at 08:38:48. > MPI 0: MPI_Test of 0 at 08:38:48. > MPI 0: MPI_Test of 0 at 08:38:48. > MPI 0: MPI_Test of 0 at 08:38:48. > MPI 0: MPI_Test of 0 at 08:38:48. > MPI 0: MPI_Test of 0 at 08:38:49. > MPI 0: MPI_Test of 0 at 08:38:49. > MPI 0: MPI_Test of 0 at 08:38:49. > MPI 0: MPI_Test of 0 at 08:38:49. > MPI 0: MPI_Test of 0 at 08:38:49. > MPI 0: MPI_Test of 0 at 08:38:49. > MPI 0: MPI_Test of 0 at 08:38:49. > MPI 0: MPI_Test of 0 at 08:38:49. > MPI 0: MPI_Test of 0 at 08:38:49. > MPI 0: MPI_Test of 0 at 08:38:49. > MPI 0: MPI_Test of 0 at 08:38:50. > MPI 0: MPI_Test of 0 at 08:38:50. > MPI 0: MPI_Test of 0 at 08:38:50. > MPI 0: MPI_Test of 0 at 08:38:50. > MPI 1: Recv of 0 finished at 08:38:50. > MPI 1: Recv of 1 started at 08:38:50. > MPI 0: MPI_Test of 0 at 08:38:50. > MPI 0: Isend of 0 finished at 08:38:50. > MPI 0: MPI_Test of 1 at 08:38:50. > MPI 0: MPI_Test of 1 at 08:38:50. > MPI 0: MPI_Test of 1 at 08:38:50. > MPI 0: MPI_Test of 1 at 08:38:50. > MPI 0: MPI_Test of 1 at 08:38:50. > MPI 0: MPI_Test of 1 at 08:38:51. > MPI 0: MPI_Test of 1 at 08:38:51. > MPI 0: MPI_Test of 1 at 08:38:51. > MPI 0: MPI_Test of 1 at 08:38:51. > MPI 0: MPI_Test of 1 at 08:38:51. > MPI 0: MPI_Test of 1 at 08:38:51. > MPI 0: MPI_Test of 1 at 08:38:51. > MPI 0: MPI_Test of 1 at 08:38:51. > MPI 0: MPI_Test of 1 at 08:38:51. > MPI 0: MPI_Test of 1 at 08:38:51. > MPI 0: MPI_Test of 1 at 08:38:52. > MPI 0: MPI_Test of 1 at 08:38:52. > MPI 0: MPI_Test of 1 at 08:38:52. > MPI 0: MPI_Test of 1 at 08:38:52. > MPI 0: MPI_Test of 1 at 08:38:52. > MPI 0: MPI_Test of 1 at 08:38:52. > MPI 0: MPI_Test of 1 at 08:38:52. > MPI 0: MPI_Test of 1 at 08:38:52. > MPI 0: MPI_Test of 1 at 08:38:52. > MPI 0: MPI_Test of 1 at 08:38:52. > MPI 0: MPI_Test of 1 at 08:38:53. > MPI 0: MPI_Test of 1 at 08:38:53. > MPI 0: MPI_Test of 1 at 08:38:53. > MPI 0: MPI_Test of 1 at 08:38:53. > MPI 0: MPI_Test of 1 at 08:38:53. > MPI 0: MPI_Test of 1 at 08:38:53. > MPI 0: MPI_Test of 1 at 08:38:53. > MPI 0: MPI_Test of 1 at 08:38:53. > MPI 0: MPI_Test of 1 at 08:38:53. > MPI 0: MPI_Test of 1 at 08:38:53. > MPI 0: MPI_Test of 1 at 08:38:54. > MPI 0: MPI_Test of 1 at 08:38:54. > MPI 0: MPI_Test of 1 at 08:38:54. > MPI 0: MPI_Test of 1 at 08:38:54. > MPI 0: MPI_Test of 1 at 08:38:54. > MPI 1: Recv of 1 finished at 08:38:54. > MPI 1: Recv of 2 started at 08:38:54. > MPI 0: MPI_Test of 1 at 08:38:54. > MPI 0: Isend of 1 finished at 08:38:54. > MPI 0: MPI_Test of 2 at 08:38:54. > MPI 0: MPI_Test of 2 at 08:38:54. > MPI 0: MPI_Test of 2 at 08:38:54. > MPI 0: MPI_Test of 2 at 08:38:55. > MPI 0: MPI_Test of 2 at 08:38:55. > MPI 0: MPI_Test of 2 at 08:38:55. > MPI 0: MPI_Test of 2 at 08:38:55. > MPI 0: MPI_Test of 2 at 08:38:55. > MPI 0: MPI_Test of 2 at 08:38:55. > MPI 0: MPI_Test of 2 at 08:38:55. > MPI 0: MPI_Test of 2 at 08:38:55. > MPI 0: MPI_Test of 2 at 08:38:55. > MPI 0: MPI_Test of 2 at 08:38:55. > MPI 0: MPI_Test of 2 at 08:38:56. > MPI 0: MPI_Test of 2 at 08:38:56. > MPI 0: MPI_Test of 2 at 08:38:56. > MPI 0: MPI_Test of 2 at 08:38:56. > MPI 0: MPI_Test of 2 at 08:38:56. > MPI 0: MPI_Test of 2 at 08:38:56. > MPI 0: MPI_Test of 2 at 08:38:56. > MPI 0: MPI_Test of 2 at 08:38:56. > MPI 0: MPI_Test of 2 at 08:38:56. > MPI 0: MPI_Test of 2 at 08:38:56. > MPI 0: MPI_Test of 2 at 08:38:57. > MPI 0: MPI_Test of 2 at 08:38:57. > MPI 0: MPI_Test of 2 at 08:38:57. > MPI 0: MPI_Test of 2 at 08:38:57. > MPI 0: MPI_Test of 2 at 08:38:57. > MPI 0: MPI_Test of 2 at 08:38:57. > MPI 0: MPI_Test of 2 at 08:38:57. > MPI 0: MPI_Test of 2 at 08:38:57. > MPI 0: MPI_Test of 2 at 08:38:57. > MPI 0: MPI_Test of 2 at 08:38:57. > MPI 0: MPI_Test of 2 at 08:38:58. > MPI 0: MPI_Test of 2 at 08:38:58. > MPI 0: MPI_Test of 2 at 08:38:58. > MPI 0: MPI_Test of 2 at 08:38:58. > MPI 0: MPI_Test of 2 at 08:38:58. > MPI 0: MPI_Test of 2 at 08:38:58. > MPI 0: MPI_Test of 2 at 08:38:58. > MPI 1: Recv of 2 finished at 08:38:58. > MPI 1: Recv of 3 started at 08:38:58. > MPI 0: MPI_Test of 2 at 08:38:58. > MPI 0: Isend of 2 finished at 08:38:58. > MPI 0: MPI_Test of 3 at 08:38:58. > MPI 0: MPI_Test of 3 at 08:38:58. > MPI 0: MPI_Test of 3 at 08:38:59. > MPI 0: MPI_Test of 3 at 08:38:59. > MPI 0: MPI_Test of 3 at 08:38:59. > MPI 0: MPI_Test of 3 at 08:38:59. > MPI 0: MPI_Test of 3 at 08:38:59. > MPI 0: MPI_Test of 3 at 08:38:59. > MPI 0: MPI_Test of 3 at 08:38:59. > MPI 0: MPI_Test of 3 at 08:38:59. > MPI 0: MPI_Test of 3 at 08:38:59. > MPI 0: MPI_Test of 3 at 08:38:59. > MPI 0: MPI_Test of 3 at 08:39:00. > MPI 0: MPI_Test of 3 at 08:39:00. > MPI 0: MPI_Test of 3 at 08:39:00. > MPI 0: MPI_Test of 3 at 08:39:00. > MPI 0: MPI_Test of 3 at 08:39:00. > MPI 0: MPI_Test of 3 at 08:39:00. > MPI 0: MPI_Test of 3 at 08:39:00. > MPI 0: MPI_Test of 3 at 08:39:00. > MPI 0: MPI_Test of 3 at 08:39:00. > MPI 0: MPI_Test of 3 at 08:39:00. > MPI 0: MPI_Test of 3 at 08:39:01. > MPI 0: MPI_Test of 3 at 08:39:01. > MPI 0: MPI_Test of 3 at 08:39:01. > MPI 0: MPI_Test of 3 at 08:39:01. > MPI 0: MPI_Test of 3 at 08:39:01. > MPI 0: MPI_Test of 3 at 08:39:01. > MPI 0: MPI_Test of 3 at 08:39:01. > MPI 0: MPI_Test of 3 at 08:39:01. > MPI 0: MPI_Test of 3 at 08:39:01. > MPI 0: MPI_Test of 3 at 08:39:01. > MPI 0: MPI_Test of 3 at 08:39:02. > MPI 0: MPI_Test of 3 at 08:39:02. > MPI 0: MPI_Test of 3 at 08:39:02. > MPI 0: MPI_Test of 3 at 08:39:02. > MPI 0: MPI_Test of 3 at 08:39:02. > MPI 0: MPI_Test of 3 at 08:39:02. > MPI 0: MPI_Test of 3 at 08:39:02. > MPI 0: MPI_Test of 3 at 08:39:02. > MPI 1: Recv of 3 finished at 08:39:02. > MPI 1: Recv of 4 started at 08:39:02. > MPI 0: MPI_Test of 3 at 08:39:02. > MPI 0: Isend of 3 finished at 08:39:02. > MPI 0: MPI_Test of 4 at 08:39:02. > MPI 0: MPI_Test of 4 at 08:39:03. > MPI 0: MPI_Test of 4 at 08:39:03. > MPI 0: MPI_Test of 4 at 08:39:03. > MPI 0: MPI_Test of 4 at 08:39:03. > MPI 0: MPI_Test of 4 at 08:39:03. > MPI 0: MPI_Test of 4 at 08:39:03. > MPI 0: MPI_Test of 4 at 08:39:03. > MPI 0: MPI_Test of 4 at 08:39:03. > MPI 0: MPI_Test of 4 at 08:39:03. > MPI 0: MPI_Test of 4 at 08:39:03. > MPI 0: MPI_Test of 4 at 08:39:04. > MPI 0: MPI_Test of 4 at 08:39:04. > MPI 0: MPI_Test of 4 at 08:39:04. > MPI 0: MPI_Test of 4 at 08:39:04. > MPI 0: MPI_Test of 4 at 08:39:04. > MPI 0: MPI_Test of 4 at 08:39:04. > MPI 0: MPI_Test of 4 at 08:39:04. > MPI 0: MPI_Test of 4 at 08:39:04. > MPI 0: MPI_Test of 4 at 08:39:04. > MPI 0: MPI_Test of 4 at 08:39:04. > MPI 0: MPI_Test of 4 at 08:39:05. > MPI 0: MPI_Test of 4 at 08:39:05. > MPI 0: MPI_Test of 4 at 08:39:05. > MPI 0: MPI_Test of 4 at 08:39:05. > MPI 0: MPI_Test of 4 at 08:39:05. > MPI 0: MPI_Test of 4 at 08:39:05. > MPI 0: MPI_Test of 4 at 08:39:05. > MPI 0: MPI_Test of 4 at 08:39:05. > MPI 0: MPI_Test of 4 at 08:39:05. > MPI 0: MPI_Test of 4 at 08:39:05. > MPI 0: MPI_Test of 4 at 08:39:06. > MPI 0: MPI_Test of 4 at 08:39:06. > MPI 0: MPI_Test of 4 at 08:39:06. > MPI 0: MPI_Test of 4 at 08:39:06. > MPI 0: MPI_Test of 4 at 08:39:06. > MPI 0: MPI_Test of 4 at 08:39:06. > MPI 0: MPI_Test of 4 at 08:39:06. > MPI 0: MPI_Test of 4 at 08:39:06. > MPI 0: MPI_Test of 4 at 08:39:06. > MPI 1: Recv of 4 finished at 08:39:06. > MPI 0: MPI_Test of 4 at 08:39:06. > MPI 0: Isend of 4 finished at 08:39:06. > > >> >> Cheers, >> >> Gilles >> >> >> On Thursday, May 5, 2016, Zhen Wang <tod...@gmail.com >> <javascript:_e(%7B%7D,'cvml','tod...@gmail.com');>> wrote: >> >>> Hi, >>> >>> I'm having a problem with Isend, Recv and Test in Linux Mint 16 Petra. >>> The source is attached. >>> >>> Open MPI 1.10.2 is configured with >>> ./configure --enable-debug --prefix=/home/<me>/Tool/openmpi-1.10.2-debug >>> >>> The source is built with >>> ~/Tool/openmpi-1.10.2-debug/bin/mpiCC a5.cpp >>> >>> and run in one node with >>> ~/Tool/openmpi-1.10.2-debug/bin/mpirun -n 2 ./a.out >>> >>> The output is in the end. What puzzles me is why MPI_Test is called so >>> many times, and it takes so long to send a message. Am I doing something >>> wrong? I'm simulating a more complicated program: MPI 0 Isends data to MPI >>> 1, computes (usleep here), and calls Test to check if data are sent. MPI 1 >>> Recvs data, and computes. >>> >>> Thanks in advance. >>> >>> >>> Best regards, >>> Zhen >>> >>> MPI 0: Isend of 0 started at 20:32:35. >>> MPI 1: Recv of 0 started at 20:32:35. >>> MPI 0: MPI_Test of 0 at 20:32:35. >>> MPI 0: MPI_Test of 0 at 20:32:35. >>> MPI 0: MPI_Test of 0 at 20:32:35. >>> MPI 0: MPI_Test of 0 at 20:32:35. >>> MPI 0: MPI_Test of 0 at 20:32:35. >>> MPI 0: MPI_Test of 0 at 20:32:35. >>> MPI 0: MPI_Test of 0 at 20:32:36. >>> MPI 0: MPI_Test of 0 at 20:32:36. >>> MPI 0: MPI_Test of 0 at 20:32:36. >>> MPI 0: MPI_Test of 0 at 20:32:36. >>> MPI 0: MPI_Test of 0 at 20:32:36. >>> MPI 0: MPI_Test of 0 at 20:32:36. >>> MPI 0: MPI_Test of 0 at 20:32:36. >>> MPI 0: MPI_Test of 0 at 20:32:36. >>> MPI 0: MPI_Test of 0 at 20:32:36. >>> MPI 0: MPI_Test of 0 at 20:32:37. >>> MPI 0: MPI_Test of 0 at 20:32:37. >>> MPI 0: MPI_Test of 0 at 20:32:37. >>> MPI 0: MPI_Test of 0 at 20:32:37. >>> MPI 0: MPI_Test of 0 at 20:32:37. >>> MPI 0: MPI_Test of 0 at 20:32:37. >>> MPI 0: MPI_Test of 0 at 20:32:37. >>> MPI 0: MPI_Test of 0 at 20:32:37. >>> MPI 0: MPI_Test of 0 at 20:32:37. >>> MPI 0: MPI_Test of 0 at 20:32:37. >>> MPI 0: MPI_Test of 0 at 20:32:38. >>> MPI 0: MPI_Test of 0 at 20:32:38. >>> MPI 0: MPI_Test of 0 at 20:32:38. >>> MPI 0: MPI_Test of 0 at 20:32:38. >>> MPI 0: MPI_Test of 0 at 20:32:38. >>> MPI 0: MPI_Test of 0 at 20:32:38. >>> MPI 0: MPI_Test of 0 at 20:32:38. >>> MPI 0: MPI_Test of 0 at 20:32:38. >>> MPI 0: MPI_Test of 0 at 20:32:38. >>> MPI 0: MPI_Test of 0 at 20:32:38. >>> MPI 0: MPI_Test of 0 at 20:32:39. >>> MPI 0: MPI_Test of 0 at 20:32:39. >>> MPI 0: MPI_Test of 0 at 20:32:39. >>> MPI 0: MPI_Test of 0 at 20:32:39. >>> MPI 0: MPI_Test of 0 at 20:32:39. >>> MPI 0: MPI_Test of 0 at 20:32:39. >>> MPI 1: Recv of 0 finished at 20:32:39. >>> MPI 0: MPI_Test of 0 at 20:32:39. >>> MPI 0: Isend of 0 finished at 20:32:39. >>> >>> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org <javascript:_e(%7B%7D,'cvml','us...@open-mpi.org');> >> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2016/05/29086.php >> > >
