per the error message, you likely misspeled vader (e.g. missed the "r")
Jeff, the behavior was initially reported on a single node, so the tcp btl is unlikely used Cheers, Gilles On Friday, May 6, 2016, Zhen Wang <tod...@gmail.com> wrote: > > > 2016-05-05 9:27 GMT-05:00 Gilles Gouaillardet < > gilles.gouaillar...@gmail.com > <javascript:_e(%7B%7D,'cvml','gilles.gouaillar...@gmail.com');>>: > >> Out of curiosity, can you try >> mpirun --mca btl self,sm ... >> > Same as before. Many MPI_Test calls. > >> and >> mpirun --mca btl self,vader ... >> > A requested component was not found, or was unable to be opened. This > means that this component is either not installed or is unable to be > used on your system (e.g., sometimes this means that shared libraries > that the component requires are unable to be found/loaded). Note that > Open MPI stopped checking at the first component that it did not find. > > Host: VirtualBox > Framework: btl > Component: vade > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > mca_bml_base_open() failed > --> Returned "Not found" (-13) instead of "Success" (0) > -------------------------------------------------------------------------- > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [VirtualBox:2188] Local abort before MPI_INIT completed successfully; not > able to aggregate error messages, and not able to guarantee that all other > processes were killed! > ------------------------------------------------------- > Primary job terminated normally, but 1 process returned > a non-zero exit code.. Per user-direction, the job has been aborted. > ------------------------------------------------------- > -------------------------------------------------------------------------- > mpirun detected that one or more processes exited with non-zero status, > thus causing > the job to be terminated. The first process to do so was: > > Process name: [[9235,1],0] > Exit code: 1 > -------------------------------------------------------------------------- > [VirtualBox:02186] 1 more process has sent help message help-mca-base.txt > / find-available:not-valid > [VirtualBox:02186] Set MCA parameter "orte_base_help_aggregate" to 0 to > see all help / error messages > > >> >> and see if one performs better than the other ? >> >> Cheers, >> >> Gilles >> >> On Thursday, May 5, 2016, Zhen Wang <tod...@gmail.com >> <javascript:_e(%7B%7D,'cvml','tod...@gmail.com');>> wrote: >> >>> Gilles, >>> >>> Thanks for your reply. >>> >>> Best regards, >>> Zhen >>> >>> On Wed, May 4, 2016 at 8:43 PM, Gilles Gouaillardet < >>> gilles.gouaillar...@gmail.com> wrote: >>> >>>> Note there is no progress thread in openmpi 1.10 >>>> from a pragmatic point of view, that means that for "large" messages, >>>> no data is sent in MPI_Isend, and the data is sent when MPI "progresses" >>>> e.g. call a MPI_Test, MPI_Probe, MPI_Recv or some similar subroutine. >>>> in your example, the data is transferred after the first usleep >>>> completes. >>>> >>> I agree. >>> >>>> >>>> that being said, it takes quite a while, and there could be an issue. >>>> what if you use MPI_Send instead () ? >>>> >>> Works as expected. >>> >>> MPI 1: Recv of 0 started at 08:37:10. >>> MPI 1: Recv of 0 finished at 08:37:10. >>> MPI 0: Send of 0 started at 08:37:10. >>> MPI 0: Send of 0 finished at 08:37:10. >>> >>> >>>> what if you send/Recv a large message first (to "warmup" connections), >>>> MPI_Barrier, and then start your MPI_Isend ? >>>> >>> Not working. For what I want to accomplish, is my code the right way to >>> go? Is there an altenative method? Thanks. >>> >>> MPI 1: Recv of 0 started at 08:38:46. >>> MPI 0: Isend of 0 started at 08:38:46. >>> MPI 0: Isend of 1 started at 08:38:46. >>> MPI 0: Isend of 2 started at 08:38:46. >>> MPI 0: Isend of 3 started at 08:38:46. >>> MPI 0: Isend of 4 started at 08:38:46. >>> MPI 0: MPI_Test of 0 at 08:38:46. >>> MPI 0: MPI_Test of 0 at 08:38:46. >>> MPI 0: MPI_Test of 0 at 08:38:46. >>> MPI 0: MPI_Test of 0 at 08:38:46. >>> MPI 0: MPI_Test of 0 at 08:38:46. >>> MPI 0: MPI_Test of 0 at 08:38:46. >>> MPI 0: MPI_Test of 0 at 08:38:46. >>> MPI 0: MPI_Test of 0 at 08:38:47. >>> MPI 0: MPI_Test of 0 at 08:38:47. >>> MPI 0: MPI_Test of 0 at 08:38:47. >>> MPI 0: MPI_Test of 0 at 08:38:47. >>> MPI 0: MPI_Test of 0 at 08:38:47. >>> MPI 0: MPI_Test of 0 at 08:38:47. >>> MPI 0: MPI_Test of 0 at 08:38:47. >>> MPI 0: MPI_Test of 0 at 08:38:47. >>> MPI 0: MPI_Test of 0 at 08:38:47. >>> MPI 0: MPI_Test of 0 at 08:38:47. >>> MPI 0: MPI_Test of 0 at 08:38:48. >>> MPI 0: MPI_Test of 0 at 08:38:48. >>> MPI 0: MPI_Test of 0 at 08:38:48. >>> MPI 0: MPI_Test of 0 at 08:38:48. >>> MPI 0: MPI_Test of 0 at 08:38:48. >>> MPI 0: MPI_Test of 0 at 08:38:48. >>> MPI 0: MPI_Test of 0 at 08:38:48. >>> MPI 0: MPI_Test of 0 at 08:38:48. >>> MPI 0: MPI_Test of 0 at 08:38:48. >>> MPI 0: MPI_Test of 0 at 08:38:48. >>> MPI 0: MPI_Test of 0 at 08:38:49. >>> MPI 0: MPI_Test of 0 at 08:38:49. >>> MPI 0: MPI_Test of 0 at 08:38:49. >>> MPI 0: MPI_Test of 0 at 08:38:49. >>> MPI 0: MPI_Test of 0 at 08:38:49. >>> MPI 0: MPI_Test of 0 at 08:38:49. >>> MPI 0: MPI_Test of 0 at 08:38:49. >>> MPI 0: MPI_Test of 0 at 08:38:49. >>> MPI 0: MPI_Test of 0 at 08:38:49. >>> MPI 0: MPI_Test of 0 at 08:38:49. >>> MPI 0: MPI_Test of 0 at 08:38:50. >>> MPI 0: MPI_Test of 0 at 08:38:50. >>> MPI 0: MPI_Test of 0 at 08:38:50. >>> MPI 0: MPI_Test of 0 at 08:38:50. >>> MPI 1: Recv of 0 finished at 08:38:50. >>> MPI 1: Recv of 1 started at 08:38:50. >>> MPI 0: MPI_Test of 0 at 08:38:50. >>> MPI 0: Isend of 0 finished at 08:38:50. >>> MPI 0: MPI_Test of 1 at 08:38:50. >>> MPI 0: MPI_Test of 1 at 08:38:50. >>> MPI 0: MPI_Test of 1 at 08:38:50. >>> MPI 0: MPI_Test of 1 at 08:38:50. >>> MPI 0: MPI_Test of 1 at 08:38:50. >>> MPI 0: MPI_Test of 1 at 08:38:51. >>> MPI 0: MPI_Test of 1 at 08:38:51. >>> MPI 0: MPI_Test of 1 at 08:38:51. >>> MPI 0: MPI_Test of 1 at 08:38:51. >>> MPI 0: MPI_Test of 1 at 08:38:51. >>> MPI 0: MPI_Test of 1 at 08:38:51. >>> MPI 0: MPI_Test of 1 at 08:38:51. >>> MPI 0: MPI_Test of 1 at 08:38:51. >>> MPI 0: MPI_Test of 1 at 08:38:51. >>> MPI 0: MPI_Test of 1 at 08:38:51. >>> MPI 0: MPI_Test of 1 at 08:38:52. >>> MPI 0: MPI_Test of 1 at 08:38:52. >>> MPI 0: MPI_Test of 1 at 08:38:52. >>> MPI 0: MPI_Test of 1 at 08:38:52. >>> MPI 0: MPI_Test of 1 at 08:38:52. >>> MPI 0: MPI_Test of 1 at 08:38:52. >>> MPI 0: MPI_Test of 1 at 08:38:52. >>> MPI 0: MPI_Test of 1 at 08:38:52. >>> MPI 0: MPI_Test of 1 at 08:38:52. >>> MPI 0: MPI_Test of 1 at 08:38:52. >>> MPI 0: MPI_Test of 1 at 08:38:53. >>> MPI 0: MPI_Test of 1 at 08:38:53. >>> MPI 0: MPI_Test of 1 at 08:38:53. >>> MPI 0: MPI_Test of 1 at 08:38:53. >>> MPI 0: MPI_Test of 1 at 08:38:53. >>> MPI 0: MPI_Test of 1 at 08:38:53. >>> MPI 0: MPI_Test of 1 at 08:38:53. >>> MPI 0: MPI_Test of 1 at 08:38:53. >>> MPI 0: MPI_Test of 1 at 08:38:53. >>> MPI 0: MPI_Test of 1 at 08:38:53. >>> MPI 0: MPI_Test of 1 at 08:38:54. >>> MPI 0: MPI_Test of 1 at 08:38:54. >>> MPI 0: MPI_Test of 1 at 08:38:54. >>> MPI 0: MPI_Test of 1 at 08:38:54. >>> MPI 0: MPI_Test of 1 at 08:38:54. >>> MPI 1: Recv of 1 finished at 08:38:54. >>> MPI 1: Recv of 2 started at 08:38:54. >>> MPI 0: MPI_Test of 1 at 08:38:54. >>> MPI 0: Isend of 1 finished at 08:38:54. >>> MPI 0: MPI_Test of 2 at 08:38:54. >>> MPI 0: MPI_Test of 2 at 08:38:54. >>> MPI 0: MPI_Test of 2 at 08:38:54. >>> MPI 0: MPI_Test of 2 at 08:38:55. >>> MPI 0: MPI_Test of 2 at 08:38:55. >>> MPI 0: MPI_Test of 2 at 08:38:55. >>> MPI 0: MPI_Test of 2 at 08:38:55. >>> MPI 0: MPI_Test of 2 at 08:38:55. >>> MPI 0: MPI_Test of 2 at 08:38:55. >>> MPI 0: MPI_Test of 2 at 08:38:55. >>> MPI 0: MPI_Test of 2 at 08:38:55. >>> MPI 0: MPI_Test of 2 at 08:38:55. >>> MPI 0: MPI_Test of 2 at 08:38:55. >>> MPI 0: MPI_Test of 2 at 08:38:56. >>> MPI 0: MPI_Test of 2 at 08:38:56. >>> MPI 0: MPI_Test of 2 at 08:38:56. >>> MPI 0: MPI_Test of 2 at 08:38:56. >>> MPI 0: MPI_Test of 2 at 08:38:56. >>> MPI 0: MPI_Test of 2 at 08:38:56. >>> MPI 0: MPI_Test of 2 at 08:38:56. >>> MPI 0: MPI_Test of 2 at 08:38:56. >>> MPI 0: MPI_Test of 2 at 08:38:56. >>> MPI 0: MPI_Test of 2 at 08:38:56. >>> MPI 0: MPI_Test of 2 at 08:38:57. >>> MPI 0: MPI_Test of 2 at 08:38:57. >>> MPI 0: MPI_Test of 2 at 08:38:57. >>> MPI 0: MPI_Test of 2 at 08:38:57. >>> MPI 0: MPI_Test of 2 at 08:38:57. >>> MPI 0: MPI_Test of 2 at 08:38:57. >>> MPI 0: MPI_Test of 2 at 08:38:57. >>> MPI 0: MPI_Test of 2 at 08:38:57. >>> MPI 0: MPI_Test of 2 at 08:38:57. >>> MPI 0: MPI_Test of 2 at 08:38:57. >>> MPI 0: MPI_Test of 2 at 08:38:58. >>> MPI 0: MPI_Test of 2 at 08:38:58. >>> MPI 0: MPI_Test of 2 at 08:38:58. >>> MPI 0: MPI_Test of 2 at 08:38:58. >>> MPI 0: MPI_Test of 2 at 08:38:58. >>> MPI 0: MPI_Test of 2 at 08:38:58. >>> MPI 0: MPI_Test of 2 at 08:38:58. >>> MPI 1: Recv of 2 finished at 08:38:58. >>> MPI 1: Recv of 3 started at 08:38:58. >>> MPI 0: MPI_Test of 2 at 08:38:58. >>> MPI 0: Isend of 2 finished at 08:38:58. >>> MPI 0: MPI_Test of 3 at 08:38:58. >>> MPI 0: MPI_Test of 3 at 08:38:58. >>> MPI 0: MPI_Test of 3 at 08:38:59. >>> MPI 0: MPI_Test of 3 at 08:38:59. >>> MPI 0: MPI_Test of 3 at 08:38:59. >>> MPI 0: MPI_Test of 3 at 08:38:59. >>> MPI 0: MPI_Test of 3 at 08:38:59. >>> MPI 0: MPI_Test of 3 at 08:38:59. >>> MPI 0: MPI_Test of 3 at 08:38:59. >>> MPI 0: MPI_Test of 3 at 08:38:59. >>> MPI 0: MPI_Test of 3 at 08:38:59. >>> MPI 0: MPI_Test of 3 at 08:38:59. >>> MPI 0: MPI_Test of 3 at 08:39:00. >>> MPI 0: MPI_Test of 3 at 08:39:00. >>> MPI 0: MPI_Test of 3 at 08:39:00. >>> MPI 0: MPI_Test of 3 at 08:39:00. >>> MPI 0: MPI_Test of 3 at 08:39:00. >>> MPI 0: MPI_Test of 3 at 08:39:00. >>> MPI 0: MPI_Test of 3 at 08:39:00. >>> MPI 0: MPI_Test of 3 at 08:39:00. >>> MPI 0: MPI_Test of 3 at 08:39:00. >>> MPI 0: MPI_Test of 3 at 08:39:00. >>> MPI 0: MPI_Test of 3 at 08:39:01. >>> MPI 0: MPI_Test of 3 at 08:39:01. >>> MPI 0: MPI_Test of 3 at 08:39:01. >>> MPI 0: MPI_Test of 3 at 08:39:01. >>> MPI 0: MPI_Test of 3 at 08:39:01. >>> MPI 0: MPI_Test of 3 at 08:39:01. >>> MPI 0: MPI_Test of 3 at 08:39:01. >>> MPI 0: MPI_Test of 3 at 08:39:01. >>> MPI 0: MPI_Test of 3 at 08:39:01. >>> MPI 0: MPI_Test of 3 at 08:39:01. >>> MPI 0: MPI_Test of 3 at 08:39:02. >>> MPI 0: MPI_Test of 3 at 08:39:02. >>> MPI 0: MPI_Test of 3 at 08:39:02. >>> MPI 0: MPI_Test of 3 at 08:39:02. >>> MPI 0: MPI_Test of 3 at 08:39:02. >>> MPI 0: MPI_Test of 3 at 08:39:02. >>> MPI 0: MPI_Test of 3 at 08:39:02. >>> MPI 0: MPI_Test of 3 at 08:39:02. >>> MPI 1: Recv of 3 finished at 08:39:02. >>> MPI 1: Recv of 4 started at 08:39:02. >>> MPI 0: MPI_Test of 3 at 08:39:02. >>> MPI 0: Isend of 3 finished at 08:39:02. >>> MPI 0: MPI_Test of 4 at 08:39:02. >>> MPI 0: MPI_Test of 4 at 08:39:03. >>> MPI 0: MPI_Test of 4 at 08:39:03. >>> MPI 0: MPI_Test of 4 at 08:39:03. >>> MPI 0: MPI_Test of 4 at 08:39:03. >>> MPI 0: MPI_Test of 4 at 08:39:03. >>> MPI 0: MPI_Test of 4 at 08:39:03. >>> MPI 0: MPI_Test of 4 at 08:39:03. >>> MPI 0: MPI_Test of 4 at 08:39:03. >>> MPI 0: MPI_Test of 4 at 08:39:03. >>> MPI 0: MPI_Test of 4 at 08:39:03. >>> MPI 0: MPI_Test of 4 at 08:39:04. >>> MPI 0: MPI_Test of 4 at 08:39:04. >>> MPI 0: MPI_Test of 4 at 08:39:04. >>> MPI 0: MPI_Test of 4 at 08:39:04. >>> MPI 0: MPI_Test of 4 at 08:39:04. >>> MPI 0: MPI_Test of 4 at 08:39:04. >>> MPI 0: MPI_Test of 4 at 08:39:04. >>> MPI 0: MPI_Test of 4 at 08:39:04. >>> MPI 0: MPI_Test of 4 at 08:39:04. >>> MPI 0: MPI_Test of 4 at 08:39:04. >>> MPI 0: MPI_Test of 4 at 08:39:05. >>> MPI 0: MPI_Test of 4 at 08:39:05. >>> MPI 0: MPI_Test of 4 at 08:39:05. >>> MPI 0: MPI_Test of 4 at 08:39:05. >>> MPI 0: MPI_Test of 4 at 08:39:05. >>> MPI 0: MPI_Test of 4 at 08:39:05. >>> MPI 0: MPI_Test of 4 at 08:39:05. >>> MPI 0: MPI_Test of 4 at 08:39:05. >>> MPI 0: MPI_Test of 4 at 08:39:05. >>> MPI 0: MPI_Test of 4 at 08:39:05. >>> MPI 0: MPI_Test of 4 at 08:39:06. >>> MPI 0: MPI_Test of 4 at 08:39:06. >>> MPI 0: MPI_Test of 4 at 08:39:06. >>> MPI 0: MPI_Test of 4 at 08:39:06. >>> MPI 0: MPI_Test of 4 at 08:39:06. >>> MPI 0: MPI_Test of 4 at 08:39:06. >>> MPI 0: MPI_Test of 4 at 08:39:06. >>> MPI 0: MPI_Test of 4 at 08:39:06. >>> MPI 0: MPI_Test of 4 at 08:39:06. >>> MPI 1: Recv of 4 finished at 08:39:06. >>> MPI 0: MPI_Test of 4 at 08:39:06. >>> MPI 0: Isend of 4 finished at 08:39:06. >>> >>> >>>> >>>> Cheers, >>>> >>>> Gilles >>>> >>>> >>>> On Thursday, May 5, 2016, Zhen Wang <tod...@gmail.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> I'm having a problem with Isend, Recv and Test in Linux Mint 16 Petra. >>>>> The source is attached. >>>>> >>>>> Open MPI 1.10.2 is configured with >>>>> ./configure --enable-debug >>>>> --prefix=/home/<me>/Tool/openmpi-1.10.2-debug >>>>> >>>>> The source is built with >>>>> ~/Tool/openmpi-1.10.2-debug/bin/mpiCC a5.cpp >>>>> >>>>> and run in one node with >>>>> ~/Tool/openmpi-1.10.2-debug/bin/mpirun -n 2 ./a.out >>>>> >>>>> The output is in the end. What puzzles me is why MPI_Test is called so >>>>> many times, and it takes so long to send a message. Am I doing something >>>>> wrong? I'm simulating a more complicated program: MPI 0 Isends data to MPI >>>>> 1, computes (usleep here), and calls Test to check if data are sent. MPI 1 >>>>> Recvs data, and computes. >>>>> >>>>> Thanks in advance. >>>>> >>>>> >>>>> Best regards, >>>>> Zhen >>>>> >>>>> MPI 0: Isend of 0 started at 20:32:35. >>>>> MPI 1: Recv of 0 started at 20:32:35. >>>>> MPI 0: MPI_Test of 0 at 20:32:35. >>>>> MPI 0: MPI_Test of 0 at 20:32:35. >>>>> MPI 0: MPI_Test of 0 at 20:32:35. >>>>> MPI 0: MPI_Test of 0 at 20:32:35. >>>>> MPI 0: MPI_Test of 0 at 20:32:35. >>>>> MPI 0: MPI_Test of 0 at 20:32:35. >>>>> MPI 0: MPI_Test of 0 at 20:32:36. >>>>> MPI 0: MPI_Test of 0 at 20:32:36. >>>>> MPI 0: MPI_Test of 0 at 20:32:36. >>>>> MPI 0: MPI_Test of 0 at 20:32:36. >>>>> MPI 0: MPI_Test of 0 at 20:32:36. >>>>> MPI 0: MPI_Test of 0 at 20:32:36. >>>>> MPI 0: MPI_Test of 0 at 20:32:36. >>>>> MPI 0: MPI_Test of 0 at 20:32:36. >>>>> MPI 0: MPI_Test of 0 at 20:32:36. >>>>> MPI 0: MPI_Test of 0 at 20:32:37. >>>>> MPI 0: MPI_Test of 0 at 20:32:37. >>>>> MPI 0: MPI_Test of 0 at 20:32:37. >>>>> MPI 0: MPI_Test of 0 at 20:32:37. >>>>> MPI 0: MPI_Test of 0 at 20:32:37. >>>>> MPI 0: MPI_Test of 0 at 20:32:37. >>>>> MPI 0: MPI_Test of 0 at 20:32:37. >>>>> MPI 0: MPI_Test of 0 at 20:32:37. >>>>> MPI 0: MPI_Test of 0 at 20:32:37. >>>>> MPI 0: MPI_Test of 0 at 20:32:37. >>>>> MPI 0: MPI_Test of 0 at 20:32:38. >>>>> MPI 0: MPI_Test of 0 at 20:32:38. >>>>> MPI 0: MPI_Test of 0 at 20:32:38. >>>>> MPI 0: MPI_Test of 0 at 20:32:38. >>>>> MPI 0: MPI_Test of 0 at 20:32:38. >>>>> MPI 0: MPI_Test of 0 at 20:32:38. >>>>> MPI 0: MPI_Test of 0 at 20:32:38. >>>>> MPI 0: MPI_Test of 0 at 20:32:38. >>>>> MPI 0: MPI_Test of 0 at 20:32:38. >>>>> MPI 0: MPI_Test of 0 at 20:32:38. >>>>> MPI 0: MPI_Test of 0 at 20:32:39. >>>>> MPI 0: MPI_Test of 0 at 20:32:39. >>>>> MPI 0: MPI_Test of 0 at 20:32:39. >>>>> MPI 0: MPI_Test of 0 at 20:32:39. >>>>> MPI 0: MPI_Test of 0 at 20:32:39. >>>>> MPI 0: MPI_Test of 0 at 20:32:39. >>>>> MPI 1: Recv of 0 finished at 20:32:39. >>>>> MPI 0: MPI_Test of 0 at 20:32:39. >>>>> MPI 0: Isend of 0 finished at 20:32:39. >>>>> >>>>> >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users >>>> Link to this post: >>>> http://www.open-mpi.org/community/lists/users/2016/05/29086.php >>>> >>> >>> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org <javascript:_e(%7B%7D,'cvml','us...@open-mpi.org');> >> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2016/05/29097.php >> > >
