I think something may be messed up with your installation. I went ahead and tested this on a Slurm 2.5.4 cluster, and got the following:
$ time mpirun -np 1 --host bend001 ./hello Hello, World, I am 0 of 1 [0 local peers]: get_cpubind: 0 bitmap 0,12 real 0m0.086s user 0m0.039s sys 0m0.046s $ time mpirun -np 1 --host bend002 ./hello Hello, World, I am 0 of 1 [0 local peers]: get_cpubind: 0 bitmap 0,12 real 0m0.528s user 0m0.021s sys 0m0.023s Which is what I would have expected. With --host set to the local host, no daemons are being launched and so the time is quite short (just spent mapping and fork/exec). With --host set to a single remote host, you have the time it takes Slurm to launch our daemon on the remote host, so you get about half of a second. IIRC, you were having some problems with the OOB setup. If you specify the TCP interface to use, does your time come down? On Aug 26, 2014, at 8:32 AM, Timur Ismagilov <tismagi...@mail.ru> wrote: > I'm using slurm 2.5.6 > > $salloc -N8 --exclusive -J ompi -p test > > $ srun hostname > node1-128-21 > node1-128-24 > node1-128-22 > node1-128-26 > node1-128-27 > node1-128-20 > node1-128-25 > node1-128-23 > > $ time mpirun -np 1 --host node1-128-21 ./hello_c > Hello, world, I am 0 of 1, (Open MPI v1.9a1, package: Open MPI > semenov@compiler-2 Distribution, ident: 1.9a1r32570, repo rev: r32570, Aug > 21, 2014 (nightly snapshot tarball), 146) > > real 1m3.932s > user 0m0.035s > sys 0m0.072s > > > > > Tue, 26 Aug 2014 07:03:58 -0700 от Ralph Castain <r...@open-mpi.org>: > hmmm....what is your allocation like? do you have a large hostfile, for > example? > > if you add a --host argument that contains just the local host, what is the > time for that scenario? > > On Aug 26, 2014, at 6:27 AM, Timur Ismagilov <tismagi...@mail.ru> wrote: > >> Hello! >> Here is my time results: >> >> $time mpirun -n 1 ./hello_c >> Hello, world, I am 0 of 1, (Open MPI v1.9a1, package: Open MPI >> semenov@compiler-2 Distribution, ident: 1.9a1r32570, repo rev: r32570, Aug >> 21, 2014 (nightly snapshot tarball), 146) >> >> real 1m3.985s >> user 0m0.031s >> sys 0m0.083s >> >> >> >> >> Fri, 22 Aug 2014 07:43:03 -0700 от Ralph Castain <r...@open-mpi.org>: >> I'm also puzzled by your timing statement - I can't replicate it: >> >> 07:41:43 $ time mpirun -n 1 ./hello_c >> Hello, world, I am 0 of 1, (Open MPI v1.9a1, package: Open MPI rhc@bend001 >> Distribution, ident: 1.9a1r32577, repo rev: r32577, Unreleased developer >> copy, 125) >> >> real 0m0.547s >> user 0m0.043s >> sys 0m0.046s >> >> The entire thing ran in 0.5 seconds >> >> >> On Aug 22, 2014, at 6:33 AM, Mike Dubman <mi...@dev.mellanox.co.il> wrote: >> >>> Hi, >>> The default delimiter is ";" . You can change delimiter with >>> mca_base_env_list_delimiter. >>> >>> >>> >>> On Fri, Aug 22, 2014 at 2:59 PM, Timur Ismagilov <tismagi...@mail.ru> wrote: >>> Hello! >>> If i use latest night snapshot: >>> $ ompi_info -V >>> Open MPI v1.9a1r32570 >>> >>> In programm hello_c initialization takes ~1 min >>> In ompi 1.8.2rc4 and ealier it takes ~1 sec(or less) >>> if i use >>> $mpirun --mca mca_base_env_list 'MXM_SHM_KCOPY_MODE=off,OMP_NUM_THREADS=8' >>> --map-by slot:pe=8 -np 1 ./hello_c >>> i got error >>> config_parser.c:657 MXM ERROR Invalid value for SHM_KCOPY_MODE: >>> 'off,OMP_NUM_THREADS=8'. Expected: [off|knem|cma|autodetect] >>> but with -x all works fine (but with warn) >>> $mpirun -x MXM_SHM_KCOPY_MODE=off -x OMP_NUM_THREADS=8 -np 1 ./hello_c >>> WARNING: The mechanism by which environment variables are explicitly >>> .............. >>> .............. >>> .............. >>> Hello, world, I am 0 of 1, (Open MPI v1.9a1, package: Open MPI >>> semenov@compiler-2 Distribution, ident: 1.9a1r32570, repo rev: r32570, Aug >>> 21, 2014 (nightly snapshot tarball), 146) >>> >>> >>> Thu, 21 Aug 2014 06:26:13 -0700 от Ralph Castain <r...@open-mpi.org>: >>> Not sure I understand. The problem has been fixed in both the trunk and the >>> 1.8 branch now, so you should be able to work with either of those nightly >>> builds. >>> >>> On Aug 21, 2014, at 12:02 AM, Timur Ismagilov <tismagi...@mail.ru> wrote: >>> >>>> Have i I any opportunity to run mpi jobs? >>>> >>>> >>>> Wed, 20 Aug 2014 10:48:38 -0700 от Ralph Castain <r...@open-mpi.org>: >>>> yes, i know - it is cmr'd >>>> >>>> On Aug 20, 2014, at 10:26 AM, Mike Dubman <mi...@dev.mellanox.co.il> wrote: >>>> >>>>> btw, we get same error in v1.8 branch as well. >>>>> >>>>> >>>>> On Wed, Aug 20, 2014 at 8:06 PM, Ralph Castain <r...@open-mpi.org> wrote: >>>>> It was not yet fixed - but should be now. >>>>> >>>>> On Aug 20, 2014, at 6:39 AM, Timur Ismagilov <tismagi...@mail.ru> wrote: >>>>> >>>>>> Hello! >>>>>> >>>>>> As i can see, the bug is fixed, but in Open MPI v1.9a1r32516 i still >>>>>> have the problem >>>>>> >>>>>> a) >>>>>> $ mpirun -np 1 ./hello_c >>>>>> >>>>>> -------------------------------------------------------------------------- >>>>>> An ORTE daemon has unexpectedly failed after launch and before >>>>>> communicating back to mpirun. This could be caused by a number >>>>>> of factors, including an inability to create a connection back >>>>>> to mpirun due to a lack of common network interfaces and/or no >>>>>> route found between them. Please check network connectivity >>>>>> (including firewalls and network routing requirements). >>>>>> -------------------------------------------------------------------------- >>>>>> >>>>>> b) >>>>>> $ mpirun --mca oob_tcp_if_include ib0 -np 1 ./hello_c >>>>>> -------------------------------------------------------------------------- >>>>>> An ORTE daemon has unexpectedly failed after launch and before >>>>>> communicating back to mpirun. This could be caused by a number >>>>>> of factors, including an inability to create a connection back >>>>>> to mpirun due to a lack of common network interfaces and/or no >>>>>> route found between them. Please check network connectivity >>>>>> (including firewalls and network routing requirements). >>>>>> -------------------------------------------------------------------------- >>>>>> >>>>>> c) >>>>>> >>>>>> $ mpirun --mca oob_tcp_if_include ib0 -debug-daemons --mca >>>>>> plm_base_verbose 5 -mca oob_base_verbose 10 -mca rml_base_verbose 10 -np >>>>>> 1 ./hello_c >>>>>> >>>>>> [compiler-2:14673] mca:base:select:( plm) Querying component [isolated] >>>>>> [compiler-2:14673] mca:base:select:( plm) Query of component [isolated] >>>>>> set priority to 0 >>>>>> [compiler-2:14673] mca:base:select:( plm) Querying component [rsh] >>>>>> [compiler-2:14673] mca:base:select:( plm) Query of component [rsh] set >>>>>> priority to 10 >>>>>> [compiler-2:14673] mca:base:select:( plm) Querying component [slurm] >>>>>> [compiler-2:14673] mca:base:select:( plm) Query of component [slurm] set >>>>>> priority to 75 >>>>>> [compiler-2:14673] mca:base:select:( plm) Selected component [slurm] >>>>>> [compiler-2:14673] mca: base: components_register: registering oob >>>>>> components >>>>>> [compiler-2:14673] mca: base: components_register: found loaded >>>>>> component tcp >>>>>> [compiler-2:14673] mca: base: components_register: component tcp >>>>>> register function successful >>>>>> [compiler-2:14673] mca: base: components_open: opening oob components >>>>>> [compiler-2:14673] mca: base: components_open: found loaded component tcp >>>>>> [compiler-2:14673] mca: base: components_open: component tcp open >>>>>> function successful >>>>>> [compiler-2:14673] mca:oob:select: checking available component tcp >>>>>> [compiler-2:14673] mca:oob:select: Querying component [tcp] >>>>>> [compiler-2:14673] oob:tcp: component_available called >>>>>> [compiler-2:14673] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4 >>>>>> [compiler-2:14673] WORKING INTERFACE 2 KERNEL INDEX 3 FAMILY: V4 >>>>>> [compiler-2:14673] WORKING INTERFACE 3 KERNEL INDEX 4 FAMILY: V4 >>>>>> [compiler-2:14673] WORKING INTERFACE 4 KERNEL INDEX 5 FAMILY: V4 >>>>>> [compiler-2:14673] WORKING INTERFACE 5 KERNEL INDEX 6 FAMILY: V4 >>>>>> [compiler-2:14673] [[49095,0],0] oob:tcp:init adding 10.128.0.4 to our >>>>>> list of V4 connections >>>>>> [compiler-2:14673] WORKING INTERFACE 6 KERNEL INDEX 7 FAMILY: V4 >>>>>> [compiler-2:14673] [[49095,0],0] TCP STARTUP >>>>>> [compiler-2:14673] [[49095,0],0] attempting to bind to IPv4 port 0 >>>>>> [compiler-2:14673] [[49095,0],0] assigned IPv4 port 59460 >>>>>> [compiler-2:14673] mca:oob:select: Adding component to end >>>>>> [compiler-2:14673] mca:oob:select: Found 1 active transports >>>>>> [compiler-2:14673] mca: base: components_register: registering rml >>>>>> components >>>>>> [compiler-2:14673] mca: base: components_register: found loaded >>>>>> component oob >>>>>> [compiler-2:14673] mca: base: components_register: component oob has no >>>>>> register or open function >>>>>> [compiler-2:14673] mca: base: components_open: opening rml components >>>>>> [compiler-2:14673] mca: base: components_open: found loaded component oob >>>>>> [compiler-2:14673] mca: base: components_open: component oob open >>>>>> function successful >>>>>> [compiler-2:14673] orte_rml_base_select: initializing rml component oob >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 30 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 15 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 32 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 33 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 5 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 10 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 12 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 9 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 34 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 2 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 21 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 22 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 45 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 46 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 1 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> [compiler-2:14673] [[49095,0],0] posting recv >>>>>> [compiler-2:14673] [[49095,0],0] posting persistent recv on tag 27 for >>>>>> peer [[WILDCARD],WILDCARD] >>>>>> Daemon was launched on node1-128-01 - beginning to initialize >>>>>> -------------------------------------------------------------------------- >>>>>> WARNING: An invalid value was given for oob_tcp_if_include. This >>>>>> value will be ignored. >>>>>> >>>>>> Local host: node1-128-01 >>>>>> Value: "ib0" >>>>>> Message: Invalid specification (missing "/") >>>>>> -------------------------------------------------------------------------- >>>>>> -------------------------------------------------------------------------- >>>>>> None of the TCP networks specified to be included for out-of-band >>>>>> communications >>>>>> could be found: >>>>>> >>>>>> Value given: >>>>>> >>>>>> Please revise the specification and try again. >>>>>> -------------------------------------------------------------------------- >>>>>> -------------------------------------------------------------------------- >>>>>> No network interfaces were found for out-of-band communications. We >>>>>> require >>>>>> at least one available network for out-of-band messaging. >>>>>> -------------------------------------------------------------------------- >>>>>> -------------------------------------------------------------------------- >>>>>> It looks like orte_init failed for some reason; your parallel process is >>>>>> likely to abort. There are many reasons that a parallel process can >>>>>> fail during orte_init; some of which are due to configuration or >>>>>> environment problems. This failure appears to be an internal failure; >>>>>> here's some additional information (which may only be relevant to an >>>>>> Open MPI developer): >>>>>> >>>>>> orte_oob_base_select failed >>>>>> --> Returned value (null) (-43) instead of ORTE_SUCCESS >>>>>> -------------------------------------------------------------------------- >>>>>> srun: error: node1-128-01: task 0: Exited with exit code 213 >>>>>> srun: Terminating job step 661215.0 >>>>>> -------------------------------------------------------------------------- >>>>>> An ORTE daemon has unexpectedly failed after launch and before >>>>>> communicating back to mpirun. This could be caused by a number >>>>>> of factors, including an inability to create a connection back >>>>>> to mpirun due to a lack of common network interfaces and/or no >>>>>> route found between them. Please check network connectivity >>>>>> (including firewalls and network routing requirements). >>>>>> -------------------------------------------------------------------------- >>>>>> [compiler-2:14673] [[49095,0],0] orted_cmd: received halt_vm cmd >>>>>> [compiler-2:14673] mca: base: close: component oob closed >>>>>> [compiler-2:14673] mca: base: close: unloading component oob >>>>>> [compiler-2:14673] [[49095,0],0] TCP SHUTDOWN >>>>>> [compiler-2:14673] mca: base: close: component tcp closed >>>>>> [compiler-2:14673] mca: base: close: unloading component tcp >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Tue, 12 Aug 2014 18:33:24 +0000 от "Jeff Squyres (jsquyres)" >>>>>> <jsquy...@cisco.com>: >>>>>> I filed the following ticket: >>>>>> >>>>>> https://svn.open-mpi.org/trac/ompi/ticket/4857 >>>>>> >>>>>> >>>>>> On Aug 12, 2014, at 12:39 PM, Jeff Squyres (jsquyres) >>>>>> <jsquy...@cisco.com> wrote: >>>>>> >>>>>> > (please keep the users list CC'ed) >>>>>> > >>>>>> > We talked about this on the weekly engineering call today. Ralph has >>>>>> > an idea what is happening -- I need to do a little investigation today >>>>>> > and file a bug. I'll make sure you're CC'ed on the bug ticket. >>>>>> > >>>>>> > >>>>>> > >>>>>> > On Aug 12, 2014, at 12:27 PM, Timur Ismagilov <tismagi...@mail.ru> >>>>>> > wrote: >>>>>> > >>>>>> >> I don't have this error in OMPI 1.9a1r32252 and OMPI 1.8.1 (with >>>>>> >> --mca oob_tcp_if_include ib0), but in all latest night snapshots i >>>>>> >> got this error. >>>>>> >> >>>>>> >> >>>>>> >> Tue, 12 Aug 2014 13:08:12 +0000 от "Jeff Squyres (jsquyres)" >>>>>> >> <jsquy...@cisco.com>: >>>>>> >> Are you running any kind of firewall on the node where mpirun is >>>>>> >> invoked? Open MPI needs to be able to use arbitrary TCP ports between >>>>>> >> the servers on which it runs. >>>>>> >> >>>>>> >> This second mail seems to imply a bug in OMPI's oob_tcp_if_include >>>>>> >> param handling, however -- it's supposed to be able to handle an >>>>>> >> interface name (not just a network specification). >>>>>> >> >>>>>> >> Ralph -- can you have a look? >>>>>> >> >>>>>> >> >>>>>> >> On Aug 12, 2014, at 8:41 AM, Timur Ismagilov <tismagi...@mail.ru> >>>>>> >> wrote: >>>>>> >> >>>>>> >>> When i add --mca oob_tcp_if_include ib0 (infiniband interface) to >>>>>> >>> mpirun (as it was here: >>>>>> >>> http://www.open-mpi.org/community/lists/users/2014/07/24857.php ) i >>>>>> >>> got this output: >>>>>> >>> >>>>>> >>> [compiler-2:08792] mca:base:select:( plm) Querying component >>>>>> >>> [isolated] >>>>>> >>> [compiler-2:08792] mca:base:select:( plm) Query of component >>>>>> >>> [isolated] set priority to 0 >>>>>> >>> [compiler-2:08792] mca:base:select:( plm) Querying component [rsh] >>>>>> >>> [compiler-2:08792] mca:base:select:( plm) Query of component [rsh] >>>>>> >>> set priority to 10 >>>>>> >>> [compiler-2:08792] mca:base:select:( plm) Querying component [slurm] >>>>>> >>> [compiler-2:08792] mca:base:select:( plm) Query of component [slurm] >>>>>> >>> set priority to 75 >>>>>> >>> [compiler-2:08792] mca:base:select:( plm) Selected component [slurm] >>>>>> >>> [compiler-2:08792] mca: base: components_register: registering oob >>>>>> >>> components >>>>>> >>> [compiler-2:08792] mca: base: components_register: found loaded >>>>>> >>> component tcp >>>>>> >>> [compiler-2:08792] mca: base: components_register: component tcp >>>>>> >>> register function successful >>>>>> >>> [compiler-2:08792] mca: base: components_open: opening oob components >>>>>> >>> [compiler-2:08792] mca: base: components_open: found loaded >>>>>> >>> component tcp >>>>>> >>> [compiler-2:08792] mca: base: components_open: component tcp open >>>>>> >>> function successful >>>>>> >>> [compiler-2:08792] mca:oob:select: checking available component tcp >>>>>> >>> [compiler-2:08792] mca:oob:select: Querying component [tcp] >>>>>> >>> [compiler-2:08792] oob:tcp: component_available called >>>>>> >>> [compiler-2:08792] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4 >>>>>> >>> [compiler-2:08792] WORKING INTERFACE 2 KERNEL INDEX 3 FAMILY: V4 >>>>>> >>> [compiler-2:08792] WORKING INTERFACE 3 KERNEL INDEX 4 FAMILY: V4 >>>>>> >>> [compiler-2:08792] WORKING INTERFACE 4 KERNEL INDEX 5 FAMILY: V4 >>>>>> >>> [compiler-2:08792] WORKING INTERFACE 5 KERNEL INDEX 6 FAMILY: V4 >>>>>> >>> [compiler-2:08792] [[42190,0],0] oob:tcp:init adding 10.128.0.4 to >>>>>> >>> our list of V4 connections >>>>>> >>> [compiler-2:08792] WORKING INTERFACE 6 KERNEL INDEX 7 FAMILY: V4 >>>>>> >>> [compiler-2:08792] [[42190,0],0] TCP STARTUP >>>>>> >>> [compiler-2:08792] [[42190,0],0] attempting to bind to IPv4 port 0 >>>>>> >>> [compiler-2:08792] [[42190,0],0] assigned IPv4 port 53883 >>>>>> >>> [compiler-2:08792] mca:oob:select: Adding component to end >>>>>> >>> [compiler-2:08792] mca:oob:select: Found 1 active transports >>>>>> >>> [compiler-2:08792] mca: base: components_register: registering rml >>>>>> >>> components >>>>>> >>> [compiler-2:08792] mca: base: components_register: found loaded >>>>>> >>> component oob >>>>>> >>> [compiler-2:08792] mca: base: components_register: component oob has >>>>>> >>> no register or open function >>>>>> >>> [compiler-2:08792] mca: base: components_open: opening rml components >>>>>> >>> [compiler-2:08792] mca: base: components_open: found loaded >>>>>> >>> component oob >>>>>> >>> [compiler-2:08792] mca: base: components_open: component oob open >>>>>> >>> function successful >>>>>> >>> [compiler-2:08792] orte_rml_base_select: initializing rml component >>>>>> >>> oob >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 30 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 15 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 32 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 33 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 5 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 10 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 12 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 9 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 34 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 2 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 21 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 22 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 45 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 46 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 1 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting recv >>>>>> >>> [compiler-2:08792] [[42190,0],0] posting persistent recv on tag 27 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> Daemon was launched on node1-128-01 - beginning to initialize >>>>>> >>> Daemon was launched on node1-128-02 - beginning to initialize >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> WARNING: An invalid value was given for oob_tcp_if_include. This >>>>>> >>> value will be ignored. >>>>>> >>> >>>>>> >>> Local host: node1-128-01 >>>>>> >>> Value: "ib0" >>>>>> >>> Message: Invalid specification (missing "/") >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> WARNING: An invalid value was given for oob_tcp_if_include. This >>>>>> >>> value will be ignored. >>>>>> >>> >>>>>> >>> Local host: node1-128-02 >>>>>> >>> Value: "ib0" >>>>>> >>> Message: Invalid specification (missing "/") >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> None of the TCP networks specified to be included for out-of-band >>>>>> >>> communications >>>>>> >>> could be found: >>>>>> >>> >>>>>> >>> Value given: >>>>>> >>> >>>>>> >>> Please revise the specification and try again. >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> None of the TCP networks specified to be included for out-of-band >>>>>> >>> communications >>>>>> >>> could be found: >>>>>> >>> >>>>>> >>> Value given: >>>>>> >>> >>>>>> >>> Please revise the specification and try again. >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> No network interfaces were found for out-of-band communications. We >>>>>> >>> require >>>>>> >>> at least one available network for out-of-band messaging. >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> No network interfaces were found for out-of-band communications. We >>>>>> >>> require >>>>>> >>> at least one available network for out-of-band messaging. >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> It looks like orte_init failed for some reason; your parallel >>>>>> >>> process is >>>>>> >>> likely to abort. There are many reasons that a parallel process can >>>>>> >>> fail during orte_init; some of which are due to configuration or >>>>>> >>> environment problems. This failure appears to be an internal failure; >>>>>> >>> here's some additional information (which may only be relevant to an >>>>>> >>> Open MPI developer): >>>>>> >>> >>>>>> >>> orte_oob_base_select failed >>>>>> >>> --> Returned value (null) (-43) instead of ORTE_SUCCESS >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> It looks like orte_init failed for some reason; your parallel >>>>>> >>> process is >>>>>> >>> likely to abort. There are many reasons that a parallel process can >>>>>> >>> fail during orte_init; some of which are due to configuration or >>>>>> >>> environment problems. This failure appears to be an internal failure; >>>>>> >>> here's some additional information (which may only be relevant to an >>>>>> >>> Open MPI developer): >>>>>> >>> >>>>>> >>> orte_oob_base_select failed >>>>>> >>> --> Returned value (null) (-43) instead of ORTE_SUCCESS >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> srun: error: node1-128-02: task 1: Exited with exit code 213 >>>>>> >>> srun: Terminating job step 657300.0 >>>>>> >>> srun: error: node1-128-01: task 0: Exited with exit code 213 >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> An ORTE daemon has unexpectedly failed after launch and before >>>>>> >>> communicating back to mpirun. This could be caused by a number >>>>>> >>> of factors, including an inability to create a connection back >>>>>> >>> to mpirun due to a lack of common network interfaces and/or no >>>>>> >>> route found between them. Please check network connectivity >>>>>> >>> (including firewalls and network routing requirements). >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> [compiler-2:08792] [[42190,0],0] orted_cmd: received halt_vm cmd >>>>>> >>> [compiler-2:08792] mca: base: close: component oob closed >>>>>> >>> [compiler-2:08792] mca: base: close: unloading component oob >>>>>> >>> [compiler-2:08792] [[42190,0],0] TCP SHUTDOWN >>>>>> >>> [compiler-2:08792] mca: base: close: component tcp closed >>>>>> >>> [compiler-2:08792] mca: base: close: unloading component tcp >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> Tue, 12 Aug 2014 16:14:58 +0400 от Timur Ismagilov >>>>>> >>> <tismagi...@mail.ru>: >>>>>> >>> Hello! >>>>>> >>> >>>>>> >>> I have Open MPI v1.8.2rc4r32485 >>>>>> >>> >>>>>> >>> When i run hello_c, I got this error message >>>>>> >>> $mpirun -np 2 hello_c >>>>>> >>> >>>>>> >>> An ORTE daemon has unexpectedly failed after launch and before >>>>>> >>> >>>>>> >>> communicating back to mpirun. This could be caused by a number >>>>>> >>> of factors, including an inability to create a connection back >>>>>> >>> to mpirun due to a lack of common network interfaces and/or no >>>>>> >>> route found between them. Please check network connectivity >>>>>> >>> (including firewalls and network routing requirements). >>>>>> >>> >>>>>> >>> When i run with --debug-daemons --mca plm_base_verbose 5 -mca >>>>>> >>> oob_base_verbose 10 -mca rml_base_verbose 10 i got this output: >>>>>> >>> $mpirun --debug-daemons --mca plm_base_verbose 5 -mca >>>>>> >>> oob_base_verbose 10 -mca rml_base_verbose 10 -np 2 hello_c >>>>>> >>> >>>>>> >>> [compiler-2:08780] mca:base:select:( plm) Querying component >>>>>> >>> [isolated] >>>>>> >>> [compiler-2:08780] mca:base:select:( plm) Query of component >>>>>> >>> [isolated] set priority to 0 >>>>>> >>> [compiler-2:08780] mca:base:select:( plm) Querying component [rsh] >>>>>> >>> [compiler-2:08780] mca:base:select:( plm) Query of component [rsh] >>>>>> >>> set priority to 10 >>>>>> >>> [compiler-2:08780] mca:base:select:( plm) Querying component [slurm] >>>>>> >>> [compiler-2:08780] mca:base:select:( plm) Query of component [slurm] >>>>>> >>> set priority to 75 >>>>>> >>> [compiler-2:08780] mca:base:select:( plm) Selected component [slurm] >>>>>> >>> [compiler-2:08780] mca: base: components_register: registering oob >>>>>> >>> components >>>>>> >>> [compiler-2:08780] mca: base: components_register: found loaded >>>>>> >>> component tcp >>>>>> >>> [compiler-2:08780] mca: base: components_register: component tcp >>>>>> >>> register function successful >>>>>> >>> [compiler-2:08780] mca: base: components_open: opening oob components >>>>>> >>> [compiler-2:08780] mca: base: components_open: found loaded >>>>>> >>> component tcp >>>>>> >>> [compiler-2:08780] mca: base: components_open: component tcp open >>>>>> >>> function successful >>>>>> >>> [compiler-2:08780] mca:oob:select: checking available component tcp >>>>>> >>> [compiler-2:08780] mca:oob:select: Querying component [tcp] >>>>>> >>> [compiler-2:08780] oob:tcp: component_available called >>>>>> >>> [compiler-2:08780] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4 >>>>>> >>> [compiler-2:08780] WORKING INTERFACE 2 KERNEL INDEX 3 FAMILY: V4 >>>>>> >>> [compiler-2:08780] [[42202,0],0] oob:tcp:init adding 10.0.251.53 to >>>>>> >>> our list of V4 connections >>>>>> >>> [compiler-2:08780] WORKING INTERFACE 3 KERNEL INDEX 4 FAMILY: V4 >>>>>> >>> [compiler-2:08780] [[42202,0],0] oob:tcp:init adding 10.0.0.4 to our >>>>>> >>> list of V4 connections >>>>>> >>> [compiler-2:08780] WORKING INTERFACE 4 KERNEL INDEX 5 FAMILY: V4 >>>>>> >>> [compiler-2:08780] [[42202,0],0] oob:tcp:init adding 10.2.251.14 to >>>>>> >>> our list of V4 connections >>>>>> >>> [compiler-2:08780] WORKING INTERFACE 5 KERNEL INDEX 6 FAMILY: V4 >>>>>> >>> [compiler-2:08780] [[42202,0],0] oob:tcp:init adding 10.128.0.4 to >>>>>> >>> our list of V4 connections >>>>>> >>> [compiler-2:08780] WORKING INTERFACE 6 KERNEL INDEX 7 FAMILY: V4 >>>>>> >>> [compiler-2:08780] [[42202,0],0] oob:tcp:init adding 93.180.7.38 to >>>>>> >>> our list of V4 connections >>>>>> >>> [compiler-2:08780] [[42202,0],0] TCP STARTUP >>>>>> >>> [compiler-2:08780] [[42202,0],0] attempting to bind to IPv4 port 0 >>>>>> >>> [compiler-2:08780] [[42202,0],0] assigned IPv4 port 38420 >>>>>> >>> [compiler-2:08780] mca:oob:select: Adding component to end >>>>>> >>> [compiler-2:08780] mca:oob:select: Found 1 active transports >>>>>> >>> [compiler-2:08780] mca: base: components_register: registering rml >>>>>> >>> components >>>>>> >>> [compiler-2:08780] mca: base: components_register: found loaded >>>>>> >>> component oob >>>>>> >>> [compiler-2:08780] mca: base: components_register: component oob has >>>>>> >>> no register or open function >>>>>> >>> [compiler-2:08780] mca: base: components_open: opening rml components >>>>>> >>> [compiler-2:08780] mca: base: components_open: found loaded >>>>>> >>> component oob >>>>>> >>> [compiler-2:08780] mca: base: components_open: component oob open >>>>>> >>> function successful >>>>>> >>> [compiler-2:08780] orte_rml_base_select: initializing rml component >>>>>> >>> oob >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 30 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 15 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 32 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 33 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 5 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 10 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 12 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 9 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 34 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 2 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 21 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 22 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 45 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 46 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 1 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting recv >>>>>> >>> [compiler-2:08780] [[42202,0],0] posting persistent recv on tag 27 >>>>>> >>> for peer [[WILDCARD],WILDCARD] >>>>>> >>> Daemon was launched on node1-130-08 - beginning to initialize >>>>>> >>> Daemon was launched on node1-130-03 - beginning to initialize >>>>>> >>> Daemon was launched on node1-130-05 - beginning to initialize >>>>>> >>> Daemon was launched on node1-130-02 - beginning to initialize >>>>>> >>> Daemon was launched on node1-130-01 - beginning to initialize >>>>>> >>> Daemon was launched on node1-130-04 - beginning to initialize >>>>>> >>> Daemon was launched on node1-130-07 - beginning to initialize >>>>>> >>> Daemon was launched on node1-130-06 - beginning to initialize >>>>>> >>> Daemon [[42202,0],3] checking in as pid 7178 on host node1-130-03 >>>>>> >>> [node1-130-03:07178] [[42202,0],3] orted: up and running - waiting >>>>>> >>> for commands! >>>>>> >>> Daemon [[42202,0],2] checking in as pid 13581 on host node1-130-02 >>>>>> >>> [node1-130-02:13581] [[42202,0],2] orted: up and running - waiting >>>>>> >>> for commands! >>>>>> >>> Daemon [[42202,0],1] checking in as pid 17220 on host node1-130-01 >>>>>> >>> [node1-130-01:17220] [[42202,0],1] orted: up and running - waiting >>>>>> >>> for commands! >>>>>> >>> Daemon [[42202,0],5] checking in as pid 6663 on host node1-130-05 >>>>>> >>> [node1-130-05:06663] [[42202,0],5] orted: up and running - waiting >>>>>> >>> for commands! >>>>>> >>> Daemon [[42202,0],8] checking in as pid 6683 on host node1-130-08 >>>>>> >>> [node1-130-08:06683] [[42202,0],8] orted: up and running - waiting >>>>>> >>> for commands! >>>>>> >>> Daemon [[42202,0],7] checking in as pid 7877 on host node1-130-07 >>>>>> >>> [node1-130-07:07877] [[42202,0],7] orted: up and running - waiting >>>>>> >>> for commands! >>>>>> >>> Daemon [[42202,0],4] checking in as pid 7735 on host node1-130-04 >>>>>> >>> [node1-130-04:07735] [[42202,0],4] orted: up and running - waiting >>>>>> >>> for commands! >>>>>> >>> Daemon [[42202,0],6] checking in as pid 8451 on host node1-130-06 >>>>>> >>> [node1-130-06:08451] [[42202,0],6] orted: up and running - waiting >>>>>> >>> for commands! >>>>>> >>> srun: error: node1-130-03: task 2: Exited with exit code 1 >>>>>> >>> srun: Terminating job step 657040.1 >>>>>> >>> srun: error: node1-130-02: task 1: Exited with exit code 1 >>>>>> >>> slurmd[node1-130-04]: *** STEP 657040.1 KILLED AT >>>>>> >>> 2014-08-12T12:59:07 WITH SIGNAL 9 *** >>>>>> >>> slurmd[node1-130-07]: *** STEP 657040.1 KILLED AT >>>>>> >>> 2014-08-12T12:59:07 WITH SIGNAL 9 *** >>>>>> >>> slurmd[node1-130-06]: *** STEP 657040.1 KILLED AT >>>>>> >>> 2014-08-12T12:59:07 WITH SIGNAL 9 *** >>>>>> >>> srun: Job step aborted: Waiting up to 2 seconds for job step to >>>>>> >>> finish. >>>>>> >>> srun: error: node1-130-01: task 0: Exited with exit code 1 >>>>>> >>> srun: error: node1-130-05: task 4: Exited with exit code 1 >>>>>> >>> srun: error: node1-130-08: task 7: Exited with exit code 1 >>>>>> >>> srun: error: node1-130-07: task 6: Exited with exit code 1 >>>>>> >>> srun: error: node1-130-04: task 3: Killed >>>>>> >>> srun: error: node1-130-06: task 5: Killed >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> An ORTE daemon has unexpectedly failed after launch and before >>>>>> >>> communicating back to mpirun. This could be caused by a number >>>>>> >>> of factors, including an inability to create a connection back >>>>>> >>> to mpirun due to a lack of common network interfaces and/or no >>>>>> >>> route found between them. Please check network connectivity >>>>>> >>> (including firewalls and network routing requirements). >>>>>> >>> -------------------------------------------------------------------------- >>>>>> >>> [compiler-2:08780] [[42202,0],0] orted_cmd: received halt_vm cmd >>>>>> >>> [compiler-2:08780] mca: base: close: component oob closed >>>>>> >>> [compiler-2:08780] mca: base: close: unloading component oob >>>>>> >>> [compiler-2:08780] [[42202,0],0] TCP SHUTDOWN >>>>>> >>> [compiler-2:08780] mca: base: close: component tcp closed >>>>>> >>> [compiler-2:08780] mca: base: close: unloading component tcp >>>>>> >>> >>>>>> >>> _______________________________________________ >>>>>> >>> users mailing list >>>>>> >>> us...@open-mpi.org >>>>>> >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>> >>> Link to this post: >>>>>> >>> http://www.open-mpi.org/community/lists/users/2014/08/24987.php >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> _______________________________________________ >>>>>> >>> users mailing list >>>>>> >>> us...@open-mpi.org >>>>>> >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>> >>> Link to this post: >>>>>> >>> http://www.open-mpi.org/community/lists/users/2014/08/24988.php >>>>>> >> >>>>>> >> >>>>>> >> -- >>>>>> >> Jeff Squyres >>>>>> >> jsquy...@cisco.com >>>>>> >> For corporate legal information go to: >>>>>> >> http://www.cisco.com/web/about/doing_business/legal/cri/ >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> > >>>>>> > >>>>>> > -- >>>>>> > Jeff Squyres >>>>>> > jsquy...@cisco.com >>>>>> > For corporate legal information go to: >>>>>> > http://www.cisco.com/web/about/doing_business/legal/cri/ >>>>>> > >>>>>> > _______________________________________________ >>>>>> > users mailing list >>>>>> > us...@open-mpi.org >>>>>> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>> > Link to this post: >>>>>> > http://www.open-mpi.org/community/lists/users/2014/08/25001.php >>>>>> >>>>>> >>>>>> -- >>>>>> Jeff Squyres >>>>>> jsquy...@cisco.com >>>>>> For corporate legal information go to: >>>>>> http://www.cisco.com/web/about/doing_business/legal/cri/ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> users mailing list >>>>>> us...@open-mpi.org >>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>>> Link to this post: >>>>>> http://www.open-mpi.org/community/lists/users/2014/08/25086.php >>>>> >>>>> >>>>> _______________________________________________ >>>>> users mailing list >>>>> us...@open-mpi.org >>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>> Link to this post: >>>>> http://www.open-mpi.org/community/lists/users/2014/08/25093.php >>>>> >>>>> _______________________________________________ >>>>> users mailing list >>>>> us...@open-mpi.org >>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>>> Link to this post: >>>>> http://www.open-mpi.org/community/lists/users/2014/08/25094.php >>>> >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>> Link to this post: >>>> http://www.open-mpi.org/community/lists/users/2014/08/25095.php >>>> >>>> >>>> >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>>> Link to this post: >>>> http://www.open-mpi.org/community/lists/users/2014/08/25105.php >>> >>> >>> >>> >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>> Link to this post: >>> http://www.open-mpi.org/community/lists/users/2014/08/25127.php >>> >>> >>> >>> -- >>> >>> Kind Regards, >>> >>> M. >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>> Link to this post: >>> http://www.open-mpi.org/community/lists/users/2014/08/25128.php >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2014/08/25129.php >> >> >> > > > > > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/08/25154.php
