FWIW: OpenMPI does support spawning of both MPI and non-MPI jobs. If you are
spawning a non-MPI job, then you have to -tell- us that so we don’t hang trying
to connect the new procs to the spawning proc as per MPI requirements.
This is done by providing an info key to indicate that the child job is
non-MPI, as explained in the MPI_Comm_spawn man page:
ompi_non_mpi bool If set to true, launching a non-MPI
application; the returned communicator
will be MPI_COMM_NULL. Failure to set
this flag when launching a non-MPI
application will cause both the child
and parent jobs to "hang".
> On Oct 9, 2015, at 2:15 AM, Lisandro Dalcin <dalc...@gmail.com> wrote:
>
> On 9 October 2015 at 12:05, simona bellavista <afy...@gmail.com
> <mailto:afy...@gmail.com>> wrote:
>>
>>
>> 2015-10-09 9:40 GMT+02:00 Lisandro Dalcin <dalc...@gmail.com>:
>>>
>>> On 8 October 2015 at 14:54, simona bellavista <afy...@gmail.com> wrote:
>>>>
>>>
>>>>>
>>>>> I cannot figure out how spawn would work with a string-command. I tried
>>>>> MPI.COMM_SELF.Spawn(cmd, args=None,maxproc=4) and it just hangs
>>>>
>>>
>>> MPI.COMM_SELF.Spawn("/bin/echo", args=["Hello",
>>> "World!"],maxprocs=1).Disconnect()
>>>
>>> Could you try the line above and confirm whether it hangs?
>>
>>
>> I have tried the line above and it hangs
>>
>
> OK, as "echo" is not an MPI application, then it seems OpenMPI does
> not support spawning.
>
>>>
>>>>
>>>> I couldn't figure out how to run Spawn with a string-like command, in
>>>> fact
>>>> the command that I want to run varies for each processor.
>>>
>>> Use maxprocs=1 and make different spawn calls.
>>>
>>> However, I have to insist. If you are using mpi4py as a tool to spawn
>>> a bunch of different processes that work in isolation and then collect
>>> result at the end, then mpi4py is likely not the right tool for the
>>> task, at least if you do not have previous experience with MPI
>>> programming.
>>>
>> Well, I don't have a big experience in MPI programming, but I do use and
>> modify existing MPI codes, and I thought MPI would be easiest choice.
>
> Have you seen these existing MPI codes calling back to the shell to
> execute commands?
>
>> Clustershells looks a bit an overshoot for the goal I would like to achieve.
>> What shall I use instead? Shall I try multiprocessing module?
>>
>
> As long as running on a single compute node is many cores is enough
> for your application, there is no reason to use MPI. Python's
> multiprocessing of perhaps the Python 3 "concurrent.futures" package
> (there is a backport for Python 2 on PyPI) would be trivial to get
> working.
>
>
> --
> Lisandro Dalcin
> ============
> Research Scientist
> Computer, Electrical and Mathematical Sciences & Engineering (CEMSE)
> Numerical Porous Media Center (NumPor)
> King Abdullah University of Science and Technology (KAUST)
> http://numpor.kaust.edu.sa/
>
> 4700 King Abdullah University of Science and Technology
> al-Khawarizmi Bldg (Bldg 1), Office # 4332
> Thuwal 23955-6900, Kingdom of Saudi Arabia
> http://www.kaust.edu.sa
>
> Office Phone: +966 12 808-0459
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/10/27855.php
> <http://www.open-mpi.org/community/lists/users/2015/10/27855.php>
