You have to call MPI_Comm_disconnect on both sides of the intercommunicator. On
the spawner processes you should call it on the intercom, while on the spawnees
you should call it on the MPI_Comm_get_parent.
George.
> On Dec 12, 2014, at 20:43 , Alex A. Schmidt <a...@ufsm.br> wrote:
>
> Gilles,
>
> MPI_comm_disconnect seem to work but not quite.
> The call to it returns almost immediatly while
> the spawn processes keep piling up in the background
> until they are all done...
>
> I think system('env -i qsub...') to launch the third party apps
> would take the execution of every call back to the scheduler
> queue. How would I track each one for their completion?
>
> Alex
>
> 2014-12-12 22:35 GMT-02:00 Gilles Gouaillardet <gilles.gouaillar...@gmail.com
> <mailto:gilles.gouaillar...@gmail.com>>:
> Alex,
>
> You need MPI_Comm_disconnect at least.
> I am not sure if this is 100% correct nor working.
>
> If you are using third party apps, why dont you do something like
> system("env -i qsub ...")
> with the right options to make qsub blocking or you manually wait for the end
> of the job ?
>
> That looks like a much cleaner and simpler approach to me.
>
> Cheers,
>
> Gilles
>
> "Alex A. Schmidt" <a...@ufsm.br <mailto:a...@ufsm.br>> wrote:
> Hello Gilles,
>
> Ok, I believe I have a simple toy app running as I think it should:
> 'n' parent processes running under mpi_comm_world, each one
> spawning its own 'm' child processes (each child group work
> together nicely, returning the expected result for an mpi_allreduce call).
>
> Now, as I mentioned before, the apps I want to run in the spawned
> processes are third party mpi apps and I don't think it will be possible
> to exchange messages with them from my app. So, I do I tell
> when the spawned processes have finnished running? All I have to work
> with is the intercommunicator returned from the mpi_comm_spawn call...
>
> Alex
>
>
>
>
> 2014-12-12 2:42 GMT-02:00 Alex A. Schmidt <a...@ufsm.br
> <mailto:a...@ufsm.br>>:
> Gilles,
>
> Well, yes, I guess....
>
> I'll do tests with the real third party apps and let you know.
> These are huge quantum chemistry codes (dftb+, siesta and Gaussian)
> which greatly benefits from a parallel environment. My code is just
> a front end to use those, but since we have a lot of data to process
> it also benefits from a parallel environment.
>
> Alex
>
>
> 2014-12-12 2:30 GMT-02:00 Gilles Gouaillardet <gilles.gouaillar...@iferc.org
> <mailto:gilles.gouaillar...@iferc.org>>:
> Alex,
>
> just to make sure ...
> this is the behavior you expected, right ?
>
> Cheers,
>
> Gilles
>
>
> On 2014/12/12 13:27, Alex A. Schmidt wrote:
>> Gilles,
>>
>> Ok, very nice!
>>
>> When I excute
>>
>> do rank=1,3
>> call MPI_Comm_spawn('hello_world','
>> ',5,MPI_INFO_NULL,rank,MPI_COMM_WORLD,my_intercomm,MPI_ERRCODES_IGNORE,status)
>> enddo
>>
>> I do get 15 instances of the 'hello_world' app running: 5 for each parent
>> rank 1, 2 and 3.
>>
>> Thanks a lot, Gilles.
>>
>> Best regargs,
>>
>> Alex
>>
>>
>>
>>
>> 2014-12-12 1:32 GMT-02:00 Gilles Gouaillardet <gilles.gouaillar...@iferc.org
>> <mailto:gilles.gouaillar...@iferc.org>
>>> :
>>>
>>> Alex,
>>>
>>> just ask MPI_Comm_spawn to start (up to) 5 tasks via the maxprocs
>>> parameter :
>>>
>>> int MPI_Comm_spawn(char *command, char *argv[], int maxprocs,
>>> MPI_Info info,
>>> int root, MPI_Comm comm, MPI_Comm *intercomm,
>>> int array_of_errcodes[])
>>>
>>> INPUT PARAMETERS
>>> maxprocs
>>> - maximum number of processes to start (integer, significant
>>> only at root)
>>>
>>> Cheers,
>>>
>>> Gilles
>>>
>>>
>>> On 2014/12/12 12:23, Alex A. Schmidt wrote:
>>>
>>> Hello Gilles,
>>>
>>> Thanks for your reply. The "env -i PATH=..." stuff seems to work!!!
>>>
>>> call system("sh -c 'env -i PATH=/usr/lib64/openmpi/bin:/bin mpirun -n 2
>>> hello_world' ")
>>>
>>> did produce the expected result with a simple openmi "hello_world" code I
>>> wrote.
>>>
>>> I might be harder though with the real third party app I have in mind. And
>>> I realize
>>> getting passed over a job scheduler with this approach might not work at
>>> all...
>>>
>>> I have looked at the MPI_Comm_spawn call but I failed to understand how it
>>> could help here. For instance, can I use it to launch an mpi app with the
>>> option "-n 5" ?
>>>
>>> Alex
>>>
>>> 2014-12-12 0:36 GMT-02:00 Gilles Gouaillardet
>>> <gilles.gouaillar...@iferc.org <mailto:gilles.gouaillar...@iferc.org>
>>>
>>> :
>>>
>>> Alex,
>>>
>>> can you try something like
>>> call system(sh -c 'env -i /.../mpirun -np 2 /.../app_name')
>>>
>>> -i start with an empty environment
>>> that being said, you might need to set a few environment variables
>>> manually :
>>> env -i PATH=/bin ...
>>>
>>> and that being also said, this "trick" could be just a bad idea :
>>> you might be using a scheduler, and if you empty the environment, the
>>> scheduler
>>> will not be aware of the "inside" run.
>>>
>>> on top of that, invoking system might fail depending on the interconnect
>>> you use.
>>>
>>> Bottom line, i believe Ralph's reply is still valid, even if five years
>>> have passed :
>>> changing your workflow, or using MPI_Comm_spawn is a much better approach.
>>>
>>> Cheers,
>>>
>>> Gilles
>>>
>>> On 2014/12/12 11:22, Alex A. Schmidt wrote:
>>>
>>> Dear OpenMPI users,
>>>
>>> Regarding to this previous
>>> post<http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php> from 2009,
>>> I wonder if the reply
>>> from Ralph Castain is still valid. My need is similar but quite simpler:
>>> to make a system call from an openmpi fortran application to run a
>>> third party openmpi application. I don't need to exchange mpi messages
>>> with the application. I just need to read the resulting output file
>>> generated
>>> by it. I have tried to do the following system call from my fortran openmpi
>>> code:
>>>
>>> call system("sh -c 'mpirun -n 2 app_name")
>>>
>>> but I get
>>>
>>> **********************************************************
>>>
>>> Open MPI does not support recursive calls of mpirun
>>>
>>> **********************************************************
>>>
>>> Is there a way to make this work?
>>>
>>> Best regards,
>>>
>>> Alex
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> users mailing listus...@open-mpi.org <mailto:listus...@open-mpi.org>
>>>
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> <http://www.open-mpi.org/mailman/listinfo.cgi/users>
>>> Link to this post:
>>> http://www.open-mpi.org/community/lists/users/2014/12/25966.php
>>> <http://www.open-mpi.org/community/lists/users/2014/12/25966.php>
>>>
>>>
>>>
>>> _______________________________________________
>>> users mailing listus...@open-mpi.org <mailto:listus...@open-mpi.org>
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> <http://www.open-mpi.org/mailman/listinfo.cgi/users>
>>> Link to this
>>> post:http://www.open-mpi.org/community/lists/users/2014/12/25967.php
>>> <http://www.open-mpi.org/community/lists/users/2014/12/25967.php>
>>>
>>>
>>>
>>> _______________________________________________
>>> users mailing listus...@open-mpi.org <mailto:listus...@open-mpi.org>
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> <http://www.open-mpi.org/mailman/listinfo.cgi/users>
>>>
>>> Link to this post:
>>> http://www.open-mpi.org/community/lists/users/2014/12/25968.php
>>> <http://www.open-mpi.org/community/lists/users/2014/12/25968.php>
>>>
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> us...@open-mpi.org <mailto:us...@open-mpi.org>
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> <http://www.open-mpi.org/mailman/listinfo.cgi/users>
>>> Link to this post:
>>> http://www.open-mpi.org/community/lists/users/2014/12/25969.php
>>> <http://www.open-mpi.org/community/lists/users/2014/12/25969.php>
>>>
>>
>>
>> _______________________________________________
>> users mailing list
>> us...@open-mpi.org <mailto:us...@open-mpi.org>
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> <http://www.open-mpi.org/mailman/listinfo.cgi/users>
>> Link to this post:
>> http://www.open-mpi.org/community/lists/users/2014/12/25970.php
>> <http://www.open-mpi.org/community/lists/users/2014/12/25970.php>
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org <mailto:us...@open-mpi.org>
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> <http://www.open-mpi.org/mailman/listinfo.cgi/users>
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2014/12/25971.php
> <http://www.open-mpi.org/community/lists/users/2014/12/25971.php>
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org <mailto:us...@open-mpi.org>
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> <http://www.open-mpi.org/mailman/listinfo.cgi/users>
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2014/12/25974.php
> <http://www.open-mpi.org/community/lists/users/2014/12/25974.php>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2014/12/25975.php
