The option system("env -i ...") has been tested earlier by me and it does
work. There is doubt though it would work along with a job scheduler.
I will reserve this as a last resort solution.
mpi_comm_spawn("/bin/sh","-c","siesta < infile",..) definitely does not
work.
Patching siesta to start as "siesta -in infile" sounds very promissing.
That option as well as a "-o outfile" option should be there to start with.
I can bring that to the attention of the siesta developers. But then we have
this other app "dftb+", and this other one "Gaussian" (just to mention a
few).
Some of them might already act like that, others don't. So, having
mpi_comm_spawn working with i/o redirection would be in fact more
simpler.
Perhaps a shell script "siesta_alt" could be written to convert
"siesta_alt infile outfile" into "siesta < infile > outfile" and then
do mpi_comm_spawn("siesta_alt","infile","outfile",...). But I am not
sure if the spawn on siesta_alt would actually lead to a spawn on siesta
as expected...
Alex
2014-12-17 22:35 GMT-02:00 Gilles Gouaillardet <
gilles.gouaillar...@gmail.com>:
>
> Alex,
>
> You do not want to spawn mpirun.
> Or if this is really what you want, then just use system("env -i ...")
>
> I think what you need is spawn a shell that do the redirection and then
> invoke your app.
> This is something like
> MPI_Comm_spawn("/bin/sh", "-c", "siesta < infile")
>
> That being said, i strongly recommend you patch siesta so it can be
> invoked like this
> siesta -in infile
> (plus the MPI_Comm_disconnect call explained by George)
> That would make everything so much easier
>
> Cheers,
>
> Gilles
>
> "Alex A. Schmidt" <a...@ufsm.br> wrote:
> Let me rephrase the previous message:
>
> Putting "/bin/sh" in command with info key "ompi_non_mpi" set to
> ".true."
> (if command is empty, mpi_comm_spawn tries to execute ' ') of
> mpi_comm_spawn and "-c" "mpirun -n 1 myapp" in args results in
> this message:
>
> **********************************************************
>
> Open MPI does not support recursive calls of mpirun
>
> **********************************************************
>
> Putting a single string in args as "-c mpirun -n 1 myapp" or "-c 'mpirun
> -n 1 myapp' "
> returns
>
> /usr/bin/sh: - : invalid option
>
> Alex
>
>
>
> 2014-12-17 21:47 GMT-02:00 Alex A. Schmidt <a...@ufsm.br>:
>>
>> Putting "/bin/sh" in command with info key "ompi_non_mpi" set to
>> ".true."
>> (if command is empty, mpi_comm_spawn tries to execute ' ') of
>> mpi_comm_spawn and "-c" "mpirun -n 1 myapp" in args results in
>> this message:
>>
>> /usr/bin/sh: -c: option requires an argument
>>
>> Putting a single string in args as "-c mpirun -n 1 myapp" or "-c 'mpirun
>> -n 1 myapp' "
>> returns
>>
>> /usr/bin/sh: - : invalid option
>>
>> Alex
>>
>> 2014-12-17 20:17 GMT-02:00 George Bosilca <bosi...@icl.utk.edu>:
>>>
>>> I don't think this has any chance of working. The redirection is
>>> something interpreted by the shell, and when Open MPI "fork-exec" a process
>>> it does not behave as the shell.
>>>
>>> Thus a potentially non-portable solution would be to instead of
>>> launching the mpirun directly to launch it through a shell. Maybe something
>>> like "/bin/sh", "-c", "mpirun -n 1 myapp".
>>>
>>> George.
>>>
>>>
>>> On Wed, Dec 17, 2014 at 5:02 PM, Alex A. Schmidt <a...@ufsm.br> wrote:
>>>
>>>> Ralph,
>>>>
>>>> Sorry, "<" as an element of argv to mpi_comm_spawn is interpreted just
>>>> the
>>>> same, as another parameter by the spawnee process.
>>>>
>>>> But I am confused: wouldn't it be redundant to put "mpirun" "-n" "1"
>>>> "myapp"
>>>> as elements of argv, considering role of the other parameters of
>>>> mpi_comm_spawn
>>>> like the 1st and 3rd ?
>>>>
>>>> Imho, it seems to me that stdin and stdout should be keys to "info" and
>>>> the
>>>> respective filenames the values of those keys, if that is possible to
>>>> implement...
>>>>
>>>> Alex
>>>>
>>>>
>>>>
>>>> 2014-12-17 15:16 GMT-02:00 Ralph Castain <r...@open-mpi.org>:
>>>>>
>>>>> Have you tried putting the "<" as a separate parameter? In other
>>>>> words, since you are specifying the argv, you have to specify each of them
>>>>> separately. So it looks more like:
>>>>>
>>>>> "mpirun", "-n", "1", "myapp", "<", "stdinfile"
>>>>>
>>>>> Does that work?
>>>>> Ralph
>>>>>
>>>>>
>>>>> On Wed, Dec 17, 2014 at 8:07 AM, Alex A. Schmidt <a...@ufsm.br> wrote:
>>>>>
>>>>>> Ralph,
>>>>>>
>>>>>> I am afraid I will have to insist on i/o redirection matter
>>>>>> for the spawnee process.
>>>>>>
>>>>>> I have a "child" mpi code that do just 2 things: read the 3 parameters
>>>>>> passed to it and print them, and then read data from stdin and show
>>>>>> it.
>>>>>> So, if "stdin_file" is a text file with two lines, say:
>>>>>>
>>>>>> 10
>>>>>> 20
>>>>>>
>>>>>> executing "mpirun -n 1 child A B < stdin_file" wiil ouput two lines:
>>>>>>
>>>>>> [A] [B] []
>>>>>> 10 20
>>>>>>
>>>>>> On the other hand , calling "child" from
>>>>>> MPI_Comm_spawn("child",args,...)
>>>>>> where
>>>>>>
>>>>>> args(1) = "A"
>>>>>> args(2) = "B"
>>>>>> args(3) ="< stdin_file"
>>>>>> args(4) = " "
>>>>>>
>>>>>> will make "child" outputs only 1 line
>>>>>>
>>>>>> [A] [B] [< stdin_file]
>>>>>>
>>>>>> and then fails because there is not stdin data to read from.
>>>>>>
>>>>>> Please, note that surprisingly the whole string "< stdin_file" is
>>>>>> interpreted
>>>>>> as a third parameter to "child" and not a stdin...
>>>>>>
>>>>>> Alex
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2014-12-15 17:26 GMT-02:00 Alex A. Schmidt <a...@ufsm.br>:
>>>>>>
>>>>>>> Ralph,
>>>>>>>
>>>>>>> I guess you mean "call mpi_comm_spawn( 'siesta', '< infile' , 2
>>>>>>> ,...)"
>>>>>>>
>>>>>>> to execute 'mpirun -n 2 siesta < infile' on the spawnee side. That
>>>>>>> was
>>>>>>> my first choice. Well, siesta behaves as if no stdin file was
>>>>>>> present...
>>>>>>>
>>>>>>> Alex
>>>>>>>
>>>>>>>
>>>>>>> 2014-12-15 17:07 GMT-02:00 Ralph Castain <r...@open-mpi.org>:
>>>>>>>>
>>>>>>>> You should be able to just include that in your argv that you pass
>>>>>>>> to the Comm_spawn API.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Dec 15, 2014 at 9:27 AM, Alex A. Schmidt <a...@ufsm.br>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> George,
>>>>>>>>>
>>>>>>>>> Thanks for the tip. In fact, calling mpi_comm_spawn right away
>>>>>>>>> with MPI_COMM_SELF
>>>>>>>>> has worked for me just as well -- no subgroups needed at all.
>>>>>>>>>
>>>>>>>>> I am testing this openmpi app named "siesta" in parallel. The
>>>>>>>>> source code is available,
>>>>>>>>> so making it "spawn ready" by adding the pair mpi_comm_get_parent
>>>>>>>>> + mpi_comm_disconnect
>>>>>>>>> into the main code can be done. If it works, maybe the siesta's
>>>>>>>>> developers can be convinced
>>>>>>>>> to add this feature in a future release.
>>>>>>>>>
>>>>>>>>> However, siesta is launched only by specifying input/output files
>>>>>>>>> with i/o redirection like
>>>>>>>>>
>>>>>>>>> mpirun -n <*some number*> siesta < infile > outfile
>>>>>>>>>
>>>>>>>>> So far, I could not find anything about how to set an stdin file
>>>>>>>>> for an spawnee process.
>>>>>>>>> Specifiyng it in a app context file doesn't seem to work. Can it
>>>>>>>>> be done? Maybe through
>>>>>>>>> an MCA parameter?
>>>>>>>>>
>>>>>>>>> Alex
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2014-12-15 2:43 GMT-02:00 George Bosilca <bosi...@icl.utk.edu>:
>>>>>>>>>>
>>>>>>>>>> Alex,
>>>>>>>>>>
>>>>>>>>>> The code looks good, and is 100% MPI standard accurate.
>>>>>>>>>>
>>>>>>>>>> I would change the way you create the subcoms in the parent. You
>>>>>>>>>> do a lot of useless operations, as you can achieve exactly the same
>>>>>>>>>> outcome
>>>>>>>>>> (one communicator per node), either by duplicating MPI_COMM_SELF or
>>>>>>>>>> doing
>>>>>>>>>> an MPI_Comm_split with the color equal to your rank.
>>>>>>>>>>
>>>>>>>>>> George.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sun, Dec 14, 2014 at 2:20 AM, Alex A. Schmidt <a...@ufsm.br>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Sorry, guys. I don't think the newbie here can follow any
>>>>>>>>>>> discussion
>>>>>>>>>>> beyond basic mpi...
>>>>>>>>>>>
>>>>>>>>>>> Anyway, if I add the pair
>>>>>>>>>>>
>>>>>>>>>>> call MPI_COMM_GET_PARENT(mpi_comm_parent,ierror)
>>>>>>>>>>> call MPI_COMM_DISCONNECT(mpi_comm_parent,ierror)
>>>>>>>>>>>
>>>>>>>>>>> on the spawnee side I get the proper response in the spawning
>>>>>>>>>>> processes.
>>>>>>>>>>>
>>>>>>>>>>> Please, take a look at the attached toy codes parent.F and
>>>>>>>>>>> child.F
>>>>>>>>>>> I've been playing with. 'mpirun -n 2 parent' seems to work as
>>>>>>>>>>> expected.
>>>>>>>>>>>
>>>>>>>>>>> Alex
>>>>>>>>>>>
>>>>>>>>>>> 2014-12-13 23:46 GMT-02:00 Gilles Gouaillardet <
>>>>>>>>>>> gilles.gouaillar...@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>> Alex,
>>>>>>>>>>>>
>>>>>>>>>>>> Are you calling MPI_Comm_disconnect in the 3 "master" tasks and
>>>>>>>>>>>> with the same remote communicator ?
>>>>>>>>>>>>
>>>>>>>>>>>> I also read the man page again, and MPI_Comm_disconnect does
>>>>>>>>>>>> not ensure the remote processes have finished or called
>>>>>>>>>>>> MPI_Comm_disconnect, so that might not be the thing you need.
>>>>>>>>>>>> George, can you please comment on that ?
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> Gilles
>>>>>>>>>>>>
>>>>>>>>>>>> George Bosilca <bosi...@icl.utk.edu> wrote:
>>>>>>>>>>>> MPI_Comm_disconnect should be a local operation, there is no
>>>>>>>>>>>> reason for it to deadlock. I looked at the code and everything is
>>>>>>>>>>>> local
>>>>>>>>>>>> with the exception of a call to PMIX.FENCE. Can you attach to your
>>>>>>>>>>>> deadlocked processes and confirm that they are stopped in the
>>>>>>>>>>>> pmix.fence?
>>>>>>>>>>>>
>>>>>>>>>>>> George.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Dec 13, 2014 at 8:47 AM, Alex A. Schmidt <a...@ufsm.br>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sorry, I was calling mpi_comm_disconnect on the group comm
>>>>>>>>>>>>> handler, not
>>>>>>>>>>>>> on the intercomm handler returned from the spawn call as it
>>>>>>>>>>>>> should be.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Well, calling the disconnect on the intercomm handler does
>>>>>>>>>>>>> halt the spwaner
>>>>>>>>>>>>> side but the wait is never completed since, as George points
>>>>>>>>>>>>> out, there is no
>>>>>>>>>>>>> disconnect call being made on the spawnee side.... and that
>>>>>>>>>>>>> brings me back
>>>>>>>>>>>>> to the beginning of the problem since, being a third party
>>>>>>>>>>>>> app, that call would
>>>>>>>>>>>>> never be there. I guess an mpi wrapper to deal with that could
>>>>>>>>>>>>> be made for
>>>>>>>>>>>>> the app, but I fell the wrapper itself, at the end, would face
>>>>>>>>>>>>> the same problem
>>>>>>>>>>>>> we face right now.
>>>>>>>>>>>>>
>>>>>>>>>>>>> My application is a genetic algorithm code that search optimal
>>>>>>>>>>>>> configuration
>>>>>>>>>>>>> (minimum or maximum energy) of cluster of atoms. The work flow
>>>>>>>>>>>>> bottleneck
>>>>>>>>>>>>> is the calculation of the cluster energy. For the cases which
>>>>>>>>>>>>> an analytical
>>>>>>>>>>>>> potential is available the calculation can be made internally
>>>>>>>>>>>>> and the workload
>>>>>>>>>>>>> is distributed among slaves nodes from a master node. This is
>>>>>>>>>>>>> also done
>>>>>>>>>>>>> when an analytical potential is not available and the energy
>>>>>>>>>>>>> calculation must
>>>>>>>>>>>>> be done externally by a quantum chemistry code like dftb+,
>>>>>>>>>>>>> siesta and Gaussian.
>>>>>>>>>>>>> So far, we have been running these codes in serial mode. No
>>>>>>>>>>>>> need to say that
>>>>>>>>>>>>> we could do a lot better if they could be executed in parallel.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am not familiar with DMRAA but it seems to be the right
>>>>>>>>>>>>> choice to deal with
>>>>>>>>>>>>> job schedulers as it covers the ones I am interested in
>>>>>>>>>>>>> (pbs/torque and loadlever).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2014-12-13 7:49 GMT-02:00 Gilles Gouaillardet <
>>>>>>>>>>>>> gilles.gouaillar...@gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> George is right about the semantic
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> However i am surprised it returns immediatly...
>>>>>>>>>>>>>> That should either work or hang imho
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The second point is no more mpi related, and is batch manager
>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You will likely find a submit parameter to make the command
>>>>>>>>>>>>>> block until the job completes. Or you can write your own wrapper.
>>>>>>>>>>>>>> Or you can retrieve the jobid and qstat periodically to get
>>>>>>>>>>>>>> the job state.
>>>>>>>>>>>>>> If an api is available, this is also an option.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Gilles
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> George Bosilca <bosi...@icl.utk.edu> wrote:
>>>>>>>>>>>>>> You have to call MPI_Comm_disconnect on both sides of the
>>>>>>>>>>>>>> intercommunicator. On the spawner processes you should call it
>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>> intercom, while on the spawnees you should call it on the
>>>>>>>>>>>>>> MPI_Comm_get_parent.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> George.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Dec 12, 2014, at 20:43 , Alex A. Schmidt <a...@ufsm.br>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Gilles,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> MPI_comm_disconnect seem to work but not quite.
>>>>>>>>>>>>>> The call to it returns almost immediatly while
>>>>>>>>>>>>>> the spawn processes keep piling up in the background
>>>>>>>>>>>>>> until they are all done...
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think system('env -i qsub...') to launch the third party
>>>>>>>>>>>>>> apps
>>>>>>>>>>>>>> would take the execution of every call back to the scheduler
>>>>>>>>>>>>>> queue. How would I track each one for their completion?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2014-12-12 22:35 GMT-02:00 Gilles Gouaillardet <
>>>>>>>>>>>>>> gilles.gouaillar...@gmail.com>:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Alex,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You need MPI_Comm_disconnect at least.
>>>>>>>>>>>>>>> I am not sure if this is 100% correct nor working.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If you are using third party apps, why dont you do something
>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>> system("env -i qsub ...")
>>>>>>>>>>>>>>> with the right options to make qsub blocking or you manually
>>>>>>>>>>>>>>> wait for the end of the job ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> That looks like a much cleaner and simpler approach to me.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Gilles
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> "Alex A. Schmidt" <a...@ufsm.br> wrote:
>>>>>>>>>>>>>>> Hello Gilles,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ok, I believe I have a simple toy app running as I think it
>>>>>>>>>>>>>>> should:
>>>>>>>>>>>>>>> 'n' parent processes running under mpi_comm_world, each one
>>>>>>>>>>>>>>> spawning its own 'm' child processes (each child group work
>>>>>>>>>>>>>>> together nicely, returning the expected result for an mpi_
>>>>>>>>>>>>>>> allreduce call).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Now, as I mentioned before, the apps I want to run in the
>>>>>>>>>>>>>>> spawned
>>>>>>>>>>>>>>> processes are third party mpi apps and I don't think it will
>>>>>>>>>>>>>>> be possible
>>>>>>>>>>>>>>> to exchange messages with them from my app. So, I do I tell
>>>>>>>>>>>>>>> when the spawned processes have finnished running? All I
>>>>>>>>>>>>>>> have to work
>>>>>>>>>>>>>>> with is the intercommunicator returned from the
>>>>>>>>>>>>>>> mpi_comm_spawn call...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2014-12-12 2:42 GMT-02:00 Alex A. Schmidt <a...@ufsm.br>:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Gilles,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Well, yes, I guess....
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'll do tests with the real third party apps and let you
>>>>>>>>>>>>>>>> know.
>>>>>>>>>>>>>>>> These are huge quantum chemistry codes (dftb+, siesta and
>>>>>>>>>>>>>>>> Gaussian)
>>>>>>>>>>>>>>>> which greatly benefits from a parallel environment. My code
>>>>>>>>>>>>>>>> is just
>>>>>>>>>>>>>>>> a front end to use those, but since we have a lot of data
>>>>>>>>>>>>>>>> to process
>>>>>>>>>>>>>>>> it also benefits from a parallel environment.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2014-12-12 2:30 GMT-02:00 Gilles Gouaillardet <
>>>>>>>>>>>>>>>> gilles.gouaillar...@iferc.org>:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Alex,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> just to make sure ...
>>>>>>>>>>>>>>>>> this is the behavior you expected, right ?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Gilles
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 2014/12/12 13:27, Alex A. Schmidt wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Gilles,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ok, very nice!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> When I excute
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> do rank=1,3
>>>>>>>>>>>>>>>>> call MPI_Comm_spawn('hello_world','
>>>>>>>>>>>>>>>>> ',5,MPI_INFO_NULL,rank,MPI_COMM_WORLD,my_intercomm,MPI_ERRCODES_IGNORE,status)
>>>>>>>>>>>>>>>>> enddo
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I do get 15 instances of the 'hello_world' app running: 5 for
>>>>>>>>>>>>>>>>> each parent
>>>>>>>>>>>>>>>>> rank 1, 2 and 3.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks a lot, Gilles.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best regargs,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2014-12-12 1:32 GMT-02:00 Gilles Gouaillardet
>>>>>>>>>>>>>>>>> <gilles.gouaillar...@iferc.org
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Alex,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> just ask MPI_Comm_spawn to start (up to) 5 tasks via the
>>>>>>>>>>>>>>>>> maxprocs
>>>>>>>>>>>>>>>>> parameter :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> int MPI_Comm_spawn(char *command, char *argv[], int
>>>>>>>>>>>>>>>>> maxprocs,
>>>>>>>>>>>>>>>>> MPI_Info info,
>>>>>>>>>>>>>>>>> int root, MPI_Comm comm, MPI_Comm
>>>>>>>>>>>>>>>>> *intercomm,
>>>>>>>>>>>>>>>>> int array_of_errcodes[])
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> INPUT PARAMETERS
>>>>>>>>>>>>>>>>> maxprocs
>>>>>>>>>>>>>>>>> - maximum number of processes to start
>>>>>>>>>>>>>>>>> (integer, significant
>>>>>>>>>>>>>>>>> only at root)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Gilles
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 2014/12/12 12:23, Alex A. Schmidt wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello Gilles,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks for your reply. The "env -i PATH=..." stuff seems to
>>>>>>>>>>>>>>>>> work!!!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> call system("sh -c 'env -i PATH=/usr/lib64/openmpi/bin:/bin
>>>>>>>>>>>>>>>>> mpirun -n 2
>>>>>>>>>>>>>>>>> hello_world' ")
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> did produce the expected result with a simple openmi
>>>>>>>>>>>>>>>>> "hello_world" code I
>>>>>>>>>>>>>>>>> wrote.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I might be harder though with the real third party app I have
>>>>>>>>>>>>>>>>> in mind. And
>>>>>>>>>>>>>>>>> I realize
>>>>>>>>>>>>>>>>> getting passed over a job scheduler with this approach might
>>>>>>>>>>>>>>>>> not work at
>>>>>>>>>>>>>>>>> all...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I have looked at the MPI_Comm_spawn call but I failed to
>>>>>>>>>>>>>>>>> understand how it
>>>>>>>>>>>>>>>>> could help here. For instance, can I use it to launch an mpi
>>>>>>>>>>>>>>>>> app with the
>>>>>>>>>>>>>>>>> option "-n 5" ?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2014-12-12 0:36 GMT-02:00 Gilles Gouaillardet
>>>>>>>>>>>>>>>>> <gilles.gouaillar...@iferc.org
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Alex,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> can you try something like
>>>>>>>>>>>>>>>>> call system(sh -c 'env -i /.../mpirun -np 2 /.../app_name')
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -i start with an empty environment
>>>>>>>>>>>>>>>>> that being said, you might need to set a few environment
>>>>>>>>>>>>>>>>> variables
>>>>>>>>>>>>>>>>> manually :
>>>>>>>>>>>>>>>>> env -i PATH=/bin ...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> and that being also said, this "trick" could be just a bad
>>>>>>>>>>>>>>>>> idea :
>>>>>>>>>>>>>>>>> you might be using a scheduler, and if you empty the
>>>>>>>>>>>>>>>>> environment, the
>>>>>>>>>>>>>>>>> scheduler
>>>>>>>>>>>>>>>>> will not be aware of the "inside" run.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> on top of that, invoking system might fail depending on the
>>>>>>>>>>>>>>>>> interconnect
>>>>>>>>>>>>>>>>> you use.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Bottom line, i believe Ralph's reply is still valid, even if
>>>>>>>>>>>>>>>>> five years
>>>>>>>>>>>>>>>>> have passed :
>>>>>>>>>>>>>>>>> changing your workflow, or using MPI_Comm_spawn is a much
>>>>>>>>>>>>>>>>> better approach.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Gilles
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 2014/12/12 11:22, Alex A. Schmidt wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Dear OpenMPI users,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regarding to this previous
>>>>>>>>>>>>>>>>> post<http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <http://www.open-mpi.org/community/lists/users/2009/06/9560.php>
>>>>>>>>>>>>>>>>> from 2009,
>>>>>>>>>>>>>>>>> I wonder if the reply
>>>>>>>>>>>>>>>>> from Ralph Castain is still valid. My need is similar but
>>>>>>>>>>>>>>>>> quite simpler:
>>>>>>>>>>>>>>>>> to make a system call from an openmpi fortran application to
>>>>>>>>>>>>>>>>> run a
>>>>>>>>>>>>>>>>> third party openmpi application. I don't need to exchange mpi
>>>>>>>>>>>>>>>>> messages
>>>>>>>>>>>>>>>>> with the application. I just need to read the resulting
>>>>>>>>>>>>>>>>> output file
>>>>>>>>>>>>>>>>> generated
>>>>>>>>>>>>>>>>> by it. I have tried to do the following system call from my
>>>>>>>>>>>>>>>>> fortran openmpi
>>>>>>>>>>>>>>>>> code:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> call system("sh -c 'mpirun -n 2 app_name")
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> but I get
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> **********************************************************
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Open MPI does not support recursive calls of mpirun
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> **********************************************************
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Is there a way to make this work?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> users mailing listus...@open-mpi.org
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Subscription:
>>>>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>>>>>>> Link to this post:
>>>>>>>>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25966.php
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> users mailing listus...@open-mpi.org
>>>>>>>>>>>>>>>>> Subscription:
>>>>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>>>>>>> Link to this
>>>>>>>>>>>>>>>>> post:http://www.open-mpi.org/community/lists/users/2014/12/25967.php
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> users mailing listus...@open-mpi.org
>>>>>>>>>>>>>>>>> Subscription:
>>>>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Link to this post:
>>>>>>>>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25968.php
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> users mailing listus...@open-mpi.org
>>>>>>>>>>>>>>>>> Subscription:
>>>>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>>>>>>> Link to this
>>>>>>>>>>>>>>>>> post:http://www.open-mpi.org/community/lists/users/2014/12/25969.php
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> users mailing listus...@open-mpi.org
>>>>>>>>>>>>>>>>> Subscription:
>>>>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>>>>>>> Link to this post:
>>>>>>>>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25970.php
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> users mailing list
>>>>>>>>>>>>>>>>> us...@open-mpi.org
>>>>>>>>>>>>>>>>> Subscription:
>>>>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>>>>>>> Link to this post:
>>>>>>>>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25971.php
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> users mailing list
>>>>>>>>>>>>>>> us...@open-mpi.org
>>>>>>>>>>>>>>> Subscription:
>>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>>>>> Link to this post:
>>>>>>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25974.php
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> users mailing list
>>>>>>>>>>>>>> us...@open-mpi.org
>>>>>>>>>>>>>> Subscription:
>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>>>> Link to this post:
>>>>>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25975.php
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> users mailing list
>>>>>>>>>>>>>> us...@open-mpi.org
>>>>>>>>>>>>>> Subscription:
>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>>>> Link to this post:
>>>>>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25978.php
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> users mailing list
>>>>>>>>>>>>> us...@open-mpi.org
>>>>>>>>>>>>> Subscription:
>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>>> Link to this post:
>>>>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25979.php
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> users mailing list
>>>>>>>>>>>> us...@open-mpi.org
>>>>>>>>>>>> Subscription:
>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>>> Link to this post:
>>>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25981.php
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> users mailing list
>>>>>>>>>>> us...@open-mpi.org
>>>>>>>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>>> Link to this post:
>>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25982.php
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> users mailing list
>>>>>>>>>> us...@open-mpi.org
>>>>>>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>>> Link to this post:
>>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/25991.php
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> users mailing list
>>>>>>>>> us...@open-mpi.org
>>>>>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>>> Link to this post:
>>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/26002.php
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> users mailing list
>>>>>>>> us...@open-mpi.org
>>>>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>> Link to this post:
>>>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/26003.php
>>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> users mailing list
>>>>>> us...@open-mpi.org
>>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>> Link to this post:
>>>>>> http://www.open-mpi.org/community/lists/users/2014/12/26010.php
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> users mailing list
>>>>> us...@open-mpi.org
>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>> Link to this post:
>>>>> http://www.open-mpi.org/community/lists/users/2014/12/26011.php
>>>>>
>>>>
>>>> _______________________________________________
>>>> users mailing list
>>>> us...@open-mpi.org
>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>> Link to this post:
>>>> http://www.open-mpi.org/community/lists/users/2014/12/26013.php
>>>>
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> us...@open-mpi.org
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> Link to this post:
>>> http://www.open-mpi.org/community/lists/users/2014/12/26014.php
>>>
>>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2014/12/26021.php
>
