Brian,
unlike Ralph, i will assume all your processes are MPI tasks.
at first glance, the MPI philosophy is the other way around :
start with mpirun -np 1 traffic_cop, and then MPI_Comm_spawn("child")
when you need more workers.
that being said, if you are fine with having idle children (e.g.
children that consume no CPU resources, but do keep memory, network and
other system resources allocated), then you can start 256 mpi tasks,
either with
mpirun -np 256 cop_children
or
mpirun -np 1 traffic_cop : -np 255 children
/*i am not 100% sure about the syntax here ... */
there is no MPI way to signal a task, but you can have your children
wait for a message from the master.
unless you are using a TCP interconnect, i do no think OpenMPI is
production ready for MPI_THREAD_MULTIPLE,
so one option is to have your children MPI_Recv() information from the
traffic cop in the main process, and do the real job is an other pthread
(so the main process can kill the working thread when MPI_Recv returns)
an other option is to MPI_Irecv(), do the job and periodically
MPI_Test() to check if there is any order from the traffic cop.
Cheers,
Gilles
On 3/18/2016 8:38 AM, Ralph Castain wrote:
Hmmm….I haven’t heard of that specific use-case, but I have seen some
similar things. Did you want the processes to be paused, or killed,
when you scale down? Obviously, I’m assuming they are not MPI procs, yes?
I can certainly see a way to make mpirun do it without too much fuss,
though it would require a message as opposed to a signal so you can
indicate how many procs to “idle/kill”.
On Mar 17, 2016, at 3:22 PM, Andrus, Brian Contractor
<bdand...@nps.edu <mailto:bdand...@nps.edu>> wrote:
All,
I have an mpi-based program that has a master process that acts as a
‘traffic cop’ in that it hands out work to child processes.
I want to be able to dynamically throttle how many child processes
are in use at any given time.
For instance, if I start it with “mpirun -n 512” I could send a
signal to tell it to only use 256 of them for a bit and then tell it
to scale back up. The upper limit being the number of processes
mpirun was launched with.
Has anyone done anything like this? Maybe a better way to do it?
Basically my program is crunching through a large text file,
examining each line for various things.
Thanks in advance for any insight,
Brian Andrus
_______________________________________________
users mailing list
us...@open-mpi.org <mailto:us...@open-mpi.org>
Subscription:http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this
post:http://www.open-mpi.org/community/lists/users/2016/03/28744.php
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/03/28745.php
