Hi, Matthieu:
Thanks for your help.
Most of your ideas show that what I want to do.
My scheduler should be able to be called from any C++ program, which can put a
list of tasks to the scheduler and then the scheduler distributes the tasks to
other client nodes.
It may work like in this way:
while(still tasks available) { myScheduler.push(tasks);
myScheduler.get(tasks results from client nodes);}
My cluster has 400 nodes with Open MPI. The tasks should be transferred b y MPI
protocol.
I am not familiar with RPC Protocol.
If I use Boost.ASIO and some Python/GCCXML script to generate the code, it can
be called from C++ program on Open MPI cluster ?
I cannot find the skeletton on your blog.
Would you please tell me where to find it ?
I really appreciate your help.
Jack
June 20 2010
> Date: Sun, 20 Jun 2010 20:13:14 +0200
> From: matthieu.bruc...@gmail.com
> To: us...@open-mpi.org
> Subject: Re: [OMPI users] Open MPI task scheduler
>
> Hi Jack,
>
> What you are seeking is the client/server pattern. Have one node act
> as a server. It will create a list of tasks or even a graph of tasks
> if you have dependencies, and then create clients that will connect to
> the server with an RPC protocol (I've done this with a SOAP+TCP
> protocol, the severance of the TCP connection meaning that the client
> is dead and that its task should be recycled, ités easy to do with
> Boost.ASIO and some Python/GCCXML scripts to automatically generate
> your code, I've written a skeletton on my blog). You may even have
> clients with different sizes or capabilities and tell the server what
> each client can do, and then the server may dispatch appropriate
> tickets to the clients.
>
> Each client and server can be a MPI process, you don't have to create
> all clients inside one MPI process (you may use several if the
> smallest resource your batch scheduler allocates is bigger that one of
> your tasks). With a batch scheduler, it's better to allocate your
> tasks as small as possible so that you can balance the resources you
> need.
>
> Matthieu
>
> 2010/6/20 Jack Bryan <dtustud...@hotmail.com>:
> > Hi, all:
> > I need to design a task scheduler (not PBS job scheduler) on Open MPI
> > cluster.
> > I need to parallelize an algorithm so that a big problem is decomposed into
> > small tasks, which can be distributed
> > to other worker nodes by the Scheduler and after being solved, the results
> > of these tasks are returned to the manager node with the Scheduler, which
> > will distribute more tasks on the base of the collected results.
> > I need to use C++ to design the scheduler.
> > I have searched online and I cannot find any scheduler available
> > for this purpose.
> > Any help is appreciated.
> > thanks
> > Jack
> > June 19 2010
> > ________________________________
> > Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
> > Learn more.
> > _______________________________________________
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
>
>
>
> --
> Information System Engineer, Ph.D.
> Blog: http://matt.eifelle.com
> LinkedIn: http://www.linkedin.com/in/matthieubrucher
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
_________________________________________________________________
The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with
Hotmail.
http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_5
