On 9/13/06, imran shaik <sk.im...@yahoo.com> wrote:
I need to run parallel jobs on a cluster typically of size 600 nodes and running SGE, but the programmers are good at perl but not C or C++. So i thought of MPI, but i dont know whether it has perl support?
Hi Imran, SGE will dispatch process among the nodes of your cluster but it does not support interprocess communication, which MPI does. If your problem is easily splittable (like parse a large apache log, read a large xml list of things) you might be able to split the data and spawn as many process as you can. I do it using LSF (another dispatcher) and a Makefile that controls the dependencies and spawn the processes (using make's -j flag) and it works quite well. But if your job need the communication (like processing big matrices, collecting and distributing data among processes etc) you'll need an interprocess communication and that's what MPI is best at. In a nutshell, you'll need the runtime environment to run MPI programs as well as you need SGE's runtime environments on every node to dispatch jobs and collect information. About MPI bindings for Perl, there's this module: http://search.cpan.org/~josh/Parallel-MPI-0.03/MPI.pm but it's far too young to be trustworthy, IMHO, and you'll probably need the MPI runtime on all nodes as well... cheers, --renato
