Am 25.10.2012 um 13:43 schrieb Guillermo Marco Puche: > After reading more. I think the only thing i need is to setup my bash job > file. > And run it with mpiexec -n 2 /home/test/mytestjob.sh
It could be, but in general: no. First: check the MPI implementation the application was compiled with - better: linked to (if you compiled it on your own, this step is trivial of course). MPI is an API (application programming interface) and not an ABI (application binary interface). You can't start a compiled MPI application with Open MPI's `mpiexec` if it was compiled and linked to MPICH2 initially. (As even a serial application like `hostname` can be started by `mpiexec`, it could have the effect that the parallel application started with the wrong `mpiexec` will start, but only in serial in each granted slot on its own. Hence no execution speedup.) Second: the version of the used MPI implementation might be important, as even inside the same implementation the libraries might vary. In Open MPI for example you can use for a compiled application any newer libraries from an uneven and the next even version of Open MPI. From http://www.open-mpi.org/software/ompi/versions/ : Starting with v1.3.2, Open MPI provides forward ABI compatibility in all versions of a given feature release series and its corresponding super stable series. For example, on a single platform, an MPI application linked against Open MPI v1.3.2 shared libraries can be updated to point to the shared libraries in any successive v1.3.x or v1.4 release and still work properly (e.g., via the LD_LIBRARY_PATH environment variable or other operating system mechanism). Third: to achieve a tight integration into the queuing system (where all tasks on all nodes are under control of the queuing system), it mihgt be necessary: - compile Open MPI with "--with-sge" - in MPICH2 it's included by default (but I don't recall exactly with which version it started) - for other MPI implementations it might be necessary to assemble an adjusted "machinefile" as argument to `mpiexec` > So it will create 2 processes then each process will be scheduled by SGE. SGE will only grant the slots. The slave tasks will be put on the node(s) by `mpiexec`'s invocation. -- Reuti > After that that each pbwa will split up into the number of threads I've > specified in the own pbwa tool. > > I think i'm right but if you could correct me I would be so happy. > Found all that stuff over here: > > http://lists.mcs.anl.gov/pipermail/mpich-discuss/2011-July/010438.html > > Regards. > > El 25/10/2012 13:23, Guillermo Marco Puche escribió: >> $ ompi_info | grep grid >> >> Returns nothing. Like i said I'm newbie to MPI. >> I didn't know that I had to compile anything. I've Rocks installation out of >> the box. >> So MPI is installed but nothing more I guess. >> >> I've found an old thread in Rocks discuss list: >> https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/2012-April/057303.html >> >> >> User asking is using this script: >> >> > >> *#$ -S /bin/bash* >> >> > >> *#* >> >> > >> *#* >> >> > >> *# Export all environment variables* >> >> > >> *#$ -V* >> >> > >> *# specify the PE and core #* >> >> > >> *#$ -pe mpi 128* >> >> > >> *# Customize job name* >> >> > >> *#$ -N job_hpl_2.0* >> >> > >> *# Use current working directory* >> >> > >> *#$ -cwd* >> >> > >> *# Join stdout and stder into one file* >> >> > >> *#$ -j y* >> >> > >> *# The mpirun command; note the lack of host names as SGE will provide them >> >> > >> on-the-fly.* >> >> > >> *mpirun -np $NSLOTS ./xhpl >> xhpl.out* >> >> >> >> But then I read this: >> >> >> in rocks sge PE >> mpi is loosely integrated >> mpich and orte are tightly integrated >> qsub require args are different for mpi mpich with orte >> >> mpi and mpich need machinefile >> >> by default >> mpi, mpich are for mpich2 >> orte is for openmpi >> regards >> -LT >> >> >> The program I need to run is pBWA: >> http://pbwa.sourceforge.net/ >> >> It uses MPI. >> >> At this moment i'm kinda confused on which is the next step. >> >> I thought i just could run with MPI and a simple SGE job pBWA with multiple >> processes. >> >> Regards, >> Guillermo. >> >> >> El 25/10/2012 13:17, Reuti escribió: >>> Am 25.10.2012 um 13:11 schrieb Guillermo Marco Puche: >>> >>> >>>> Hello Reuti, >>>> >>>> I got stoned here. I've no idea what MPI library I've got. I'm using Rocks >>>> Cluster Viper 5.4.3 which comes out with Centos 5.6, SGE, SPM, OPEN MPI >>>> and MPI. >>>> >>>> How can i check which library i got installed? >>>> >>>> I found this: >>>> >>>> $ mpirun -V >>>> mpirun (Open MPI) 1.4.3 >>>> >>>> Report bugs to >>>> >>>> http://www.open-mpi.org/community/help/ >>> Good, and this one you also used to compile the application? >>> >>> The check whether Open MPI was build with SGE support: >>> >>> $ ompi_info | grep grid >>> MCA ras: gridengine (MCA v2.0, API v2.0, Component v1.6.2) >>> >>> -- Reuti >>> >>> >>> >>>> Thanks, >>>> >>>> Best regards, >>>> Guillermo. >>>> >>>> El 25/10/2012 13:05, Reuti escribió: >>>> >>>>> Am 25.10.2012 um 10:37 schrieb Guillermo Marco Puche: >>>>> >>>>> >>>>> >>>>>> Hello ! >>>>>> >>>>>> I found a new version of my tool which supports multi-threading but also >>>>>> MPI or OPENMPI for more additional processes. >>>>>> >>>>>> I'm kinda new to MPI with SGE. What would be the good command for qsub >>>>>> or config inside a job file to ask SGE to work with 2 MPI processes? >>>>>> >>>>>> Will the following code work in a SGE job file? >>>>>> >>>>>> #$ -pe mpi 2 >>>>>> >>>>>> That's supposed to make job work with 2 processes instead of 1. >>>>>> >>>>>> >>>>> Not out of the box: it will grant 2 slots for the job according to the >>>>> allocation rules of the PE. But how to start your application in the >>>>> jobscript inside the granted allocation is up to you. Fortunately the MPI >>>>> libraries got an (almost) automatic integration into queuing systems >>>>> nowadays without further user intervention. >>>>> >>>>> Which MPI library do you use when you compile your application of the >>>>> mentioned ones above? >>>>> >>>>> -- Reuti >>>>> >>>>> >>>>> >>>>> >>>>>> Regards, >>>>>> Guillermo. >>>>>> >>>>>> El 22/10/2012 17:19, Reuti escribió: >>>>>> >>>>>> >>>>>>> Am 22.10.2012 um 16:31 schrieb Guillermo Marco Puche: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> I'm using a program where I can specify the number of threads I want >>>>>>>> to use. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> Only threads and not additional processes? Then you are limited to one >>>>>>> node, unless you add something like >>>>>>> http://www.kerrighed.org/wiki/index.php/Main_Page or >>>>>>> http://www.scalemp.com >>>>>>> >>>>>>> >>>>>>> to get a cluster wide unique process and memory space. >>>>>>> >>>>>>> -- Reuti >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> I'm able to launch multiple instances of that tool in separate nodes. >>>>>>>> For example: job_process_00 in compute-0-0, job_process_01 in >>>>>>>> compute-1 etc.. each job is calling that program which splits up in 8 >>>>>>>> threads (each of my nodes has 8 CPUs). >>>>>>>> >>>>>>>> When i setup 16 threads i can't split 8 threads per node. So I would >>>>>>>> like to split them between 2 compute nodes. >>>>>>>> >>>>>>>> Currently I've 4 compute nodes and i would like to speed up the >>>>>>>> process setting 16 threads of my program splitting between more than >>>>>>>> one compute node. At this moment I'm stuck using only 1 compute node >>>>>>>> per process with 8 threads. >>>>>>>> >>>>>>>> Thank you ! >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Guillermo. >>>>>>>> _______________________________________________ >>>>>>>> users mailing list >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> [email protected] >>>>>>>> https://gridengine.org/mailman/listinfo/users >>>>>> _______________________________________________ >>>>>> users mailing list >>>>>> >>>>>> >>>>>> [email protected] >>>>>> https://gridengine.org/mailman/listinfo/users >> > _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
