Hello everybody
I am trying to run a hybrid mpi + openmp program in a cluster. I created a
queue with 14 machines, each one with 16 cores. The program divides the work
among the 14 processors with MPI and within each processor a loop is also
divided into 8 threads for example, using openmp. The problem is that when I
submit the job to the queue the MPI processes don't divide the work into
threads and the program prints the number of threads that are working within
each process as one.
I made a simple test program that uses openmp and I logged in one machine of
the fourteen. I compiled it using gfortran -fopenmp program.f -o exe, set the
OMP_NUM_THREADS environment variable equal to 8 and when I ran directly in the
terminal the loop was effectively divided among the cores and for example in
this case the program printed the number of threads equal to 8
This is my Makefile # Start of the makefile# Defining variablesobjects =
inv_grav3d.o funcpdf.o gr3dprm.o fdjac.o dsvd.o#f90comp =
/opt/openmpi/bin/mpif90f90comp = /usr/bin/mpif90#switch = -O3executable =
inverse.exe# Makefileall : $(executable)$(executable) : $(objects)
$(f90comp) -fopenmp -g -O -o $(executable) $(objects) rm $(objects)%.o: %.f
$(f90comp) -c $<# Cleaning everythingclean: rm $(executable) # rm
$(objects)# End of the makefile
and the script that i am using is
#!/bin/bash#$ -cwd#$ -j y#$ -S /bin/bash#$ -pe orte 14#$ -N job#$ -q new.q
export OMP_NUM_THREADS=8/usr/bin/time -f "%E" /opt/openmpi/bin/mpirun -v -np
$NSLOTS ./inverse.exe
am I forgetting something?
Thanks,
Oscar Fabian Mojica Ladino
Geologist M.S. in Geophysics
