Thanks for the response. May be I am wrong. However my argument is as follows: our test shows that a 100^3 grid per core performs 10 times faster (normalised in proper unit) than 200^3. Both of these sizes are not fitting in cache. 100^3 run is benefiting due to smaller size where compiler is guessing access pattern in slightly better way. So, in case of running one large job of 200^3 per core if I oversubscribe the core with smaller jobs of size comparable to 100^3, high saving due to better memory access should compensate for thread compition. Best, Santosh On Mon, Dec 26, 2011 at 10:31 PM, Matthieu Brucher <matthieu.bruc...@gmail.com> wrote: > Hi, > > If your problem is memory bound and if you don't use the whole memory > capacity of one node, it means that you are limited by your memory > bandwidth. In this case oversubscribing the number of processes will lead to > worse behavior, as all processes will fight for the same memory bandwidth. > > Just my opinion. > > Matthieu Brucher > > 2011/12/23 Santosh Ansumali <ansum...@gmail.com> >> >> Dear All, >> We are running a PDE solver which is memory bound. Due to >> cache related issue, smaller number of grid point per core leads to >> better performance for this code. Thus, though available memory per >> core is more than 2 GB, we are able to good performance by using >> less than 1 GB per core. >> >> I want to know whether oversubscribing the cores can potentially >> improve performance of such a code. My thinking is that if I >> oversubscribe the cores, each thread will be using less than 1 GB so >> cache related problems will be less severe. Is this logic correct or >> due to cache conflict performance will deteriorate further? >> In case, over-subscription can help, how shall I modify >> submission file (using sun grid engine) to enable over-subscription of >> cores? >> my current submission file is written as follows >> #!/bin/bash >> #$ -N first >> #$ -S /bin/bash >> #$ -cwd >> #$ -e $JOB_ID.$JOB_NAME.ERROR >> #$ -o $JOB_ID.$JOB_NAME.OUTPUT >> #$ -P faculty_prj >> #$ -p 0 >> #$ -pe orte 8 >> /opt/mpi/openmpi/1.3.3/gnu/bin/mpirun -np $NSLOTS ./test_vel.out >> >> Is it possible to allow over-subscription by modifying submission file >> itself? Or do I need to change hostfiles somehow? >> Thanks for your help! >> Best Regards >> Santosh Ansumali, >> Faculty Fellow, >> Engineering Mechanics Unit >> Jawaharlal Nehru Centre for Advanced Scientific Research (JNCASR) >> Jakkur, Bangalore-560 064, India >> Tel: + 91 80 22082938 >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users > > > > > -- > Information System Engineer, Ph.D. > Blog: http://matt.eifelle.com > LinkedIn: http://www.linkedin.com/in/matthieubrucher > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
-- Santosh Ansumali, Faculty Fellow, Engineering Mechanics Unit Jawaharlal Nehru Centre for Advanced Scientific Research (JNCASR) Jakkur, Bangalore-560 064, India Tel: + 91 80 22082938
