Thanks Carsten,
I have started updating my jumpshot so will let you know as soon as I
have some ideas on whats going on.
G.
ps. I am going offline now for 2 days while travelling
On Wed, 4 Jan 2006, Carsten Kutzner wrote:
Hi Graham,
here are the all-to-all test results with the modification to the decision
routine you suggested yesterday. Now the routine behaves nicely for 128
and 256 float messages on 128 CPUs! For the other sizes one probably wants
to keep the original algorithm, since it is faster there. However I have
the feeling that for messages >= 4096 floats there still exists the old
problem since the execution times are so variable there (note that the
standard deviation rises by more than a factor of 10 when going from 2048
to 4096 floats.) If you need additional test results to tune the decision
functions please let me know.
Carsten
OMPI tuned all-to-all with modification:
========================================
mesg size time in seconds
#CPUs floats average std.dev. min. max.
128 1 0.001253 0.000074 0.001141 0.001470
128 2 0.023507 0.000563 0.022562 0.024761
128 4 0.023435 0.000426 0.022582 0.024166
128 8 0.023438 0.000359 0.022904 0.024104
128 16 0.023664 0.000438 0.022844 0.024670
128 32 0.024136 0.000463 0.023297 0.025117
128 64 0.024704 0.000535 0.023727 0.026030
128 128 0.025750 0.000525 0.024592 0.026799 *
128 256 0.028862 0.000683 0.027389 0.030168 *
128 512 0.035869 0.001214 0.034067 0.038655
128 1024 0.046528 0.001722 0.043549 0.050432
128 2048 0.072388 0.007032 0.066708 0.104358
128 4096 0.217678 0.097312 0.135113 0.409431
128 8192 0.378586 0.090267 0.297878 0.577771
128 16384 0.567473 0.105083 0.483573 0.735509
128 32768 1.151343 0.146547 0.937150 1.404478
128 65536 2.298998 0.169669 1.983286 2.572027
128 131072 4.070989 0.159958 3.691039 4.373587
OMPI tuned all-to-all:
======================
mesg size time in seconds
#CPUs floats average std.dev. min. max.
128 1 0.001288 0.000102 0.001077 0.001512
128 2 0.008391 0.000400 0.007861 0.009958
128 4 0.008403 0.000237 0.008095 0.009018
128 8 0.008228 0.000942 0.003801 0.008810
128 16 0.008503 0.000191 0.008233 0.008839
128 32 0.008656 0.000271 0.008084 0.009177
128 64 0.009085 0.000209 0.008757 0.009603
128 128 0.251414 0.073069 0.011547 0.506703 !
128 256 0.385515 0.127661 0.251431 0.578955 !
128 512 0.035111 0.000872 0.033358 0.036262
128 1024 0.046028 0.002116 0.043381 0.052602
128 2048 0.073392 0.007745 0.066432 0.104531
128 4096 0.165052 0.072889 0.124589 0.404213
128 8192 0.341377 0.041815 0.309457 0.530409
128 16384 0.507200 0.050872 0.492307 0.750956
128 32768 1.050291 0.132867 0.954496 1.344978
128 65536 2.213977 0.154987 1.962907 2.492560
128 131072 4.026107 0.147103 3.800191 4.336205
alternative all-to-all:
======================
128 1 0.012584 0.000724 0.011073 0.015331
128 2 0.012506 0.000444 0.011707 0.013461
128 4 0.012412 0.000511 0.011157 0.013413
128 8 0.012488 0.000455 0.011767 0.013746
128 16 0.012664 0.000416 0.011745 0.013362
128 32 0.012878 0.000410 0.012157 0.013609
128 64 0.013138 0.000417 0.012452 0.013826
128 128 0.014016 0.000505 0.013195 0.014942 +
128 256 0.015843 0.000521 0.015107 0.016725 +
128 512 0.052240 0.079323 0.027019 0.320653 !
128 1024 0.123884 0.121560 0.038062 0.308929 !
128 2048 0.176877 0.125229 0.074457 0.387276 !
128 4096 0.305030 0.121716 0.176640 0.496375 !
128 8192 0.546405 0.108007 0.415272 0.899858 !
128 16384 0.604844 0.056576 0.558657 0.843943 !
128 32768 1.235298 0.097969 1.094720 1.451241 !
128 65536 2.926902 0.312733 2.458742 3.895563 !
128 131072 6.208087 0.472115 5.354304 7.317153 !
---------------------------------------------------
Dr. Carsten Kutzner
Max Planck Institute for Biophysical Chemistry
Theoretical and Computational Biophysics Department
Am Fassberg 11
37077 Goettingen, Germany
Tel. +49-551-2012313, Fax: +49-551-2012302
eMail ckut...@gwdg.de
http://www.gwdg.de/~ckutzne
Thanks,
Graham.
----------------------------------------------------------------------
Dr Graham E. Fagg | Distributed, Parallel and Meta-Computing
Innovative Computing Lab. PVM3.4, HARNESS, FT-MPI, SNIPE & Open MPI
Computer Science Dept | Suite 203, 1122 Volunteer Blvd,
University of Tennessee | Knoxville, Tennessee, USA. TN 37996-3450
Email: f...@cs.utk.edu | Phone:+1(865)974-5790 | Fax:+1(865)974-8296
Broken complex systems are always derived from working simple systems
----------------------------------------------------------------------
