Hi Try with -dd 2 x 2 x 2 option with 8 nodes and check the performance. I too got the same problems, i could overcome with symmetrical -dd option.
Srinivas. On Fri, May 7, 2010 at 8:57 AM, irene farabella < i.farabe...@mail.cryst.bbk.ac.uk> wrote: > Hello! > > I am new to Gromacs and especially at parallel runs. I have some problem > running my system using domain decomposition. I apologize but it will > be a long mail... > > My system is made up by a membrane protein embedded is a mixed lipid > bilayer (POPE/POPG). > I tried to run it on 8 nodes but the simulation crashed due to "A charge > group moved too far between two domain decomposition steps" and > high % of load imbalance between nodes. I then tested the same run on a > single node and it worked. I tried then different amount of nodes, > changing DD (using mdrun -dd option) and it seems that more node I am > using the less the performance in terms of ns/day although the load > imbalance % is highly variable. During this test I found that the > optimal nodes for my system is running it on 6 node with a DD 3:2:1 (vol > 0.80 imb F 1% ) > > ---- from the log file: > > Initializing Domain Decomposition on 6 nodes > Dynamic load balancing: auto > Will sort the charge groups at every domain (re)decomposition > Initial maximum inter charge-group distances: > two-body bonded interactions: 0.703 nm, LJ-14, atoms 21579 21587 > multi-body bonded interactions: 2.038 nm, Proper Dih., atoms 20434 20405 > Minimum cell size due to bonded interactions: 2.241 nm > Maximum distance for 5 constraints, at 120 deg. angles, all-trans: 0.820 nm > Estimated maximum distance required for P-LINCS: 0.820 nm > Domain decomposition grid 3 x 2 x 1, separate PME nodes 0 > Domain decomposition nodeid 0, coordinates 0 0 0 > > Table routines are used for coulomb: TRUE > Table routines are used for vdw: FALSE > Will do PME sum in reciprocal space. > ... > > Making 2D domain decomposition grid 3 x 2 x 1, home cell index 0 0 0 > > Center of mass motion removal mode is Linear > We have the following groups for center of mass motion removal: > 0: Protein_POPE_POPG > 1: SOL_NA+_CL- > There are: 82711 Atoms > Charge group distribution at step 2000000: 5512 5533 5668 5694 5535 5626 > Grid: 7 x 10 x 14 cells > Initial temperature: 309.243 K > -------- > > Using this setting I finally managed to equilibrate my system by going > trough a > series of restrained runs. > > Surprisingly after 6,5 ns of non-restrained run (Step 3289500) the > simulation > crashes with : > "Fatal error: > A charge group moved too far between two domain decomposition steps > This usually means that your system is not well equilibrated ". > > It seems strange that it crashes only at 3289500 steps of a non-restrained > run. > I am now running a short run starting from a short while before the > crash step using a single processor and, as suspected, it is going > smoothly. My guess is that something is going wrong with the domain > decomposition of a such non-homogeneous system, considering that there > are also charged lipids that complicate it but I have no idea how to > solve/improve it. > > I am using gromacs-4.0.5. > > Any suggestion are welcome. > > -- > Irene Farabella > Wellcome Trust PhD student > > Institute of Structural and Molecular Biology > Department of of Biological Sciences > Birkbeck > University of London > Malet Street > London WC1E 7HX > Telephone +44 (0)20 7631 6815 > -- > gmx-users mailing list gmx-users@gromacs.org > http://lists.gromacs.org/mailman/listinfo/gmx-users > Please search the archive at http://www.gromacs.org/search before posting! > Please don't post (un)subscribe requests to the list. Use the > www interface or send it to gmx-users-requ...@gromacs.org. > Can't post? Read http://www.gromacs.org/mailing_lists/users.php > -- ********************************************* J. Srinivasa Rao Post-doctoral Research Associate C/o Prof. Luis R Cruz Cruz Computational Biophysics Group Department of Physics Drexel University 3141 Chestnut St Philadelphia, PA 19104, USA. Ph: Off: 215-895-1989 Mob: 704-706-4191 Web:http://jsrao.web.officelive.com/default.aspx **********************************************
-- gmx-users mailing list gmx-users@gromacs.org http://lists.gromacs.org/mailman/listinfo/gmx-users Please search the archive at http://www.gromacs.org/search before posting! Please don't post (un)subscribe requests to the list. Use the www interface or send it to gmx-users-requ...@gromacs.org. Can't post? Read http://www.gromacs.org/mailing_lists/users.php
