On 6/13/12 10:20 AM, ms wrote:
Hi,
I am trying to prepare a simple system for tests with CUDA. My guinea pig is the
lysozyme system from this tutorial:
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin/gmx-tutorials/lysozyme/01_pdb2gmx.html
but I prepared it using the AMBER99sb-ildn force field and the SPCE water model.
After extensive minimization steps, I started a (CPU) very short test run (3 ns)
to check if everything was ok before beginning to deal with CUDA.
However I found that there is a strong drift in energy and temperature:
>>>>>g_energy_d output:
Opened 1AKI_production_GPU.edr as double precision energy file
Select the terms you want from the following list by
selecting either (part of) the name or the number or a combination.
End your selection with an empty line or a zero.
-------------------------------------------------------------------
1 Bond 2 Angle 3 Proper-Dih. 4 Improper-Dih.
5 LJ-14 6 Coulomb-14 7 LJ-(SR) 8 Coulomb-(SR)
9 Coul.-recip. 10 Potential 11 Kinetic-En. 12 Total- Energy
13 Temperature 14 Pressure 15 Box-X 16 Box-Y
17 Box-Z 18 Volume 19 Density 20 pV
21 Enthalpy 22 Vir-XX 23 Vir-XY 24 Vir-XZ
25 Vir-YX 26 Vir-YY 27 Vir-YZ 28 Vir-ZX
29 Vir-ZY 30 Vir-ZZ 31 Pres-XX 32 Pres-XY
33 Pres-XZ 34 Pres-YX 35 Pres-YY 36 Pres-YZ
37 Pres-ZX 38 Pres-ZY 39 Pres-ZZ 40 #Surf*SurfTen
41 Box-Vel-XX 42 Box-Vel-YY 43 Box-Vel-ZZ 44 Mu-X
45 Mu-Y 46 Mu-Z
47 Coul-SR:Protein-Protein 48 LJ-SR:Protein-Protein
49 Coul-14:Protein-Protein 50 LJ-14:Protein-Protein
51 Coul-SR:Protein-non-Protein 52 LJ-SR:Protein-non-Protein
53 Coul-14:Protein-non-Protein 54 LJ-14:Protein-non-Protein
55 Coul-SR:non-Protein-non-Protein 56 LJ-SR:non-Protein-non-Protein
57 Coul-14:non-Protein-non-Protein 58 LJ-14:non-Protein-non-Protein
59 T-System
10
11
12
13
14
Last energy frame read 30000 time 5000.000
Statistics over 3000001 steps [ 2000.0000 through 5000.0000 ps ], 5 data sets
All statistics are over 1500001 points
Energy Average Err.Est. RMSD Tot-Drift
-------------------------------------------------------------------------------
Potential -529618 1500 3004.68 -10322.5 (kJ/mol)
Kinetic En. 86141.8 610 1295.7 -4294.67 (kJ/mol)
Total Energy -443476 2100 4220.26 -14617.1 (kJ/mol)
Temperature 292.49 2.1 4.39949 -14.5823 (K)
Pressure 1.02119 0.0046 133.601 -0.0134421 (bar)
You may want to use the -driftcorr flag in order to correct
for spurious drift in the graphs. Note that this is not
a substitute for proper equilibration and sampling!
WARNING: nmol = 1, this may not be what you want.
Temperature dependent fluctuation properties at T = 292.49.
The MDP is:
>>>>>production_GPU.mdp
; PREPROCESSING OPTIONS
title = production run 1 for GPU usage
include =
define =
; RUN CONTROL PARAMETERS
integrator = md ; for GPUs: "Option md is accepted but keep in
mind that the actual algorithm is not leap-frog."
tinit = 2000
dt = 0.001 ; 1 fs
nsteps = 3000000 ; 3 ns
init_step = 0
; CENTER OF MASS MOTION REMOVAL
nstcomm = 1
comm_mode = linear
comm_grps = protein non-protein
; OUTPUT CONTROL
nstxout = 2000 ; Doubled these because disk output is a strong
bottleneck apparently
nstvout = 2000
nstfout = 0
nstlog = 100
nstenergy = 100
nstxtcout = 100
nstcalcenergy = -1
xtcprecision = 1000
energygrps = protein non-protein
; NEIGHBOR SEARCHING PARAMETERS
nstlist = 2.
ns_type = grid
pbc = xyz
rlist = 1.0
; OPTIONS FOR ELECTROSTATICS AND VDW
coulombtype = PME ; this is OK for GPUs
rcoulomb_switch = 0.
rcoulomb = 1.0
epsilon_r = 1
vdwtype = cut-off
rvdw-switch = 0.
rvdw = 1.0
DispCorr = no
fourierspacing = 0.12
fourier_nx = 0
fourier_ny = 0
fourier_nz = 0
pme_order = 4
ewald_rtol = 1e-05
epsilon_surface = 0
optimize_fft = no
; TEMPERATURE COUPLING
tcoupl = andersen ; All values of this are equivalent to
"andersen" in GPU mode
tc_grps = system
tau_t = 0.1
ref_t = 300.00
; PRESSURE COUPLING
pcoupl = parrinello-rahman ; "OpenMM implements the Monte Carlo
barostat. All values for Pcoupl are thus accepted."
pcoupltype = isotropic
tau_p = 1.0
compressibility = 4.5e-05
ref_p = 1.0
; VELOCITY GENERATION
gen_vel = no
<<<<<<<<
Command lines are:
grompp_d -f production_GPU.mdp -c 1AKI_em4sol.gro -p topol.top -o
1AKI_production_GPU.tpr
Here, you're not preserving any of the previous state information. You're
picking up from 2 ns, but not passing a .cpt file to grompp - the previous state
is lost. Is that what you want? In conjunction with "gen_vel = no" I suspect
you could see some instabilities.
mpirun -np 8 mdrun_d -v -deffn 1AKI_production_GPU -s 1AKI_production_GPU.tpr
-g 1AKI_production_GPU.log -c 1AKI_production_GPU.gro -o 1AKI_production_GPU.trr
-g 1AKI_production_GPU.log -e 1AKI_production_GPU.edr
As an aside, proper use of -deffnm (not -deffn) saves you all of this typing :)
mpirun -np 8 mdrun_d -v -deffnm 1AKI_production_GPU
That's all you need.
I am using Gromacs 4.5.5 compiled in double precision.
I am very rusty with Gromacs, since I last dealt molecular dynamics more than 1
year ago :) , so probably I am missing something obvious. Any hint on where
should I look for to solve the problem? (Also, advice on if the .mdp is indeed
correct for CUDA simulations are welcome)
I see the same whenever I run on GPU, but my systems are always implicit
solvent. Do you get reasonable performance with an explicit solvent PME system
on GPU? I thought that was supposed to be really slow.
Do you observe similar effects on CPU? My tests have always indicated that
equivalent systems on CPU are far more stable (energetically and structurally)
than on GPU. I have never had any real luck on GPU. I get great performance,
and then crashes ;)
Maybe others can share their experiences.
-Justin
--
========================================
Justin A. Lemkul, Ph.D.
Research Scientist
Department of Biochemistry
Virginia Tech
Blacksburg, VA
jalemkul[at]vt.edu | (540) 231-9080
http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin
========================================
--
gmx-users mailing list gmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
