Re: [gmx-users] mdrun-gpu

[Szilárd Páll]

Dear Natalia,

The current mdrun-gpu (which btw uses OpenMM) is capable to run a
single simulation, on a single node, using a single GPU only.

Cheers,
--
Szilárd



On Fri, Mar 4, 2011 at 6:28 PM, Nathalia Garces  wrote:
> Hello,
> I want to know if it is possible to use "mdrun-gpu" with the command
> "-multi", meaning to use GPUs using paralell simulation.
> When I used it (mdrun-gpu -multi 2 -replex 2 ), it appears me an error shown
> bellow
>  Fatal error:
> This binary is compiled without MPI support, can not do multiple
> simulations.
>
> this bewilder me because I'm not sure I can use gpu and mpi at the same
> time.
>
>
> Thank you for four answer
>
> Nathalia
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Tweeking MDP file options to run on GPU

[Szilárd Páll]

Hi,

What is the error you are getting? What is unfortunate about
temperature coupling?

Have you checked out the part of the documentation, especially the
supported features on GPUs part
(http://www.gromacs.org/gpu#Supported_features)?

--
Szilárd



On Mon, Mar 7, 2011 at 3:43 PM, kala  wrote:
> Dear friends
>                 I am trying to run a ternary complex simulation using
> gromacs. so far the simulation is time taking on my dual-core maching
> 36hrs/ns. Fortunately or unfortunately I have a fermi graphics card wherein
> I can run the simulation quite fast. Now the unfortunate thing is the
> temperature coulpling. I am new to gmx and tweeking the mdp files is out of
> my head. I seek for advice for tweeking this mdp file (which i now use for
> cpu calculations)  for fast md using mdrun-gpu.
> My system
> 725 Amino acids
> 2 ligand molecules
> 1 co-ordinate zn Ion
>
> my MDP-file
>
> title   = Protein-ligand complex NVT equilibration
> ; Run parameters
> integrator  = md; leap-frog integrator
> nsteps  = 50; 2 * 50 = 1000 ps (1 ns)
> dt  = 0.002 ; 2 fs
> ; Output control
> nstxout = 0 ; suppress .trr output
> nstvout = 0 ; suppress .trr output
> nstenergy   = 1000  ; save energies every 2 ps
> nstlog  = 1000  ; update log file every 2 ps
> nstxtcout   = 1000  ; write .xtc trajectory every 2 ps
> energygrps  = Protein JZ4
> ; Bond parameters
> continuation= yes   ; first dynamics run
> constraint_algorithm = lincs; holonomic constraints
> constraints = all-bonds ; all bonds (even heavy atom-H bonds)
> constrained
> lincs_iter  = 1 ; accuracy of LINCS
> lincs_order = 4 ; also related to accuracy
> ; Neighborsearching
> ns_type = grid  ; search neighboring grid cells
> nstlist = 5 ; 10 fs
> rlist   = 0.9   ; short-range neighborlist cutoff (in nm)
> rcoulomb= 0.9   ; short-range electrostatic cutoff (in nm)
> rvdw= 1.4   ; short-range van der Waals cutoff (in nm)
> ; Electrostatics
> coulombtype = PME   ; Particle Mesh Ewald for long-range
> electrostatics
> pme_order   = 4 ; cubic interpolation
> fourierspacing  = 0.16  ; grid spacing for FFT
> ; Temperature coupling is on
> tcoupl  = V-rescale ; modified Berendsen thermostat
> tc-grps = Protein_JZ4 Water_and_ions; two coupling groups - more
> accurate
> tau_t   = 0.1   0.1 ; time constant, in ps
> ref_t   = 300   300 ; reference temperature, one for
> each group, in K
> ; Pressure coupling is off
> pcoupl  = Parrinello-Rahman ; pressure coupling is on for
> NPT
> pcoupltype  = isotropic ; uniform scaling of box vectors
> tau_p   = 2.0   ; time constant, in ps
> ref_p   = 1.0   ; reference pressure, in bar
> compressibility = 4.5e-5; isothermal compressibility of
> water, bar^-1
> ; Periodic boundary conditions
> pbc = xyz   ; 3-D PBC
> ; Dispersion correction
> DispCorr= EnerPres  ; account for cut-off vdW scheme
> ; Velocity generation
> gen_vel = yes   ; assign velocities from Maxwell distribution
> gen_temp= 300   ; temperature for Maxwell distribution
> gen_seed= -1; generate a random seed
>
> thanks and regards
>
> bharath
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] no output dgdl file

[Szilárd Páll]

> 1- I am wondering for each interval say
>
> Interval 2:
> init_lambda = 0.05
> foreign_lambda = 0 0.1
>
> how this foreign_lambda 0 is related to the init_lambda = 0 from interval 1
> and the same for foreign_lambda = 0.1 in interval 2 and init_lambda = 0.1 in
> interval 3? Is there a physical meaning?

Intermediate states are non-physical. For the details on how do
intermediate state contribute to the final free energy estimate, see
the original BAR paper [*].

> 2- If one needs to run multiple native lambda for both methods what is the
> adnatage of BAR over TI. I thought using foreign_lambda facilitates FE
> calculations in terms of amount of work but it seems it is only a different
> method (BAR).

Regarding 2), to summarize the advantage of BAR: it's been proved that
it's the best asymptotically unbiased method which the its greatest
strength.

In practice, unlike with TI where in order to sample efficiently you
need containment relationship between the "important" states of B wrt
the important states of A, for BAR good overlap is enough. Also, the
method itself provides with an estimate for overlap which gives a good
indication where you need more lambda points. Again, for details check
[*].

A few more points:
- You don't need all other states as foreign_lambda for a certain
lambda, but if you put them all there it won't hurt, but it will
increase your runtime. A rule of thumb often used is to use as foreign
lambdas a few lambda values around the respective lambda.
-  There is a very handy feature which makes possible writing the free
energy data to the binary edr files (separate_dhdl_file = no mdp
option). The advantages are much faster processing with g_bar and way
less disk usage (note that you can write energy frames relatively
rarely if you wish, while sampling the dhdl more often).


[*] Bennett, Efficient estimation of free energy differences from
Monte Carlo data; JCP (1976); 22:(2), 245-268.

--
Szilárd

>>
>>
>> Emanuel Birru wrote:
>>>
>>> Yeah, I am using IT and do analyse the result using another method not
>>> bar. But I used g_bar when I was using the foreign_lambda and simulate
>>> all in a single file. I have already sent my suspect few weeks back.
>>
>> Is there an active redmine issue?  I cannot find any post from you in the
>> list archive from the last few months.  What is the issue?
>>
>>> I am a bit confused on the g_bar part, when it says -f expects multiple
>>> dhdl files. Do we need to run still multiple independent simulations
>>> using different foreign_lambda values? I do not see why we should run
>>> independent simulations, if we use for couple-lambda0 and couple-lambda1
>>> vdw-q and none respectively.
>>>
>>
>> As I mentioned to the OP of this thread, simultaneous (de)coupling of vdW
>> and Coulombic interactions is not stable.  The output energies are not
>> trustworthy, in my experience due to physically unreasonable configurations
>> and the potential for numerical singularities.
>>
>> The multiple files that g_bar expects are not simply from two separate
>> processes, however.  It expects dhdl.xvg files from multiple values of
>> native lambda that have corresponding foreign_lambda values in it, i.e.:
>>
>> Interval 1:
>> init_lambda = 0
>> foreign_lambda = 0.05
>>
>> Interval 2:
>> init_lambda = 0.05
>> foreign_lambda = 0 0.1
>>
>> Interval 3:
>> init_lambda = 0.1
>> foreign_lambda = 0.05 0.15
>>
>> etc.
>>
>> -Justin
>>
>>> For IT I am using 4.5.3 and it is working good so far.
>>>
>>> Cheers,
>>>
>>> -Original Message-
>>> From: gmx-users-boun...@gromacs.org
>>> [mailto:gmx-users-boun...@gromacs.org] On Behalf Of Justin A. Lemkul
>>> Sent: Thursday, 24 March 2011 1:36 PM
>>> To: Gromacs Users' List
>>> Subject: Re: [gmx-users] no output dgdl file
>>>
>>>
>>>
>>> Emanuel Birru wrote:

 Hi Justin,

 It good that the issue is solved. As per my experience if you want to
>>>
>>> do

 a series of simulations it is not necessary to use foreign_lambda, for
 each simulation we can give different lambda values starting from 0 to
 1( from fully interactive to non-interactive) no need of foreign_lamda
>>>
>>> This is certainly a viable method, but it is thermodynamic integration,
>>> not BAR.   The purpose of lambda/foreign_lambda is to use the BAR method.
>>>
 and certainly no need of generating dhdl.xvg to calculate FE. I think
 -dhdl is an optional it will not be generated just because
>>>
>>> "free_energy

 = yes" is present. The problem with the 4.5.3 is not the problem of
>>>
>>> It has always been my experience (in versions 3.3.3 and 4.5.3) that if
>>> the free energy code is activated, this file is written.  I never specify
>>> the
>>> file names independently.
>>>
 generating dhdl data by using single simulation with all
>>>
>>> foreign_lambda

 values. The problem is with g_bar, when I tried to analyse the
>>>
>>> dhdl.xvg

 output (with all the necessary data in it) using g_bar, it do

Re: [gmx-users] no output dgdl file

[Szilárd Páll]

> I am a bit confused on the g_bar part, when it says -f expects multiple
> dhdl files. Do we need to run still multiple independent simulations
> using different foreign_lambda values? I do not see why we should run
> independent simulations, if we use for couple-lambda0 and couple-lambda1
> vdw-q and none respectively.
>

As Justin said, you do need multiple simulations. That is because you
need overlap between state lam_i lam_j to sample and estimate the dG
efficiently . If you only have two states, lambda 0 and 1 you will
most probably have a very poor overlap and therefor a poor dG
estimate.

g_bar expects you to provide a set of dhdl (with -f) or edr (with -e)
files corresponding to you individual simulations at the chosen lambda
values and for every lambda it needs the foreign lambdas provided in
the mdp file.

--
Szilárd


> -Original Message-
> From: gmx-users-boun...@gromacs.org
> [mailto:gmx-users-boun...@gromacs.org] On Behalf Of Justin A. Lemkul
> Sent: Thursday, 24 March 2011 1:36 PM
> To: Gromacs Users' List
> Subject: Re: [gmx-users] no output dgdl file
>
>
>
> Emanuel Birru wrote:
>> Hi Justin,
>>
>> It good that the issue is solved. As per my experience if you want to
> do
>> a series of simulations it is not necessary to use foreign_lambda, for
>> each simulation we can give different lambda values starting from 0 to
>> 1( from fully interactive to non-interactive) no need of foreign_lamda
>
> This is certainly a viable method, but it is thermodynamic integration,
> not BAR.
>  The purpose of lambda/foreign_lambda is to use the BAR method.
>
>> and certainly no need of generating dhdl.xvg to calculate FE. I think
>> -dhdl is an optional it will not be generated just because
> "free_energy
>> = yes" is present. The problem with the 4.5.3 is not the problem of
>
> It has always been my experience (in versions 3.3.3 and 4.5.3) that if
> the free
> energy code is activated, this file is written.  I never specify the
> file names
> independently.
>
>> generating dhdl data by using single simulation with all
> foreign_lambda
>> values. The problem is with g_bar, when I tried to analyse the
> dhdl.xvg
>> output (with all the necessary data in it) using g_bar, it doesn't
>> function properly. The error is related with the source code. I guess
>
> If you suspect a bug, you should report it.  If you don't use
> foreign_lambda,
> you can't use the BAR method.  You're doing TI, not BAR.  They are
> fundamentally
> different, and the analysis for TI is done independently of any Gromacs
> tool.
>
>> the main advantage of using 4.5.3 to calculated FE using
> foreign_lambda
>> was to avoid running series of independent simulations. May be it is
>> solved on the newer version 4.5.4., didn't try it yet.
>>
>
> Version 4.5.3 has worked just fine for all free energy calculations I
> have done.
>   Perhaps BAR can be done with a single simulation and multiple
> foreign_lambda,
> but that was not my understanding of the proper procedure (based on some
> posts
> by developers a few months ago).  Multiple simulations, each at "native"
> lambda,
> are conducted with values of foreign_lambda above and below the native
> value at
> some lambda spacing (except for end points).  Invoking g_bar gives the
> total
> DeltaG over all lambda intervals according to the BAR algorithm.
>
> -Justin
>
>> Cheers,
>>
>>
>>
>> -Original Message-
>> From: gmx-users-boun...@gromacs.org
>> [mailto:gmx-users-boun...@gromacs.org] On Behalf Of Justin A. Lemkul
>> Sent: Thursday, 24 March 2011 12:54 PM
>> To: Gromacs Users' List
>> Subject: Re: [gmx-users] no output dgdl file
>>
>>
>>
>> Emanuel Birru wrote:
>>> Hi Justin,
>>>
>>> Sure what you wrote is correct, what I am trying to tell to him is
>> that
>>> if he has more than one foreign lambda it is better to put all of
> them
>>> as it is not logical to use only one foreign lambda to calculate FE
> (0
>> -
>>> 0.1) and he doesn't have delta lambda too. -Deffnm is sure all about
>>
>> You can do a series of simulations at many values of lambda, each
>> specifying
>> their own foreign_lambda.  It seems to me that this method would be
> more
>>
>> reliable, but I have not tested simply attempting a single simulation
>> with all
>> values of foreign_lambda.  Using delta_lambda is (from all that I have
>> read) not
>> reliable, as there are known issues with "slow growth" methods.
>>
>>> file names but it also generate all the necessary files including the
>>> xvg's without the need of using -dhdl.
>>>
>>
>> Whether or not one specifies -dhdl or -deffnm is independent of
> whether
>> or not
>> dhdl.xvg (or whatever name) is written.  It is controlled purely by
> the
>> presence
>> of "free_energy = yes" in the input file.
>>
>>> I would appreciate if you come up with a new solution for him.
>>>
>>
>> As reported by the OP, this issue had been effectively solved already:
>>
>> http://lists.gromacs.org/pipermail/gmx-users/2011-March/059631.html
>>
>> -Justin
>>
>>> Cheers
>

Re: [gmx-users] problems when installing gromacs 4.5.3 or 4.5.4 with GPU's

[Szilárd Páll]

Hi Jordi,

I've never seen this error or anything similar, but I can give you hints.

The CMake build system first generates binaries in the build directory
which are also runnable from there (are linked against libs located in
the build three). When you do a "make install[-mdrun]", howerver,
these binaries get relinked to rewrite the library dependencies. lt
seems that in your case this last step fails.

What you can do is to try to run
$ make install-mdrun VERBOSE=1 ,
maybe you can see something on the verbose output. For reference I've
attached a verbose output of the above command I just generated.

Otherwise, you could set BUILD_SHARED_LIBS=OFF, case in which internal
Gromacs libraries will be linked statically.

Cheers,
--
Szilárd



2011/3/28 Jordi Inglés :
> Hi,
>
> I'm compiling gromacs with gcc compiler (v 4.3.2), cmake (2.8.3) and OpenMM
> 2.0 in a SLES 11.0 Linux.
>
> I follow the instructions in the page and it seems that the compilation goes
> fine.
>
> gpu01:/scratch/jingles/gcc/gromacs-4.5.3 # ls -ltra src/kernel/mdrun-gpu
> -rwxr-xr-x 1 root root 518294 Mar 28 10:16 src/kernel/mdrun-gpu
>
> But when I try to install the application I obtain this error:
>
> make install-mdrun
>
> -- Using default binary suffix: "-gpu"
> -- Using default library suffix: "_gpu"
> -- Using internal FFT library - fftpack
> -- Configuring done
> -- Generating done
> -- Build files have been written to: /scratch/jingles/gcc/gromacs-4.5.3
> Linking CXX static library libgmx_gpu_utils.a
> [  0%] Built target gmx_gpu_utils
> [ 78%] Built target gmx
> [ 92%] Built target md
> [ 98%] Built target gmxpreprocess
> [ 98%] Built target openmm_api_wrapper
> Linking CXX executable mdrun-gpu
> [100%] Built target mdrun
> Scanning dependencies of target install-mdrun
> [100%] Installing mdrun
> -- Install configuration: "Release"
> -- Install component: "libraries"
> CMake Error at
> /scratch/jingles/gcc/gromacs-4.5.3/src/gmxlib/cmake_install.cmake:38 (FILE):
>  file INSTALL cannot find
>  "/scratch/jingles/gcc/gromacs-4.5.3/src/gmxlib/CMakeFiles/CMakeRelink.dir/libgmx_gpu.so.6".
> Call Stack (most recent call first):
>  /scratch/jingles/gcc/gromacs-4.5.3/src/cmake_install.cmake:37 (INCLUDE)
>  /scratch/jingles/gcc/gromacs-4.5.3/cmake_install.cmake:40 (INCLUDE)
>
>
> make[3]: *** [src/kernel/CMakeFiles/install-mdrun] Error 1
> make[2]: *** [src/kernel/CMakeFiles/install-mdrun.dir/all] Error 2
> make[1]: *** [src/kernel/CMakeFiles/install-mdrun.dir/rule] Error 2
> make: *** [install-mdrun] Error 2
>
> It seems that CMakeRelink.dir it's empty (I supose that the own compilation
> cleans it). the libgmx_gpu.so.6 exist in her source directory:
>
> gpu01:/scratch/jingles/gcc/gromacs-4.5.3 # ls -ltra
> src/gmxlib/libgmx_gpu.so*
> -rwxr-xr-x 1 root root 3378435 Mar 28 10:11 src/gmxlib/libgmx_gpu.so.6
> lrwxrwxrwx 1 root root      15 Mar 28 10:11 src/gmxlib/libgmx_gpu.so ->
> libgmx_gpu.so.6
>
>
> Have you got any idea about the problem? I tried to compile with the
> --debut-output --debug-trycompile options of cmake that don't clean the
> compilation after finishing for checking that CMakeRelink.dir is empty or
> not, but it seems that still continues empty.
>
> Thanks for any advise.
>
> Jordi Inglés Camats
>
> --
>
> __
> Jordi Inglés Camats
> Institut de Química Teòrica Computacional (IQTCUB)
> Facultat de Física i Química
> Despatx 766
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the www interface
> or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>


make-install-VERBOSE.out
Description: Binary data
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] problems when installing gromacs 4.5.3 or 4.5.4 with GPU's

[Szilárd Páll]

Hi,

I've diff-ed the cache file you sent me against the one I generated
and I couldn't see anything relevant. Neither does the verbose CMake
output suggest anything useful, but that it tries to copy the
mdrun-gpu binary from a location where it obviously does not exist. I
don't even have this "CMakeRelink.dir" directory in my build tree.

I googled around a found the same issue being mentioned in a few
places, but couldn't figure out what the reason/solution is.

A few more things you could try if you haven't tried yet:
1) 4.5.4 (does it produce the same error?);
2) Run cmake in a _clean_ build directory;
3) Try a different (possibly earlier) CMake version, for me it works
fine with 2.8.0 and actually the latest version I used was AFAIR
2.8.2.

Cheers,
--
Szilárd



On Tue, Mar 29, 2011 at 8:52 AM, Jordi Inglés  wrote:
> Thanks Szilárd for your fast mail,
>
> but it seems that is still failing and don't appears any new error. With
> BUILD_SHARED_LIBS flag it seems that it pass the library error but I receive
> a similar error with the executable (see attached file) mdrun-gpu, but this
> executable exist in other directory:
>
> -rwxr-xr-x  1 root   root  3433307 Mar 29 08:31 mdrun-gpu
>
> I attach also the CMakeCache list if it can help.
>
> Thanks for any advise!
>
> jordi inglés
>
> El 28/03/11 21:50, Szilárd Páll escribió:
>>
>> Hi Jordi,
>>
>> I've never seen this error or anything similar, but I can give you hints.
>>
>> The CMake build system first generates binaries in the build directory
>> which are also runnable from there (are linked against libs located in
>> the build three). When you do a "make install[-mdrun]", howerver,
>> these binaries get relinked to rewrite the library dependencies. lt
>> seems that in your case this last step fails.
>>
>> What you can do is to try to run
>> $ make install-mdrun VERBOSE=1 ,
>> maybe you can see something on the verbose output. For reference I've
>> attached a verbose output of the above command I just generated.
>>
>> Otherwise, you could set BUILD_SHARED_LIBS=OFF, case in which internal
>> Gromacs libraries will be linked statically.
>>
>> Cheers,
>> --
>> Szilárd
>>
>>
>>
>> 2011/3/28 Jordi Inglés:
>>>
>>> Hi,
>>>
>>> I'm compiling gromacs with gcc compiler (v 4.3.2), cmake (2.8.3) and
>>> OpenMM
>>> 2.0 in a SLES 11.0 Linux.
>>>
>>> I follow the instructions in the page and it seems that the compilation
>>> goes
>>> fine.
>>>
>>> gpu01:/scratch/jingles/gcc/gromacs-4.5.3 # ls -ltra src/kernel/mdrun-gpu
>>> -rwxr-xr-x 1 root root 518294 Mar 28 10:16 src/kernel/mdrun-gpu
>>>
>>> But when I try to install the application I obtain this error:
>>>
>>> make install-mdrun
>>>
>>> -- Using default binary suffix: "-gpu"
>>> -- Using default library suffix: "_gpu"
>>> -- Using internal FFT library - fftpack
>>> -- Configuring done
>>> -- Generating done
>>> -- Build files have been written to: /scratch/jingles/gcc/gromacs-4.5.3
>>> Linking CXX static library libgmx_gpu_utils.a
>>> [  0%] Built target gmx_gpu_utils
>>> [ 78%] Built target gmx
>>> [ 92%] Built target md
>>> [ 98%] Built target gmxpreprocess
>>> [ 98%] Built target openmm_api_wrapper
>>> Linking CXX executable mdrun-gpu
>>> [100%] Built target mdrun
>>> Scanning dependencies of target install-mdrun
>>> [100%] Installing mdrun
>>> -- Install configuration: "Release"
>>> -- Install component: "libraries"
>>> CMake Error at
>>> /scratch/jingles/gcc/gromacs-4.5.3/src/gmxlib/cmake_install.cmake:38
>>> (FILE):
>>>  file INSTALL cannot find
>>>
>>>  "/scratch/jingles/gcc/gromacs-4.5.3/src/gmxlib/CMakeFiles/CMakeRelink.dir/libgmx_gpu.so.6".
>>> Call Stack (most recent call first):
>>>  /scratch/jingles/gcc/gromacs-4.5.3/src/cmake_install.cmake:37 (INCLUDE)
>>>  /scratch/jingles/gcc/gromacs-4.5.3/cmake_install.cmake:40 (INCLUDE)
>>>
>>>
>>> make[3]: *** [src/kernel/CMakeFiles/install-mdrun] Error 1
>>> make[2]: *** [src/kernel/CMakeFiles/install-mdrun.dir/all] Error 2
>>> make[1]: *** [src/kernel/CMakeFiles/install-mdrun.dir/rule] Error 2
>>> make: *** [install-mdrun] Error 2
>>>
>>> It seems that CMakeRelink.dir it's empty (I supose that the own
>>> compilation
>>> cleans it). the libgmx_gpu.so.

Re: [gmx-users] problems when installing gromacs 4.5.3 or 4.5.4 with GPU's

[Szilárd Páll]

Great! I would really appreciate if you could file a bug on
redmine.gromacs.org against the build system.

Thanks,
--
Szilárd



On Tue, Mar 29, 2011 at 2:27 PM, Jordi Inglés  wrote:
> Hi!
>
> tried with cmake 2.8.0 and it seems that is working
>
> Thanks for all Szilárd :)
>
> Jordi Inglés
>
> El 29/03/11 13:00, Szilárd Páll escribió:
>>
>> Hi,
>>
>> I've diff-ed the cache file you sent me against the one I generated
>> and I couldn't see anything relevant. Neither does the verbose CMake
>> output suggest anything useful, but that it tries to copy the
>> mdrun-gpu binary from a location where it obviously does not exist. I
>> don't even have this "CMakeRelink.dir" directory in my build tree.
>>
>> I googled around a found the same issue being mentioned in a few
>> places, but couldn't figure out what the reason/solution is.
>>
>> A few more things you could try if you haven't tried yet:
>> 1) 4.5.4 (does it produce the same error?);
>> 2) Run cmake in a _clean_ build directory;
>> 3) Try a different (possibly earlier) CMake version, for me it works
>> fine with 2.8.0 and actually the latest version I used was AFAIR
>> 2.8.2.
>>
>> Cheers,
>> --
>> Szilárd
>>
>>
>>
>> On Tue, Mar 29, 2011 at 8:52 AM, Jordi Inglés
>>  wrote:
>>>
>>> Thanks Szilárd for your fast mail,
>>>
>>> but it seems that is still failing and don't appears any new error. With
>>> BUILD_SHARED_LIBS flag it seems that it pass the library error but I
>>> receive
>>> a similar error with the executable (see attached file) mdrun-gpu, but
>>> this
>>> executable exist in other directory:
>>>
>>> -rwxr-xr-x  1 root   root  3433307 Mar 29 08:31 mdrun-gpu
>>>
>>> I attach also the CMakeCache list if it can help.
>>>
>>> Thanks for any advise!
>>>
>>> jordi inglés
>>>
>>> El 28/03/11 21:50, Szilárd Páll escribió:
>>>>
>>>> Hi Jordi,
>>>>
>>>> I've never seen this error or anything similar, but I can give you
>>>> hints.
>>>>
>>>> The CMake build system first generates binaries in the build directory
>>>> which are also runnable from there (are linked against libs located in
>>>> the build three). When you do a "make install[-mdrun]", howerver,
>>>> these binaries get relinked to rewrite the library dependencies. lt
>>>> seems that in your case this last step fails.
>>>>
>>>> What you can do is to try to run
>>>> $ make install-mdrun VERBOSE=1 ,
>>>> maybe you can see something on the verbose output. For reference I've
>>>> attached a verbose output of the above command I just generated.
>>>>
>>>> Otherwise, you could set BUILD_SHARED_LIBS=OFF, case in which internal
>>>> Gromacs libraries will be linked statically.
>>>>
>>>> Cheers,
>>>> --
>>>> Szilárd
>>>>
>>>>
>>>>
>>>> 2011/3/28 Jordi Inglés:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm compiling gromacs with gcc compiler (v 4.3.2), cmake (2.8.3) and
>>>>> OpenMM
>>>>> 2.0 in a SLES 11.0 Linux.
>>>>>
>>>>> I follow the instructions in the page and it seems that the compilation
>>>>> goes
>>>>> fine.
>>>>>
>>>>> gpu01:/scratch/jingles/gcc/gromacs-4.5.3 # ls -ltra
>>>>> src/kernel/mdrun-gpu
>>>>> -rwxr-xr-x 1 root root 518294 Mar 28 10:16 src/kernel/mdrun-gpu
>>>>>
>>>>> But when I try to install the application I obtain this error:
>>>>>
>>>>> make install-mdrun
>>>>>
>>>>> -- Using default binary suffix: "-gpu"
>>>>> -- Using default library suffix: "_gpu"
>>>>> -- Using internal FFT library - fftpack
>>>>> -- Configuring done
>>>>> -- Generating done
>>>>> -- Build files have been written to: /scratch/jingles/gcc/gromacs-4.5.3
>>>>> Linking CXX static library libgmx_gpu_utils.a
>>>>> [  0%] Built target gmx_gpu_utils
>>>>> [ 78%] Built target gmx
>>>>> [ 92%] Built target md
>>>>> [ 98%] Built target gmxpreprocess
>>>>> [ 98%] Built target openmm_api_wrapper
>>>>> Linking 

Re: [gmx-users] speed issue, gmx runtime estimate = f(t)

[Szilárd Páll]

Hi,

As Justin said, it's probably the imbalance which is causing the
slowdown. You can take a look at the statistics at the end of the log
file. One simple thing you could do is to compare the log file of your
long, gradually slowing down run with a shorter run to see which part
takes more time.

--
Szilárd



On Wed, Apr 6, 2011 at 3:26 PM, Michael Brunsteiner  wrote:
>
> Dear all,
>
> I am trying to optimize runtimes/speed for a series of
> calculations i plan to do with a rather complex system.
> When looking at runtimes I observed one peculiar issue:
> the estimated runtime written to stderr by mdrun with the "-v"
> option keepsgrowing ... in my experience this estimate normally
> converges to a fairly accurate value within a few hundred
> steps. not so here ..
> for example, starting a job at 12:14 gromacs writes to stderr:
>
> step 100, will finish Wed Apr  6 12:40:35 2011vol 0.97  imb F  1%
> step 200, will finish Wed Apr  6 12:39:42 2011vol 0.95  imb F  2%
> step 300, will finish Wed Apr  6 12:39:47 2011vol 0.95  imb F  1%
> step 400, will finish Wed Apr  6 12:39:52 2011vol 0.96  imb F  1%
> ...
> step 38400, will finish Wed Apr  6 14:09:11 2011vol 0.91  imb F 29%
>
> so the estimated run time grows steadily from 25 to 110 minutes
> within about one hour...
> this is mdrun_d (gmx-4.5.3) running on a linux PC with
> 2 processores/4 nodes/8 threads. There's a couple of other programs (X, 
> editor,
> ...)
> running at the same time but none of them uses a lot of CPU.
>
> Anyway, what i see lets me expect there might be a way
> to speed up these calculations. I'd appreciate any suggestions!
> (below is my mdp file.)
>
> cheers,
> Michael
>
>
> mdp file:
>
> integrator           = md
> ;
> dt                   = 0.001
> tinit                = 0
> nsteps               = 10
> nstxtcout            = 0
> nstxout              = 1000
> nstvout              = 0
> nstfout              = 0
> nstenergy            = 1000
> ;
> constraint_algorithm = lincs
> constraints          = hbonds
> ;
> comm_mode            = Linear
> nstcomm              = 1
> pbc                  = xyz
> nstlist              = 10
> ns_type              = grid
> rlist                = 1.4
> rcoulomb             = 1.4
> rvdw                 = 1.4
> ;
> coulombtype          = PME
> vdwtype              = User
> fourierspacing       = 0.2
> optimize_fft         = yes
> ;
> Tcoupl               = nose-hoover
> tc_grps              = System
> ref_t                = 300.0
> tau_t                = 0.4
> gen_vel              = yes
> gen_temp             = 240.0
> gen_seed             = 22
> ;
> pull                 = umbrella
> pull_geometry        = distance
> pull_dim             = N N Y
> pull_start           = yes
> pull_ngroups         = 1
> pull_group0          = f1
> pull_group1          = f2
> pull_rate1           = -0.015
> pull_k1              = 1000
> pull_nstxout         = 100
> pull_nstfout         = 100
> pull_pbcatom0        = 241
> pull_pbcatom1        = 482
> ;
> freezegrps           = c1 c2
> freezedim            = Y Y N Y Y N
> ;
> energygrps           = SOL CC CF
> energygrp_table      = CF CF  CF SOL  CF CC
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] no CUDA-capable device detected

[Szilárd Páll]

Hi Sebastian,

Are you sure that the NVIDIA driver works and is compatible with the
CUDA version you have? What does "nvidia-smi -L -a" output? Have you
tried running something, for instance an SDK sample?

Cheers,
--
Szilárd



2011/4/18 SebastianWaltz :
> Dear gromacs user,
>
> I have some problems getting a simulation running on a tesla T10 with
> CUDA driver version 3.20 running.
> My .mdp file looks like:
>
> ;
> title                    = ttt
> cpp                    =  /lib/cpp
> include               = -I../top
> constraints         =  none
> ;define                =  -DPOSRES            ; for possition restraints
> integrator          =  md-vv            ; leap-frog integrator
> emtol               =  100.0             ; max-force to converge
> emstep              =  0.005            ; initial step size
> dt                  =  0.002            ; ps !
> nsteps              =  50              ; total 2 ps
> nstcomm             =  5000            ; frequency for center of mass
> motion removal
>
> nstxout             =  500            ; frequency for writting the
> trajectory
> nstvout             =  500            ; frequency for writting the velocity
> nstfout             =  500            ; frequency to write forces to
> output trajectory
> nstlog              =  5000            ; frequency to write the log file
> nstenergy           =  5000
> nstxtcout           =  500            ; frequency to write energies to
> energy file
>
> nstlist             =  5            ; Frequency to update the neighbor list
> ns_type             =  grid            ; Make a grid in the box and only
> check atoms in neighboring grid cells when constructing a new neighbor
> rlist               =  1.2            ; cut-off distance for the
> short-range neighbor list
>
> coulombtype         =  PME            ; Fast Particle-Mesh Ewald
> electrostatics
> vdwtype            =  cut-off
> rcoulomb            =  1.2            ; cut-off distance for the coulomb
> field
> rvdw                =  1.2            ; cut-off distance for the vdw field
> fourierspacing      =  0.12            ; The maximum grid spacing for
> the FFT grid
> pme_order           =  4            ; Interpolation order for PME
> optimize_fft        =  yes
> pbc                =  xyz
>
> Tcoupl              =  andersen
> tc-grps             =  System
> tau_t               =  0.1
> ref_t               =  300
>
> energygrps          =  System
>
> Pcoupl              =  no
> tau_p               =  1
> compressibility     =  4.1e-5
> ref_p               =  1.0
>
> gen_vel             =  yes
> gen_temp            =  300
> gen_seed            =  171113
>
> I'm starting the simulation using the comand:
>
> mdrun-gpu -device 
> "OpenMM:Platform=CUDA,DeviceID=0,Memtest=15,Force-device=yes" -deffnm 
> GPU_test.tpr
>
> When I get the warning that OpenMM only works with the Andersen
> thermostat and so on, and stops with:
>
> Error getting CUDA device no CUDA-capable device detected.
>
> Every help is very welcome
>
> Sebastian
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Modified Gromacs and OPENMM

[Szilárd Páll]

Hi,

> Thanks for the reply. The part I modified is the implicit solvent
> part, particularly the Still model. Also I modified a part of nonbonded
> kernel, not sse ones. So I assumed I have to link GROMACS with OPENMM
> using customGBforce?

You mean that you modified the CUDA kernel inside OpenMM, right? If
you did that than it should be as simple as recompiling OpenMM and
using this version to link against mdrun-gpu. However, as you suggest
you can use the CustomGBForce class to define a GB model in a  general
way. For that you should look at the OpenMM wrapper module (see
below). I'm not entirely sure what you mean so part of this is
guessing...

> Is there a guide of how currently GROMACS is linked to OPENMM?

That's "fairly" straightforward. In runner.c:mdruner(), when an
OpenMM-enabled mdrun-gpu is used, all function pointers in the
integrator[] array simply point to md_openmm.c:do_md_openmm() (which
is a stripped-down version of md.c:do_md()). This contains the main MD
loop which besides all the Gromacs-magic essentially does:

openmm_init();
while(!bLastStep) {
openmm_copy_state();
openmm_take_one_step();
}
openmm_cleanup();

Finally, all the logic of interfacing with OpenMM is implemented in
src/kernel/openmm_wrapper.cpp, so I think that should be pretty much
the only place where you need to modify things to use
additional/custom OpenMM functionality in Gromacs.

Cheers,
--
Szilárd

> On Fri, Apr 29, 2011 at 3:08 PM, Justin A. Lemkul  wrote:
>>
>>
>> Chi-cheng Chiu wrote:
>>>
>>> Hi Gromacs users & developers,
>>> If I modified gromacs, will my modification works if I complied it with
>>> OPENMM and use GPU?
>>
>> Presumably, if you have written sound code that uses compatible functions
>> that are actually implemented in the OpenMM library, then yes.  If you want
>> a more specific answer, you'll have to be more specific about what "modified
>> Gromacs" actually means.
>>
>> -Justin
>>
>>> Thanks!
>>>
>>> Regards,
>>>
>>> Chi-cheng
>>>
>>
>> --
>> 
>>
>> Justin A. Lemkul
>> Ph.D. Candidate
>> ICTAS Doctoral Scholar
>> MILES-IGERT Trainee
>> Department of Biochemistry
>> Virginia Tech
>> Blacksburg, VA
>> jalemkul[at]vt.edu | (540) 231-9080
>> http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin
>>
>> 
>> --
>> gmx-users mailing list    gmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> Please don't post (un)subscribe requests to the list. Use the www
>> interface or send it to gmx-users-requ...@gromacs.org.
>> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] openmm 3.0, opencl support

[Szilárd Páll]

Hi Claus,

> be supported in future versions. Yet, in the openmm website,
> the new version of openmm (3.0 that is) is supposed to support both cuda and
> opencl framework alongside gromacs:
> (https://simtk.org/project/xml/downloads.xml?group_id=161)

What do you mean by "alongside gromacs"?

> 1) does gromacs support the opencl framework?

No.

> 3) if the answer to the 1st question is negative, could you please be more
> specific about the context "future"?

I'm afraid I can't. If there would be somebody willing to contribute
code + test & support it, it could work out in the foreseeable future.
Otherwise, I'm not sure.

--
Szilárd
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Fwd: Compiling issue : Compiling gromacs 4.5.4 with intel compiler suite 12.0

[Szilárd Páll]

Hi,

> Is there any common problem for an compilation with the intel compiler
> suite 12.0 and FFTW 3.2.2/Gromacs 4.5.4

Not that I know of, I've never had/head about such issues.

I just checked and I couldn't reproduce your issue. Tried with both
autoconf and CMake and with:
Intel Compilers: v12.0.2 20110112.
OS: Ubuntu 10.04.2 x86_64

I ran pdb2gmx on share/tutor/speptide/speptide.pdb located in the
install directory of Gromacs . Is your problem reproducible with all
available force-fields/water models?

I've no clue what could be causing the segfault, but I'd try the following:
- other ff/water model combinations;
- some other pdb inputs;
- binary built with CMake;
- as a last resort you could get a backtrace (build with debug symbols
and run with gdb).

Cheers,
--
Szilárd



On Sun, May 8, 2011 at 1:12 PM,   wrote:
> Dear GMX-users,
>
> in a first trial, I successfully managed to compile gromacs 4.5.4 and
> fftw 3.2.2 by following the installation instructions on the gromacs
> main page. For these compilation, I did not specify the CC CXX or F77
> (Compilation was accomplished by GNU compiler 4.5). I was forced to use
> --enable-shared for the configuration of FFTW, else the "make" command
> of gromacs stopped and showed the described common -fPIC error.
> At least, this led me to an compiled and functioning version.
>
> In a second step, I tried to compile FFTW 3.2.2 and Gromacs 4.5.4 with
> the intel compiler suite 12.0, which would probably suit my processors
> best. Both compilations finished without an error. However, the newly
> built pdb2gmx tool quits in the first line with a segmentation fault,
> while or after reading the *.pdb file:
>
> "Reading 1OMB.pdb..."
>
> The official gromacs tests showed segmentation faults too.
>
> For FFTW:
> "./configure --enable-threads --enable-float --enable-shared
> --prefix=/usr/gromacs_4_5_4/install_tests/fftw-3.2.2/fftw_fertig/ CC=icc
> F77=ifort CXX=icpc
> make -j4
> make install -j4"
>
> For Gromacs 4.5.4:
> "./configure --with-fft=fftw3
> --prefix=/usr/gromacs_4_5_4/install_tests/gromacs_fftw3
> CPPFLAGS="-I/usr/gromacs_4_5_4/install_tests/fftw-3.2.2/fftw_fertig/include"
> LDFLAGS="-L/usr/gromacs_4_5_4/install_tests/fftw-3.2.2/fftw_fertig/lib"
> CC=icc F77=ifort CXX=icpc
>
> make -j4
> make install -j4 "
>
> Is there any common problem for an compilation with the intel compiler
> suite 12.0 and FFTW 3.2.2/Gromacs 4.5.4
>
> thanks for all your support,
> M.Kalavera
> --
> Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
> belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Fwd: Compiling issue : Compiling gromacs 4.5.4 with intel compiler suite 12.0 SOLVED !

[Szilárd Páll]

Hi,

Good that managed to solve the issue!

Even though it might have been caused by a bug in the autoconf
scripts, as these are the final days for the autoconf support in
Gromacs,  I see a very slim change that it will get
investigated/fixed.

--
Szilárd



On Tue, May 10, 2011 at 4:19 PM,   wrote:
> Dear Gromacs user,
>
> my intel compiler issue got solved with a combination off autoconfiger and 
> cmake. In a first step, I used autoconfigure for FFTW-3.2.2 and solved my 
> compilation/segfault problems for gromacs-4.5.4 with cmake:
>
>
> ##For FFTW compilation ##
>
> ./configure --enable-threads --enable-float --enable-shared 
> --prefix=/usr/fftw-3.2.2/fftw_fertig/ CC=icc F77=ifort CXX=icpc
>
> make -j4
> make install
>
> ##For Gromacs compilation ##
>
> export CXX=icpc
> export CC=icc
>
> cmake -D CMAKE_INSTALL_PREFIX=/usr/gromacs_cmake -D 
> FFTW3F_INCLUDE_DIR=/usr/fftw-3.2.2/fftw_fertig/include -D 
> FFTW3F_LIBRARIES=/usr/fftw-3.2.2/fftw_fertig/lib/libfftw3f.a ../gromacs-4.5.4
>
> make -j4
> make install
>
> ##
>
>
> This worked fine for me and might be a solution for some people with similar 
> problems. Thanks for your kind support.
>
> M.Kalavera
>
> --
> Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
> belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs make error

[Szilárd Páll]

Just some hints:

- Does the target platform (32/64 bit) or the fftw libs and Gromacs
build match?

- Try to do a make clean first and (/or?) and reconfigure with
--enable-shared. Not sure that it will help, but I vaguely remember
something.

- Try CMake.

Cheers,
--
Szilárd



On Tue, May 10, 2011 at 9:03 AM, Sajad Ahrari  wrote:
> dear users
> while installing gromacs and after running ./configure, i tried to run the
> "make" command. but soon i was faced with an error witch is typed below. it
> seems that fftw libraries are not found. i downloaded and installed
> fftw3.2.2 as mentioned in gromacs website. and put the extracted files of
> gromacs and fftw both in /user/local directory. how should i come along this
> problem?
> thanks for your help!
> sajad
>
>
> /user/local/lib/libfftw3f .a; could not read symbols; bad value
> collect2: ld returned 1 exit status
> make[3]: *** [libmd.la] error 1
> make[3]: leaving directory '/user/local/gromacs-4.5.4/src/mdlib'
> make[2]: *** [all-recursive] error 1
> make[2]: leaving directory '/user/local/gromacs-4.5.4/src/mdlib'
> make[1]: *** [all-recursive] error 2
> make[1]: leaving directory '/user/local/gromacs-4.5.4/src'
> make: *** [all-recursive] error 1
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] P-LINCS and nstlist

[Szilárd Páll]

> nstlist is more directly related to dt, and is often connected to the force
> field, as well.  Values of 5-10 are standard.  Setting nstlist = 1 is
> usually only necessary for EM, not MD.  Excessive updating of the neighbor
> list can result in performance loss, I believe.

Not only can, but it surely will.

If you use nstlist=1 instead of say 10 which is a common value, you
essentially increase the neighbor search cost by a factor of 10. As
neighbor search contributes 6-8% to a typical PME runtime
(nstlist=10), that will be up to around 30-40% with nstlist=1.

--
Szilárd
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] RHEL OS, g_mdrun

[Szilárd Páll]

Hi,

You should really use fftw, it's *much* faster! (Btw: fftpack comes
with Gromacs.)

I'm not sure what versions of Gromacs are available through the RHEL
packages, but as installing from source is not very difficult
(especially for a sysadmin) you might be better of with getting a
fresh installation. You can find the instructions here:
http://www.gromacs.org/Downloads/Installation_Instructions

Cheers,
--
Szilárd



On Wed, May 18, 2011 at 10:24 PM, ram bio  wrote:
> Dear Justin,
>
> Thanks for the suggestion.
> Do we need to install FFTW libraries also while installing gromacs as
> I think FFT pack got installed as default when the gromacs is
> installed by the system admin using yum i.e. yum install gromacs.
>
>
> yum install gromacs_openmpi /
> yum install gromacs_mpich2,
>
> because when i asked my system administrator he mentioned that he
> installed mpich2 version of gromacs, but when i tried to locate
> g_mdrun_mpich2, i could not find it in the path, instead i found
> g_mdrun_openmpi and so i tried to run the job with g_mdrun_openmpi,
> but could not run the job.
>
> So it is difficult now to figure out why I could not use
> g_mdrun_openmpi, even though the file exists in the
> path:/usr/lib64/openmpi/bin/g_mdrun_openmpi
>
> Ram
>
> On Wed, May 18, 2011 at 3:00 PM, Justin A. Lemkul  wrote:
>>
>>
>> ram bio wrote:
>>>
>>> Dear Gromacs Users,
>>>
>>> I am trying to run a gromacs mdrun  job (pbs script) on the cluster
>>> (RHEL operating system) using the recently installed gromacs version
>>> 4.5.3, but  i think i could not run it paralleled on the nodes, as it
>>> is taking a long time to finish the job.
>>>
>>> The information regarding the  installed gromacs on the server is as
>>> follows:
>>>
>>> Scheduler: Moab/Torque
>>> Mpi Version: Mpich2
>>> OS: RHEL 5.5
>>> Compiler: gcc  version 4.4 enterprise linux 5
>>> FFT – FFT Pack
>>>
>>> I am running a batch job using the script as under:
>>>
>>> #PBS -S /bin/bash
>>> #PBS -N aarontest
>>> #PBS -o aaronout.txt
>>> #PBS -j oe
>>> #PBS -l nodes=2:ppn=4
>>> #PBS -l walltime=336:00:00
>>>
>>> #cat $PBS_NODEFILE
>>> NP=`wc -l < $PBS_NODEFILE`
>>> cd $PBS_O_WORKDIR
>>> #export MPI_GROUP_MAX=128
>>>
>>>
>>> /usr/local/bin/mpirun -np 8 /usr/bin/g_mdrun -nt 8 -deffnm testnpt3ns
>>> -v dlb yes -rdd 1.2
>>>
>>> Please let me know your suggestions for correcting the error.
>>>
>>
>> MPI and threading are mutually exclusive; you can't launch a multi-thread
>> process (mdrun -nt) via mpirun.  The mdrun executable must be MPI-enabled to
>> run on this system.  Perhaps the pre-compiled binary is not, in which case
>> you should install from source with --enable-mpi and --disable-threads.
>>
>> -Justin
>>
>> --
>> 
>>
>> Justin A. Lemkul
>> Ph.D. Candidate
>> ICTAS Doctoral Scholar
>> MILES-IGERT Trainee
>> Department of Biochemistry
>> Virginia Tech
>> Blacksburg, VA
>> jalemkul[at]vt.edu | (540) 231-9080
>> http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin
>>
>> 
>> --
>> gmx-users mailing list    gmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> Please don't post (un)subscribe requests to the list. Use the www interface
>> or send it to gmx-users-requ...@gromacs.org.
>> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Re: Re: Re: Gromacs Installation Error on Powerbook G4 Running OS 10.5.8

[Szilárd Páll]

Hi Matt,

Just wanted to warn you that AFAIK Gromacs 4.5.x was not tested with
gcc 4.0.1, we've done testing only with 4.1 and later. This doesn't
mean that it shouldn't work, but if you encounter anything strange,
the first thing you should do is to get a newer gcc.

Also, your tMPI errors might be related to gcc 4.0.1, but as the
Powerbook G4 has a single core that's irrelevant.

--
Szilárd



On Thu, May 26, 2011 at 1:31 AM, Matthew Bick
 wrote:
> Justin.  Thank you very much.  using --disable-threads fixed the problem.
>  Installation completed without error.  And by the way, my gcc version is
> 4.0.1.
> Matt
>
>
> On May 24, 2011, at 8:22 PM, gmx-users-requ...@gromacs.org wrote:
>
> 1. Re: Re: Gromacs Installation Error on Powerbook G4 Running    OS
>  10.5.8
>
> Matthew Bick wrote:
>
> On May 23, 2011, at 4:24 PM, gmx-users-requ...@gromacs.org
>
>  wrote:
>
> Re: Gromacs Installation Error on Powerbook G$ Running    OS
>
> 10.5.8
>
>
> Hi Justin.  Thanks for you response.  See my responses below, embedded
>
> in the original message:
>
> Matthew Bick wrote:
>
>
> Dear Gromacs community.  I am attempting to install the latest version
>
> of Gromcs (4.5.4) on my Mac Powerbook G4 (1.67 MHz, OS 10.5.8).  I
>
> have successfully installed FFTW following the instructions provided
>
> on the Gromacs installation page.
>
> "./configure --with-fft=fftw3" appears to work properly, but when I
>
> perform  "Make" I get the following errors:
>
> ld: symbol(s) not found
>
> collect2: ld returned 1 exit status
>
> make[4]: *** [libgmx.la] Error 1
>
> make[3]: *** [all-recursive] Error 1
>
> make[2]: *** [all-recursive] Error 1
>
> make[1]: *** [all] Error 2
>
> make: *** [all-recursive] Error 1
>
>
> |The information about the actual missing symbols should be just above this
>
> |output, so that would be useful information.  Also pertinent would be the
>
> |compiler versions you're using.  From what you've posted below it looks
>
> like
>
> |GCC, but which version?
>
> I am using the latest Developer's tools for Leopard 10.5.8.  So that
>
> comes with gcc 4.0 and 4.2.  I
>
>    can't say for certain which version was used during
>
> configuration.  I do know that when 'configuration'
>
>    looks for gcc it finds it.
>
>
> If you're not specifying a custom PATH, then your default gcc is being
> detected.
>  gcc -v will tell you which is being used.
>
>    Here are the missing symbols, plus the error codes once again:
>
> Undefined symbols:
>
> "___sync_lock_test_and_set", referenced from:
>
> _tMPI_Lock_trylock in lock.o
>
> _tMPI_Lock_lock in lock.o
>
> _tMPI_Lock_lock in lock.o
>
> _tMPI_Lock_lock in lock.o
>
> _tMPI_Lock_lock in lock.o
>
> _tMPI_Lock_lock in lock.o
>
> _tMPI_Lock_lock in lock.o
>
> _tMPI_Lock_lock in lock.o
>
> _tMPI_Lock_lock in lock.o
>
> _tMPI_Type_commit in type.o
>
> _tMPI_Type_contiguous in type.o
>
> "___sync_bool_compare_and_swap", referenced from:
>
> _tMPI_Stack_detach in list.o
>
> _tMPI_Stack_detach in list.o
>
> _tMPI_Stack_detach in list.o
>
> _tMPI_Stack_detach in list.o
>
> _tMPI_Stack_detach in list.o
>
> _tMPI_Stack_detach in list.o
>
> _tMPI_Stack_detach in list.o
>
> _tMPI_Stack_detach in list.o
>
> _tMPI_Stack_push in list.o
>
> _tMPI_Stack_push in list.o
>
> _tMPI_Stack_push in list.o
>
> _tMPI_Stack_push in list.o
>
> _tMPI_Stack_push in list.o
>
> _tMPI_Stack_push in list.o
>
> _tMPI_Stack_push in list.o
>
> _tMPI_Stack_push in list.o
>
> _tMPI_Stack_pop in list.o
>
> _tMPI_Once_wait in once.o
>
> _tMPI_Once in once.o
>
> _tMPI_Prep_send_envelope in p2p_protocol.o
>
> _tMPI_Prep_send_envelope in p2p_protocol.o
>
> _tMPI_Prep_send_envelope in p2p_protocol.o
>
> _tMPI_Prep_send_envelope in p2p_protocol.o
>
> _tMPI_Prep_send_envelope in p2p_protocol.o
>
> _tMPI_Prep_send_envelope in p2p_protocol.o
>
> _tMPI_Prep_send_envelope in p2p_protocol.o
>
> _tMPI_Prep_send_envelope in p2p_protocol.o
>
> _tMPI_Post_send in p2p_protocol.o
>
> _tMPI_Post_send in p2p_protocol.o
>
> _tMPI_Post_send in p2p_protocol.o
>
> _tMPI_Post_send in p2p_protocol.o
>
> _tMPI_Post_send in p2p_protocol.o
>
> _tMPI_Post_send in p2p_protocol.o
>
> _tMPI_Post_send in p2p_protocol.o
>
> _tMPI_Post_send in p2p_protocol.o
>
> _tMPI_Wait_process_incoming in p2p_protocol.o
>
> _tMPI_Wait_process_incoming in p2p_protocol.o
>
> _tMPI_Wait_process_incoming in p2p_protocol.o
>
> _tMPI_Wait_process_incoming in p2p_protocol.o
>
> "___sync_fetch_and_add", referenced from:
>
> _tMPI_Once_wait in once.o
>
> "___sync_lock_release", referenced from:
>
> _tMPI_Lock_unlock in lock.o
>
> _tMPI_Type_commit in type.o
>
> _tMPI_Type_contiguous in type.o
>
> "___sync_add_and_fetch", referenced from:
>
> _tMPI_Alltoallv in alltoall.o
>
> _tMPI_Alltoall in alltoall.o
>
> _tMPI_Barrier_wait in barrier.o
>
> _tMPI_Mult_recv in collective.o
>
> _tMPI_Mult_recv in collective.o
>
> _tMPI_Post_multi in collective.o
>
> _tMPI_Post_multi in collective.o
>
> _tMPI_Post_send in p2p_p

Re: [gmx-users] Error of installing gromacs v4.5.4

[Szilárd Páll]

Hi Hsin-Lin,

Your problem are caused by a missing header file which is included by
the nobonded SSE kernels which is indicated by the first error in your
output:

nb_kernel400_ia32_sse.c:22:23: emmintrin.h: No such file or directory

This header is needed for SSE and SSE2, but for some reason you don't
have it. What platform are you compiling on/for and what compiler are
you using?

Alternatively, you can turn off acceleration and you'll be able to
compile the code, but it will run *much* slower than with the
accelerated kernels.

--
Szilárd



2011/5/26 Hsin-Lin Chiang :
> Hi,
>
> I met some problem on installing gromacs 4.5.4
> Below is the log.
> I'm soory I can't distinguish the important part and I post all of them.
> I'll appreciate for any help.
>
> Hsin-Lin
> -
> Making all in include
> make[1]: Entering directory `/stathome/jiangsl/src/gromacs-4.5.4/include'
> Making all in .
> make[2]: Entering directory `/stathome/jiangsl/src/gromacs-4.5.4/include'
> make[2]: Nothing to be done for `all-am'.
> make[2]: Leaving directory `/stathome/jiangsl/src/gromacs-4.5.4/include'
> Making all in types
> make[2]: Entering directory
> `/stathome/jiangsl/src/gromacs-4.5.4/include/types'
> make[2]: Nothing to be done for `all'.
> make[2]: Leaving directory
> `/stathome/jiangsl/src/gromacs-4.5.4/include/types'
> Making all in thread_mpi
> make[2]: Entering directory
> `/stathome/jiangsl/src/gromacs-4.5.4/include/thread_mpi'
> Making all in atomic
> make[3]: Entering directory
> `/stathome/jiangsl/src/gromacs-4.5.4/include/thread_mpi/atomic'
> make[3]: Nothing to be done for `all'.
> make[3]: Leaving directory
> `/stathome/jiangsl/src/gromacs-4.5.4/include/thread_mpi/atomic'
> make[3]: Entering directory
> `/stathome/jiangsl/src/gromacs-4.5.4/include/thread_mpi'
> make[3]: Nothing to be done for `all-am'.
> make[3]: Leaving directory
> `/stathome/jiangsl/src/gromacs-4.5.4/include/thread_mpi'
> make[2]: Leaving directory
> `/stathome/jiangsl/src/gromacs-4.5.4/include/thread_mpi'
> make[1]: Leaving directory `/stathome/jiangsl/src/gromacs-4.5.4/include'
> Making all in src
> make[1]: Entering directory `/stathome/jiangsl/src/gromacs-4.5.4/src'
> make  all-recursive
> make[2]: Entering directory `/stathome/jiangsl/src/gromacs-4.5.4/src'
> Making all in gmxlib
> make[3]: Entering directory `/stathome/jiangsl/src/gromacs-4.5.4/src/gmxlib'
> Making all in nonbonded
> make[4]: Entering directory
> `/stathome/jiangsl/src/gromacs-4.5.4/src/gmxlib/nonbonded'
> Making all in nb_kernel_ia32_sse
> make[5]: Entering directory
> `/stathome/jiangsl/src/gromacs-4.5.4/src/gmxlib/nonbonded/nb_kernel_ia32_sse'
> /bin/sh ../../../../libtool --tag=CC   --mode=compile cc  -DHAVE_CONFIG_H
> -I. -I../../../../src -I/usr/include/libxml2 -I../../../../include
> -DGMXLIBDIR=\"/stathome/jiangsl/soft/gromacs-4.5.4/share/top\"
> -I/stathome/jiangsl/soft/fftw-3.2.2/include
> -I/stathome/jiangsl/soft/mpich2-1.2.1p1/include  -O3 -fomit-frame-pointer
> -finline-functions -Wall -Wno-unused -msse2 -funroll-all-loops -std=gnu99
> -pthread -I./include -MT nb_kernel400_ia32_sse.lo -MD -MP -MF
> .deps/nb_kernel400_ia32_sse.Tpo -c -o nb_kernel400_ia32_sse.lo
> nb_kernel400_ia32_sse.c
>  cc -DHAVE_CONFIG_H -I. -I../../../../src -I/usr/include/libxml2
> -I../../../../include
> -DGMXLIBDIR=\"/stathome/jiangsl/soft/gromacs-4.5.4/share/top\"
> -I/stathome/jiangsl/soft/fftw-3.2.2/include
> -I/stathome/jiangsl/soft/mpich2-1.2.1p1/include -O3 -fomit-frame-pointer
> -finline-functions -Wall -Wno-unused -msse2 -funroll-all-loops -std=gnu99
> -pthread -I./include -MT nb_kernel400_ia32_sse.lo -MD -MP -MF
> .deps/nb_kernel400_ia32_sse.Tpo -c nb_kernel400_ia32_sse.c  -fPIC -DPIC -o
> .libs/nb_kernel400_ia32_sse.o
> nb_kernel400_ia32_sse.c:22:23: emmintrin.h: No such file or directory
> In file included from nb_kernel400_ia32_sse.c:24:
> ../../../../include/gmx_sse2_single.h:39:34: emmintrin.h: No such file or
> directory
> In file included from nb_kernel400_ia32_sse.c:24:
> ../../../../include/gmx_sse2_single.h:128: syntax error before "__m128i"
> ../../../../include/gmx_sse2_single.h: In function `printxmmi':
> ../../../../include/gmx_sse2_single.h:132: warning: implicit declaration of
> function `_mm_storeu_si128'
> ../../../../include/gmx_sse2_single.h:132: `__m128i' undeclared (first use
> in this function)
> ../../../../include/gmx_sse2_single.h:132: (Each undeclared identifier is
> reported only once
> ../../../../include/gmx_sse2_single.h:132: for each function it appears in.)
> ../../../../include/gmx_sse2_single.h:132: syntax error before ')' token
> ../../../../include/gmx_sse2_single.h:133: `s' undeclared (first use in this
> function)
> ../../../../include/gmx_sse2_single.h: In function `gmx_mm_log_ps':
> ../../../../include/gmx_sse2_single.h:215: warning: implicit declaration of
> function `_mm_set_epi32'
> ../../../../include/gmx_sse2_single.h:215: can't convert between vector
> values of different size
> ../../../../include/gmx_sse2_single.h:2

Re: [gmx-users] Re: Error of installing gromacs v4.5.4

[Szilárd Páll]

Hi,

That compiler is ancient (thought it might have SSE2 support) as well
as the OS, I guess (RHEL 3?). Still, the CPU does support SSE2 so if
you can get a gcc 4.1 or later on it you should still be able compile
and run the code without a problem.

--
Szilárd



2011/5/26 Hsin-Lin Chiang :
> Hi,
>
> Thank you for your reply.
> I'm not so good in computer.
> I think the platform you ask me is "Linux", and kernel is 2.4.21-60.ELsmp
> The compiler is gcc v3.2.3 in the machine in my institute.
>
> Sincerely yours,
> Hsin-Lin
>> Message: 1
>> Date: Thu, 26 May 2011 18:03:09 +0200
>> From: Szil?rd P?ll 
>> Subject: Re: [gmx-users] Error of installing gromacs v4.5.4
>> To: Discussion list for GROMACS users 
>> Message-ID: 
>> Content-Type: text/plain; charset=ISO-8859-1
>>
>> Hi Hsin-Lin,
>>
>> Your problem are caused by a missing header file which is included by
>> the nobonded SSE kernels which is indicated by the first error in your
>> output:
>>
>> nb_kernel400_ia32_sse.c:22:23: emmintrin.h: No such file or directory
>>
>> This header is needed for SSE and SSE2, but for some reason you don't
>> have it. What platform are you compiling on/for and what compiler are
>> you using?
>>
>> Alternatively, you can turn off acceleration and you'll be able to
>> compile the code, but it will run *much* slower than with the
>> accelerated kernels.
>>
>> --
>> Szil嫫d
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Installation of gromacs-gpu on windows

[Szilárd Páll]

Dear Andrew,

Compiling on Windows was tested only using MSVC and I have no idea if
it works or not under cygwin. You should just try, both cmake and gcc
is available for cygwin so you might be lucky and get mdrun-gpu
compiled without any additional effort.

All binaries on the Gromacs webpage _are outdated_ which is clearly
stated on the respective page.

Cheers,
--
Szilárd



On Fri, Jun 24, 2011 at 9:54 AM, Андрей Гончар  wrote:
> Hello! I have a misunderstood about the installation of gpu-enabled
> gromacs under windows.
> I'll try to explain: in system requirements of gromacs-gpu it is wrote
> that Nvidia CUDA libraries have to be installed. But is it possible to
> do under cygwin? This quertion appears because we run gromacs under
> windows and so under cygwin.
> Is there a way to compile gromacs-gpu on cygwin? Will the binaries
> from gromacs-gpu page work and if so which of them to prefer to run
> under cygwin? Those for windows or for linux?
>
> --
>
> Andrew Gontchar
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] tesla vs gtx

[Szilárd Páll]

Hi,

Teslas should be superior when it comes to reliability, but otherwise,
for Gromacs 4.6 GeForce cards are perfectly fine and no Tesla-specific
features will provide performance benefits. The only exception is
GPUDirect for InfiniBand which might not work with consumer boards --
, although I don't see why it wouldn't, except if it's simply disabled
in the driver.

Here are the reasons (based on the list on the page you provided):
- Faster Double Precision Performance - not needed;
- Faster Data Transfers to/from CPU (with full duplex PCI-E) - will
not help in 4.6;
- Direct GPU to GPU communication - will not be used in 4.6;
- Larger Memory Size- not needed (simulation with millions of atoms
will run on a GeForce);
- Optimized for InfiniBand - not sure about it, but it might not work;
- Performance Drivers- simple solution: don't use Windows.

Cheers,
--
Szilárd



On Thu, Jan 5, 2012 at 11:46 AM, Andrzej Rzepiela
 wrote:
> Hey,
>
> From the previous posts on the list I got the feeling that gtx are fine
> enough for gromacs calculations and  4 times more expensive teslas are
> actually  not necessary. However the hardware providers discourage us from
> using gtx in a small cluster. The number of reasons is listed on the
> website http://www.fluidyna.de/en/tesla_geforce .  For the future versions
> of parallel gromacs are the listed points ( in particular  faster data
> transfers to/from CPU, faster GPU-GPU communication and less memory errors
> on teslas) relevant ?
>
> Thank you for comments
>
> Andrzej
>
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] mdrun-gpu error

[Szilárd Páll]

Hi,

Most of those are just warnings, the only error I see there comes from
the shell, probably an error in your script.

Cheers,
--
Szilárd



On Wed, Jan 18, 2012 at 12:27 PM, aiswarya pawar
 wrote:
> Hi users,
>
> Am running mdrun on gpu . I receive an error such as=
>
> WARNING: This run will generate roughly 38570 Mb of data
>
>
> WARNING: OpenMM does not support leap-frog, will use velocity-verlet
> integrator.
>
>
> WARNING: OpenMM supports only Andersen thermostat with the
> md/md-vv/md-vv-avek integrators.
>
>
> WARNING: OpenMM supports only Monte Carlo barostat for pressure coupling.
>
>
> WARNING: OpenMM provides contraints as a combination of SHAKE, SETTLE and
> CCMA. Accuracy is based on the SHAKE tolerance set by the "shake_tol"
> option.
>
> /bin/cat: file_loc: No such file or directory
>
>
> and the job is running but the nothing written into .xtc, .trr, .edr files .
> What could have gone wrong?
>
> --
> Aiswarya  B Pawar
>
> Bioinformatics Dept,
> Indian Institute of Science
> Bangalore
>
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] mdrun-gpu error

[Szilárd Páll]

And sorting out where the /bin/cat error comes from because that is
surely not a Gromacs message!

Cheers,
--
Szilárd



On Thu, Jan 19, 2012 at 1:12 PM, Mark Abraham  wrote:
> On 19/01/2012 8:45 PM, aiswarya pawar wrote:
>
> Mark,
>
> THe normal mdrun also hangs thus not generating any output.
>
>
> OK. It's your problem to solve... keep simplifying stuff until you can
> isolate a small number of possible causes. Top of the list is file system
> availability.
>
> Mark
>
>
>
> On Thu, Jan 19, 2012 at 3:09 AM, Mark Abraham 
> wrote:
>>
>> On 19/01/2012 2:59 AM, aiswarya.pa...@gmail.com wrote:
>>
>> Hi,
>>
>> Its going into the running mode but gets hang there for long hours which
>> generating any data in the output file. And am not able to figure out the
>> error file_doc. Please anyone knows what's going wrong.
>>
>>
>> No, but you should start trying to simplify what you're doing to see where
>> the problem lies. Does normal mdrun work?
>>
>> Mark
>>
>>
>> Thanks
>> Sent from my BlackBerry® on Reliance Mobile, India's No. 1 Network. Go for
>> it!
>>
>> -Original Message-
>> From: Szilárd Páll 
>> Sender: gmx-users-boun...@gromacs.org
>> Date: Wed, 18 Jan 2012 14:47:59
>> To: Discussion list for GROMACS users
>> Reply-To: Discussion list for GROMACS users 
>> Subject: Re: [gmx-users] mdrun-gpu error
>>
>> Hi,
>>
>> Most of those are just warnings, the only error I see there comes from
>> the shell, probably an error in your script.
>>
>> Cheers,
>> --
>> Szilárd
>>
>>
>>
>> On Wed, Jan 18, 2012 at 12:27 PM, aiswarya pawar
>>  wrote:
>>
>> Hi users,
>>
>> Am running mdrun on gpu . I receive an error such as=
>>
>> WARNING: This run will generate roughly 38570 Mb of data
>>
>>
>> WARNING: OpenMM does not support leap-frog, will use velocity-verlet
>> integrator.
>>
>>
>> WARNING: OpenMM supports only Andersen thermostat with the
>> md/md-vv/md-vv-avek integrators.
>>
>>
>> WARNING: OpenMM supports only Monte Carlo barostat for pressure coupling.
>>
>>
>> WARNING: OpenMM provides contraints as a combination of SHAKE, SETTLE and
>> CCMA. Accuracy is based on the SHAKE tolerance set by the "shake_tol"
>> option.
>>
>> /bin/cat: file_loc: No such file or directory
>>
>>
>> and the job is running but the nothing written into .xtc, .trr, .edr files
>> .
>> What could have gone wrong?
>>
>> --
>> Aiswarya  B Pawar
>>
>> Bioinformatics Dept,
>> Indian Institute of Science
>> Bangalore
>>
>>
>>
>> --
>> gmx-users mailing list    gmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-requ...@gromacs.org.
>> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-requ...@gromacs.org.
>> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>>
>>
>>
>>
>> --
>> gmx-users mailing list    gmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-requ...@gromacs.org.
>> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
>
>
>
> --
> Aiswarya  B Pawar
>
> Bioinformatics Dept,
> Indian Institute of Science
> Bangalore
>
>
>
>
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] mdrun-gpu error

[Szilárd Páll]

That's a generic GPU kernel launch failure which can mean anything,
from faulty hardware to bad driver to messed up installation.

Does the memory test run? Try to compile/install again and see if it works.

--
Szilárd



On Thu, Jan 19, 2012 at 4:18 PM, aiswarya pawar
 wrote:
> when i tired running it again, i got an error as
>
> Cuda error in file
> '/home/staff/sec/secdpal/software/openmm_tesla/platforms/cuda/./src/kernels//bbsort.cu'
> in line 176 : unspecified launch failure.
>
> /bin/cat: file_loc: No such file or directory
>
>
> On Thu, Jan 19, 2012 at 8:45 PM, aiswarya pawar 
> wrote:
>>
>> Has the tesla card got to do anything with the error. Am using Nvidia
>> Tesla S1070 1U server.
>>
>>
>> On Thu, Jan 19, 2012 at 8:37 PM, Szilárd Páll 
>> wrote:
>>>
>>> And sorting out where the /bin/cat error comes from because that is
>>> surely not a Gromacs message!
>>>
>>> Cheers,
>>> --
>>> Szilárd
>>>
>>>
>>>
>>> On Thu, Jan 19, 2012 at 1:12 PM, Mark Abraham 
>>> wrote:
>>> > On 19/01/2012 8:45 PM, aiswarya pawar wrote:
>>> >
>>> > Mark,
>>> >
>>> > THe normal mdrun also hangs thus not generating any output.
>>> >
>>> >
>>> > OK. It's your problem to solve... keep simplifying stuff until you can
>>> > isolate a small number of possible causes. Top of the list is file
>>> > system
>>> > availability.
>>> >
>>> > Mark
>>> >
>>> >
>>> >
>>> > On Thu, Jan 19, 2012 at 3:09 AM, Mark Abraham 
>>> > wrote:
>>> >>
>>> >> On 19/01/2012 2:59 AM, aiswarya.pa...@gmail.com wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> Its going into the running mode but gets hang there for long hours
>>> >> which
>>> >> generating any data in the output file. And am not able to figure out
>>> >> the
>>> >> error file_doc. Please anyone knows what's going wrong.
>>> >>
>>> >>
>>> >> No, but you should start trying to simplify what you're doing to see
>>> >> where
>>> >> the problem lies. Does normal mdrun work?
>>> >>
>>> >> Mark
>>> >>
>>> >>
>>> >> Thanks
>>> >> Sent from my BlackBerry® on Reliance Mobile, India's No. 1 Network. Go
>>> >> for
>>> >> it!
>>> >>
>>> >> -Original Message-
>>> >> From: Szilárd Páll 
>>> >> Sender: gmx-users-boun...@gromacs.org
>>> >> Date: Wed, 18 Jan 2012 14:47:59
>>> >> To: Discussion list for GROMACS users
>>> >> Reply-To: Discussion list for GROMACS users 
>>> >> Subject: Re: [gmx-users] mdrun-gpu error
>>> >>
>>> >> Hi,
>>> >>
>>> >> Most of those are just warnings, the only error I see there comes from
>>> >> the shell, probably an error in your script.
>>> >>
>>> >> Cheers,
>>> >> --
>>> >> Szilárd
>>> >>
>>> >>
>>> >>
>>> >> On Wed, Jan 18, 2012 at 12:27 PM, aiswarya pawar
>>> >>  wrote:
>>> >>
>>> >> Hi users,
>>> >>
>>> >> Am running mdrun on gpu . I receive an error such as=
>>> >>
>>> >> WARNING: This run will generate roughly 38570 Mb of data
>>> >>
>>> >>
>>> >> WARNING: OpenMM does not support leap-frog, will use velocity-verlet
>>> >> integrator.
>>> >>
>>> >>
>>> >> WARNING: OpenMM supports only Andersen thermostat with the
>>> >> md/md-vv/md-vv-avek integrators.
>>> >>
>>> >>
>>> >> WARNING: OpenMM supports only Monte Carlo barostat for pressure
>>> >> coupling.
>>> >>
>>> >>
>>> >> WARNING: OpenMM provides contraints as a combination of SHAKE, SETTLE
>>> >> and
>>> >> CCMA. Accuracy is based on the SHAKE tolerance set by the "shake_tol"
>>> >> option.
>>> >>
>>> >> /bin/cat: file_loc: No such file or directory
>>> >>
>>> >>
>>> >> and the job is running but th

Re: [gmx-users] BornSum: cudaMalloc in CUDAStream::Allocate failed out of memory

[Szilárd Páll]

HI Adam,


> I'm trying to run a mdrun-gpu simulation on a 64-bit Ubuntu system.
> I'm using a 15GB GTX 580 NVIDIA GPU card with all the appropriate drivers
> and cuda toolkit.
> However, when I run the command:
>
> mdrun-gpu -s inpufile.tpr -c inputfile.gro -x outputfile.xtc -g
> outputfile.log -nice - 0
>
> I get the following error message:
>
> BornSum: cudaMalloc in CUDAStream::Allocate failed out of memory
>
> What is the problem? Is my 15GB GPU card insufficient for my 138596 atom
> system?

Note that the 580 has 1.5 Gb, not 15! As the message says, most
probably you're running out of memory.

If you want to confirm this you could:
- Use the "nvidia-smi" tool (installed with the driver) which can
monitor GPUs and luckily the memory usage has not been disabled (yet)
by NVIDIA on consumer cards.
- Try a gradually smaller sytems, my wild guess is that the limit
should be somewhere around 120k.

Cheers,
--
Szilárd


> Appreciate any help,
> Adam
>
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Gromacs-GPU benchmark test killed after exhausting the memory

[Szilárd Páll]

Hi,

First I thought that there might be a memory leak which could have
caused this if you ran for really long time. However, I've just ran
the very same benchmark (dhfr with PME) for one hour, monitored the
memory usage and I couldn't see any change whatsoever (see the plot
attached).

I've attached a script I used to monitor the memory usage, feel free
to use if you want check again.

Cheers,
--
Szilárd



On Wed, Feb 29, 2012 at 9:50 AM, Efrat Exlrod  wrote:
> Hi,
>
> I have Gromacs-GPU version 4.5.5 and GTX 580.
> I ran dhfr-solv-PME benchmark test (see below) and my run was killed after
> couple of hours exhausting all the computer memory, including the swap (2G
> + 4G swap).
> Has anyone encountered this problem? What is wrong?
>
> Thanks, Efrat
>
>> mdrun-gpu -device
>> "OpenMM:platform=Cuda,memtest=15,deviceid=0,force-device=yes" -deffnm md
>  :-)  G  R  O  M  A  C  S  (-:
>
>    Great Red Oystrich Makes All Chemists Sane
>
>     :-)  VERSION 4.5.5  (-:
>
>     Written by Emile Apol, Rossen Apostolov, Herman J.C. Berendsen,
>   Aldert van Buuren, Pär Bjelkmar, Rudi van Drunen, Anton Feenstra,
> Gerrit Groenhof, Peter Kasson, Per Larsson, Pieter Meulenhoff,
>    Teemu Murtola, Szilard Pall, Sander Pronk, Roland Schulz,
>     Michael Shirts, Alfons Sijbers, Peter Tieleman,
>
>    Berk Hess, David van der Spoel, and Erik Lindahl.
>
>    Copyright (c) 1991-2000, University of Groningen, The Netherlands.
>     Copyright (c) 2001-2010, The GROMACS development team at
>     Uppsala University & The Royal Institute of Technology, Sweden.
>     check out http://www.gromacs.org for more information.
>
>  This program is free software; you can redistribute it and/or
>   modify it under the terms of the GNU General Public License
>  as published by the Free Software Foundation; either version 2
>  of the License, or (at your option) any later version.
>
>   :-)  mdrun-gpu  (-:
>
> Option Filename  Type Description
> 
>   -s md.tpr  Input    Run input file: tpr tpb tpa
>   -o md.trr  Output   Full precision trajectory: trr trj cpt
>   -x md.xtc  Output, Opt. Compressed trajectory (portable xdr
> format)
> -cpi md.cpt  Input, Opt.  Checkpoint file
> -cpo md.cpt  Output, Opt. Checkpoint file
>   -c md.gro  Output   Structure file: gro g96 pdb etc.
>   -e md.edr  Output   Energy file
>   -g md.log  Output   Log file
> -dhdlmd.xvg  Output, Opt. xvgr/xmgr file
> -field   md.xvg  Output, Opt. xvgr/xmgr file
> -table   md.xvg  Input, Opt.  xvgr/xmgr file
> -tablep  md.xvg  Input, Opt.  xvgr/xmgr file
> -tableb  md.xvg  Input, Opt.  xvgr/xmgr file
> -rerun   md.xtc  Input, Opt.  Trajectory: xtc trr trj gro g96 pdb cpt
> -tpi md.xvg  Output, Opt. xvgr/xmgr file
> -tpidmd.xvg  Output, Opt. xvgr/xmgr file
>  -ei md.edi  Input, Opt.  ED sampling input
>  -eo md.edo  Output, Opt. ED sampling output
>   -j md.gct  Input, Opt.  General coupling stuff
>  -jo md.gct  Output, Opt. General coupling stuff
> -ffout   md.xvg  Output, Opt. xvgr/xmgr file
> -devout  md.xvg  Output, Opt. xvgr/xmgr file
> -runav   md.xvg  Output, Opt. xvgr/xmgr file
>  -px md.xvg  Output, Opt. xvgr/xmgr file
>  -pf md.xvg  Output, Opt. xvgr/xmgr file
> -mtx md.mtx  Output, Opt. Hessian matrix
>  -dn md.ndx  Output, Opt. Index file
> -multidirmd  Input, Opt., Mult. Run directory
>
> Option   Type   Value   Description
> --
> -[no]h   bool   no  Print help info and quit
> -[no]version bool   no  Print version info and quit
> -niceint    0   Set the nicelevel
> -deffnm  string md  Set the default filename for all file options
> -xvg enum   xmgrace  xvg plot formatting: xmgrace, xmgr or none
> -[no]pd  bool   no  Use particle decompostion
> -dd  vector 0 0 0   Domain decomposition grid, 0 is optimize
> -npmeint    -1  Number of separate nodes to be used for PME, -1
>     is guess
> -ddorder enum   interleave  DD node order: interleave, pp_pme or
> cartesian
> -[no]ddcheck bool   yes Check for all bonded interactions with DD
> -rdd real   0   The maximum distance for bonded interactions
> with
>     DD (nm), 0 is determine from initial coordinates
> -rcon    real   0   Maximum distance for P-LINCS (nm), 0 is estimate
> -dlb enum   auto    Dynamic load balancing (with DD): auto, no or
> yes
> -dds real   0.8 Minimum allowed dlb scaling of the DD cell size
> -gcom  

Re: [gmx-users] Re: Gromacs-GPU benchmark test killed after exhausting the memory

[Szilárd Páll]

Hi Efrat,

It indeed looks like a memory leak.

Could you please file a bug on redmine.gromacs.org?

Cheers,
--
Szilárd



On Sun, Mar 4, 2012 at 12:21 PM, Efrat Exlrod  wrote:
> Hi Szilard,
>
> Thanks for your reply.
> I used your script and I think it does look as a memory leak. Please look at 
> the attached runchkmem.out
> Is it possible this problem exists in version 4.5.5 and was solved in version 
> 4.6 you are using?
> When will version 4.6 be released?
>
> Thanks, Efrat
>
>>Message: 2
>>Date: Fri, 2 Mar 2012 14:46:25 +0100
>>From: Szil?rd P?ll 
>>Subject: Re: [gmx-users] Gromacs-GPU benchmark test killed after
>>        exhausting      the memory
>>To: Discussion list for GROMACS users 
>>Message-ID:
>>        
>>Content-Type: text/plain; charset="iso-8859-1"
>>
>>Hi,
>>
>>First I thought that there might be a memory leak which could have
>>caused this if you ran for really long time. However, I've just ran
>>the very same benchmark (dhfr with PME) for one hour, monitored the
>>memory usage and I couldn't see any change whatsoever (see the plot
>>attached).
>>
>>I've attached a script I used to monitor the memory usage, feel free
>>to use if you want check again.
>>
>>Cheers,
>>--
>>Szil?rd
>>
>>
>>
>>On Wed, Feb 29, 2012 at 9:50 AM, Efrat Exlrod  wrote:
>>> Hi,
>>>
>>> I have?Gromacs-GPU version 4.5.5 and?GTX 580.
>>> I ran?dhfr-solv-PME benchmark test (see below) and my run was killed after
>>> couple of hours?exhausting all the computer memory, including the swap (2G
>>> +?4G swap).
>>> Has anyone encountered this problem? What?is wrong?
>>>
>>> Thanks, Efrat
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GROMACS stalls for NPT simulation when using -npme and -dd flags

[Szilárd Páll]

Hi,

That sounds like a memory leak. Could you please file a bug report in
redmine preferably with:
- configure options (+config.log or CMakeCache.txt would be ideal)
- compiler version, external library versions (fft, blas, lapack)
- input (tpr at least)
- full mdrun command line

Cheers,
--
Szilárd



On Tue, Mar 13, 2012 at 10:18 AM, Stephen Cox  wrote:
> My Gmail doesn't show the subject line when hitting reply, so apologies for
> the double post, but I thought I'd put it out there with an amended subject
> line in case it's of use to anyone else
>
>
>
> Hi,
>
> so further to my last email, I have run a quick test on my desktop machine
> (4 cores, 12Gb RAM). It seems that when running the parrinello-rahman
> barostat with domain decomposition (-dd 2 2 1) that I'm still getting
> memory leak (this time with GNU compilers). I followed proper equilibration
> with the berendsen barostat (although I didn't think this was the problem)
> and also added the "constraints = all-bonds" line, but this has made no
> difference to my results (I'm using "[ constraints ]" in my topology file).
>
> To give an idea of the rate of memory loss, initial memory consumption was
> 0.1% total memory per process, which rose steadily to 0.5% total memory
> after 5 minutes. After 17mins, memory consumption is 1.7% total memory and
> rising. Running with "-pd", memory usage is constant at 0.1% total memory.
>
> The system is 328 TIP4P/ice + 64 all-atomic methane. This problem has
> occurred for me on different architectures and with different compilers
> (and different system sizes). It would be good if anybody familiar with the
> source could take a look, or if anybody knows any compiler flags that would
> prevent memory leak.
>
> Thanks,
> Steve
>
>
>
> On 9 March 2012 13:32, Stephen Cox  wrote:
>
>>
>>
>> On 9 March 2012 13:03, gmx-users-requ...@gromacs.org <
>> gmx-users-requ...@gromacs.org> wrote:
>>
>>> Send gmx-users mailing list submissions to
>>>        gmx-users@gromacs.org
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>        http://lists.gromacs.org/mailman/listinfo/gmx-users
>>> or, via email, send a message with subject or body 'help' to
>>>        gmx-users-requ...@gromacs.org
>>>
>>> You can reach the person managing the list at
>>>        gmx-users-ow...@gromacs.org
>>>
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of gmx-users digest..."
>>>
>>>
>>> Today's Topics:
>>>
>>>   1. Re: GROMACS stalls for NPT simulation when using -npme    and
>>>      -dd flags (Mark Abraham)
>>>   2. Error Message in Make clean command for installation. (a a)
>>>   3. Re: Error Message in Make clean command for installation.
>>>      (Mark Abraham)
>>>
>>>
>>> --
>>>
>>> Message: 1
>>> Date: Fri, 09 Mar 2012 23:42:33 +1100
>>> From: Mark Abraham 
>>> Subject: Re: [gmx-users] GROMACS stalls for NPT simulation when using
>>>        -npme   and     -dd flags
>>> To: Discussion list for GROMACS users 
>>> Message-ID: <4f59fab9.6010...@anu.edu.au>
>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>
>>> On 9/03/2012 9:43 PM, Stephen Cox wrote:
>>> > Dear users,
>>> >
>>> > I'm trying to run an isotropic NPT simulation on a cubic cell
>>> > containing TIP4P/ice water and methane. I'm using the
>>> > Parrinello-Rahman barostat. I've been playing around with the
>>> > different decomposition flags of mdrun to get better performance and
>>> > scaling and have found that the standard -npme (half number of cores)
>>> > works pretty well. I've also tried using the -dd flags, and I appear
>>> > to get decent performance and scaling. However, after a few
>>> > nanoseconds (corresponding to about 3 hours run time), the program
>>> > just stalls; no output and no error messages. I realise NPT may cause
>>> > domain decompositon some issues if the cell vectors vary wildly, but
>>> > this isn't happening in my system.
>>> >
>>> > Has anybody else experienced issues with domain decomposition and NPT
>>> > simulations? If so, are there any workarounds? For the moment, I've
>>> > had to resort to using -pd, which is giving relatively poor
>>> > performance and scaling, but at least it isn't dying!
>>> >
>>> > I'm using GROMACS 4.5.5 with an intel compiler (I followed the
>>> > instructions online, with static linking) and using the command:
>>> >
>>> > #!/bin/bash -f
>>> > # ---
>>> > #$ -V
>>> > #$ -N test
>>> > #$ -S /bin/bash
>>> > #$ -cwd
>>> > #$ -l vf=2G
>>> > #$ -pe ib-ompi 32
>>> > #$ -q infiniband.q
>>> >
>>> > mpirun mdrun_mpi -cpnum -cpt 60 -npme 16 -dd 4 2 2
>>> >
>>> > Below is my grompp.mdp.
>>> >
>>> > Thanks,
>>> > Steve
>>> >
>>> > P.S. I think that there may be an issue with memory leak that occurs
>>> > for domain decomposition with NPT. I seem to remember seeing this
>>> > happening on my desktop and my local cluster. I don't see this with
>>> > NVT simu

Re: [gmx-users] Gromacs 4.5.5 GPU Error

[Szilárd Páll]

Hi,

Could you compile in Debug mode and run mdurn-gpu in gdb, it will tell
more about the location and type of exception.

--
Szilárd



On Mon, Mar 12, 2012 at 8:47 AM, TH Chew  wrote:
> Hi all,
>
> I am trying to get the GPU version of gromacs running. I manage to install
> OpenMM, compile and install mdrun-gpu. However when I ran mdrun-gpu on any
> of the benchmark files, I got this:
>
> ./run.sh: line 2: 14224 Floating point
> exception/usr/local/gromacs-4.5.5-cuda/bin/mdrun-gpu -v
>
> Any idea what goes wrong here?
>
> --
> Regards,
> THChew
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Announcement: Large biomolecule benchmark report

[Szilárd Páll]

I fully agree with David, it's great to have independent benchmarks!

In fact, already the the previous version of the report has been of
great use for us, we have referred to the results in a few occasions.

--
Szilárd



On Thu, Mar 15, 2012 at 2:37 PM, Hannes Loeffler
 wrote:
> Dear all,
>
> we proudly announce our third benchmarking report on (large)
> biomolecular systems carried out on various HPC platforms.  We have
> expanded our repertoire to five MD codes (AMBER, CHARMM, GROMACS,
> LAMMPS and NAMD) and to five protein and protein-membrane systems
> ranging from 20 thousand to 3 million atoms.
>
> Please find the report on
> http://www.stfc.ac.uk/CSE/randd/cbg/Benchmark/25241.aspx
> where we also offer the raw runtime data.  We also plan to release
> the complete joint benchmark suite at a later date (as soon as we
> have access to a web server with sufficient storage space).
>
> We are open to any questions or comments related to our reports.
>
>
> Kind regards,
> Hannes Loeffler
> STFC Daresbury
> --
> Scanned by iCritical.
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Scaling/performance on Gromacs 4

[Szilárd Páll]

Hi Sara,

The bad performance you are seeing is most probably caused by the
combination of the new AMD "Interlagos" CPUs, compiler, operating
system and it is very likely the the old Gromacs version also
contributes.

In practice these new CPUs don't perform as well as expected, but that
is partly due to compilers and operating systems not having full
support for the new architecture. However, based on the quite
extensive benchmarking I've done, the with such a large system should
be considerably better than what your numbers show.

This is what you should try:
- compile Gromacs with gcc 4.6 using the "-march=bdver1" optimization flag;
- have at least 3.0 or preferably newer Linux kernel;
- if you're not required to use 4.0.x, use 4.5.

Note that you have to be careful with drawing conclusions from
benchmarking on small number of cores with large systems; you will get
artifacts from caching effects.


And now a bit of fairly technical explanation, for more details ask Google ;)

The machine you are using has AMD Interlagos CPUs based on the
Bulldozer micro-architecture. This is a new architecture, a departure
from previous AMD processors and in fact quite different from most
current CPUs. "Bulldozer cores" are not the traditional physical
cores. In fact the hardware unit is the "module" which consists of two
"half cores" (at least when it comes to floating point units). and
enable a special type of multithreading called "clustered
multithreading". This is slightly similar to the Intel cores with
Hyper-Threading.


Cheers,
--
Szilárd



On Mon, Feb 20, 2012 at 5:12 PM, Sara Campos  wrote:
> Dear GROMACS users
>
> My group has had access to a quad processor, 64 core machine (4 x Opteron
> 6274 @ 2.2 GHz with 16 cores)
> and I made some performance tests, using the following specifications:
>
> System size: 299787 atoms
> Number of MD steps: 1500
> Electrostatics treatment: PME
> Gromacs version: 4.0.4
> MPI: LAM
> Command ran: mpirun -ssi rpi tcp C mdrun_mpi ...
>
> #CPUS          Time (s)   Steps/s
> 64             195.000     7.69
> 32             192.000     7.81
> 16             275.000     5.45
> 8              381.000     3.94
> 4              751.000     2.00
> 2             1001.000     1.50
> 1             2352.000     0.64
>
> The scaling is not good. But the weirdest is the 64 processors performing
> the same as 32. I see the plots from Dr. Hess on the GROMACS 4 paper on JCTC
> and I do not understand why this is happening. Can anyone help?
>
> Thanks in advance,
> Sara
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] 4.6 development version

[Szilárd Páll]

The docs actually tells you:

"Native GPU acceleration is supported with the verlet cut-off scheme
(not with the group scheme) with PME, reaction-field, and plain
cut-off electrostatics."
(http://www.gromacs.org/Documentation/Parallelization_and_acceleration#GPU_acceleration)

Use the cutoff-scheme = verlet mdp option, but before you start number
crunching make sure to read the docs carefully!

Cut-off schemes: http://www.gromacs.org/Documentation/Cut-off_schemes

--
Szilárd



On Thu, Mar 15, 2012 at 10:33 AM, SebastianWaltz
 wrote:
> Dear Gromacs user,
>
> since a few days we try to get the heterogeneous parallelization on a Dell
> blade server with Tesla M2090 GPUs to work using the worksheet on the page:
>
> http://www.gromacs.org/Documentation/Acceleration_and_parallelization
>
> we only get the OpenMM pure GPU version with mdrun-gpu running. Actually is
> the heterogeneous parallelization already working in the development version
> you can download using the link on the page:
>
>  http://www.gromacs.org/Developer_Zone/Roadmap/GROMACS_4.6
>
> and how can we get it running? Just adding the CMake variable GMX_GPU=ON
> when compiling mdrun did not enable the  heterogeneous parallelization.
>
> We want to use the heterogeneous parallelization used in the 4.6 version to
> find out which is the optimal GPU/CPU ratio for our studied systems, since
> we soon have to buy machines for the upgrade of our cluster.
>
> Thanks a lot
>
> yours
>
> Sebastian
>
>
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Vi plugin for Gromacs files

[Szilárd Páll]

Hi Hubert,

> With a coworker, we recently developed a plugin Vi to manipulate easily
> Gromacs files.
> It enables syntax highlighting for Gromacs files. For the moment, it works
> with mdp, gro, top/itp and ndx files.
> It contains also macros to comment/uncomment easily selections of a file.

That's pretty cool, thanks for sharing it!

> Further informations and download can be found in this GitHub repository:
> https://github.com/HubLot/vim-gromacs
> Any feedback will be appreciated.

I'll get back with feedback when I'll have used it a bit.

> I hope this is the right place to post it.

Yes, it is, but I'll forward your mail to the devel list just in case
if some devs have missed it.

We should also consider putting a link on the Gromacs page to your github page!

Cheers,
--
Szilárd


> Best Regards,
>
> --
> Hubert SANTUZ
> DSIMB Team
> UMR-S 665, INSERM-Paris Diderot, INTS
> 6 rue Alexandre Cabanel 75015 Paris, France
> phone : +33 1 44 49 31 53
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the www interface
> or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] About the function of "-v' in mdrun

[Szilárd Páll]

Just try and you'll see, it's not like -v is a dangerous command line option!
--
Szilárd



On Fri, Mar 30, 2012 at 1:10 AM, Acoot Brett  wrote:
> Dear All,
>
> There is a tutorial says the function of "-v" in mdrun is to make the md
> process visible in the screen, which is correct.
>
> However from the introduction of mdrun in the new version of gromacs, it
> says the function of "-v" is to make "noisy and loud".
>
> Will you please tell me what is the meaning of "noisy and loud" here?
>
> I am looking forward to getting your reply.
>
> Cheers,
>
> Acoot
>
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] large scale simulation?

[Szilárd Páll]

On a single quad 16-core Interlagos node with system size 34k atoms I
get 55-60 ns/day and even with 24k atoms ~70 ns/day (solvated protein
amber99 + tip3p) . On three of these nodes I think you can get close
to 100 ns/day, but it will be close to the scaling limit.

Have you considered vsites with larger timesteps?

Additionally, the new cut-off scheme which will come in 4.6 you can
get better scaling due to the more even nonbonded workload.

--
Szilárd



On Fri, Mar 30, 2012 at 7:02 PM, Albert  wrote:
> Hello:
>
>   I am wondering does anybody have experience with GROMACS for large scale
> simulation? I've heard lot of people said that it would be difficult for
> Gromacs to do so. eg: I've got a 60,000 atoms system, is it possible for
> GROMACS to produce 100 ns/days or even more? suppose I can use as much CPU
> as possible My recent experience for such system, Gromacs can only
> produce up to 20ns/day If I would like to produce 1000 ns, I have to
> wait for 50 days..
>
> thank you very much
>
> best
> A.
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] Re: GPU support on GROMACS

[Szilárd Páll]

Hi,

> thought you can help with this...

I'd appreciate if you kept future conversation on the mailing list.

> The GROMACS website says Geforce GTX 560 Ti is supported. How about GTX 560?

Yes, they are. All Fermi-class cards are supported.

--
Szilárd


> Thanks,
> G
>
> --
> Gaurav Goel, PhD
> Assistant Professor
> Department of Chemical Engineering
> Indian Institute of Technology, Delhi
> Hauz Khas, New Delhi 110016
> India
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Re: GROMACS (w. OpenMPI) fails to run with -np larger than 10

[Szilárd Páll]

On Wed, Apr 11, 2012 at 6:16 PM, Mark Abraham  wrote:
> On 12/04/2012 1:42 AM, haadah wrote:
>>
>> Could you clarify what you mean by "Sounds like an MPI configuration
>> program.
>> I'd get a test program running on 18 cores before worrying about anything
>> else."? My problem is that i can't get anything to work with -np set to
>> more
>> than 10.
>
>
> Only you can configure your hardware to use your MPI software correctly. You
> should use some simple non-GROMACS test program to probe whether it is
> working correctly. Only then is testing GROMACS a reasonable thing to do,
> given your existing observations.

That's a good point, test a (few) simple, but not "hello world"
trivial, program(s) first. If that works, it might be a GROMACS issue.

Above >10 processes separate PME nodes are used. Try the -npme 0
option to see if the error occurs without separate PME nodes on >10
processes.

Cheers,
--
Szilárd


> Mark
>
>
>>
>>
>> The “Cannot rename checkpoint file; maybe you are out of quota?” problem
>> is
>> fixed, i needed to set the NFS to sync instead of async.
>>
>>
>>
>> Mark Abraham wrote
>>>
>>> On 11/04/2012 11:03 PM, Seyyed Mohtadin Hashemi wrote:

 Hello,

 I have a very peculiar problem: I have a micro cluster with three
 nodes (18 cores total); the nodes are clones of each other and
 connected to a frontend via Ethernet. I am using Debian squeeze as the
 OS for all nodes.

 I have compiled a GROMACS v.4.5.5 environment with FFTW v.3.3.1 and
 OpenMPI v.1.4.2 (OpenMPI version that is standard for Debian). On the
 nodes, individually, I can do simulations of any size and complexity;
 however, as soon as I want to do a parallel job the whole thing crashes.
 Since my own simulations can be non-ideal for a parallel situation I
 have used the gmxbench d.dppc files, this is the result I get:

 For a simple parallel job I use: path/mpirun --hostfile
 path/machinefile --np XX path/mdrun_mpi --p tcp --s path/topol.tpr --o
 path/output.trr
 For --np XX being smaller than or 10 it works, however as soon as I
 make use of 11 or larger the whole thing crashes and I get:
 [host:] Signal: Bus error (7)
 [host:] Signal code: Non-existant physical address (2)
 [host]] Lots of lines with libmpi.so.0

 I have tried using different versions of OpenMPI, v.1.4.5 and all the
 way to beta v.1.5.5, they all behave exactly the same. This is making
 no sense.
>>>
>>> Sounds like an MPI configuration program. I'd get a test program running
>>> on 18 cores before worrying about anything else.
>>>
 When I use threads over the OpenMPI interface I can get all cores
 engaged and the simulation works until 5-7 minutes in then it gives an
 error "Cannot rename checkpoint file; maybe you are out of quota?"
 even though I have more than 500gb left on each node.
>>>
>>> Sounds like a filesystem availability problem. Checkpoint files are
>>> written in the working directory, so the available local disk space is
>>> not strictly relevant.
>>>
>>> Mark
>>>
 I hope somebody can help me figure out what is wrong and maybe a
 possible solution.

 regards,
 Mohtadin


>>>
>>> --
>>> gmx-users mailing list    gmx-users@
>>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>>> Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>>> Please don't post (un)subscribe requests to the list. Use the
>>> www interface or send it to gmx-users-request@.
>>> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>
>> --
>> View this message in context:
>> http://gromacs.5086.n6.nabble.com/GROMACS-w-OpenMPI-fails-to-run-with-np-larger-than-10-tp4832034p4832579.html
>> Sent from the GROMACS Users Forum mailing list archive at Nabble.com.
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the www interface
> or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] single X double precision

[Szilárd Páll]

2012/4/17 Pedro Alexandre de Araújo Gomes Lapido Loureiro :
> Hi,
>
> I've come across a discussion about the "single X double precision" issue in
> a NAMD users list e-mail
> (http://www.ks.uiuc.edu/Research/namd/mailing_list/namd-l/9166.html).
> I would like to know what do you think about these 3 specific points:
>
> 1) "there are some places where single precision (FFT, force/energy
> computation), where single precision can be applied, but in other
> places (summations) where even double precision may result in
> artefacts, due to limited numerical accuracy for large enough
> systems. there are some applications that show little sensitivity
> to single/double precision issues (homogeneous bulk lennard-jones
> systems), but others are notoriously difficult (systems with
> significant potential drops in one or two dimensions, e.g. metal
> surface slabs or lipid bilayers) since you don't have that much
> error cancellation anymore."
>
> 2) "if you store coordinates in single precision, you have to take
> extra precautions for handling very large systems, since you
> would store coordinates with different relative precision,
> depending on how close you are to the origin. e.g., with domain
> decomposition, you can define per domain offsets, but that
> would work only well in case of a large enough number of domains. "
>
> 3) "the best way to study the impact of single vs. double would be
> by running tests with the gromacs code. gromacs _can_ be compiled
> in single or double precision. for a proper comparison, you'll have
> to turn off the assembly innerloops though, since they may use single
> precision even in double precision mode."
>
> Specifically this last point startled me. In fact, is this true?

AFAIK no, the assembly (SSE intrinsic in 4.6) kernels have both single
and double versions. If double precision is requested that's always
respected in the code.

Additionally, single precision in GROMACS is in fact not *pure* single
precision, in some parts of the code where needed double precision is
used (e.g. constraints). Therefore, technically what we call single
corresponds to what some MD packages call mixed precision.

--
Szilárd


> Cheers,
>
> Pedro.
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] single X double precision

[Szilárd Páll]

Manual Appendix A.2; note that the last one among the reasons listed
for needing double will soon not be the case anymore (see verlet
scheme in 4.6 http://www.gromacs.org/Documentation/Cut-off_schemes).

--
Szilárd



2012/4/17 Pedro Alexandre de Araújo Gomes Lapido Loureiro :
> Thank you, Szilárd.
>
> Do you know where (say, in a piece of code or some documentation) I can be
> sure about that?
> Cheers,
>
> Pedro.
> Em 17 de abril de 2012 15:43, Szilárd Páll 
> escreveu:
>
>> 2012/4/17 Pedro Alexandre de Araújo Gomes Lapido Loureiro
>> :
>> > Hi,
>> >
>> > I've come across a discussion about the "single X double precision"
>> > issue in
>> > a NAMD users list e-mail
>> > (http://www.ks.uiuc.edu/Research/namd/mailing_list/namd-l/9166.html).
>> > I would like to know what do you think about these 3 specific points:
>> >
>> > 1) "there are some places where single precision (FFT, force/energy
>> > computation), where single precision can be applied, but in other
>> > places (summations) where even double precision may result in
>> > artefacts, due to limited numerical accuracy for large enough
>> > systems. there are some applications that show little sensitivity
>> > to single/double precision issues (homogeneous bulk lennard-jones
>> > systems), but others are notoriously difficult (systems with
>> > significant potential drops in one or two dimensions, e.g. metal
>> > surface slabs or lipid bilayers) since you don't have that much
>> > error cancellation anymore."
>> >
>> > 2) "if you store coordinates in single precision, you have to take
>> > extra precautions for handling very large systems, since you
>> > would store coordinates with different relative precision,
>> > depending on how close you are to the origin. e.g., with domain
>> > decomposition, you can define per domain offsets, but that
>> > would work only well in case of a large enough number of domains. "
>> >
>> > 3) "the best way to study the impact of single vs. double would be
>> > by running tests with the gromacs code. gromacs _can_ be compiled
>> > in single or double precision. for a proper comparison, you'll have
>> > to turn off the assembly innerloops though, since they may use single
>> > precision even in double precision mode."
>> >
>> > Specifically this last point startled me. In fact, is this true?
>>
>> AFAIK no, the assembly (SSE intrinsic in 4.6) kernels have both single
>> and double versions. If double precision is requested that's always
>> respected in the code.
>>
>> Additionally, single precision in GROMACS is in fact not *pure* single
>> precision, in some parts of the code where needed double precision is
>> used (e.g. constraints). Therefore, technically what we call single
>> corresponds to what some MD packages call mixed precision.
>>
>> --
>> Szilárd
>>
>>
>> > Cheers,
>> >
>> > Pedro.
>> >
>> >
>> > --
>> > gmx-users mailing list    gmx-users@gromacs.org
>> > http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > Please search the archive at
>> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> > Please don't post (un)subscribe requests to the list. Use the
>> > www interface or send it to gmx-users-requ...@gromacs.org.
>> > Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> --
>> gmx-users mailing list    gmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-requ...@gromacs.org.
>> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
>
>
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Gromacs 4.6 with CUDA 4.2

[Szilárd Páll]

On Wed, Apr 25, 2012 at 11:43 AM, SebastianWaltz
 wrote:
> Dear all,
>
> will the new version 4.6 work together with CUDA 4.2? Would be good to
> know, since this is needed for the new NVIDIA Gemini cards with Kepler
> technology.

Yes it will. I would like to point out that the "needed" part is not
exactly true, binaries compiled with pre-4.2 can run equally fast on
Kepler especially if JIT compilation of PTX code is enabled.

Moreover, current 4.2 pre-release compiler produces slower code than
CUDA 4.0 not only for Fermi but also for Kepler. We'll have to see
brins the final CUDA 4.2 release + official drivers.

--
Szilárd



> Thanks,
>
> Basti
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at 
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] some hardware questions

[Szilárd Páll]

On Wed, May 2, 2012 at 11:51 AM, melichercik  wrote:
> Hi people,
> we plan to buy some computer mostly for gromacs, so I want to ask you for
> some hints. I'd tried to do some research, but no success. :(
>
> Have someone ever compared AMD Bulldozer and Sandy/Ivy Bridge architectures?
> Are there planed any optimizations in gmx core in asm parts? (if there are
> need).
> Most interesting for me whould be to compare FX-8150 and i5-2550, I7-2600,
> i7-3570K or i7-3770 processors.

I can't comment on Ivy bridge, but when it comes to Sandy Bridge, with
the new NxX non-bonded kernels this is roughly the performance you
should expect:
- an FX-8150 should have a performance close to the SB i7-2600 (with
PME I've seen up to 10-15% advantage to the AMD)
- an SB-E will have slightly higher per core performance than SB, e.g.
in the benchmark I've done an i7-3930K was 1.6x faster than SB i7-2600
even though the latter has higher stock frequency.

> GPGPU - are there some real plans for supporting Radeon GPUs? Because OpenMP
> 3.x supports them - but I don't have any idea how complicate is to use newer
> OpenMP in GMX (and another work about it).

Not even OpenMP 3.1 supports GPUs. We will be considering OpenCL
support in the future and with it support for AMD GPUs, but don't
expect it anytime soon. However, rest assured that as soon as AMD GPUs
become more widely adopted on HPC, GROMACS will make use of them.

> And are there bigger differences (if someone knows) in use 1 Gbps onboard
> NIC vs. "better"? add-on NIC cards e.g. Intel Pro/1000 CT or similar? In MPI
> calculations with use 4 to 8 cores per computing node.

There are differences mostly when it comes to latency and reliability,
but I can't quantify it. However, instead of struggling with 1 Gb
ehternet you might want to consider sticking to single-node runs using
say a dual 16-core Interlagos setup.

Cheers,
Szilárd

> Thanks a lot.
>
> Milan Melichercik
> --
> gmx-users mailing list    gmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the www interface
> or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] some hardware questions

[Szilárd Páll]

On Wed, May 2, 2012 at 12:17 PM, Thomas Evangelidis wrote:

> For CPU performance comparison look at:
>
> www.spec.org/cpu2006/results
>
> for each CPU look at the statistics for floating point operations, namely
> for SPECfp_base2006 score for the programs of interest (GAMESS, GROMACS,
> NAMD, etc.).
>

That's not bad as a reference, but a) it doesn't use the latest GROMACS
version b) it shows performance one a single input with some settings that
will not reflect the production settings for everyone.


> For graphics cards you can look at the same site for GPU availability and
> memory...
>

Memory doesn't matter (much).

Cheers,
--
Szilárd


> HTH,
> Thomas
>
>
>
> On 2 May 2012 12:51, melichercik  wrote:
>
>> Hi people,
>> we plan to buy some computer mostly for gromacs, so I want to ask you for
>> some hints. I'd tried to do some research, but no success. :(
>>
>> Have someone ever compared AMD Bulldozer and Sandy/Ivy Bridge
>> architectures? Are there planed any optimizations in gmx core in asm parts?
>> (if there are need).
>> Most interesting for me whould be to compare FX-8150 and i5-2550,
>> I7-2600, i7-3570K or i7-3770 processors.
>>
>> GPGPU - are there some real plans for supporting Radeon GPUs? Because
>> OpenMP 3.x supports them - but I don't have any idea how complicate is to
>> use newer OpenMP in GMX (and another work about it).
>>
>> And are there bigger differences (if someone knows) in use 1 Gbps onboard
>> NIC vs. "better"? add-on NIC cards e.g. Intel Pro/1000 CT or similar? In
>> MPI calculations with use 4 to 8 cores per computing node.
>>
>>
>> Thanks a lot.
>>
>> Milan Melichercik
>> --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/**mailman/listinfo/gmx-users
>> Please search the archive at http://www.gromacs.org/**
>> Support/Mailing_Lists/Searchbefore
>>  posting!
>> Please don't post (un)subscribe requests to the list. Use the www
>> interface or send it to gmx-users-requ...@gromacs.org.
>> Can't post? Read 
>> http://www.gromacs.org/**Support/Mailing_Lists
>>
>
>
>
> --
>
> ==
>
> Thomas Evangelidis
>
> PhD student
>
> Biomedical Research Foundation, Academy of Athens
>
> 4 Soranou Ephessiou , 115 27 Athens, Greece
>
> email: tev...@bioacademy.gr
>
>   teva...@gmail.com
>
>
> website: https://sites.google.com/site/thomasevangelidishomepage/
>
>
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] some hardware questions

[Szilárd Páll]

On Wed, May 2, 2012 at 5:44 PM, Peter C. Lai  wrote:

> When considering compute hardware today, you're gonna have to shop like a
> gamer. A rig that can get high Battlefield 3 or Arma 2/3 framerates should
> also be able to get you some nice ns/day.
>

(To a great extent) wong! While many if not most games are GPU-bound and
not well multi-threaded, GROMACS uses highly efficient CPU code and can
make use of virtually any amount of CPU juice (i.e. extra cores and
frequency).


> I think most people consider Bulldozer mostly a flop. AMD masks lower
> cycle speeds with more cores per chip, but that just shifts the burden onto
> interprocessor communication, to which MPI is the most sensitive.
>

Partly true, but GROMACS has no issues whatsoever with scaling pretty much
linearly to any number of cores currently available per node  (with at
least 500 atoms/core).


> You can wait for ivy bridge, then stick some Kepler GPUs (nvidia gtx 680)
> in it. That should max the performance. asm is pretty much stagnant for
> general purpose procs since core2 came out.
>

AFAIK IB doesn't improve much on flop/s performance over SB, but it's a
power efficiency king. However, it's not at all sure that when it comes to
performance/$ AMD doesn't win. Also, Kepler might not be the best choice,
we'll see what can we get out with some more tuning.

What do you mean by asm?


> Keep an eye on the I7-3770K. It has the most cache and fastest core speed
> of the intel ones. The K chips are preferred for adventurous overclocking.
> Although hyperthreading and onboard intel gpu is useless for hpc. The
> I5-3570K is slower and has 2M less cache - it's basically an ivy bridge
> drop in replacement for the I5-2500K (just pci-e 3.0 upgrade)
>

Ehmm, HT *is* useful, it will boost GROMACS performance by up to 10-15% or
so.


> gmx 4.6 is supposed come with native gpu support; you should look at some
> list archives to see what is planned to be supported. Waiting for 4.6 to
> come out is going to extend your wait on hardware considerations, but it
> also means prices should drop.
>
> for nics find the best/most compatible tcp offloading engine, also while
> using jumbo frames. Personally I've always found Intel better than
> Broadcom. Stay away from anything Realtek. If you can, try to begin to look
> at going to fiber earlier (with an eye on splitting out storage from
> compute).
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>
>
> melichercik  wrote:
>>
>> Hi people,
>> we plan to buy some computer mostly for gromacs, so I want to ask you
>> for some hints. I'd tried to do some research, but no success. :(
>>
>> Have someone ever compared AMD Bulldozer and Sandy/Ivy Bridge
>> architectures? Are there planed any optimizations in gmx core in asm
>> parts? (if there are need).
>> Most interesting for me whould be to compare FX-8150 and i5-2550,
>> I7-2600, i7-3570K or i7-3770 processors.
>>
>> GPGPU - are there some real plans for supporting Radeon GPUs? Because
>> OpenMP 3.x supports them - but I don't have any idea how complicate is
>> to use newer OpenMP in GMX (and another work about it).
>>
>> And are there bigger differences (if someone knows) in use 1 Gbps
>> onboard NIC vs. "better"? add-on NIC cards e.g. Intel Pro/1000 CT or
>> similar? In MPI calculations with use 4 to 8 cores per computing node.
>>
>>
>> Thanks a lot.
>>
>> Milan Melichercik
>> --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> Please search the archive at 
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-requ...@gromacs.org.
>> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] some hardware questions

[Szilárd Páll]

On Wed, May 2, 2012 at 6:13 PM, Mirco Wahab <
mirco.wa...@chemie.tu-freiberg.de> wrote:

> Hello Peter,
>
> Am 02.05.2012 17:44, schrieb Peter C. Lai:
>
>
>  You can wait for ivy bridge, then stick some Kepler GPUs (nvidia gtx
>> 680) in it. That should max the performance. asm is pretty much stagnant
>> for general purpose procs since core2 came out.
>>
>
> Is this true?
>
> To my knowledge, the Fermi GPU (GF-110, eg. GTX-580) is
> a 16 processor (streaming multiprocessor, SM) system, each
> processor having 32 cores running at 1.5 GHz, and the
> Kepler-1 (GK-104, eg. GTX-680) an 8 processor system
> with 192 cores per processor at 1GHz.
> Because on the Fermi each SM has more L1 cache than
> each SM on the Kepler-1 and because it might be harder
> to saturate 192 cores in a compute-scenario, I'd expect
> the 580 (GF-110) to be significant(?) faster in the
> next Gromacs (4.6). Maybe somebody tested this already.
>

The current experimental Kepler kernels have about the same performance as
a GTX 580. We hope to squeeze out more performance, but we'll have to see.



> (Here in Germany, I can by GTX-580/3GB for ~325€ + Tax.)
>
> Regards,
>
> M.
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] some hardware questions

[Szilárd Páll]

On Fri, May 4, 2012 at 8:58 PM, Oliver Stueker  wrote:

> Hi all,
>
> This week Nvidia has also announced the GeForce GTX 690 with two
> Kepler Chips [1].
> That will make a total of 3072 CUDA cores @ 915MHz to 1019MHz and will
> perform like two GTX 680s in SLI mode [2], but more energy efficient.
>
> I'd really be interested if parallelization would be a bottleneck for
> the Kepler chips compared to the Fermi ones. Has anyone already run
> some benchmarks on Kepler?
>

If you stick the GTX 690 into PCi-E 3.0, it should have a performance quite
close to 2x GTX 680 on PCI-E 2.0. Don't expect to gain more than 5% from
having GTX 680 on PCI-E 2.0 vs PCI-E 3.0.

Cheers,
--
Szilárd



> Best,
> Oliver
>
>
> [1]
> http://nvidianews.nvidia.com/Releases/NVIDIA-Unveils-GeForce-GTX-690-Dual-Graphics-Card-Combines-World-s-Fastest-Gaming-Performance-With-Sleek-Sexy-Design-7c1.aspx
>
> [2] http://www.geforce.com/whats-new/articles/article-keynote/
>
> On Wed, May 2, 2012 at 10:16 AM, Peter C. Lai  wrote:
> > On 2012-05-02 06:13:04PM +0200, Mirco Wahab wrote:
> >> Hello Peter,
> >>
> >> Am 02.05.2012 17:44, schrieb Peter C. Lai:
> >>
> >> > You can wait for ivy bridge, then stick some Kepler GPUs (nvidia gtx
> >> > 680) in it. That should max the performance. asm is pretty much
> stagnant
> >> > for general purpose procs since core2 came out.
> >>
> >> Is this true?
> >>
> >> To my knowledge, the Fermi GPU (GF-110, eg. GTX-580) is
> >> a 16 processor (streaming multiprocessor, SM) system, each
> >> processor having 32 cores running at 1.5 GHz, and the
> >> Kepler-1 (GK-104, eg. GTX-680) an 8 processor system
> >> with 192 cores per processor at 1GHz.
> >> Because on the Fermi each SM has more L1 cache than
> >> each SM on the Kepler-1 and because it might be harder
> >> to saturate 192 cores in a compute-scenario, I'd expect
> >> the 580 (GF-110) to be significant(?) faster in the
> >> next Gromacs (4.6). Maybe somebody tested this already.
> >>
> >> (Here in Germany, I can by GTX-580/3GB for ~325€ + Tax.)
> >>
> >> Regards,
> >>
> >> M.
> >
> > You might be. I do have a collaborator in France who chooses to use GTX
> 580
> > with NAMD and is eschewing the Keplers. I think I saw a bench somewhere
> > for Kepler compute; not sure if I saw that much difference (but you will
> > be paying the price though). In that case stick with Fermis for now :)
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Gromacs 4.6 with CUDA 4.2

[Szilárd Páll]

On Fri, May 4, 2012 at 9:20 PM, Roland Schulz  wrote:
>
>
> On Wed, Apr 25, 2012 at 9:38 AM, Szilárd Páll 
> wrote:
>>
>> On Wed, Apr 25, 2012 at 11:43 AM, SebastianWaltz
>>  wrote:
>> > Dear all,
>> >
>> > will the new version 4.6 work together with CUDA 4.2? Would be good to
>> > know, since this is needed for the new NVIDIA Gemini cards with Kepler
>> > technology.
>>
>> Yes it will. I would like to point out that the "needed" part is not
>> exactly true, binaries compiled with pre-4.2 can run equally fast on
>> Kepler especially if JIT compilation of PTX code is enabled.
>>
>> Moreover, current 4.2 pre-release compiler produces slower code than
>> CUDA 4.0 not only for Fermi but also for Kepler. We'll have to see
>> brins the final CUDA 4.2 release + official drivers.
>
>
> Szilard, had you already a chance to test the 4.2 final release and can
> recommend which version to use?
> Is the difference in performance significant?

Yes, I've tried the 4.2 release (4.2.9) and the performance regression
is still present, the kernels are unfortunately up to 5% slower on
Fermi compared to CUDA 4.0.

--
Szilárd


> Roland
>
>>
>>
>> --
>> Szilárd
>>
>>
>>
>>
>> > Thanks,
>> >
>> > Basti
>> > --
>> > gmx-users mailing list    gmx-users@gromacs.org
>> > http://lists.gromacs.org/mailman/listinfo/gmx-users
>> > Please search the archive at
>> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> > Please don't post (un)subscribe requests to the list. Use the
>> > www interface or send it to gmx-users-requ...@gromacs.org.
>> > Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>> --
>> gmx-users mailing list    gmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-requ...@gromacs.org.
>> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>>
>>
>>
>
>
>
> --
> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> 865-241-1537, ORNL PO BOX 2008 MS6309
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

[gmx-users] nbnxn_hybrid_acc branch rewritten

[Szilárd Páll]

Hi,

Due to a mistake during commit cleanup we had to rewrite part of the
history of the nbnxn_hybrid_acc branch. As a consequence, pulling from
branches that track nbnxn_hybrid_acc will not work (it will result in
conflicts) and a forced update will be required:

$ git branch
nbnxn_hybrid_acc   # current branch is tracking nbnxn_hybrid_acc!
$ git remote update
[...]
$ git reset --hard origin/nbnxn_hybrid_acc

Sorry for the inconvenience!

Cheers,
--
Szilárd
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Re: Possible bug: energy changes with the number of nodes for energy minimization

[Szilárd Páll]

Or you can just use the git version.
--
Szilárd


On Wed, May 30, 2012 at 5:51 PM, Stephen Cox  wrote:
> Hi Justin and Mark,
>
> Thanks once again for getting back.
>
> I've found the problem - it's actually a known bug already:
>
> http://redmine.gromacs.org/issues/901
>
> The dispersion correction is multiplied my the number of processes (I found
> this after taking a closer look at my md.log files to see where the energy
> was changing)! I guess this means I should use the serial version for
> meaningful binding energies. It also looks like it will be fixed for version
> 4.5.6
>
> Thank you again, I really appreciate your help.
>
> Steve
>
>
>>
>> On 30/05/2012 9:42 PM, Stephen Cox wrote:
>> > Hi Justin,
>> >
>> > Thanks for getting back and posting the links.
>> >
>> >
>> >     On 5/29/12 6:22 AM, Stephen Cox wrote:
>> >     > Hi,
>> >     >
>> >     > I'm running a number of energy minimizations on a clathrate
>> >     supercell and I get
>> >     > quite significantly different values for the total energy
>> >     depending on the
>> >     > number of mpi processes / number of threads I use. More
>> >     specifically, some
>> >     > numbers I get are:
>> >     >
>> >     > #cores      energy
>> >     > 1        -2.41936409202696e+04
>> >     > 2        -2.43726425776809e+04
>> >     > 3        -2.45516442350804e+04
>> >     > 4        -2.47003944216983e+04
>> >     >
>> >     > #threads    energy
>> >     > 1        -2.41936409202696e+04
>> >     > 2        -2.43726425776792e+04
>> >     > 3        -2.45516442350804e+04
>> >     > 4        -2.47306458924815e+04
>> >     >
>> >     >
>> >     > I'd expect some numerical noise, but these differences seem to0
>> >     large for that.
>> >
>> >     The difference is only 2%, which by MD standards, is quite good,
>> >     I'd say ;)
>> >     Consider the discussion here:
>> >
>> >
>> > I agree for MD this wouldn't be too bad, but I'd expect energy
>> > minimization to get very close to the same local minimum from a given
>> > starting configuration. The thing is I want to compute a binding curve
>> > for my clathrate and compare to DFT values for the binding energy
>> > (amongst other things), and the difference in energy between different
>> > number of cores is rather significant for this purpose.
>>
>> Given the usual roughness of the PE surface to which you are minimizing,
>> some variation in end point is expected.
>>
>>
>> >
>>
>> > Furthermore, if I only calculate the energy for nsteps = 0 (i.e. a
>> > single point energy for identical structures) I get the same trend as
>> > above (both mpi/openmp with domain/particle decomposition). Surely
>> > there shouldn't be such a large difference in energy for a single
>> > point calculation?
>>
>> nsteps = 0 is not strictly a single-point energy, since the constraints
>> act before EM step 0. mdrun -s -rerun will give a single point. This
>> probably won't change your observations. You should also be sure you're
>> making observations with the latest release (4.5.5).
>>
>>
>> If you can continue to observe this trend for more processors
>> (overallocating?), then you may have evidence of a problem - but a full
>> system description and an .mdp file will be in order also.
>>
>> Mark
>
>
>> >
>> >
>> >     http://www.gromacs.org/Documentation/Terminology/Reproducibility
>> >
>> >     To an extent, the information here may also be relevant:
>> >
>> >
>> > http://www.gromacs.org/Documentation/How-tos/Extending_Simulations#Exact_vs_binary_identical_continuation
>> >
>> >     > Before submitting a bug report, I'd like to check:
>> >     > a) if someone has seen something similar;
>> >
>> >     Sure.  Energies can be different due to a whole host of factors
>> >     (discussed
>> >     above), and MPI only complicates matters.
>> >
>> >     > b) should I just trust the serial version?
>> >
>> >     Maybe, but I don't know that there's evidence to say that any of
>> >     the above tests
>> >     are more or less accurate than the others.  What happens if you
>> >     run with mdrun
>> >     -reprod on all your tests?
>> >
>> >
>> > Running with -reprod produces the same trend as above. If it was
>> > numerical noise, I would have thought that the numbers would fluctuate
>> > around some average value, not follow a definite trend where the
>> > energy decreases with the number of cores/threads...
>> >
>> >
>> >     > c) have I simply done something stupid (grompp.mdp appended
>> > below);
>> >     >
>> >
>> >     Nope, looks fine.
>> >
>> >     -Justin
>> >
>> > Thanks again for getting back to me.
>> >
>> >
>>
>> -- next part --
>> An HTML attachment was scrubbed...
>> URL:
>> http://lists.gromacs.org/pipermail/gmx-users/attachments/20120530/a4ed4a18/attachment-0001.html
>>
>> --
>>
>> Message: 2
>> Date: Wed, 30 May 2012 07:51:02 -0400
>> From: "Justin A. Lemkul" 
>> Subject: Re: [gmx-users] Re: Possible bug: energy changes with the
>>        number  of     

Re: [gmx-users] gromacs 4.6 on NVIDIA K10

[Szilárd Páll]

Hi,

The fact that GPUs are detected at configure-time does not imply that these
GPUs will also work fine at run-time and you see this in action in your
case. The current CMake configure script only looks at whether there is any
NVIDIA GPU device connected, but does not check other conditions that might
prevent a CUDA application to work, like: badly configured or broken GPU
driver or incompatible driver and CUDA toolkit.

In your case, as the second line of the error message states, no GPUs were
detected by the CUDA runtime itself (this message is returned by libcudart).

Please check your CUDA/driver installation, I'm pretty cure there is
something wrong with it. You can try to first run the "nvidia-smi" GPU
monitoring tool to see whether this can detect any device at all.

Cheers,
--
Szilárd


On Fri, Dec 14, 2012 at 4:09 PM, sebastian <
sebastian.wa...@physik.uni-freiburg.de> wrote:

> Dear gromacs users,
>
> I run into problems when I try to use the NVIDIA tesla K10 GPUs with mdrun.
> Looking at the cmake log file I find this lines:
>
> -- Looking for NVIDIA GPUs present in the system
> -- Number of NVIDIA GPUs detected: 2
> -- Found CUDA: /usr/local/cuda (found suitable version "4.2", required is
> "3.2")
> -- Enabling native GPU acceleration
>
> so everything seems to work fine during configuration and since running
> make also does not complain also during compilation.
> Now if I run mdrun on a test system I get the following error:
>
> Error occurred during GPU detection:
>   no CUDA-capable device is detected
>   Can not use GPU acceleration, will fall back to CPU kernels
>
> How can I get mdrun working on the K10?
>
> Any hint is welcomed.
>
> Dears
>
> Sebastian
>
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Szilárd Páll]

Hi,

That unfortunately tell exactly about the reason why mdrun is stuck. Can
you reproduce the issue on another machines or with different launch
configurations? At which step does it get stuck (-stepout 1 can help)?

Please try the following:
- try running on a single GPU;
- try running on CPUs only (-nb cpu and to match closer the GPU setup with
-ntomp 12);
- try running in GPU emulation mode with the GMX_EMULATE_GPU=1 env. var
set (and to match closer the GPU setup with -ntomp 12)
- provide a backtrace (using gdb).

Cheers,

--
Szilárd



On Mon, Dec 17, 2012 at 5:37 PM, Albert  wrote:

> hello:
>
>  I am running GMX-4.6 beta2 GPU work in a 24 CPU core workstation with two
> GTX590, it stacked there without any output i.e the .xtc file size is
> always 0 after hours of running. Here is the md.log file I found:
>
>
> Using CUDA 8x8x8 non-bonded kernels
>
> Potential shift: LJ r^-12: 0.112 r^-6 0.335, Ewald 1.000e-05
> Initialized non-bonded Ewald correction tables, spacing: 7.82e-04 size:
> 1536
>
> Removing pbc first time
> Pinning to Hyper-Threading cores with 12 physical cores in a compute node
> There are 1 flexible constraints
>
> WARNING: step size for flexible constraining = 0
>  All flexible constraints will be rigid.
>  Will try to keep all flexible constraints at their original
> length,
>  but the lengths may exhibit some drift.
>
> Initializing Parallel LINear Constraint Solver
> Linking all bonded interactions to atoms
> There are 161872 inter charge-group exclusions,
> will use an extra communication step for exclusion forces for PME
>
> The initial number of communication pulses is: X 1
> The initial domain decomposition cell size is: X 1.83 nm
>
> The maximum allowed distance for charge groups involved in interactions is:
>  non-bonded interactions   1.200 nm
> (the following are initial values, they could change due to box
> deformation)
> two-body bonded interactions  (-rdd)   1.200 nm
>   multi-body bonded interactions  (-rdd)   1.200 nm
>   atoms separated by up to 5 constraints  (-rcon)  1.826 nm
>
> When dynamic load balancing gets turned on, these settings will change to:
> The maximum number of communication pulses is: X 1
> The minimum size for domain decomposition cells is 1.200 nm
> The requested allowed shrink of DD cells (option -dds) is: 0.80
> The allowed shrink of domain decomposition cells is: X 0.66
> The maximum allowed distance for charge groups involved in interactions is:
>  non-bonded interactions   1.200 nm
> two-body bonded interactions  (-rdd)   1.200 nm
>   multi-body bonded interactions  (-rdd)   1.200 nm
>   atoms separated by up to 5 constraints  (-rcon)  1.200 nm
>
> Making 1D domain decomposition grid 4 x 1 x 1, home cell index 0 0 0
>
> Center of mass motion removal mode is Linear
> We have the following groups for center of mass motion removal:
>   0:  Protein_LIG_POPC
>   1:  Water_and_ions
>
>  PLEASE READ AND CITE THE FOLLOWING REFERENCE 
> G. Bussi, D. Donadio and M. Parrinello
> Canonical sampling through velocity rescaling
> J. Chem. Phys. 126 (2007) pp. 014101
>   --- Thank You ---  
>
>
>
> THX
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Szilárd Páll]

> distance (nm)
> pbc = xyz   ; Direction in which to use Perodic
> Boundary Conditions (xyz, xy, no)
> cutoff-scheme   =Verlet  ; GPU running
>
> ; Parameters for treating bonded interactions
> continuation= no; Whether a fresh start or a continuation
> from a previous run (yes/no)
> constraint_algorithm = LINCS; Constraint algorithm (LINCS / SHAKE)
> constraints = all-bonds ; Which bonds/angles to constrain
> (all-bonds / hbonds / none / all-angles / h-angles)
> lincs_iter  = 1 ; Number of iterations to correct for
> rotational lengthening in LINCS (related to accuracy)
> lincs_order = 4 ; Highest order in the expansion of the
> constraint coupling matrix (related to accuracy)
>
> ; Parameters for treating electrostatic interactions
> coulombtype = PME   ; Long range electrostatic interactions
> treatment (cut-off, Ewald, PME)
> pme_order   = 4 ; Interpolation order for PME (cubic
> interpolation is represented by 4)
> fourierspacing  = 0.12  ; Maximum grid spacing for FFT grid using
> PME (nm)
>
> ; Temperature coupling parameters
> tcoupl  = V-rescale ; Modified Berendsen thermostat
> using velocity rescaling
> tc-grps = Protein_LIG POPC Water_and_ions ; Define groups to be
> coupled separately to temperature bath
> tau_t   = 0.1   0.1 0.1 ; Group-wise coupling time
> constant (ps)
> ref_t   = 303   303 303 ; Group-wise reference temperature
> (K)
>
> ; Pressure coupling parameters
> pcoupl  = no; Under NVT conditions pressure coupling
> is not done
>
> ; Miscellaneous control parameters
> ; Dispersion correction
> DispCorr= EnerPres  ; Dispersion corrections for Energy and
> Pressure for vdW cut-off
> ; Initial Velocity Generation
> gen_vel = yes   ; Generate velocities from Maxwell
> distribution at given temperature
> gen_temp= 303   ; Specific temperature for Maxwell
> distribution (K)
> gen_seed= -1; Use random seed for velocity generation
> (integer; -1 means seed is calculated from the process ID number)
> ; Centre of mass (COM) motion removal relative to the specified groups
> nstcomm = 1 ; COM removal frequency (steps)
> comm_mode   = Linear; Remove COM translation (linear /
> angular / no)
> comm_grps   = Protein_LIG_POPC Water_and_ions ; COM removal relative
> to the specified groups
>
> THX
>
>
>
>
>
>
> On 12/17/2012 05:45 PM, Szilárd Páll wrote:
>
>> Hi,
>>
>> That unfortunately tell exactly about the reason why mdrun is stuck. Can
>> you reproduce the issue on another machines or with different launch
>> configurations? At which step does it get stuck (-stepout 1 can help)?
>>
>> Please try the following:
>> - try running on a single GPU;
>> - try running on CPUs only (-nb cpu and to match closer the GPU setup with
>> -ntomp 12);
>> - try running in GPU emulation mode with the GMX_EMULATE_GPU=1 env. var
>> set (and to match closer the GPU setup with -ntomp 12)
>> - provide a backtrace (using gdb).
>>
>> Cheers,
>>
>> --
>> Szilárd
>>
>>
>>
>> On Mon, Dec 17, 2012 at 5:37 PM, Albert  wrote:
>>
>>  hello:
>>>
>>>   I am running GMX-4.6 beta2 GPU work in a 24 CPU core workstation with
>>> two
>>> GTX590, it stacked there without any output i.e the .xtc file size is
>>> always 0 after hours of running. Here is the md.log file I found:
>>>
>>>
>>> Using CUDA 8x8x8 non-bonded kernels
>>>
>>> Potential shift: LJ r^-12: 0.112 r^-6 0.335, Ewald 1.000e-05
>>> Initialized non-bonded Ewald correction tables, spacing: 7.82e-04 size:
>>> 1536
>>>
>>> Removing pbc first time
>>> Pinning to Hyper-Threading cores with 12 physical cores in a compute node
>>> There are 1 flexible constraints
>>>
>>> WARNING: step size for flexible constraining = 0
>>>   All flexible constraints will be rigid.
>>>   Will try to keep all flexible constraints at their original
>>> length,
>>>   but the lengths may exhibit some drift.
>>>
>>> Initializing Parallel LINear Constraint Solver
>>> Linking all bonded interactions to atoms
>>> There are 161872 inter charge-group exclusions,
>>> will use an extra communication step for exclusion forces for PME
>>>
>>> The initial number of communication pulses is: X 1
>&g

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Szilárd Páll]

Hi Albert,

Thanks for the testing.

Last questions.
- What version are you using? Is it beta2 release or latest git? if it's
the former, getting the latest git might help if...
-  (do) you happen to be using GMX_GPU_ACCELERATION=None (you shouldn't!)?
A bug triggered only with this setting has been fixed recently.

If the above doesn't help, please file a bug report and attach a tpr so we
can reproduce.

Cheers,

--
Szilárd



On Mon, Dec 17, 2012 at 6:21 PM, Albert  wrote:

> On 12/17/2012 06:08 PM, Szilárd Páll wrote:
>
>> Hi,
>>
>> How about GPU emulation or CPU-only runs? Also, please try setting the
>> number of therads to 1 (-ntomp 1).
>>
>>
>> --
>> Szilárd
>>
>>
> hello:
>
> I am running in GPU emulation mode with the GMX_EMULATE_GPU=1 env. var
> set (and to match closer the GPU setup with -ntomp 12), it failed with log:
>
> Back Off! I just backed up step33b.pdb to ./#step33b.pdb.2#
>
> Back Off! I just backed up step33c.pdb to ./#step33c.pdb.2#
>
> Wrote pdb files with previous and current coordinates
> [CUDANodeA:20753] *** Process received signal ***
> [CUDANodeA:20753] Signal: Segmentation fault (11)
> [CUDANodeA:20753] Signal code: Address not mapped (1)
> [CUDANodeA:20753] Failing at address: 0x106ae6a00
>
> [1]Segmentation faultmdrun_mpi -v -s nvt.tpr -c nvt.gro -g
> nvt.log -x nvt.xtc -ntomp 12
>
>
>
>
> I also tried , number of therads to 1 (-ntomp 1), it failed with following
> messages:
>
>
> Back Off! I just backed up step33c.pdb to ./#step33c.pdb.1#
>
> Wrote pdb files with previous and current coordinates
> [CUDANodeA:20740] *** Process received signal ***
> [CUDANodeA:20740] Signal: Segmentation fault (11)
> [CUDANodeA:20740] Signal code: Address not mapped (1)
> [CUDANodeA:20740] Failing at address: 0x1f74a96ec
> [CUDANodeA:20740] [ 0] /lib64/libpthread.so.0(+**0xf2d0) [0x2b351d3022d0]
> [CUDANodeA:20740] [ 1] /opt/gromacs-4.6/lib/libmd_**mpi.so.6(+0x11020f)
> [0x2b351a99c20f]
> [CUDANodeA:20740] [ 2] /opt/gromacs-4.6/lib/libmd_**mpi.so.6(+0x111c94)
> [0x2b351a99dc94]
> [CUDANodeA:20740] [ 3] 
> /opt/gromacs-4.6/lib/libmd_**mpi.so.6(gmx_pme_do+0x1d2e)
> [0x2b351a9a1bae]
> [CUDANodeA:20740] [ 4] /opt/gromacs-4.6/lib/libmd_**
> mpi.so.6(do_force_lowlevel+**0x1eef) [0x2b351a97262f]
> [CUDANodeA:20740] [ 5] /opt/gromacs-4.6/lib/libmd_**
> mpi.so.6(do_force_cutsVERLET+**0x1756) [0x2b351aa04736]
> [CUDANodeA:20740] [ 6] /opt/gromacs-4.6/lib/libmd_**mpi.so.6(do_force+0x3bf)
> [0x2b351aa0a0df]
> [CUDANodeA:20740] [ 7] mdrun_mpi(do_md+0x8133) [0x4334c3]
> [CUDANodeA:20740] [ 8] mdrun_mpi(mdrunner+0x19e9) [0x411639]
> [CUDANodeA:20740] [ 9] mdrun_mpi(main+0x17db) [0x4373db]
> [CUDANodeA:20740] [10] /lib64/libc.so.6(__libc_start_**main+0xfd)
> [0x2b351d52ebfd]
> [CUDANodeA:20740] [11] mdrun_mpi() [0x407f09]
> [CUDANodeA:20740] *** End of error message ***
>
> [1]Segmentation faultmdrun_mpi -v -s nvt.tpr -c nvt.gro -g
> nvt.log -x nvt.xtc -ntomp 1
>
>
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users<http://lists.gromacs.org/mailman/listinfo/gmx-users>
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Search<http://www.gromacs.org/Support/Mailing_Lists/Search>before
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists<http://www.gromacs.org/Support/Mailing_Lists>
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU running problem with GMX-4.6 beta2

[Szilárd Páll]

On Mon, Dec 17, 2012 at 7:56 PM, Mark Abraham wrote:

> On Mon, Dec 17, 2012 at 6:01 PM, Albert  wrote:
>
> > hello:
> >
> >  I reduced the GPU to two, and it said:
> >
> > Back Off! I just backed up nvt.log to ./#nvt.log.1#
> > Reading file nvt.tpr, VERSION 4.6-dev-20121004-5d6c49d (single precision)
> >
>
> This is a development version from October 1. Please use the mdrun version
> you think you're using :-)
>

Thanks Mark, good catch!

--
Szilárd



>
> Mark
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

[Szilárd Páll]

Hi James,

Don't use the OpenMM accelerated mdrun for explicit solvent simulations.
For such runs any modern CPU with 4-8 cores will probably be as fast or
faster than most GPUs.

However, do consider the 4.6 *native* heterogeneous GPU-accelerated code.
This new efficient code can provide 2-4x speedup and uses both CPU and GPU.

To get the latest 4.6 version use the 4.6-beta3 release (http://goo.gl/XFPoe)
or even better, get the code from git (git clone git://
git.gromacs.org/gromacs -b release-4-6).

Let us know how things worked out!

Cheers,

--
Szilárd


On Mon, Jan 7, 2013 at 8:05 AM, James Starlight wrote:

> Today I've tried to test gpu performance on the benchmark system (
> explicit solvent with pme )
>
> grompp-gpu -f md_sd.mdp -c conf -p topol.top -o md_test
> mdrun-gpu -device
> "OpenMM:platform=Cuda,DeviceID=1,Memtest=15,force-device=no" -v
> -deffnm md_test
>
> that produce the error
>
> Program mdrun-gpu, VERSION 4.5.5
> Source code file: /home/own/gromacs_gpu/src/kernel/openmm_wrapper.cpp,
> line: 1365
>
> Fatal error:
> OpenMM exception caught while initializating: cudaMemcpyToSymbol:
> SetSim copy to cSim failed invalid device symbol
> For more information and tips for troubleshooting, please check the GROMACS
> website at http://www.gromacs.org/Documentation/Errors
>
>
> In the mailing list archive I've found the same error with the same
> card posted some time ago
>
> http://article.gmane.org/gmane.science.biology.gromacs.user/55577/match=openmm+exception+caught+while+initializating+cudaMemcpyToSymbol+setsim+copy+cSim+failed+invalid+device+symbol
>
> >From that topic I've understood that 670 graphic card is not supported
> yet in Gromacs (4.55). I've checked the manual for the 4.6 beta
> version and does not found anything about supporting of that card in
> the lattest gromacs realeses as well. IS there any ways to solve that
> problem ? Could that error be linked with some others ?  E.g I've
> installed nvidia drivers + cuda first and openMM 4.01 than. During
> gromacs compilation I've not forced with any errors.
>
> Thanks for help
> James
>
>  2013/1/6 Justin Lemkul :
> >
> >
> > On 1/6/13 1:53 PM, James Starlight wrote:
> >>
> >> OK!
> >>
> >> I've compilated gromacs-gpu from the source using that tutorial for
> >> the Debian OS
> >>
> >>
> http://verahill.blogspot.ru/2012/01/debian-testing-64-wheezy-compiling_20.html
> >>
> >> The only thing that it lack is the installation mdrun_d-gpu  but i'm
> >> not sure that double precission can be used with gpu.
> >>
> >> Could someone provide me with some tutorial which would show me basic
> >> mdrun options with the calculation on my gpu as well as some testing
> >> system?
> >>
> >
> > Everything you need to know (including benchmark systems) is posted at
> > http://www.gromacs.org/Documentation/Installation_Instructions_4.5/GPUs.
> >
> > The invocation of mdrun-gpu is like any other mdrun, with GPU-specific
> > information available by reading mdrun-gpu -h.
> >
> >
> > -Justin
> >
> > --
> > 
> >
> > Justin A. Lemkul, Ph.D.
> > Research Scientist
> > Department of Biochemistry
> > Virginia Tech
> > Blacksburg, VA
> > jalemkul[at]vt.edu | (540) 231-9080
> > http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin
> >
> > 
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the www
> > interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

[Szilárd Páll]

On Tue, Jan 8, 2013 at 3:22 PM, James Starlight wrote:

> So could someone provide me more about gpu-accelerated MD implemented
> in the 4.6 gromacs ? Does it require openMM (what version is supported
>

FYI, if nobody can, trust G:
http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
http://lmgtfy.com/?q=gromacs+4.6+installation+instructions

The wiki and mailing list contains quite extensive information (indexed by
G).

Otherwise, release notes (not final):
http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x

Install guide is at the expected location:
http://www.gromacs.org/Documentation/Installation_Instructions

Cheers,
--
Szilárd


> for that gromacs release ?) installed? By the way at present time I
> force with the problem of compilation 4.1.1 openMM (i need to compile
> openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
> +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.
>
>
> James
>
> 2013/1/7 James Starlight :
> > Hi Szilárd!
> >
> > As I understood you correctly gromacs-4.6 have specific algorithm
> > (independent on openMM?) for gpu-based calculations havent ? If it
> > true how I should compilate such new gpu-based gromacs? In the
> > gromacs-4.6-beta-3 folder I've found instructuon for the standart
> > installation via cmake
> >
> > cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
> >
> >
> > James
> >
> > 2013/1/7 Szilárd Páll :
> >> Szilárd
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

[Szilárd Páll]

On Wed, Jan 9, 2013 at 9:17 AM, James Starlight wrote:

> As I understood that gromacs version already has included openMM so
> the installation of the external openMM sources is not needed, isnt it
> ?
>

The manual and wiki guides are meant to be read. Although they are a bit
dated, the main body of information is still valid. Please read the
documentation; aka RT(F)M:
http://www.gromacs.org/gpu#Compiling_and_installation_of_GROMACS-GPU_from_source


> also I wounder to know what exactly CUDA version is needed ? For
> example I've tried lattest cuda-5.0 but with that version i've obtain
> error from mdrun-openmm that platform cuda was not detected (gromacs
> was compilated without any errors). I've seen that erorr in other
> posts in the mailing list but could not found any possible sollutions.
>

It surely works with the (admittedly dated) versions the docs suggest,
you mileage may vary with other versions, but in principle everything
should work with the latest compilers, CUDA, and OpenMM (myself and others
have built and ran successfully).


>
> by the way is it possible to compilate gromacs-4.6 agains other
> platgorm ( e.g openCL) ? I have no problems with the compatibility of

the openCL and openMM.
>

No, RTM. OpenMM does have an OpenCL plugin, but that's not supported in
GROMACS, if you want to use it you'll have to write your own code around
the OpenMM library. We
have no resources to implement it ATM, but as GROMACS is OSS anyone is
welcome to contribute it.

And let me reiterate, if you're running explicit solvent you should be
using the native GPU acceleration as you've been told before.

Cheers,
Szilard


> James
>
> 2013/1/9 Szilárd Páll :
> > On Tue, Jan 8, 2013 at 3:22 PM, James Starlight  >wrote:
> >
> >> So could someone provide me more about gpu-accelerated MD implemented
> >> in the 4.6 gromacs ? Does it require openMM (what version is supported
> >>
> >
> > FYI, if nobody can, trust G:
> > http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
> > http://lmgtfy.com/?q=gromacs+4.6+installation+instructions
> >
> > The wiki and mailing list contains quite extensive information (indexed
> by
> > G).
> >
> > Otherwise, release notes (not final):
> > http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x
> >
> > Install guide is at the expected location:
> > http://www.gromacs.org/Documentation/Installation_Instructions
> >
> > Cheers,
> > --
> > Szilárd
> >
> >
> >> for that gromacs release ?) installed? By the way at present time I
> >> force with the problem of compilation 4.1.1 openMM (i need to compile
> >> openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
> >> +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.
> >>
> >>
> >> James
> >>
> >> 2013/1/7 James Starlight :
> >> > Hi Szilárd!
> >> >
> >> > As I understood you correctly gromacs-4.6 have specific algorithm
> >> > (independent on openMM?) for gpu-based calculations havent ? If it
> >> > true how I should compilate such new gpu-based gromacs? In the
> >> > gromacs-4.6-beta-3 folder I've found instructuon for the standart
> >> > installation via cmake
> >> >
> >> > cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
> >> >
> >> >
> >> > James
> >> >
> >> > 2013/1/7 Szilárd Páll :
> >> >> Szilárd
> >> --
> >> gmx-users mailing listgmx-users@gromacs.org
> >> http://lists.gromacs.org/mailman/listinfo/gmx-users
> >> * Please search the archive at
> >> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> >> * Please don't post (un)subscribe requests to the list. Use the
> >> www interface or send it to gmx-users-requ...@gromacs.org.
> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

[Szilárd Páll]

On Wed, Jan 9, 2013 at 10:27 AM, James Starlight wrote:

> I've solved previous problem but havent been able to launch
> mdrun-openmm. Below you can find mdrun's output
>
> Back Off! I just backed up md_test.log to ./#md_test.log.3#
> Reading file md_test.tpr, VERSION 4.6-beta3 (single precision)
> Changing nstlist from 10 to 40, rlist from 1 to 1.146
>
> Compiled acceleration: None (Gromacs could use AVX_256 on this
> machine, which is better)
>

Have you set the CPU acceleration manually to "None"? If you did *don't* do
that, let the build system pick what's best for you. However, if you did
not do that, there is some issue with our auto-detection; a log file as a
start would possibly help in identifying the issue.


>
> 1 GPU detected:
>   #0: NVIDIA GeForce GTX 670, compute cap.: 3.0, ECC:  no, stat: compatible
>
> 1 GPU auto-selected to be used for this run: #0
>
> Program mdrun-openmm, VERSION 4.6-beta3
> Source code file:
> /home/own/gromacs-4.6-beta3/src/kernel/openmm_wrapper.cpp, line: 1367
>
> Fatal error:
> OpenMM exception caught while initializating: Error setting device
> flags cannot set while device is active in this process
> For more information and tips for troubleshooting, please check the GROMACS
> website at http://www.gromacs.org/Documentation/Errors
>
>
> I'm still using cuda-5-toolkit with the 304.54 driver. Also my
> question about openCL is still valid. I've browesed manual but didnt
> found how I can compile gromacs with openCL instead of Cuda. On openMM
> group developers told me that openCL would give much performance for
> gpu-accelerated simulation in comparison to the CUDA.
>
> James
>
> 2013/1/9 James Starlight :
> > As I understood that gromacs version already has included openMM so
> > the installation of the external openMM sources is not needed, isnt it
> > ?
> >
> > also I wounder to know what exactly CUDA version is needed ? For
> > example I've tried lattest cuda-5.0 but with that version i've obtain
> > error from mdrun-openmm that platform cuda was not detected (gromacs
> > was compilated without any errors). I've seen that erorr in other
> > posts in the mailing list but could not found any possible sollutions.
> >
> > by the way is it possible to compilate gromacs-4.6 agains other
> > platgorm ( e.g openCL) ? I have no problems with the compatibility of
> > the openCL and openMM.
> >
> > James
> >
> > 2013/1/9 Szilárd Páll :
> >> On Tue, Jan 8, 2013 at 3:22 PM, James Starlight  >wrote:
> >>
> >>> So could someone provide me more about gpu-accelerated MD implemented
> >>> in the 4.6 gromacs ? Does it require openMM (what version is supported
> >>>
> >>
> >> FYI, if nobody can, trust G:
> >> http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
> >> http://lmgtfy.com/?q=gromacs+4.6+installation+instructions
> >>
> >> The wiki and mailing list contains quite extensive information (indexed
> by
> >> G).
> >>
> >> Otherwise, release notes (not final):
> >> http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x
> >>
> >> Install guide is at the expected location:
> >> http://www.gromacs.org/Documentation/Installation_Instructions
> >>
> >> Cheers,
> >> --
> >> Szilárd
> >>
> >>
> >>> for that gromacs release ?) installed? By the way at present time I
> >>> force with the problem of compilation 4.1.1 openMM (i need to compile
> >>> openMM because of cuda-5.0 ). If someone have done it (openMM 4.11
> >>> +cuda 5.0 + gromacs-4.5 for lattest geforces) please let me know.
> >>>
> >>>
> >>> James
> >>>
> >>> 2013/1/7 James Starlight :
> >>> > Hi Szilárd!
> >>> >
> >>> > As I understood you correctly gromacs-4.6 have specific algorithm
> >>> > (independent on openMM?) for gpu-based calculations havent ? If it
> >>> > true how I should compilate such new gpu-based gromacs? In the
> >>> > gromacs-4.6-beta-3 folder I've found instructuon for the standart
> >>> > installation via cmake
> >>> >
> >>> > cmake PATH_TO_SOURCE_DIRECTORY -DGMX_OPENMM=ON -DGMX_THREADS=OFF
> >>> >
> >>> >
> >>> > James
> >>> >
> >>> > 2013/1/7 Szilárd Páll :
> >>> >> Szilárd
> >>> --
> >>> gmx-users mailing listgmx-users@gromacs.org
> &

Re: [gmx-users] gromacs on GPU

[Szilárd Páll]

Dear James,

On Wed, Jan 9, 2013 at 6:17 PM, James Starlight wrote:

> Roland,
>
> indeed the error was that I'have compilate mdrun-openmm which is not
> the native gpu.
>
> now I've made mdrun via
>
> cmake CMakeLists.txt -DGMX_GPU=ON
> -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0
>
> and obtain workable gromacs.
>
> My test system consist of calmodulin (charmm27) solvated in tip3p
> water. total size of the system: 67752 atoms.
> I have set below values on the mdp file as grompp told me
>
> nstlist = 20
> cutoff-scheme = Verlet   ; !not quite sure whai is this!
>

Again RTM:
http://www.gromacs.org/Documentation/Cut-off_schemes<http://www.gromacs.org/Documentation/Cut-off_schemes?highlight=verlet+scheme>


> vdwtype = cut-off
>
>
>  Now on the mdrun output I have
>
> Using 1 MPI thread
> Using 4 OpenMP threads
>
> 1 GPU detected:
>   #0: NVIDIA GeForce GTX 670, compute cap.: 3.0, ECC:  no, stat: compatible
>
> 1 GPU auto-selected to be used for this run: #0
>
> does it mean that all 4 cores of my CPU + gpu are used at same time ?
>

Yes.


> is there any other ways to increase performance ? ( e.g I'm not quite
> sure if open_mpi is used with that build because during compilation
> cmake asked me only about fftw3 libs).
>

There could be, but I/we can't well without more information on what and
how you compiled and ran. The minimum we need is a log file.


>
> Is there any ways to monitor total performance ( e.g separate cpu and
> gpu usage ) ?
>

See the performance table and summary at the end of the log file. For
further CPU monitoring you can use top, for GPUs the nvidia-smi tool that
comes together with the driver has some monitoring capabilities.


Finally, I would strongly suggest that you check the links I posted in the
first mail. There is plenty of documentation and general description on
the heterogeneous (CPU+GPU) acceleration on the wiki and mailing list; the
google results of the searches I hinted will get you exactly that and more.

Let me reiterate: while questions and general interest are appreciated, the
*minimum* expectation is that you at least read the relevant part of the
documentation and related mailing list posts, especially if this has been
already suggested to you. Please do us and yourself a favor and browse
through the available resources.

Cheers,
--
Szilárd


>
>
> Thanks for suggestions,
>
> James
>
> 2013/1/9 Roland Schulz :
> > On Wed, Jan 9, 2013 at 3:17 AM, James Starlight  >wrote:
> >
> >> As I understood that gromacs version already has included openMM so
> >> the installation of the external openMM sources is not needed, isnt it
> >> ?
> >>
> >
> > No the new build in GPU implementation and openMM are two different
> things.
> > The Gromacs-OpenMM interface isn't actively maintained and thus not
> > recommended.
> >
> >
> >> also I wounder to know what exactly CUDA version is needed ? For
> >> example I've tried lattest cuda-5.0 but with that version i've obtain
> >> error from mdrun-openmm that platform cuda was not detected (gromacs
> >> was compilated without any errors).
> >>
> > with the natived gpu implementation (GMX_GPU) cuda 5.0 works fine.
> >
> >
> >> by the way is it possible to compilate gromacs-4.6 agains other
> >> platgorm ( e.g openCL) ? I have no problems with the compatibility of
> >> the openCL and openMM.
> >>
> > GMX_GPU doesn't support openCL.
> >
> > Roland
> >
> >
> >>
> >> James
> >>
> >> 2013/1/9 Szilárd Páll :
> >> > On Tue, Jan 8, 2013 at 3:22 PM, James Starlight <
> jmsstarli...@gmail.com
> >> >wrote:
> >> >
> >> >> So could someone provide me more about gpu-accelerated MD implemented
> >> >> in the 4.6 gromacs ? Does it require openMM (what version is
> supported
> >> >>
> >> >
> >> > FYI, if nobody can, trust G:
> >> > http://lmgtfy.com/?q=gromacs+4.6+gpu+acceleration
> >> > http://lmgtfy.com/?q=gromacs+4.6+installation+instructions
> >> >
> >> > The wiki and mailing list contains quite extensive information
> (indexed
> >> by
> >> > G).
> >> >
> >> > Otherwise, release notes (not final):
> >> > http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x
> >> >
> >> > Install guide is at the expected location:
> >> > http://www.gromacs.org/Documentation/Installation_Instructions
> >> >
> >> > Cheers,
> 

Re: [gmx-users] gromacs on GPU

[Szilárd Páll]

Hi James,

The build looks mostly fine except that you are using fftw3 compiled with
AVX which is slower than with only SSE (even on AVX-capable CPUs) - you
should have been warned about this at configure-time.

Now, performance-wise everything looks fine except that with a 1.2 nm
cut-off your GPU is not able to keep up with the CPU and finish the
non-bonded work before the CPU is done with Bonded + PME. That's why you
see the "Wait GPU" taking 20% of the total time and that's also why you see
some cores idling (because for 20% of the run-time thread 0 on core 0
is blocked waiting for the GPU while the rest idle).

As the suggestion at the end of the log file point out, you can consider
using a shorter cut-off which will push more work back to the PME on the
CPU, but whether you can do this it depends on your very problem.

There is one more alternative of running two MPI processes on the GPU
(mpirun -np 2 mdrun -gpu_id 00) and using the -nb gpu_cpu mode which will
execute part of the nonbonded on the CPU, but this might not help.

Cheers,

--
Szilárd


On Wed, Jan 9, 2013 at 8:27 PM, James Starlight wrote:

> Dear Szilárd, thanks for help again!
>
> 2013/1/9 Szilárd Páll :
>
> >
> > There could be, but I/we can't well without more information on what and
> > how you compiled and ran. The minimum we need is a log file.
> >
> I've compilated gromacs 4.6-3 beta via simple
>
>
> cmake CMakeLists.txt -DGMX_GPU=ON
> -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-5.0
> make
> sudo make install
>
> I have not added any special params to the grompp or mdrun.
>
> After that I've run tested simulation of the calmodulin in explicit
> water ( 60k atoms ) 100ps and obtain next output
>
> Host: starlight  pid: 21028  nodeid: 0  nnodes:  1
> Gromacs version:VERSION 4.6-beta3
> Precision:  single
> MPI library:thread_mpi
> OpenMP support: enabled
> GPU support:enabled
> invsqrt routine:gmx_software_invsqrt(x)
> CPU acceleration:   AVX_256
> FFT library:fftw-3.3.2-sse2-avx
> Large file support: enabled
> RDTSCP usage:   enabled
> Built on:   Wed Jan  9 20:44:51 MSK 2013
> Built by:   own@starlight [CMAKE]
> Build OS/arch:  Linux 3.2.0-2-amd64 x86_64
> Build CPU vendor:   GenuineIntel
> Build CPU brand:Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
> Build CPU family:   6   Model: 58   Stepping: 9
> Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm
> mmx msr nonstop_tsc pcid pclmuldq pdcm popcnt pse rdrnd rdtscp sse2
> sse3 sse4.1 sse4.2 ssse3 tdt x2apic
> C compiler: /usr/bin/gcc GNU gcc (Debian 4.6.3-11) 4.6.3
> C compiler flags:   -mavx  -Wextra -Wno-missing-field-initializers
> -Wno-sign-compare -Wall -Wno-unused -Wunused-value
> -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3
> -DNDEBUG
> C++ compiler:   /usr/bin/c++ GNU c++ (Debian 4.6.3-11) 4.6.3
> C++ compiler flags: -mavx  -Wextra -Wno-missing-field-initializers
> -Wno-sign-compare -Wall -Wno-unused -Wunused-value
> -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -O3
> -DNDEBUG
> CUDA compiler:  nvcc: NVIDIA (R) Cuda compiler driver;Copyright
> (c) 2005-2012 NVIDIA Corporation;Built on
> Fri_Sep_21_17:28:58_PDT_2012;Cuda compilation tools, release 5.0,
> V0.2.1221
> CUDA driver:5.0
> CUDA runtime:   5.0
>
> 
>
>Core t (s)   Wall t (s)(%)
>Time: 2770.700 1051.927  263.4
>  (ns/day)(hour/ns)
> Performance:8.2142.922
>
> full log can be found here http://www.sendspace.com/file/inum84
>
>
> Finally when I check CPU usage I notice that only 1 CPU was full
> loaded ( 100%) and 2-4 cores were loaded on only 60% but  gave me
> strange results that GPU is not used (I've only monitored temperature
> of video card and noticed increase of the temperature up to 65 degrees
> )
>
> +--+
> | NVIDIA-SMI 4.304.54   Driver Version: 304.54 |
>
> |---+--+--+
> | GPU  Name | Bus-IdDisp.  | Volatile Uncorr.
> ECC |
> | Fan  Temp  Perf  Pwr:Usage/Cap| Memory-Usage | GPU-Util  Compute
> M. |
>
> |===+==+==|
> |   0  GeForce GTX 670  | :02:00.0 N/A |
>  N/A |
> | 38%   63C  N/A N/A /  N/A |   9%  174MB / 2047MB | N/A
>  Default |
>
> +---+--+--+
>
>
> +---

Re: [gmx-users] gromacs on GPU

[Szilárd Páll]

On Thu, Jan 10, 2013 at 7:25 AM, James Starlight wrote:

> Szilárd ,
>
>  thanks again for explanation!
>
> Today I've performed some tests on my calmodulin in water system with
> different cutt-offs (I've used all cutt-ooffs 1.0 , 0.9 and 0.8
> respectually)
>
> Below you can see that the highest performance was in case of 0.8 cut-offs
>
> all cut-offs 1.0
>  Force evaluation time GPU/CPU: 6.134 ms/4.700 ms = 1.305
>
> NOTE: The GPU has >20% more load than the CPU. This imbalance causes
>   performance loss, consider using a shorter cut-off and a finer PME
> grid.
>
>
>Core t (s)   Wall t (s)(%)
>Time: 1313.420  464.035  283.0
>  (ns/day)(hour/ns)
> Performance:9.3102.578
> Finished mdrun on node 0 Thu Jan 10 09:39:23 2013
>
>
> all cut-offs 0.9
> Force evaluation time GPU/CPU: 4.951 ms/4.675 ms = 1.059
>
>Core t (s)   Wall t (s)(%)
>Time: 2414.930  856.179  282.1
>  (ns/day)(hour/ns)
> Performance:   10.0922.378
> Finished mdrun on node 0 Thu Jan 10 10:09:52 2013
>
> all cut-offs 0.8
>  Force evaluation time GPU/CPU: 4.001 ms/4.659 ms = 0.859
>
>Core t (s)   Wall t (s)(%)
>Time: 1166.390  413.598  282.0
>  (ns/day)(hour/ns)
> Performance:   10.4452.298
> Finished mdrun on node 0 Thu Jan 10 09:50:33 2013
>
> Also I've noticed that 2-4 CPU cores usage in 2 and 3rd case was only
> 67%. Is there any other ways to increase performance by means of
> neighboor search parameters ( e.g nstlist etc) ?
>

You can tweak nstlist and it often helps to increase it with GPUs,
especially in parallel. However, as increasing nstlist requires larger
rlist and more non-bonded calculations, this will not help you. You can try
to decrease it to 10-15 which will increase the NS cost but decrease the
GPU time, but it won't change the performance dramatically.

What's strange is that your Core time/Wall time = (%) is quite low. If
you're running on four threads on an otherwise empty machine, you should
get close to 400 if the threads are not idling, e.g waiting for the GPU.
For instance in the rc=0.8 case you can see that the GPU/CPU balance is
<1.0 meaning that the GPU has less work than the CPU, case in which there
should be no idling and you should be getting (%) = 400.

Long story short: are you sure you're not running anything else on the
computer while simulating? What do you get if you run on CPU only?

Might such reduced cut-off be used with the force fields ( e,g charmm)
> where initially usage of longest cut-offs have given better results
> (e,g in charmm27 and gromos56 I always use 1.2 and 1.4 nm for rvdw,
> respectually) ?
>

No, at least not without *carefully* checking whether a shorter LJ cut-off
makes sense and that it does not break the physics of your simulation.

Although we advise you to consider decreasing your cut-off - mostly because
these days a large number of simulations are carried out with overly long
cut-off chosen by the rule of thumb or folclore -, you should always either
make sure that this makes sense before doing it or not do it at all.

Cheers,
--
Szilárd


>
>
> James
>
> 2013/1/10 Szilárd Páll :
> > Hi James,
> >
> > The build looks mostly fine except that you are using fftw3 compiled with
> > AVX which is slower than with only SSE (even on AVX-capable CPUs) - you
> > should have been warned about this at configure-time.
> >
> > Now, performance-wise everything looks fine except that with a 1.2 nm
> > cut-off your GPU is not able to keep up with the CPU and finish the
> > non-bonded work before the CPU is done with Bonded + PME. That's why you
> > see the "Wait GPU" taking 20% of the total time and that's also why you
> see
> > some cores idling (because for 20% of the run-time thread 0 on core 0
> > is blocked waiting for the GPU while the rest idle).
> >
> > As the suggestion at the end of the log file point out, you can consider
> > using a shorter cut-off which will push more work back to the PME on the
> > CPU, but whether you can do this it depends on your very problem.
> >
> > There is one more alternative of running two MPI processes on the GPU
> > (mpirun -np 2 mdrun -gpu_id 00) and using the -nb gpu_cpu mode which will
> > execute part of the nonbonded on the CPU, but this might not help.
> >
> > Cheers,
> >
> > --
> > Szilárd
> >
> >
> > On Wed, Jan 9, 2013 at 8:27 PM, James Starlight  >wrote:
> >
> >> Dear Szilárd, thanks for help agai

Re: [gmx-users] Gromacs 4.6 turn off ecc

[Szilárd Páll]

Hi,

Note that the CUDA NxN non-bonded kernels are programmer and
performance-tuned in such a way that they have very high compute/memory
access ratio and therefore are not sensitive to GPU memory performance.
Hence with the native GPU acceleration you won't see any improvement from
turning off ECC.

Cheers,

--
Szilárd


On Thu, Jan 10, 2013 at 12:30 PM, Justin Lemkul  wrote:

>
>
> On 1/10/13 5:43 AM, sebastian wrote:
>
>> Dear Gromacs user,
>>
>> is there a way to manually  turn off ECC (gromacs version 4.6)?
>>
>>
> That's outside of Gromacs.  Google has lots to say...
>
> http://nvidia.helpmax.net/en/**workstation/ecc-state-control/**
> how-do-i/turn-my-gpu-ecc-on-**or-off/
>
> -Justin
>
> --
> ==**==
>
> Justin A. Lemkul, Ph.D.
> Research Scientist
> Department of Biochemistry
> Virginia Tech
> Blacksburg, VA
> jalemkul[at]vt.edu | (540) 231-9080
> http://www.bevanlab.biochem.**vt.edu/Pages/Personal/justin
>
> ==**==
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] gromacs on GPU

[Szilárd Páll]

Hi,

On Thu, Jan 10, 2013 at 8:30 PM, James Starlight wrote:

> Szilárd,
>
> There are no any others cpu-usage tasks. Below you can see log from the
> TOP.
>
> 26553 own   20   0 28.4g 106m  33m S 285.6  0.7   2263:57 mdrun
>

This still shows that the average CPU utilization is only 285.6 iso 400 and
that matches with what mdrun's log shows. Try to run with a very short
cut-off, one which leads to <=1 GPU/CPU balance (i.e no waiting) and if you
still don't get 400, something weird is going on.


>  1611 root  20   0  171m  65m  24m S   3.0  0.4   7:43.05 Xorg
> 29647 own   20   0  381m  22m  17m S   3.0  0.1   0:01.77
> mate-system-mon
>  2344 own   20   0  358m  17m  11m S   1.3  0.1   0:33.76 mate-terminal
> 29018 root  20   0 000 S   0.3  0.0   0:04.99 kworker/0:0
> 29268 root  20   0 000 S   0.3  0.0   0:00.22 kworker/u:2
> 29705 root  20   0 000 S   0.3  0.0   0:00.03 kworker/3:0
> 29706 own   20   0 23284 1648 1188 R   0.3  0.0   0:00.05 top
> 1 root  20   0  8584  872  736 S   0.0  0.0   0:02.34 init
> 2 root  20   0 000 S   0.0  0.0   0:00.02 kthreadd
> 3 root  20   0 000 S   0.0  0.0   0:00.57 ksoftirqd/0
> 6 root  rt   0 000 S   0.0  0.0   0:00.00 migration/0
> 7 root  rt   0 000 S   0.0  0.0   0:00.17 watchdog/0
> 8 root  rt   0 000 S   0.0  0.0   0:00.00 migration/1
>10 root  20   0 000 S   0.0  0.0   0:00.43 ksoftirqd/1
>12 root  rt   0 000 S   0.0  0.0   0:00.17 watchdog/1
>13 root  rt   0 000 S   0.0  0.0   0:00.00 migration/2
>15 root  20   0 000 S   0.0  0.0   0:00.37 ksoftirqd/2
>16 root  rt   0 000 S   0.0  0.0   0:00.16 watchdog/2
>17 root  rt   0 000 S   0.0  0.0   0:00.00 migration/3
>19 root  20   0 000 S   0.0  0.0   0:00.38 ksoftirqd/3
>20 root  rt   0 000 S   0.0  0.0   0:00.16 watchdog/3
>21 root   0 -20 000 S   0.0  0.0   0:00.00 cpuset
>22 root   0 -20 000 S   0.0  0.0   0:00.00 khelper
>23 root  20   0 000 S   0.0  0.0   0:00.00 kdevtmpfs
>
>
> Usually I run my simulations by means of simple mdrun -v -deffnm md.
> Should I specify number of cores manually by means of -nt (or -ntmpi)
>

If you just want to run on the full machine, simply running like that
should in most cases still be the optimal run configuration or very close
to the optimal, i.e. in your case:
mdrun
<=>
mdrun -ntmpi 1 -ntomp 4 -gpu_id 0 -pinht


> flagg? Also I notice that  -pinht flagg could give me Hyper-Threading
> support. Does it reasonable in the simulation on cpu+gpu ? What
>

Correctly using HT is also fully automatic and optimal as long as you are
using the full machine.


> another possible options of md_run should I consider ? Finally is it
> possible that problems due to openMP (4.7.2) or open-mpi (1.4.5)
> drivers ?
>

No, you are using the latest version of compilers which is good. Other than
my earlier suggestions, there isn't much you can do to eliminate the idling
on the CPU (I assume that's what bugs you) - except getting a faster GPU.
Btw, have you tried the hybrid GPU-CPU mode (although I expect it to not be
faster)?

Cheers,
--
Szilárd



>
>
> Thanks for help
>
> James
>
>
> 2013/1/10 Szilárd Páll :
> > On Thu, Jan 10, 2013 at 7:25 AM, James Starlight  >wrote:
> >
> >> Szilárd ,
> >>
> >>  thanks again for explanation!
> >>
> >> Today I've performed some tests on my calmodulin in water system with
> >> different cutt-offs (I've used all cutt-ooffs 1.0 , 0.9 and 0.8
> >> respectually)
> >>
> >> Below you can see that the highest performance was in case of 0.8
> cut-offs
> >>
> >> all cut-offs 1.0
> >>  Force evaluation time GPU/CPU: 6.134 ms/4.700 ms = 1.305
> >>
> >> NOTE: The GPU has >20% more load than the CPU. This imbalance causes
> >>   performance loss, consider using a shorter cut-off and a finer PME
> >> grid.
> >>
> >>
> >>Core t (s)   Wall t (s)(%)
> >>Time: 1313.420  464.035  283.0
> >>  (ns/day)(hour/ns)
> >> Performance:9.3102.578
> >> Finished mdrun on node 0 Thu Jan 10 09:39:23 2013
> >>
> >>
> >> all cut-offs 0.9
> >> Force evaluation time GPU/CPU: 4.951 ms/4.675 ms = 1.059
> >>
> >>Core t (s)   Wall t (

Re: [gmx-users] gromacs 4.6 segfault

[Szilárd Páll]

Hi,

Please try to run the very same simulation that crashes on CPUs only (-nb
cpu). If that also crashes, please file a bug report because we surely have
a software bug.

If that does not crash, we will have to dig a bit deeper into what could be
causing the issue.

 Log files from the two runs would be useful, could you share them?

Cheers,

--
Szilárd


On Tue, Jan 15, 2013 at 12:10 AM, sebastian <
sebastian.wa...@physik.uni-freiburg.de> wrote:

> Hey GROMACS users,
>
> using mdrun (version 4.6-beta3) on a GPU node (1 nvidia K10 with cuda
> drivers and runtime 4.2 + 2 times intel 6 core E5 with hyper threading and
> SSE4.1) I get allways after a few or few 100 ns the following segfault:
>
> line 15: 28957 Segmentation fault mdrun -deffnm pdz_trans_NVT_equi_4 -maxh
> 95
>
> I can restart the system using the cpt file and run the it for the next
> few or few 100 ns and when I get the same segfault again.
> The same system runs on a different cluster (mdrun version 4.6-beta3 on a
> GPU node (1 nvidia M2090 with cuda drivers and runtime 4.2 + 2 times intel
> 6 core X5 and SSE4.1) fine for 1μs without any complains.
>
> My system consists of a 95 residue protein solvated in approx 6000 spc
> water molecules.
> .mdp parameters:
>
> ;
> title = ttt
> cpp = /lib/cpp
> include = -I../top
> constraints = hbonds
> integrator = md
> cutoff-scheme = verlet
>
> dt = 0.002 ; ps !
> nsteps = 5 ; total 5 ns
> nstcomm = 25 ; frequency for center of mass motion removal
> nstcalcenergy = 25
> nstxout = 10 ; frequency for writting the trajectory
> nstvout = 10 ; frequency for writting the velocity
> nstfout = 10 ; frequency to write forces to output trajectory
> nstlog = 100 ; frequency to write the log file
> nstenergy = 1 ; frequency to write energies to energy file
> nstxtcout = 1
>
> xtc_grps = System
>
> nstlist = 25 ; Frequency to update the neighbor list
> ns_type = grid ; Make a grid in the box and only check atoms in
> neighboring grid cells when constructing a new neighbor
> rlist = 1.4 ; cut-off distance for the short-range neighbor list
>
> coulombtype = PME ; Fast Particle-Mesh Ewald electrostatics
> rcoulomb = 1.4 ; cut-off distance for the coulomb field
> vdwtype = cut-off
> rvdw = 1.4 ; cut-off distance for the vdw field
> fourierspacing = 0.12 ; The maximum grid spacing for the FFT grid
> pme_order = 6 ; Interpolation order for PME
> optimize_fft = yes
> pbc = xyz
> Tcoupl = v-rescale
> tc-grps = System
> tau_t = 0.1
> ref_t = 300
>
> energygrps = Protein Non-Protein
>
> Pcoupl = no;berendsen
> tau_p = 0.1
> compressibility = 4.5e-5
> ref_p = 1.0
> nstpcouple = 5
> refcoord_scaling = all
> Pcoupltype = isotropic
> gen_vel = no
> gen_temp = 300
> gen_seed = -1
>
> Since I have no clue on which paramter should be tuned any guess would be
> very welcomed.
>
> Thanks
>
> Sebastian
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] >60% slowdown with GPU / verlet and sd integrator

[Szilárd Páll]

Hi Floris,

Great feedback, this needs to be looked into. Could you please file a bug
report, preferably with a tpr (and/or all inputs) as well as log files.

Thanks,

--
Szilárd


On Tue, Jan 15, 2013 at 3:50 AM, Floris Buelens wrote:

> Hi,
>
>
> I'm seeing MD simulation running a lot slower with the sd integrator than
> with md - ca. 10 vs. 30 ns/day for my 47000 atom system. I found no
> documented indication that this should be the case.
> Timings and logs pasted in below - wall time seems to be accumulating up
> in Update and Rest, adding up to >60% of total. The effect is still there
> without GPU, ca. 40% slowdown when switching from group to Verlet with the
> SD integrator
> System: Xeon E5-1620, 1x GTX 680, gromacs
> 4.6-beta3-dev-20130107-e66851a-unknown, GCC 4.4.6 and 4.7.0
>
> I didn't file a bug report yet as I don't have much variety of testing
> conditions available right now, I hope someone else has a moment to try to
> reproduce?
>
> Timings:
>
> cpu (ns/day)
> sd / verlet: 6
> sd / group: 10
> md / verlet: 9.2
> md / group: 11.4
>
> gpu (ns/day)
> sd / verlet: 11
> md / verlet: 29.8
>
>
>
> **MD integrator, GPU / verlet
>
> M E G A - F L O P S   A C C O U N T I N G
>
>  NB=Group-cutoff nonbonded kernelsNxN=N-by-N cluster Verlet kernels
>  RF=Reaction-Field  VdW=Van der Waals  QSTab=quadratic-spline table
>  W3=SPC/TIP3p  W4=TIP4p (single or pairs)
>  V&F=Potential and force  V=Potential only  F=Force only
>
>  Computing:   M-Number M-Flops  % Flops
>
> -
>  Pair Search distance check1244.988096   11204.893 0.1
>  NxN QSTab Elec. + VdW [F]   194846.615488 7988711.23591.9
>  NxN QSTab Elec. + VdW [V&F]   2009.923008  118585.457 1.4
>  1,4 nonbonded interactions  31.6163222845.469 0.0
>  Calc Weights   703.010574   25308.381 0.3
>  Spread Q Bspline 14997.558912   29995.118 0.3
>  Gather F Bspline 14997.558912   89985.353 1.0
>  3D-FFT   47658.567884  381268.543 4.4
>  Solve PME   20.5808961317.177 0.0
>  Shift-X  9.418458  56.511 0.0
>  Angles  21.8793753675.735 0.0
>  Propers 48.599718   11129.335 0.1
>  Virial  23.498403 422.971 0.0
>  Stop-CM  2.436616  24.366 0.0
>  Calc-Ekin   93.8097162532.862 0.0
>  Lincs   12.147284 728.837 0.0
>  Lincs-Mat  131.328750 525.315 0.0
>  Constraint-V   246.6336141973.069 0.0
>  Constraint-Vir  23.486379 563.673 0.0
>  Settle  74.129451   23943.813 0.3
>
> -
>  Total 8694798.114   100.0
>
> -
>
>
>  R E A L   C Y C L E   A N D   T I M E   A C C O U N T I N G
>
>  Computing: Nodes   Th. Count  Wall t (s) G-Cycles   %
>
> -
>  Neighbor search18201   0.944   27.206 3.3
>  Launch GPU ops.18   5001   0.371   10.690 1.3
>  Force  18   5001   2.185   62.987 7.7
>  PME mesh   18   5001  15.033  433.44152.9
>  Wait GPU local 18   5001   1.551   44.719 5.5
>  NB X/F buffer ops. 18   9801   0.538   15.499 1.9
>  Write traj.18  2   0.725   20.912 2.6
>  Update 18   5001   2.318   66.826 8.2
>  Constraints18   5001   2.898   83.55110.2
>  Rest   1   1.832   52.828 6.5
>
> -
>  Total  1  28.394  818.659   100.0
>
> -
>
> -
>  PME spread/gather  18  10002   8.745  252.14430.8
>  PME 3D-FFT 18  10002   5.392  155.45819.0
>  PME solve  18   5001   0.869   25.069 3.1
>
> 

Re: [gmx-users] >60% slowdown with GPU / verlet and sd integrator

[Szilárd Páll]

Hi,

Just to note for the users who might read this: the report is valid, some
non-thread-parallel code is the reason and we hope to have a fix for 4.6.0.

For updates, follow the issue #1211.

Cheers,

--
Szilárd


On Wed, Jan 16, 2013 at 4:45 PM, Berk Hess  wrote:

>
> The issue I'm referring to is about a factor of 2 in update and
> constraints, but here it's much more.
> I just found out that the SD update is not OpenMP threaded (and I even
> noted in the code why this is).
> I reopened the issue and will find a solution.
>
> Cheers.
>
> Berk
>
> 
> > Date: Wed, 16 Jan 2013 16:20:32 +0100
> > Subject: Re: [gmx-users] >60% slowdown with GPU / verlet and sd
> integrator
> > From: mark.j.abra...@gmail.com
> > To: gmx-users@gromacs.org
> >
> > We should probably note this effect on the wiki somewhere?
> >
> > Mark
> >
> > On Wed, Jan 16, 2013 at 3:44 PM, Berk Hess  wrote:
> >
> > >
> > > Hi,
> > >
> > > Unfortunately this is not a bug, but a feature!
> > > We made the non-bondeds so fast on the GPU that integration and
> > > constraints take more time.
> > > The sd1 integrator is almost as fast as the md integrator, but slightly
> > > less accurate.
> > > In most cases that's a good solution.
> > >
> > > I closed the redmine issue:
> > > http://redmine.gromacs.org/issues/1121
> > >
> > > Cheers,
> > >
> > > Berk
> > >
> > > 
> > > > Date: Wed, 16 Jan 2013 17:26:18 +0300
> > > > Subject: Re: [gmx-users] >60% slowdown with GPU / verlet and sd
> > > integrator
> > > > From: jmsstarli...@gmail.com
> > > > To: gmx-users@gromacs.org
> > > >
> > > > Hi all!
> > > >
> > > > I've also done some calculations with the SD integraator used as the
> > > > thermostat ( without t_coupl ) with the system of 65k atoms I
> obtained
> > > > 10ns\day performance on gtc 670 and 4th core i5.
> > > > I haventrun any simulations with MD integrator yet so It should test
> it.
> > > >
> > > > James
> > > >
> > > > 2013/1/15 Szilárd Páll :
> > > > > Hi Floris,
> > > > >
> > > > > Great feedback, this needs to be looked into. Could you please
> file a
> > > bug
> > > > > report, preferably with a tpr (and/or all inputs) as well as log
> files.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > --
> > > > > Szilárd
> > > > >
> > > > >
> > > > > On Tue, Jan 15, 2013 at 3:50 AM, Floris Buelens <
> > > floris_buel...@yahoo.com>wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >>
> > > > >> I'm seeing MD simulation running a lot slower with the sd
> integrator
> > > than
> > > > >> with md - ca. 10 vs. 30 ns/day for my 47000 atom system. I found
> no
> > > > >> documented indication that this should be the case.
> > > > >> Timings and logs pasted in below - wall time seems to be
> accumulating
> > > up
> > > > >> in Update and Rest, adding up to >60% of total. The effect is
> still
> > > there
> > > > >> without GPU, ca. 40% slowdown when switching from group to Verlet
> > > with the
> > > > >> SD integrator
> > > > >> System: Xeon E5-1620, 1x GTX 680, gromacs
> > > > >> 4.6-beta3-dev-20130107-e66851a-unknown, GCC 4.4.6 and 4.7.0
> > > > >>
> > > > >> I didn't file a bug report yet as I don't have much variety of
> testing
> > > > >> conditions available right now, I hope someone else has a moment
> to
> > > try to
> > > > >> reproduce?
> > > > >>
> > > > >> Timings:
> > > > >>
> > > > >> cpu (ns/day)
> > > > >> sd / verlet: 6
> > > > >> sd / group: 10
> > > > >> md / verlet: 9.2
> > > > >> md / group: 11.4
> > > > >>
> > > > >> gpu (ns/day)
> > > > >> sd / verlet: 11
> > > > >> md / verlet: 29.8
> > > > >>
> > > > >>
> > > > >>
> 

Re: [gmx-users] Problem with gromacs-4.6 compilation on Debian

[Szilárd Páll]

Hi,

Not sure why, but it looks like libcudart.so is linked against a glibc that
not compatible with what you have (perhaps much newer)?

Alternatively you could try adding "--add-needed" to the linker flags, but
I doubt it will help.

Cheers,

--
Szilárd


On Mon, Jan 21, 2013 at 5:09 PM, James Starlight wrote:

> Dear Gromacs Users!
>
>
> I've successfully installed Gromacs-4.6 on my Debian mint system but
> on another ( classic Debian) I've obtained error during make step (in
> both cases I'm using 64 bit debian):
>
> [ 55%] Building C object
> share/template/CMakeFiles/template.dir/template.c.o
> Linking CXX executable template
> /usr/local/cuda-5.0/lib64/libcudart.so: undefined reference to
> `std::__detail::_List_node_base::_M_unhook()@GLIBCXX_3.4.15'
> /usr/local/cuda-5.0/lib64/libcudart.so: undefined reference to
>
> `std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*)@GLIBCXX_3.4.15'
> collect2: ld returned 1 exit status
> make[2]: *** [share/template/template] Error 1
> make[1]: *** [share/template/CMakeFiles/template.dir/all] Error 2
> make: *** [all] Error 2
>
>
> In both cases I've used gcc-4.4 for compilation with cuda-5.
> What's wrong?
>
>
>
> James
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Problem with gromacs-4.6 compilation on Debian

[Szilárd Páll]

On Tue, Jan 22, 2013 at 12:45 PM, James Starlight wrote:

> Szilárd,
>
> Today I've tried to re-install cuda+gromacs and do apt-get
> distr-upgrade but the same error was obtained during gromacs
>

I'm don't see how does the distribution upgrade relate to the issues you
had (except if you have updated glibs to a newer version which is *still*
incompatible with libcuda. You should probably try getting a more recent
glibc and GCC version, one that libcuda is linked against. However, I'm not
sure what the source of your problem is, so I can't suggest a certain
solution.

> compilation. By the way where I could provide --add-needed option ?

CMAKE_EXE_LINKER_FLAGS=-wl,--add-needed

Cheers,
Szilárd


>
>
> James
>
>
> 2013/1/21 Szilárd Páll :
> > Szilárd
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Problem with gromacs-4.6 compilation on Debian

[Szilárd Páll]

Hi all,

Let me clarify one thing: I highly doubt that the *gcc version* upgrade is
what fixes your the issue, but rather the standard C++ library's version,
as you can see the undefined symbols refer to "GLIBCXX_3.4.15".

Of course, updating the compiler is a perfectly fine solution if you get
the new (enough version of the) standard C++ library by doing so.

Just wanted to clarify this for users bumping into this issue later.

Cheers,

--
Szilárd


On Wed, Jan 23, 2013 at 5:47 PM, Ricardo  wrote:

> On 01/22/2013 06:02 PM, Christoph Junghans wrote:
>
>> Message: 5
>>> Date: Tue, 22 Jan 2013 19:42:01 +0100
>>> From: Szil?rd P?ll
>>> Subject: Re: [gmx-users] Problem with gromacs-4.6 compilation on
>>>  Debian
>>> To: Discussion list for GROMACS users
>>> Message-ID:
>>>  >> gmail.com <3yncu1aegv7hcac3mut_75...@mail.gmail.com>>
>>> Content-Type: text/plain; charset=ISO-8859-1
>>>
>>> On Tue, Jan 22, 2013 at 12:45 PM, James Starlight>> com >wrote:
>>>
>>>  Szilárd,
>>>>
>>>> Today I've tried to re-install cuda+gromacs and do apt-get
>>>> distr-upgrade but the same error was obtained during gromacs
>>>>
>>>>  I'm don't see how does the distribution upgrade relate to the issues
>>> you
>>> had (except if you have updated glibs to a newer version which is *still*
>>> incompatible with libcuda. You should probably try getting a more recent
>>> glibc and GCC version, one that libcuda is linked against. However, I'm
>>> not
>>> sure what the source of your problem is, so I can't suggest a certain
>>> solution.
>>>
>> I had a similar problem on Gentoo Linux
>> (https://bugs.gentoo.org/show_**bug.cgi?id=452386<https://bugs.gentoo.org/show_bug.cgi?id=452386>),
>> it helped to use a
>> newer version of gcc (>=4.6).
>>
>> Cheers,
>>
>> Christoph
>>
>
>
>
> I also had this problem, and found that (manually) updating GCC to version
> 4.6 or higher has fully solved some installation issues with GROMACS 4.6.
> Please note that, for older linux dists, updating via console 'apt-get
> update' does not assure that you'd get newer versions of GCC, therefore, in
> this case, it should be downloaded and installed manually.
>
> Cheers
>
>
>  compilation. By the way where I could provide --add-needed option ?
>>>>
>>> CMAKE_EXE_LINKER_FLAGS=-wl,--**add-needed
>>>
>>> Cheers,
>>> Szilárd
>>>
>>>
>>>
>>>> James
>>>>
>>>>
>>>> 2013/1/21 Szilárd Páll:
>>>>
>>>>> Szilárd
>>>>>
>>>> --
>>>> gmx-users mailing listgmx-users@gromacs.org
>>>> http://lists.gromacs.org/**mailman/listinfo/gmx-users<http://lists.gromacs.org/mailman/listinfo/gmx-users>
>>>> * Please search the archive at
>>>> http://www.gromacs.org/**Support/Mailing_Lists/Search<http://www.gromacs.org/Support/Mailing_Lists/Search>before
>>>>  posting!
>>>> * Please don't post (un)subscribe requests to the list. Use the
>>>> www interface or send it to gmx-users-requ...@gromacs.org.
>>>> * Can't post? Read 
>>>> http://www.gromacs.org/**Support/Mailing_Lists<http://www.gromacs.org/Support/Mailing_Lists>
>>>>
>>>>  --
>> Christoph Junghans
>> Web: http://www.compphys.de
>>
>
> Ricardo O S Soares
> http://fisbio.fcfrp.usp.br/**joomla/ <http://fisbio.fcfrp.usp.br/joomla/>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users<http://lists.gromacs.org/mailman/listinfo/gmx-users>
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Search<http://www.gromacs.org/Support/Mailing_Lists/Search>before
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists<http://www.gromacs.org/Support/Mailing_Lists>
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Parallelization scheme and terminology help

[Szilárd Páll]

Hi,

Here's a bit more explanation, hopefully a bit more practical and give for
you and others a better view of what's going on under mdrun's hood.


thread-MPI, or in other contexts referred to as "thread_mpi" or abbreviated
as "tMPI", is functionally equivalent with the standard MPI you'd use on a
cluster. The difference that tMPI implements MPI "ranks" using
platform-native threads (i.e pthreads on Linux, Mac OS, etc. and Win
threads on Windozz), while most standard MPI implementations use processes
for ranks. Note that threads always belong to a single process. From a
practical point of view the implication is that you don't need to use an
external MPI library (thread-MPI comes with the GROMACS source) nor the mpi
command launcher (mpirun) if you want to run on a *single machine* - a
single mdrun processes' MPI threads can only span across a single machine's
cores, but not across machines.

In other words, running the following two is equivalent:
mpirun -np 4 mdrun_mpi
mdrun -ntmpi 4
except that the former uses four MPI processes while the latter the same
amount of MPI threads. In both cases you'll have four domains, typically
with a 4x1x1 domain-decomposition, each domain assigned to an MPI
process/thread.

While thread-MPI does essentially provide multi-threading capability to
mdrun and uses efficient multi-core optimized communication and
synchronization, it does not allow exploiting the efficient core-to-core
data-sharing (through cache) advantages of multi-core CPUs as it relies on
domain-decomposition which localizes data to threads (running on a single
core).

To achieve data sharing we use OpenMP multi-threading and the data
decomposed is practically equivalent with particle-decomposition. To
illustrate this, let's take as example the update (integration) with N
particles running on four cores: the first core will compute the new
coordinates for the particles [0..N/4), the second for [N/4,N/2), etc. In
contrast, with domain-decomposition four *spatial* domains are assigned to
the four cores.

The two parallelization schemes can, of course, be combined and one can use
domain-decomposition with this particle-decompostion-like parallelization
together. In implementation this means in (t)MPI + OpenMP parallelization
(=hybrid or multi-level).

Now, if you have a machine with two sockets and two CPUs, these communicate
through a fast link (QPI on Intel HyperTransport on AMD) which does allow
inter-core communication and therefore can be used without explicit (MPI)
communication. Hence, you can run OpenMP-only parallelization, that is
2x4x2=16 threads, on your machine* by:
mdrun -ntmpi 1 -ntomp 16
as well as tMPI-only:
mdrun -ntmpi 16 -ntomp 1
or mixed:
mdrun -ntmpi 2 -ntomp 8

Mixing MPI+OpenMP has a non-negligible overhead and in our current
algorithms/implementation, it only becomes considerably useful with GPUs,
or in highly parallel runs where communication (mostly PME) becomes
bottleneck or when the domain-decomposition limits the parallelization (by
e.g halving the number of domains needed with -ntomp 2).

Note that Intel machines, especially Sandy Bridge, have much more efficient
inter-core/socket communication than AMD. On a 2x8-core Sandy Bridge
machine one can run 16 OpenMP threads typically with much better
performance than 16 thread-MPI, but on AMD above 4-8 OpenMP becomes
inefficient and running across sockets is out of question. As noted before,
except with GPUs and/or at high parallelization mixing MPI and OpenMP is
rarely advantageous.


We tried to come up with a decent level of automation for mdrun launch
configuration (i.e. #of threads). So the launch configuration should, in
most cases, give the best performance and mdrun will always try to use all
cores "filling up" the machine with OpenMP threads, i.e.
mdrun -ntmpi 2
on a four core machines will result in 2 tMPI x 2 OpenMP and is equivalent
with
mdrun -ntmpi 2 -ntomp 2, but (for the aforementioned reasons) "mdrun"
without arguments with be equivalent with
mdrun -ntomp 4.
Note that with MPI you need to specify a fixed number of processes, so if
you request 4x8-core nodes but only 8 processes (mpirun -np 8), you'll get
4 OpenMP threads started per process (which gives 8x4=32 threads).

NOTE: Full OpenMP parallelization is available only with the Verlet scheme,
the group scheme supports OpenMP for PME and can be used to improve scaling
at high parallelization on clusters.


Coming back to your machine, depending on the Intel CPU generation, OpenMP
can be the default. On "Nehalem" (e.g. Wesmere Xeon) CPUs by default we use
OpenMP only with max 12 threads, above tMPI. You can certainly try -ntomp
16, for you it *could* be faster than -ntmpi 16.

I hope this helps, I'll try to add some more clarification to the
acceleration/parallelization page later.

Cheers,
--
Szilárd

On Mon, Jan 21, 2013 at 11:50 PM, Brad Van Oosten  wrote:

> I have been lost in the sea of terminology for installing gromacs with
> multi-processors.   The plan is

Re: [gmx-users] Problem with gromacs-4.6 compilation on Debian

[Szilárd Páll]

On Thu, Jan 24, 2013 at 3:28 PM, Justin Lemkul  wrote:

>
>
> On 1/24/13 9:23 AM, James Starlight wrote:
>
>> oh that was simply solved by upgrading of G++ :)
>>
>> the only problem which remains is the missing of support of mu GPU :(
>> That time I've tried to start simulation on simply 2 cores CPU + geforce
>> 9500
>>
>>  From md run I've obtained
>>>
>>
>> NOTE: Error occurred during GPU detection:
>>CUDA driver version is insufficient for CUDA runtime version
>>Can not use GPU acceleration, will fall back to CPU kernels.
>>
>> ED sampling will be performed!
>> ED: Reading edi file new_a2a_pca_biased.edi
>> ED: Note: Reference and average structure are composed of the same atom
>> indices.
>> ED: Found 1 ED group.
>> Using 1 MPI thread
>> Using 2 OpenMP threads
>>
>> No GPUs detected
>>
>>
>> I've installed cuda-5.0 with driver
>> NVRM version: NVIDIA UNIX x86_64 Kernel Module  304.54  Sat Sep 29
>> 00:05:49 PDT 2012
>>
>> that system works fine with the recent nvidia GPU's but has no support
>> for GeForce 9500.
>>
>>
> GeForce 9500 cards have a compute capability of 1.1.  The minimum required
> for Gromacs is 2.0 (noted in the installation instructions).


Exactly! Anything below CC 2.0 lacks certain features that make those
devices rather  slow for our algorithms and therefore pretty much
impractical.

Cheers,
--
Szilárd





>
>
> -Justin
>
> --
> ==**==
>
> Justin A. Lemkul, Ph.D.
> Research Scientist
> Department of Biochemistry
> Virginia Tech
> Blacksburg, VA
> jalemkul[at]vt.edu | (540) 231-9080
> http://www.bevanlab.biochem.**vt.edu/Pages/Personal/justin
>
> ==**==
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Problem with gromacs-4.6 compilation on Debian

[Szilárd Páll]

Let me add two more things.

Note that we *always* compare performance and acceleration to our
super-tuned state-of-the-art CPU code, which I can confidently say that is
among the fastest if not the fastest to date, and never to some slow (CPU)
implementation. Therefore, while other codes might be able to get 10-20x
with GPUs and many x-es even with pre-Fermi (CC <2.0) cards, for obvious
reasons we simply can't.

Still you'll get high *absolute performance* regardless whether you can use
CPU only or CPU+GPU, so this should be a decent deal, right?

If anybody volunteers to add software-based floating point atomic
operations for CC <2.0 and try running the current kernels on earlier
hardware, I'm willing to give pointers, but I simply had no time to try it
myself. This would enable using earlier cards, with a considerable
performance penalty of emulated atomic ops (plus these cards are anyway
slower), but it *might* be worth a try!

Feel free to drop me a mail if anybody is interested.

Cheers,

--
Szilárd


On Thu, Jan 24, 2013 at 3:48 PM, Szilárd Páll wrote:

> On Thu, Jan 24, 2013 at 3:28 PM, Justin Lemkul  wrote:
>
>>
>>
>> On 1/24/13 9:23 AM, James Starlight wrote:
>>
>>> oh that was simply solved by upgrading of G++ :)
>>>
>>> the only problem which remains is the missing of support of mu GPU :(
>>> That time I've tried to start simulation on simply 2 cores CPU + geforce
>>> 9500
>>>
>>>  From md run I've obtained
>>>>
>>>
>>> NOTE: Error occurred during GPU detection:
>>>CUDA driver version is insufficient for CUDA runtime version
>>>Can not use GPU acceleration, will fall back to CPU kernels.
>>>
>>> ED sampling will be performed!
>>> ED: Reading edi file new_a2a_pca_biased.edi
>>> ED: Note: Reference and average structure are composed of the same atom
>>> indices.
>>> ED: Found 1 ED group.
>>> Using 1 MPI thread
>>> Using 2 OpenMP threads
>>>
>>> No GPUs detected
>>>
>>>
>>> I've installed cuda-5.0 with driver
>>> NVRM version: NVIDIA UNIX x86_64 Kernel Module  304.54  Sat Sep 29
>>> 00:05:49 PDT 2012
>>>
>>> that system works fine with the recent nvidia GPU's but has no support
>>> for GeForce 9500.
>>>
>>>
>> GeForce 9500 cards have a compute capability of 1.1.  The minimum
>> required for Gromacs is 2.0 (noted in the installation instructions).
>
>
> Exactly! Anything below CC 2.0 lacks certain features that make those
> devices rather  slow for our algorithms and therefore pretty much
> impractical.
>
> Cheers,
> --
> Szilárd
>
>
>
>
>
>>
>>
>> -Justin
>>
>> --
>> ==**==
>>
>> Justin A. Lemkul, Ph.D.
>> Research Scientist
>> Department of Biochemistry
>> Virginia Tech
>> Blacksburg, VA
>> jalemkul[at]vt.edu | (540) 231-9080
>> http://www.bevanlab.biochem.**vt.edu/Pages/Personal/justin<http://www.bevanlab.biochem.vt.edu/Pages/Personal/justin>
>>
>> ==**==
>>
>> --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/**mailman/listinfo/gmx-users<http://lists.gromacs.org/mailman/listinfo/gmx-users>
>> * Please search the archive at http://www.gromacs.org/**
>> Support/Mailing_Lists/Search<http://www.gromacs.org/Support/Mailing_Lists/Search>before
>>  posting!
>> * Please don't post (un)subscribe requests to the list. Use the www
>> interface or send it to gmx-users-requ...@gromacs.org.
>> * Can't post? Read 
>> http://www.gromacs.org/**Support/Mailing_Lists<http://www.gromacs.org/Support/Mailing_Lists>
>>
>
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Problem with gromacs-4.6 compilation on Debian

[Szilárd Páll]

On Thu, Jan 24, 2013 at 7:09 PM, Christoph Junghans wrote:

> > Date: Thu, 24 Jan 2013 01:07:04 +0100
> > From: Szil?rd P?ll 
> > Subject: Re: [gmx-users] Problem with gromacs-4.6 compilation on
> > Debian
> > To: Discussion list for GROMACS users 
> > Message-ID:
> >  xwqyn3xwoeujjpubh4bfc1r7ynhs...@mail.gmail.com>
> > Content-Type: text/plain; charset=ISO-8859-1
> >
> > Hi all,
> >
> > Let me clarify one thing: I highly doubt that the *gcc version* upgrade
> is
> > what fixes your the issue, but rather the standard C++ library's version,
> > as you can see the undefined symbols refer to "GLIBCXX_3.4.15".
> Usually that is the case but for std::, the symbols are defined in
> libstdc++, which is part of the gcc package.
>
> $ strings /usr/lib/gcc/i686-pc-linux-gnu/4.6.3/libstdc++.so | grep _M_hook
> _ZNSt6__norm15_List_node_base7_M_hookEPS0_
> _ZNSt15_List_node_base7_M_hookEPS_
> _ZNSt8__detail15_List_node_base7_M_hookEPS0_
> _ZNSt9__cxx199815_List_node_base7_M_hookEPS0_
>

Christoph, you're right I was just looking at the Ubuntu setup where
libstdc++ comes as a separate package - although it is in a similar
location as the one you suggest and it is a hard dependency for g++.

So folks & fellow gglers: the conclusion is that if you see the
initially reported errors, you should update your g++ (which will get you
new libstdc++).

Cheers,
--
Szilárd



>
> Christoph
>
> >
> > Of course, updating the compiler is a perfectly fine solution if you get
> > the new (enough version of the) standard C++ library by doing so.
> >
> > Just wanted to clarify this for users bumping into this issue later.
> >
> > Cheers,
> >
> > --
> > Szilárd
>
> --
> Christoph Junghans
> Web: http://www.compphys.de
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] On the choosing of GPU for MD runs

[Szilárd Páll]

Hi,

Typically, the fastest the GPU the better. On a more detailed level, less
multiprocessors (that's not "cores"!) with faster clock speeds help in
achieving higher performance with small systems. As GPUs are highly
parallel processors, 20-30k atoms/GPU can be considered small-ish -
especially with Kepler GPUs.

Our algorithms neither require large amount of memory nor are affected very
much by the speed of the on-board GPU main memory. For this reason ECC,
which is available on Tesla compute cards, has a negligible effect on the
performance.

The ideal CPU and GPU depends on the type of simulation and the amount/mix
of force computational work determined by system and settings.  With PME is
used, the bonded force and long-range electrostatics is calculated on the
CPU while the GPU computes non-bonded forces (see the wiki for more
details). Therefore, the longer the cut-off you provide in the mdp file
(which will result in coarser the PME grid) the relatively higher the GPU
non-bonded workload compared to the CPU PME workload will be. mdrun does
automated particle-particle - PME load balancing but I will not get into
those details now. The point is that if for a given cut-off the GPU is not
fast enough to finish before the CPU, the CPU will have to wait and
therefore idle; the opposite case will be handled by the PP-PME load
balancing. Therefore, for a modern desktop CPU, as the GROMACS CPU code is
also highly tuned, you will typically need a mid to high-end desktop
GeForce card or a Tesla GPU. If you use long cut-offs (>1.2 nm), my guess
is that you might need a faster GPU for the pretty fast Intel CPU you have,
but that you can see by looking at the amount of "Wait for GPU" time in the
log file.

For reference, on an Intel i7-3930K and a GTX 680 with a 134k solvated
protein system with PME, rc=1.0, dt=5fs (with virtual sites) I get 33
ns/day (15 ns/day with dt=0.2 without virtual sites) with pretty good
CPU-GPU balance (only a few % waiting for GPU). If I increase the cut-off
to 1.2 nm, the performance drops to 26.8 ns/day caused by GPU taking much
more time to compute due to the larger cut off (the force-only kernel takes
30% more time) and the CPU ends up waiting for 35% of the total time.

Cheers,

--
Szilárd


On Mon, Jan 14, 2013 at 9:00 AM, James Starlight wrote:

> Dear Gromacs Users!
>
>
> I wounder to know some detailes about choosing of the gpu for md with
> gromacs. In particular on what properties of the videoadapter should I
> pay most attention ? What modern gpu nvidia-series might give best
> performance (gtx 6xx, tesla or quadro series) ? Could you provide me
> with some bechmarks besides the information present on the Gromacs web
> ?
>
> For instance with the gpu Geforce GTX 670 + core i5 (4 cores) I have
> performance 10ns\day for explicit system with 67000 atoms ( protein in
> tip3p water). Have someone better results with common home-like
> desktop? :)
>
>
> James
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Re: On the choosing of GPU for MD runs

[Szilárd Páll]

On Mon, Jan 14, 2013 at 7:20 PM, James Starlight wrote:

> 1)As I understood from nvidia site at present time GPU's from the
> tesla serie are the best for the calculations like md. For instasnce
> NVIDIA Tesla C2075 have performance ~ 1 TFLOPS in the
> single-precission calculations. I wounder to know how many 8-core CPU
> must have typical cluster to obtain such performance ? Have someone
> tried to use tesla GPU with gromacs ? What real performance in ns\day
> have been obtained with the explicit solvent systems?
>
>
> 2) Today I performed test gpu-based calculation with 2 different gpu's
> ( GTX 670 vs GT 640). Both of that gpu's have the same GPU frequency
> but GTX 670 has 256 bit memory interface width with ddr5 ram worked on
> higher frequency. As the result I've obtained 50% more perfomance with
> the GTX 670. Does the memory interface width as well as ram frequency
> affect on total GPU performance in addition to GPU frequency?
>
>
As mentioned before, the memory does not matter as much as the number of
processing cores on the GPUs and their frequency.
- GTX 670: 1344 "CUDA cores" (7 multiprocessors * 192 cores) running at 915
Mhz.
- GTX 640: 384 "CUDA cores" (2 multiprocessors * 192 cores) running at 900
Mhz.

The raw GPU performance with our current algorithms has a roughly linear
relationship with the number of cores and frequency, so the raw GPU kernel
performance difference between 670 and 640 should be around 3.5x. It is a
bit surprising that you've only got 50% higher total performance with the
670.

--
Szilárd



> James
>
>
>
> 2013/1/14 James Starlight :
> > Dear Gromacs Users!
> >
> >
> > I wounder to know some detailes about choosing of the gpu for md with
> > gromacs. In particular on what properties of the videoadapter should I
> > pay most attention ? What modern gpu nvidia-series might give best
> > performance (gtx 6xx, tesla or quadro series) ? Could you provide me
> > with some bechmarks besides the information present on the Gromacs web
> > ?
> >
> > For instance with the gpu Geforce GTX 670 + core i5 (4 cores) I have
> > performance 10ns\day for explicit system with 67000 atoms ( protein in
> > tip3p water). Have someone better results with common home-like
> > desktop? :)
> >
> >
> > James
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Gromacs 4.6 compilation with Open64

[Szilárd Páll]

Hi,

Earlier in the development of 4.6 I have tried Open64 (4.5.1, I guess)
hoping that it would help with testing/improving Bulldozer support, but I
quickly found a compiler bug or two that prevented me from being able to
actually use it. I reported the bugs to AMD so they might have fixed them,
but have not tried it since.

Regarding your errors, it looks like the include test on immintrin.h
(required for AVX support) fails which might be because your include path
is not set up correctly. I suggest that you first try
-DGMX_CPU_ACCELERATION=SSE4.1 and see if it compiles and passes regression
tests successfully, then try to get that include detected. You could
perhaps add the include path manually to a cloned gcc toolchain file.

Btw, is there any particular reason why you want to use Open64?

Cheers,

--
Szilárd


On Tue, Jan 29, 2013 at 1:06 PM, David McGiven wrote:

> Hi there,
>
> Has anyone succeded compiling gromacs 4.6 with Open64 ?
>
> I'm using the following configuration :
> export PATH=/opt/open64-4.5.2/bin:$PATH
> export CC=opencc; export CXX=opencc; export F90=openf90; F95=openf95
> export TOOLROOT=/opt//open64-4.5.2
> /opt/cmake-2.8.10.2/bin/cmake /opt/gromacs-4.6 -DGMX_BUILD_OWN_FFTW=ON
> -DBUILD_SHARED_LIBS=OFF -DGMX_PREFER_STATIC_LIBS=ON
>
> I'm getting the following error :
>
> CMake Error at CMakeLists.txt:791 (message):
>   Cannot find immintrin.h, which is required for AVX intrinsics support.
>   Consider switching compiler.
>
> -- Configuring incomplete, errors occurred!
>
> I have tried adding the folder where immintrin.h is located :
> export
>
> CMAKE_INCLUDE_PATH=/opt/open64-4.5.2/include/4.5.2/:/opt/open64-4.5.2/open64-gcc-4.2.0/lib/gcc/x86_64-redhat-linux/4.2.0/include/
>
> But I still get the same error.
>
> Cheers,
> David
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] question re; building Gromacs 4.6

[Szilárd Páll]

On Tue, Jan 29, 2013 at 4:39 PM, Susan Chacko  wrote:

>
> Sorry for a newbie question -- I've built several versions of Gromacs in
> the
> past but am not very familiar with the new cmake build system.
>
> In older versions, the procedure was:
> - build the single-threaded version
>

FYI there is really no point in compiling GROMACS with all parallelization
disabled, and by default you get (thread-)MPI and OpenMP which work just
fine within a node (and mdrun as well as a few tools support
multi-threading).


> - then build the MPI version of mdrun only. No need to build the other
> executables with MPI.
>
> Is this still how it should be done, or should one just build everything
> once with MPI?
>

That is how I usually do it so that I have tools and a multi-threaded mdrun
without without MPI dependency, then configure again with MPI and install
to the same location mdrun only (make install-mdrun) - which by deafult
will have the _mpi suffix.


> Likewise, if I want a separate GPU version (only a few nodes on our
> cluster have GPUs), do I build the whole tree separately with -DGMX_GPU=ON,
> or just a GPU-enabled version of mdrun?
>

You only need to build separate GPU-enabled version of mdrun if you can not
have CUDA available on the non-GPU nodes (as mdrun links against the CUDA
libraries). Otherwise, a GPU-acceleration enabled mdrun on a node without
GPUs works just fine.


Cheers,
--
Szilard


> Thanks for any suggestions,
> Susan.
>
>
>
>
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Decreasing of the box size during membrane protein simulation

[Szilárd Páll]

Great discussion!

While (seemingly) small changes in cut-off can give considerable
performance improvements and therefore it is worth considering such tweaks
in simulation settings, these finding highlight the importance of not
changing settings blindly.

Cheers,

--
Szilárd


On Wed, Jan 30, 2013 at 2:35 PM, Justin Lemkul  wrote:

>
>
> On 1/29/13 11:57 PM, James Starlight wrote:
>
>> Justin,
>>
>>
>> with cut-offs=1.2 both in cpu+gpu or with cpu only I've obtained only
>> small decrease along the Z-dim ( from 10 to 9 nm). With cut-offs 1.0
>> I've obtained decreae from 10 to 8.5. So it might be concluded that
>> the observable is cut-off sensetive.
>>
>>
> Good to know.  Most "normal" systems of proteins in water will probably be
> insensitive to most changes with the new cutoff scheme, but I was
> suspicious about membranes.  I'll have to do some of my own testing, as
> well.
>
>
>  By the way during simulation of such system in gromos-54a7 united atom
>> ( the same number of lipids and water)  ff with 1.2 cut-offs I didnt
>> observed any decrease of the cutoffs.
>>
>>
> Different force fields behave differently.  I wouldn't expect to
> necessarily be able to draw a parallel between different parameter sets
> that inherently use different run parameters, but this is also good to know.
>
> -Justin
>
>
>  2013/1/29 Justin Lemkul :
>>
>>>
>>>
>>> On 1/29/13 2:19 AM, James Starlight wrote:
>>>

 One important point:

 in that simulations I've used decreased cut-offs with charmm36 ff
 because that systems have been modelled with CPU+GPU so I had some
 imbalance in cpu\gpu loadings with common (1.2) cutoffs.

 rlist   = 0.8   ; Cut-off for making neighbor list
 (short
 range forces)
 rlistlong   = 1.4
 rcoulomb= 0.8   ; long range electrostatic cut-off
 rvdw= 0.8


 Might that affect on the compresbility of my system? Might  I prevent
 such compression by means of changing ref_p or compressibility on the
 Z using  emiisotropic pcoupltype?


>>> Membranes are very sensitive to cutoff settings.  Can you test by
>>> running on
>>> CPU only with the normal CHARMM cutoff settings?  I would be curious to
>>> see
>>> if the shortened cutoffs produce bad results.
>>>
>>> You should not have to make any special changes to the compressibility or
>>> pressure along Z in order for the simulation to be stable.  What's
>>> happening
>>> to the x and y box vectors?  Is the membrane becoming distorted or
>>> compressed?  Or is it expanding laterally?
>>>
>>>
>>>
>>>  2013/1/29 James Starlight :

>
> Its intresting that on the same system which was equilibrated longer
> the decrease on the Z dim was smaller (from 10 to 9nm). By the way
> does it possible to simulate membrane proteins (with explicit
> membrane) in the nvt enssemble without explicit barostat ? What
> options in the mdp should be added for such simulation ?
>
>
>>> Certainly you can run in an NVT ensemble if the force field will produce
>>> good results.  Normally NPT or NPAT ensembles are most appropriate for
>>> membranes, depending on the parameterization of the lipids.  Running in
>>> NVT
>>> is as simple as setting "pcoupl = no" while leaving other
>>> thermostat-related
>>> items the same.
>>>
>>> -Justin
>>>
>>>
>>>
> James
>
> 2013/1/28 Justin Lemkul :
>
>>
>>
>>
>> On 1/28/13 8:45 AM, James Starlight wrote:
>>
>>>
>>>
>>> Justin,
>>>
>>> yes, 2 A for C atoms.
>>>
>>> The dims are 8.68740   8.41864  10.0
>>>
>>>
>> Well, with such a dramatic change, it should be fairly easy to simply
>> watch
>> the trajectory and see what went wrong.
>>
>>
>> -Justin
>>
>> --
>> ==**==
>>
>> Justin A. Lemkul, Ph.D.
>> Research Scientist
>> Department of Biochemistry
>> Virginia Tech
>> Blacksburg, VA
>> jalemkul[at]vt.edu | (540) 231-9080
>> http://www.bevanlab.biochem.**vt.edu/Pages/Personal/justin
>>
>> ==**==
>> --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/**mailman/listinfo/gmx-users
>> * Please search the archive at
>> http://www.gromacs.org/**Support/Mailing_Lists/Searchbefore
>>  posting!
>> * Please don't post (un)subscribe requests to the list. Use the www
>> interface or send it to gmx-users-requ...@gromacs.org.
>> * Can't post? Read 
>> http://www.gromacs.org/**Support/Mailing_Lists
>>
>
>>>
>>> --
>>> ==**==
>>>
>>> Justin A. Lemkul, Ph.D.
>>> Research

Re: [gmx-users] Re: question about fftw3 in gromacs 4.6 installation

[Szilárd Páll]

On Sat, Feb 2, 2013 at 2:38 PM, Tomek Wlodarski
wrote:

> On Sat, Feb 2, 2013 at 12:34 PM, Mark Abraham  >wrote:
>
> > On Sat, Feb 2, 2013 at 1:00 AM, Justin Lemkul  wrote:
> >
> > >
> > >
> > > On 2/1/13 5:26 PM, didymos wrote:
> > >
> > >> Hi Justin!
> > >>
> > >> I had the same problem and I used your advise but I am not sure if
> this
> > >> entirely worked for me...
> > >>
> > >> I used:
> > >> cmake -DGMX_MPI=ON DCMAKE_INSTALL_PREFIX=$HOME/**gromacs
> > >> -DFFTWF_INCLUDE_DIR=$HOME/**fftw/include
> > >> -DFFTWF_LIBRARY=$HOME/fftw/**lib/libfftw3f.so
> > -DCMAKE_PREFIX_PATH=$HOME/*
> > >> *fftw/
> > >> ..
> > >>
> > >>
> > > The use of -DCMAKE_PREFIX_PATH eliminates the need for
> > -DFFTWF_INCLUDE_DIR
> > > and -DFFTWF_LIBRARY.  Not that this is the source of your problem, but
> > you
> > > can certainly trim down your command.
> >
> >
> > Indeed. Also missing hyphen before "DCMAKE_INSTALL_PREFIX"
> >
>
> Thanks a lot Mark and Justin - I fixed it.
>
> >
> >  and I got:
> > >> -- checking for module 'fftw3f'
> > >> --   found fftw3f, version 3.2.2
> > >> -- Looking for fftwf_plan_r2r_1d in
> > >> /home/users/didymos/fftw/lib/**libfftw3f.so
> > >> -- Looking for fftwf_plan_r2r_1d in
> > >> /home/users/didymos/fftw/lib/**libfftw3f.so - found
> > >> -- Looking for fftwf_have_simd_avx in
> > >> /home/users/didymos/fftw/lib/**libfftw3f.so
> > >> -- Looking for fftwf_have_simd_avx in
> > >> /home/users/didymos/fftw/lib/**libfftw3f.so - not found
> > >> -- Looking for fftwf_have_simd_sse2 in
> > >> /home/users/didymos/fftw/lib/**libfftw3f.so
> > >> -- Looking for fftwf_have_simd_sse2 in
> > >> /home/users/didymos/fftw/lib/**libfftw3f.so - found
> > >>
> > >> So it seems ok but I am not sure about first two lines:
> > >> -- checking for module 'fftw3f'
> > >> --   found fftw3f, version 3.2.2
> > >> it should be 3.3.3
> > >>
> > >
> > > Is there a different FFTW installed anywhere else on your system?  I
> > > believe the first "found fftw3f" statement comes from pkgconfig
> > detection,
> > > but this is superseded by what you specify manually.  Perhaps someone
> who
> > > is more familiar with this element of the build system can comment.
> >
> >
> > Yeah the first message is CMake finding a system fftw via pkgconfig, but
> > then there is further detection and the one found via pkconfig need not
> be
> > the one used. I guess if we have the two layers of detection we need to
> be
> > more clear about what is going on.
> >
>
>
> Yes, I am installing gromacs on a server and I want to use my local version
> of FFTW rather than the one founded by pkgconfig.
> So to make it clear for me :), which version of FFTW in the end will be
> used in my case? this detected by pkgconfig or from DCMAKE_PREFIX_PATH
> Thanks!
>

I can't tell, but you can check it yourself by having a look at the "mdrun
-version" output which contains the FFTW version. :)


>
> tomek
>
>
> > Mark
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Re: question about fftw3 in gromacs 4.6 installation

[Szilárd Páll]

Tomek,

That looks like your assembler does not support the XOP instruction set (
http://goo.gl/nMSX0) and chokes on XOP instructions generated by the
compiler. As you did not post much detail on the compilers you are using
and cmake cache, but my guess is that you'll need to either update
your assembler or use SSE4.1 acceleration (AVX128_FMA also enables XOP).

Cheers,

--
Szilárd


On Sat, Feb 2, 2013 at 8:48 PM, Tomek Wlodarski
wrote:

> Thanks Szilárd, good idea:)
> I tried to build gromacs and I got errors..:
>
> /tmp/ccBvi1my.s: Assembler messages:
> /tmp/ccBvi1my.s:110: Error: no such instruction: `vpcmov
> %xmm13,%xmm5,%xmm14,%xmm10'
> [...]
> make[2]: *** [src/gmxlib/CMakeFiles/gmx.dir/ewald_util.c.o] Error 1
> make[2]: *** Waiting for unfinished jobs
> /tmp/ccncRj9M.s: Assembler messages:
> /tmp/ccncRj9M.s:431: Error: no such instruction: `vpshad %xmm3,%xmm0,%xmm1'
> []
> make[2]: *** [src/gmxlib/CMakeFiles/gmx.dir/bondfree.c.o] Error 1
> make[1]: *** [src/gmxlib/CMakeFiles/gmx.dir/all] Error 2
> make: *** [all] Error 2
>
> Any idea?
> Gromacs confirguration with cmake went ok appart from:
>
> -- Check size of bool - failed
> -- Performing Test _isfinite_compile_ok - Failed
> -- Performing Test _finite_compile_ok - Failed
>
> I do not know if those are related to building errors..
> Thanks!
> Best!
>
> tomek
>
>
> On Sat, Feb 2, 2013 at 3:40 PM, Szilárd Páll 
> wrote:
>
> > On Sat, Feb 2, 2013 at 2:38 PM, Tomek Wlodarski
> > wrote:
> >
> > > On Sat, Feb 2, 2013 at 12:34 PM, Mark Abraham <
> mark.j.abra...@gmail.com
> > > >wrote:
> > >
> > > > On Sat, Feb 2, 2013 at 1:00 AM, Justin Lemkul 
> wrote:
> > > >
> > > > >
> > > > >
> > > > > On 2/1/13 5:26 PM, didymos wrote:
> > > > >
> > > > >> Hi Justin!
> > > > >>
> > > > >> I had the same problem and I used your advise but I am not sure if
> > > this
> > > > >> entirely worked for me...
> > > > >>
> > > > >> I used:
> > > > >> cmake -DGMX_MPI=ON DCMAKE_INSTALL_PREFIX=$HOME/**gromacs
> > > > >> -DFFTWF_INCLUDE_DIR=$HOME/**fftw/include
> > > > >> -DFFTWF_LIBRARY=$HOME/fftw/**lib/libfftw3f.so
> > > > -DCMAKE_PREFIX_PATH=$HOME/*
> > > > >> *fftw/
> > > > >> ..
> > > > >>
> > > > >>
> > > > > The use of -DCMAKE_PREFIX_PATH eliminates the need for
> > > > -DFFTWF_INCLUDE_DIR
> > > > > and -DFFTWF_LIBRARY.  Not that this is the source of your problem,
> > but
> > > > you
> > > > > can certainly trim down your command.
> > > >
> > > >
> > > > Indeed. Also missing hyphen before "DCMAKE_INSTALL_PREFIX"
> > > >
> > >
> > > Thanks a lot Mark and Justin - I fixed it.
> > >
> > > >
> > > >  and I got:
> > > > >> -- checking for module 'fftw3f'
> > > > >> --   found fftw3f, version 3.2.2
> > > > >> -- Looking for fftwf_plan_r2r_1d in
> > > > >> /home/users/didymos/fftw/lib/**libfftw3f.so
> > > > >> -- Looking for fftwf_plan_r2r_1d in
> > > > >> /home/users/didymos/fftw/lib/**libfftw3f.so - found
> > > > >> -- Looking for fftwf_have_simd_avx in
> > > > >> /home/users/didymos/fftw/lib/**libfftw3f.so
> > > > >> -- Looking for fftwf_have_simd_avx in
> > > > >> /home/users/didymos/fftw/lib/**libfftw3f.so - not found
> > > > >> -- Looking for fftwf_have_simd_sse2 in
> > > > >> /home/users/didymos/fftw/lib/**libfftw3f.so
> > > > >> -- Looking for fftwf_have_simd_sse2 in
> > > > >> /home/users/didymos/fftw/lib/**libfftw3f.so - found
> > > > >>
> > > > >> So it seems ok but I am not sure about first two lines:
> > > > >> -- checking for module 'fftw3f'
> > > > >> --   found fftw3f, version 3.2.2
> > > > >> it should be 3.3.3
> > > > >>
> > > > >
> > > > > Is there a different FFTW installed anywhere else on your system?
>  I
> > > > > believe the first "found fftw3f" statement comes from pkgconfig
> > > > detection,
> > > > > but this is superseded by what you specify manually.  Perhaps
> someone
> > > who
>

Re: [gmx-users] Re: question about fftw3 in gromacs 4.6 installation

[Szilárd Páll]

On Sat, Feb 2, 2013 at 10:12 PM, Mark Abraham wrote:

> On Sat, Feb 2, 2013 at 8:48 PM, Tomek Wlodarski
> wrote:
>
> > Thanks Szilárd, good idea:)
> > I tried to build gromacs and I got errors..:
> >
> > /tmp/ccBvi1my.s: Assembler messages:
> > /tmp/ccBvi1my.s:110: Error: no such instruction: `vpcmov
> > %xmm13,%xmm5,%xmm14,%xmm10'
> > [...]
> > make[2]: *** [src/gmxlib/CMakeFiles/gmx.dir/ewald_util.c.o] Error 1
> > make[2]: *** Waiting for unfinished jobs
> > /tmp/ccncRj9M.s: Assembler messages:
> > /tmp/ccncRj9M.s:431: Error: no such instruction: `vpshad
> %xmm3,%xmm0,%xmm1'
> > []
> > make[2]: *** [src/gmxlib/CMakeFiles/gmx.dir/bondfree.c.o] Error 1
> > make[1]: *** [src/gmxlib/CMakeFiles/gmx.dir/all] Error 2
> > make: *** [all] Error 2
> >
> > Any idea?
> >
>
> Your compiler is probably buggy. Try a more recent one.
>
>
I'm not entirely sure about that especially that the above is an assembler
error, not a compiler one.


>
> > Gromacs confirguration with cmake went ok appart from:
> >
> > -- Check size of bool - failed
> > -- Performing Test _isfinite_compile_ok - Failed
> > -- Performing Test _finite_compile_ok - Failed
> >
> > I do not know if those are related to building errors..
> >
>
> Software build systems run a bunch of tests and react to what they learn. A
> failed test is not of itself indicative of a problem (unless the authors
> screw up, of course...)
>

Just to extend Mark's comment: certain functionalities are implemented by
different compilers with functions/variables/macros that have different
names. Instead of guessing what the name of e.g a function should be, these
test simply check all possibilities and typically fail for all but the one
name that works. Therefore, the above two failure don't mean that the
"isfinite" function is not available and in fact with gcc 4.7.2 the
"isfinite" test should pass and the "_isfinite" and "_finite" are expected
to fail.

Cheers,
--
Szilárd



>
> Mark
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Re: question about fftw3 in gromacs 4.6 installation

[Szilárd Páll]

On Sat, Feb 2, 2013 at 10:16 PM, Tomek Wlodarski
wrote:

> On Sat, Feb 2, 2013 at 10:12 PM, Mark Abraham  >wrote:
>
> > On Sat, Feb 2, 2013 at 8:48 PM, Tomek Wlodarski
> > wrote:
> >
> > > Thanks Szilárd, good idea:)
> > > I tried to build gromacs and I got errors..:
> > >
> > > /tmp/ccBvi1my.s: Assembler messages:
> > > /tmp/ccBvi1my.s:110: Error: no such instruction: `vpcmov
> > > %xmm13,%xmm5,%xmm14,%xmm10'
> > > [...]
> > > make[2]: *** [src/gmxlib/CMakeFiles/gmx.dir/ewald_util.c.o] Error 1
> > > make[2]: *** Waiting for unfinished jobs
> > > /tmp/ccncRj9M.s: Assembler messages:
> > > /tmp/ccncRj9M.s:431: Error: no such instruction: `vpshad
> > %xmm3,%xmm0,%xmm1'
> > > []
> > > make[2]: *** [src/gmxlib/CMakeFiles/gmx.dir/bondfree.c.o] Error 1
> > > make[1]: *** [src/gmxlib/CMakeFiles/gmx.dir/all] Error 2
> > > make: *** [all] Error 2
> > >
> > > Any idea?
> > >
> >
> > Your compiler is probably buggy. Try a more recent one.
> >
>
> Actually, I am using gcc 4.7.2 :)
>
> I could use less recent ;)
>

Could you try compiling some other code with the "-mxop" compiler flag.
With this flag the compiler should generate the XOP instructions that cause
problems above (but it's not entirely sure that it will, I think).

--
Szilárd


>
> >
> >
> > > Gromacs confirguration with cmake went ok appart from:
> > >
> > > -- Check size of bool - failed
> > > -- Performing Test _isfinite_compile_ok - Failed
> > > -- Performing Test _finite_compile_ok - Failed
> > >
> > > I do not know if those are related to building errors..
> > >
> >
> > Software build systems run a bunch of tests and react to what they
> learn. A
> > failed test is not of itself indicative of a problem (unless the authors
> > screw up, of course...)
> >
> > Mark
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] compiling on different architecture than the compute nodes architecture

[Szilárd Páll]

On Wed, Feb 6, 2013 at 6:03 PM, Richard Broadbent <
richard.broadben...@imperial.ac.uk> wrote:

> Dear All,
>
> I would like to compile gromacs 4.6 to run with the correct acceleration
> on the compute nodes on our local cluster. Some of the nodes have intel
> sandy-bridge whilst others only have sse4.1 and some (including the login
> and single core job nodes) are still stuck on ssse3 (gmx would use sse2
> acceleration here).
>
> Installing several versions is not a problem however, I'm not sure how to
> make cmake build a version of the code that is not using the acceleration
> for the system on which the code is being compiled. Restrictions on job
> sizes makes running the compilation on the sandy-bridge nodes almost
> impossible. Can anyone let me know which flags cmake needs to enable
> avx-256 acceleration?
>
> my standard cmake line is:
>
> $ CC=mpiicc CXX=mpiicpc ; cmake -DGMX_OPENMP=ON  -DGMX_MPI=ON
> -DGMX_DOUBLE=ON -DGMX_GPU=OFF -DGMX_PREFER_STATIC_LIBS=ON
> -DGMX_FFT_LIBRARY=mkl -DMKL_INCLUDE_DIR=$MKLROOT/**include
> -DMKL_LIBRARIES="$MKLROOT/lib/**intel64/libmkl_core.so;$**
> MKLROOT/lib/intel64/libmkl_**intel_lp64.so;$MKLROOT/lib/**intel64/libmkl_sequential.so"
> -DCMAKE_INSTALL_PREFIX=$HOME/**libs/gromacs  ../
>

Note that MKL (without the fftw wrappers) is known to not work out of the
box. Making it work requires a fairly simple workaround described here:
http://redmine.gromacs.org/issues/1110#note-3

Additionally, note that FFTW is in most cases faster than MKL (but if you
find the contrary do let us know).

--
Szilárd



>
>
>
> Thanks,
>
> Richard
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] problems for GPU simulations

[Szilárd Páll]

On Thu, Feb 7, 2013 at 10:16 AM, Albert  wrote:

> Hello:
>
>  I got a workstation with two GTX590 which have two core for each GPU. I
> can submit gromacs GPU jobs with command:
>
> mpirun  -np 4 mdrun .
>
> with such running, I can get 26ns/day for Gromacs-4.6 beta version.
>
> However, I found that for Gromacs-4.6 final version (which is the latest
> one), it claimed that I only have two GPU, it asked me to adjust -np to 2.
>

Please make sure that nvididia-smi or the deviceQuery SDK tool show all
four GPUs. If that is the case and mdrun still shows only two, please file
a bug report with you OS info and a log file attached.

Cheers,
--
Szilárd



>
> so I submit the jobs with command:
>
> mpirun -np 2 mdrun...
>
> for the same system with the same paramters (of course the tpr file must
> be regenerated). I found that I can get only half of the speed, something
> around 10 ns/day.
>
>
> So I am just wondering what's happening?
>
> thank you very much
> best
> Albert
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] problems for GPU simulations

[Szilárd Páll]

Hi,

If you have two GTX 590-s four devices should show up in nvidia-smi and
mdrun should also show four devices detected. As nvidia-smi shows only two
GPUs means that one of your cards is not functioning properly.

You can try to check what GPU devices does you operating system "see"
independently form the driver using the lspci command, e.g:
lspci | grep -i ".*VGA.*NVIDIA.*"

If you see two PCI devices in this output that means that both cards are
detected by the operating system. If nvidia-smi does not show all four
GPUs, there must be something wrong with your driver.

Cheers,

--
Szilárd


On Thu, Feb 7, 2013 at 5:08 PM, Albert  wrote:

> On 02/07/2013 01:34 PM, Szilárd Páll wrote:
>
>> Please make sure that nvididia-smi or the deviceQuery SDK tool show all
>> four GPUs. If that is the case and mdrun still shows only two, please file
>> a bug report with you OS info and a log file attached.
>>
>> Cheers,
>> --
>> Szilárd
>>
>
> no, it showed two. I don't know why it only work in beta version it can
> recognize 4 GPU, but in final version only 2 The fact is that the beta
> version use -np 4 can get double speed. The GTX590 have two core, so two
> GPU have 4 core.
>
>
> here is the log for nvidia-sim:
>
> Thu Feb  7 17:27:10 2013
> +-**-+
> | NVIDIA-SMI 2.285.05   Driver Version: 285.05.33 |
> |-**--+--+**
> --+
> | Nb.  Name | Bus IdDisp.  | Volatile ECC SB /
> DB |
> | Fan   Temp   Power Usage /Cap | Memory Usage | GPU Util. Compute
> M. |
> |=**==+==+**
> ==|
> | 0.  GeForce GTX 590   | :0C:00.0  N/A| N/AN/A |
> |   0%   55 C  N/A   N/A /  N/A |  22%  336MB / 1535MB |  N/A Default|
> |-**--+--+**
> --|
> | 1.  GeForce GTX 590   | :0B:00.0  N/A| N/AN/A |
> |  43%   57 C  N/A   N/A /  N/A |   0%5MB / 1535MB |  N/A Default|
> |-**--+--+**
> --|
> | Compute processes: GPU Memory |
> |  GPU  PID Process name Usage  |
> |=**==**
> ==|
> |  0.   ERROR: Not Supported
>|
> |  1.   ERROR: Not Supported
>|
> +-**--**
> --+
>
>
>
>
> here is the log for mdrun:
>
>
> Program mdrun_mpi, VERSION 4.6
> Source code file: /home/albert/Documents/2013-**
> 02-06/gromacs-4.6/src/gmxlib/**gmx_detect_hardware.c, line: 356
>
> Fatal error:
> Incorrect launch configuration: mismatching number of PP MPI processes and
> GPUs per node.
> mdrun_mpi was started with 4 PP MPI processes per node, but only 2 GPUs
> were detected.
> For more information and tips for troubleshooting, please check the GROMACS
> website at 
> http://www.gromacs.org/**Documentation/Errors<http://www.gromacs.org/Documentation/Errors>
> --**-
>
> "I Like You. I Will Kill You Last" (Tyler in Fishtank)
>
> Error on node 0, will try to stop all the nodes
> Halting parallel program mdrun_mpi on CPU 0 out of 4
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users<http://lists.gromacs.org/mailman/listinfo/gmx-users>
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Search<http://www.gromacs.org/Support/Mailing_Lists/Search>before
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists<http://www.gromacs.org/Support/Mailing_Lists>
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] 4.6 seems improved the efficiency

[Szilárd Páll]

Additionally to the new non-bonded SIMD kernels:
- Bonded force calculation got some SIMD acceleration which should
considerably improve performance;
- There have been be major improvements in PME (courtesy of Berk Hess &
Roland Schulz), you should see anywhere between up to 1.5-2x performance
improvement. OpenMP multi-threading is supported with separate PME nodes
also with the group scheme which should reduce communication cost at high
parallelization (turn it on with -npme N -ntomp_pme M). Note that
g_tune_pme does not support tuning the number of OpenMP threads (yet).


--
Szilárd


On Mon, Feb 11, 2013 at 4:39 PM, Mark Abraham wrote:

> Indeed - that's about the order of improvement expected for just switching
> to 4.6 and proceeding as before on CPU-only hardware. Better use of SIMD
> instructions and calling force-only kernels when energies are not required
> are likely to be the code improvements leading to the performance
> improvement you note. (However, the new automated PME load balancing in
> mdrun may mean you don't need to use g_tune_pme for 4.6... see your log
> file.)
>
> Mark
>
> On Sat, Feb 9, 2013 at 9:38 AM, Albert  wrote:
>
> > Hello :
> >
> >  I found the new released 4.6 probably improved very obviously for the
> > efficiency. With the same system, I used 4.5.5 it can get 26 ns/day (144
> > CPU, 55,000 atoms, Amber FF) with g_tume_pme optimization. Now I used
> 4.6,
> > even without g_tune_pme, the  pme mesh/force can get 0.8-1.0 with
> > efficiency 32 ns/day. That's really nice.
> >
> > I don't know whether other users have similar experiences for this new
> > version.
> >
> > Albert
> >
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/**mailman/listinfo/gmx-users<
> http://lists.gromacs.org/mailman/listinfo/gmx-users>
> > * Please search the archive at http://www.gromacs.org/**
> > Support/Mailing_Lists/Search<
> http://www.gromacs.org/Support/Mailing_Lists/Search>before posting!
> > * Please don't post (un)subscribe requests to the list. Use the www
> > interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/**Support/Mailing_Lists<
> http://www.gromacs.org/Support/Mailing_Lists>
> >
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Gromacs 4.6 Installation under Cygwin

[Szilárd Páll]

On Sat, Feb 23, 2013 at 3:41 AM, Mirco Wahab <
mirco.wa...@chemie.tu-freiberg.de> wrote:

> On 23.02.2013 01:59, toma0...@umn.edu wrote:
>
>> Hello,
>>  I am trying to install Gromacs 4.6 on a Windows workstation under
>> cygwin. After I install everything, when executing g_luck I come up with
>> the error:
>> 'error while loading shared libraries: cyggmx-6.dll: cannot open shared
>> object file: No such file or directory'
>>  There does exist the file cyggmx-6.dll in /usr/local/gromacs/lib
>> and I have tried: export LD_LIBRARY_PATH=/usr/local/**gromacs/lib as well
>> as using the flag -BUILD_SHARED_LIBS=OFF with cmake, but neither seem to
>> help. What could be the cause of this?
>>
>
> For cygwin, you need to have the .dll directory path,
> which  is located at the position given by:
>
>   -DCMAKE_INSTALL_PREFIX=/my/**install/path  trailed by /lib
>
> in your PATH variable:
>
>   PATH=/my/install/path/lib:$**PATH   mdrun
>
> (or set it permanent somewhere)
>
>
> BTW: with the current cygwin (1.7.17-1), Gromacs 4.6 *does*
> indeed compile fine:
>
> cmake  -DCMAKE_INSTALL_PREFIX=/usr/**local -DGMX_GPU=OFF
> -DGMX_BUILD_OWN_FFTW=OFF  ../gromacs-4.6
>
> and, more interestingly, runs multithreaded at similar
> performance as compiled with visual studio (tested on Win8/x64).


Thanks for the feedback! Would you mind adding the information on the wiki.
It could help many others in getting GROMACS to work with cygwin and we
have no development of testing that covers that platform - at least not
that I know of.

Cheers,
--
Szilárd



>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Re: Gromacs 4.6 Installation under Cygwin

[Szilárd Páll]

That's strange, it seems that mdrun gets stuck somewhere. This should not
happen, but as we don't actively test cygwin, we can't be sure what's
happening. It would be great if you could help us figure out what is going
wrong.

Could you try doing the following:
- run with -ntmpi 1 -ntomp 1 (i.e single-threaded);
- run with OpenMP (-only) multhithreading, that is with -ntmpi 1 (this
should start with 8 OpenMP threads)
- run with -debug 1 which will produce the mdrun.debug output, please
upload this to e.g pastebin and post a link;

On a side-note: you would be better off using a newer gcc, 4.7 should be
considerably faster than 4.5 - especially on this Sandy Bridge Xeon
processor.

Cheers,

--
Szilárd


On Mon, Feb 25, 2013 at 5:25 PM,  wrote:

> Hello,
> Thanks for the help. After setting the library path properly, I seem
> to be able to get gromacs up and running. However, I have run into another
> problem with mdrun and actually running any jobs. When I execute
> mdrun -v -deffnm Clp_Test -nt
> The output is: Reading file Clp_Test.tpr, VERSION 4.6 (single precision)
> Using 8 MPI threads
>
> Followed by several occurrences of:
> Can not set thread affinities on the current platform. On NUMA systems this
> can cause performance degradation. If you think your platform should
> support
> setting affinities, contact the GROMACS developers.
>
> Then:
> starting mdrun 'Martini system for ClpX'
> 1 steps,200.0 ps.
>
> After this however, the simulation never actually begins. I can get rid of
> the error messages by using -pin off, but that doesn't seem to actually fix
> anything. Is there something that has not been installed properly? Below is
> the seemingly relevant portions of the log file generated by the above
> mdrun command.
>
>
> Log file opened on Mon Feb 25 11:21:08 2013
> Host: Theory-Monster  pid: 3192  nodeid: 0  nnodes:  1
> Gromacs version:VERSION 4.6
> Precision:  single
> Memory model:   32 bit
> MPI library:thread_mpi
> OpenMP support: enabled
> GPU support:disabled
> invsqrt routine:gmx_software_invsqrt(x)
> CPU acceleration:   AVX_256
> FFT library:fftw-3.3.3-sse2
> Large file support: enabled
> RDTSCP usage:   enabled
> Built on:   Mon, Feb 25, 2013 10:38:04 AM
> Built by:   Mike@Theory-Monster [CMAKE]
> Build OS/arch:  CYGWIN_NT-6.1-WOW64 1.7.17(0.262/5/3) i686
> Build CPU vendor:   GenuineIntel
> Build CPU brand:Intel(R) Xeon(R) CPU E5-2687W 0 @ 3.10GHz
> Build CPU family:   6   Model: 45   Stepping: 7
> Build CPU features: aes apic avx clfsh cmov cx8 cx16 htt lahf_lm mmx msr
> nonstop_tsc pcid pclmuldq pdcm pdpe1gb popcnt pse rdtscp sse2 sse3 sse4.1
> sse4.2 ssse3 tdt x2apic
> C compiler: /usr/bin/gcc.exe GNU gcc (GCC) 4.5.3
> C compiler flags: -mavx -Wextra -Wno-missing-field-**initializers
> -Wno-sign-compare -Wall -Wno-unused -Wunused-value -fomit-frame-pointer
> -funroll-all-loops -fexcess-precision=fast -O3 -DNDEBUG
>
> Initializing Domain Decomposition on 8 nodes
> Dynamic load balancing: auto
> Will sort the charge groups at every domain (re)decomposition
> Initial maximum inter charge-group distances:
>two-body bonded interactions: 0.988 nm, Bond, atoms 997 1005
>  multi-body bonded interactions: 1.042 nm, G96Angle, atoms 1938 1942
> Minimum cell size due to bonded interactions: 1.146 nm
> Maximum distance for 5 constraints, at 120 deg. angles, all-trans: 0.810 nm
> Estimated maximum distance required for P-LINCS: 0.810 nm
> Scaling the initial minimum size with 1/0.8 (option -dds) = 1.25
> Optimizing the DD grid for 8 cells with a minimum initial size of 1.433 nm
> The maximum allowed number of cells is: X 11 Y 11 Z 9
> Domain decomposition grid 4 x 2 x 1, separate PME nodes 0
> Domain decomposition nodeid 0, coordinates 0 0 0
>
> Using 8 MPI threads
>
> Detecting CPU-specific acceleration.
> Present hardware specification:
> Vendor: GenuineIntel
> Brand:  Intel(R) Xeon(R) CPU E5-2687W 0 @ 3.10GHz
> Family:  6  Model: 45  Stepping:  7
> Features: aes apic avx clfsh cmov cx8 cx16 htt lahf_lm mmx msr nonstop_tsc
> pcid pclmuldq pdcm pdpe1gb popcnt pse rdtscp sse2 sse3 sse4.1 sse4.2 ssse3
> tdt x2apic
> Acceleration most likely to fit this hardware: AVX_256
> Acceleration selected at GROMACS compile time: AVX_256
>
> Table routines are used for coulomb: TRUE
> Table routines are used for vdw: TRUE
> Using shifted Lennard-Jones, switch between 0.9 and 1.2 nm
> Cut-off's:   NS: 1.4   Coulomb: 1.2   LJ: 1.2
> System total charge: 0.000
> Generated table with 1200 data points for Shift.
> Tabscale = 500 points/nm
> Generated table with 1200 data points for LJ6Shift.
> Tabscale = 500 points/nm
> Generated table with 1200 data points for LJ12Shift.
> Tabscale = 500 points/nm
> Potential shift: LJ r^-12: 0.000 r^-6 0.000
> Removing pbc first time
>
> Can not set thread affinities on the current platform. On NUMA systems this
> can cause performance degradation. If you think your platfo

Re: [gmx-users] Re: Gromacs 4.6 Installation under Cygwin

[Szilárd Páll]

Hi,

That is a likely cause. I can not comment with full certainty on the use of
MARTINI with the Verlet scheme & without shifts, but I know that it is a
topic that's being investigated and hopefully other can comment on it.

However, the hanging should definitely not happen, but instead an error
should be triggered, Now that we have quite well-defined problem, could you
please file a bug report (redmine.gromacs.org)? A brief problem description
with:
- inputs;
- a link to this discussion (
http://www.mail-archive.com/gmx-users@gromacs.org/msg57828.html);
- link to mdrun.debug in pastebin
would be very helpful.

Thanks,

--
Szilárd


On Tue, Feb 26, 2013 at 10:11 PM,  wrote:

> Hello,
> It looks like the problem might be in my mdp settings. I am using the
> MARTINI force field which uses shift functions for both electrostatics and
> vdw which are only available in the group-based cut-off scheme. OpenMP
> doesn't seem to run with the group-based cut-off scheme, just Verlet.
> If I switch the cut-off scheme to Verlet and change the vdw and
> coulombtype to cut-off and then add in potential-shift-verlet as modifiers
> I can get the system to run. However, in doing this, am I mucking up the
> force field?
>
> Thanks,
> Mike
>
>
>
> On Feb 26 2013, Mirco Wahab wrote:
>
>  Hi Mike
>>
>> On 25.02.2013 17:25, toma0...@umn.edu wrote:
>>
>>> ...
>>> Estimated maximum distance required for P-LINCS: 0.810 nm
>>> ...
>>> Domain decomposition grid 4 x 2 x 1, separate PME nodes 0
>>>
>>
>> Do your runs use PME and/or P-LINCS?
>>
>> Maybe you could point out the problem by disabling
>> each or both in yout .mdp file?
>>
>> I tested some systems (without PME) on cygwin/gmx
>> 4.6 that worked ok. Probably glitches with the
>> cygwin-provided fftw3?
>>
>> Maybe you could try to compile fftwf 3.3.3
>> in single precision from source and link it to
>> your gromacs build.
>>
>  http://www.fftw.org/fftw3_doc/**Installation-on-Unix.html#**
> Installation-on-Unix
>
>>
>>
>>
>> M.
>>
>>
>>  --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/**mailman/listinfo/gmx-users
> * Please search the archive at http://www.gromacs.org/**
> Support/Mailing_Lists/Searchbefore
>  posting!
> * Please don't post (un)subscribe requests to the list. Use the www
> interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read 
> http://www.gromacs.org/**Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Re: Gromacs 4.6 Installation under Cygwin

[Szilárd Páll]

I've a few more comments. First of all, please upload to redmine the full
mdrun.debug output!

It looks like the detection of the number of cores in your machine does not
work with cygwin because when you only set "-ntmp 1" N OpenMP threads
should be auto-set (N=#cores), but based on your output it seems that only
one is used. Could you please verify this? I am not sure whether this is an
inherent problem with cygwin or a bug (and as I don't have access to cygwin
I can't test).

Cheers,

--
Szilárd


On Tue, Feb 26, 2013 at 10:55 PM, Szilárd Páll wrote:

> Hi,
>
> That is a likely cause. I can not comment with full certainty on the use
> of MARTINI with the Verlet scheme & without shifts, but I know that it is a
> topic that's being investigated and hopefully other can comment on it.
>
> However, the hanging should definitely not happen, but instead an error
> should be triggered, Now that we have quite well-defined problem, could you
> please file a bug report (redmine.gromacs.org)? A brief problem
> description with:
> - inputs;
> - a link to this discussion (
> http://www.mail-archive.com/gmx-users@gromacs.org/msg57828.html);
> - link to mdrun.debug in pastebin
> would be very helpful.
>
> Thanks,
>
> --
> Szilárd
>
>
> On Tue, Feb 26, 2013 at 10:11 PM,  wrote:
>
>> Hello,
>> It looks like the problem might be in my mdp settings. I am using the
>> MARTINI force field which uses shift functions for both electrostatics and
>> vdw which are only available in the group-based cut-off scheme. OpenMP
>> doesn't seem to run with the group-based cut-off scheme, just Verlet.
>> If I switch the cut-off scheme to Verlet and change the vdw and
>> coulombtype to cut-off and then add in potential-shift-verlet as modifiers
>> I can get the system to run. However, in doing this, am I mucking up the
>> force field?
>>
>> Thanks,
>> Mike
>>
>>
>>
>> On Feb 26 2013, Mirco Wahab wrote:
>>
>>  Hi Mike
>>>
>>> On 25.02.2013 17:25, toma0...@umn.edu wrote:
>>>
>>>> ...
>>>> Estimated maximum distance required for P-LINCS: 0.810 nm
>>>> ...
>>>> Domain decomposition grid 4 x 2 x 1, separate PME nodes 0
>>>>
>>>
>>> Do your runs use PME and/or P-LINCS?
>>>
>>> Maybe you could point out the problem by disabling
>>> each or both in yout .mdp file?
>>>
>>> I tested some systems (without PME) on cygwin/gmx
>>> 4.6 that worked ok. Probably glitches with the
>>> cygwin-provided fftw3?
>>>
>>> Maybe you could try to compile fftwf 3.3.3
>>> in single precision from source and link it to
>>> your gromacs build.
>>>
>>  http://www.fftw.org/fftw3_doc/**Installation-on-Unix.html#**
>> Installation-on-Unix<http://www.fftw.org/fftw3_doc/Installation-on-Unix.html#Installation-on-Unix>
>>
>>>
>>>
>>>
>>> M.
>>>
>>>
>>>  --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/**mailman/listinfo/gmx-users<http://lists.gromacs.org/mailman/listinfo/gmx-users>
>> * Please search the archive at http://www.gromacs.org/**
>> Support/Mailing_Lists/Search<http://www.gromacs.org/Support/Mailing_Lists/Search>before
>>  posting!
>> * Please don't post (un)subscribe requests to the list. Use the www
>> interface or send it to gmx-users-requ...@gromacs.org.
>> * Can't post? Read 
>> http://www.gromacs.org/**Support/Mailing_Lists<http://www.gromacs.org/Support/Mailing_Lists>
>>
>
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GROMACS4.6 on Intel MIC

[Szilárd Páll]

Hi,

Yes, it can. If you use the MIC in symmetric mode, it will just look and
behave like another compute node. However, the performance will be
horrible, because we don't have optimized kernels  for MIC. Work on porting
the new SIMD-friendly Verlet NxN kernels to MIC ins ongoing, but it is
highly unlikely that production-level release code will include this work
before 5.0. A stable 4.6-based development branch with decent performance
could should up in a few months.

Cheers,

--
Szilárd


On Wed, Feb 27, 2013 at 12:21 PM, Anirban wrote:

> Hi ALL,
>
> Can GROMACS4.6 or a lower version be compiled on Intel MIC processor?
>
> Any suggestion is welcome.
>
> Regards,
>
> Anirban
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Problem with OpenMP+MPI

[Szilárd Páll]

Jesmin,

First of all, have you read the following pages?
http://www.gromacs.org/Documentation/Acceleration_and_parallelization
http://www.gromacs.org/Documentation/Cut-off_schemes

To summarize:
- OpenMP parallelization is supported in Verlet scheme for the full mdrun
and for separate PME nodes also with the group scheme (for improving
scaling);
- GB only works with group scheme and right now is rather limited by
parallelization issues. Additionally, fixing it does not have a very high
priority due to the lack of active maintainers of this code with lots of
free time as well as because there have been fairly limited user feedback
on (the need for) GB. Please express your wish of getting this fixed +
support for OpenMP by commenting on existing bug reports of filing new
bugs/feature requests on redmine.gromacs.org.

Thanks,

--
Szilárd


On Wed, Feb 27, 2013 at 6:13 PM, jesmin jahan  wrote:

> Dear Justin,
>
> I want to compare the performance of MPI+OpenMP implementation of
> Gromacs GB-energy with other MPI+X implementation of GB-energy.
>
> So, my question is:
> Is there any way to get sensible result for the implicit solvent
> simulation using Gromacs that employs both MPI and OpenMP?
> If yes, then what should be the parameter set?
>
> Thanks for your time.
>
> Best Regards,
> Jesmin
>
> On Wed, Feb 27, 2013 at 11:54 AM, Justin Lemkul  wrote:
> >
> >
> > On 2/27/13 11:27 AM, jesmin jahan wrote:
> >>
> >> Thanks again Justin,
> >>
> >> I added
> >>
> >> cutoff-scheme   = group
> >>
> >> But still I am getting
> >>
> >> OpenMP threads have been requested with cut-off scheme Group, but
> >> these are only supported with cut-off scheme Verlet
> >>
> >> which really says, openmp threads are supported only in Verlet!
> >>
> >
> > Then you need to run without OpenMP threads.
> >
> > -Justin
> >
> >
> >> constraints =  none
> >> integrator  =  md
> >> cutoff-scheme   = group
> >>
> >> pbc =  no
> >> ;verlet-buffer-drift = -1
> >> dt  =  0.001
> >> nsteps  =  0
> >> ns_type = simple
> >> comm-mode   = angular
> >> rlist   = 0
> >> rcoulomb= 0
> >> rvdw= 0
> >> nstlist = 0
> >> rgbradii= 0
> >> nstgbradii  = 1
> >> coulombtype = cutoff
> >> vdwtype = cutoff
> >> implicit_solvent=  GBSA
> >> gb_algorithm=  HCT ;
> >> sa_algorithm=  None
> >> gb_dielectric_offset= 0.02
> >>
> >> optimize_fft = yes
> >> energygrps   = protein
> >>
> >> Best Regards,
> >> Jesmin
> >>
> >>
> >> On Wed, Feb 27, 2013 at 10:57 AM, Justin Lemkul 
> wrote:
> >>>
> >>>
> >>>
> >>> On 2/27/13 10:54 AM, jesmin jahan wrote:
> 
> 
>  Many thanks to you Justin for your help.
> 
>  Now my .mdp file looks like:
> 
>  constraints =  none
>  integrator  =  md
> 
>  pbc =  no
>  verlet-buffer-drift = -1
>  dt  =  0.001
>  nsteps  =  0
>  ns_type = simple
>  comm-mode   = angular
>  rlist   = 0
>  rcoulomb= 0
>  rvdw= 0
>  nstlist = 0
>  rgbradii= 0
>  nstgbradii  = 1
>  coulombtype = cutoff
>  vdwtype = cutoff
>  implicit_solvent=  GBSA
>  gb_algorithm=  HCT ;
>  sa_algorithm=  None
>  gb_dielectric_offset= 0.02
> 
>  optimize_fft = yes
>  energygrps   = protein
> 
>  And when I run mdrun, it says
> 
>  "OpenMP threads have been requested with cut-off scheme Group, but
>  these are only supported with cut-off scheme Verlet"
> 
>  Now if I add
> 
>  cutoff-scheme   = Verlet
> 
>  It says
> 
> With Verlet lists only full pbc or pbc=xy with walls is supported
> 
>  With Verlet lists nstlist should be larger than 0
> 
> 
>  So, what is the solution??
> 
> >>>
> >>> Use the group cutoff scheme rather than Verlet, otherwise we wind up
> >>> right
> >>> back where we were before.
> >>>
> >>>
>  Also,  you told me
> 
>  "Note that implicit solvent calculations can be run on no more than 2
>  processors.  You'll get a fatal error if you try to use more."
> 
>  Is there any documentation for this? Can I cite you if I want to add
>  this information in an article?
> 
> >>>
> >>> It's a bug, listed on Redmine somewhere.  I doubt that requires any
> sort
> >>> of
> >>> citation, but if anyone asks, there are discussions in the list archive
> >>> and
> >>> on redmine.gromacs.org.
> >>>
> >>>
> >>> -Justin
> >>>
> >>> --
> >>> 
> >>>
> >>> Justin A. Lemkul, Ph.D.
> >>> Research Scientist
> >>> Department of Biochemistry
> >>> Virginia Tech
> >>> Bla

Re: [gmx-users] Gromacs 4.6 crashes w/AVX acceleration on cygwin

[Szilárd Páll]

Hi,

Thanks for the detailed information!

The only reason why cygwin is cumbersome is that we are not testing it
actively, have we had a developer or contributor working with cygwin there
would be no issues.

In any case, especially because there is performance involved, we should
try to get the AVX issue solved. First and foremost, would you mind filing
a redmine issue on the issue (or if you consider the dll problem annoying,
feel free to file a second one).

However, before filing the AVX bug report, please try the following:
- A newer gcc, preferably 4.7; although in *nix 4.5 seems to work with AVX,
it's best if we have things tested with the latest gcc.
- SSE4.1 acceleration; beside that this should be faster than SSE2, but if
that doesn't work either something is seriously broken.

If you manage to get performance numbers with the above, please provide
them, here for the sake of completeness. I would be surprised if VS2010
would outperform gcc 4.7, though.

Cheers,

--
Szilárd


On Wed, Feb 27, 2013 at 8:31 PM, Mirco Wahab <
mirco.wa...@chemie.tu-freiberg.de> wrote:

> Out of curiosity, I installed cygwin  1.7.17-1 on my AMD/FX-8350
> workplace box (W8Pro/x64). This is what I found out:
>
> - Gromacs 4.6 compiles smooth but crashes w/native(AVX_128) on
>simulation start with loads of fp errors (non-reproducible,
>sometimes stops w/no error at all, just stops)
>
> - Gromacs 4.6 runs fine if compiled w/GMX_CPU_ACCELERATION=SSE2,
>even if PME involved (see below)
>
> - Gromacs shared executables (runtime-dlls) didn't start due to
>some erratic dlopen problems with I didn't understand. By
>using GMX_PREFER_STATIC_LIBS=ON, this was solved
>
> I testet a "Martini"-system of ~1 Mio particles (a 50^3nm water
> box with a vesicle). Performance comparison:
>
>   VS2010(x64)  - md/cut-off/Reaction-Field : 37.460 ns/day
>   cygwin/gcc 4.5.3 - md/cut-off/Reaction-Field : 30.304 ns/day
>
>   VS2010(x64)  - md/cut-off/PME : 10.874 ns/day
>   cygwin/gcc 4.5.3 - md/cut-off/PME :  9.283 ns/day
>
> The fftw3f used was installed before from source. I'll attach
> a recipe for both builds and some more hints below.
>
> Resumé: cygwin installation of Gromacs 4.6 is (still) cumbersome
> and may throw up suprising errors. For windows users, looks much
> more promising (imho!) to build Gromacs with the Visual Studio
> compiler (which is, through combination of VS2010 express +
> SDK 7.1, free of cost).
>
> "works on my machine"-instructions: =>
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Gromacs 4.6 crashes w/AVX acceleration on cygwin

[Szilárd Páll]

PS: If you have fftw in the standard location it should be picked up by
cmake; otherwise, -DCMAKE_PREFIX_PATH=/usr/local should work as well.




--
Szilárd


On Wed, Feb 27, 2013 at 9:21 PM, Szilárd Páll wrote:

> Hi,
>
> Thanks for the detailed information!
>
> The only reason why cygwin is cumbersome is that we are not testing it
> actively, have we had a developer or contributor working with cygwin there
> would be no issues.
>
> In any case, especially because there is performance involved, we should
> try to get the AVX issue solved. First and foremost, would you mind filing
> a redmine issue on the issue (or if you consider the dll problem annoying,
> feel free to file a second one).
>
> However, before filing the AVX bug report, please try the following:
> - A newer gcc, preferably 4.7; although in *nix 4.5 seems to work with
> AVX, it's best if we have things tested with the latest gcc.
> - SSE4.1 acceleration; beside that this should be faster than SSE2, but if
> that doesn't work either something is seriously broken.
>
> If you manage to get performance numbers with the above, please provide
> them, here for the sake of completeness. I would be surprised if VS2010
> would outperform gcc 4.7, though.
>
> Cheers,
>
> --
> Szilárd
>
>
> On Wed, Feb 27, 2013 at 8:31 PM, Mirco Wahab <
> mirco.wa...@chemie.tu-freiberg.de> wrote:
>
>> Out of curiosity, I installed cygwin  1.7.17-1 on my AMD/FX-8350
>> workplace box (W8Pro/x64). This is what I found out:
>>
>> - Gromacs 4.6 compiles smooth but crashes w/native(AVX_128) on
>>simulation start with loads of fp errors (non-reproducible,
>>sometimes stops w/no error at all, just stops)
>>
>> - Gromacs 4.6 runs fine if compiled w/GMX_CPU_ACCELERATION=SSE2,
>>even if PME involved (see below)
>>
>> - Gromacs shared executables (runtime-dlls) didn't start due to
>>some erratic dlopen problems with I didn't understand. By
>>using GMX_PREFER_STATIC_LIBS=ON, this was solved
>>
>> I testet a "Martini"-system of ~1 Mio particles (a 50^3nm water
>> box with a vesicle). Performance comparison:
>>
>>   VS2010(x64)  - md/cut-off/Reaction-Field : 37.460 ns/day
>>   cygwin/gcc 4.5.3 - md/cut-off/Reaction-Field : 30.304 ns/day
>>
>>   VS2010(x64)  - md/cut-off/PME : 10.874 ns/day
>>   cygwin/gcc 4.5.3 - md/cut-off/PME :  9.283 ns/day
>>
>> The fftw3f used was installed before from source. I'll attach
>> a recipe for both builds and some more hints below.
>>
>> Resumé: cygwin installation of Gromacs 4.6 is (still) cumbersome
>> and may throw up suprising errors. For windows users, looks much
>> more promising (imho!) to build Gromacs with the Visual Studio
>> compiler (which is, through combination of VS2010 express +
>> SDK 7.1, free of cost).
>>
>> "works on my machine"-instructions: =>
>>
>>
>> --
>> gmx-users mailing listgmx-users@gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-users
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
>> * Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-users-requ...@gromacs.org.
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU version of GROMACS 4.6 in MacOS cluster

[Szilárd Páll]

HI,

That looks like the driver does not work or is incompatible with the
runtime. Please get the SDK, compile a simple program, e.g. deviceQuery and
see if that works (I suspect that it won't).

Regarding your machines, just FYI, the Quadro 4000 is a pretty slow card
(somewhat slower than a GTX 460) so you'll hava a quite strong resource
imbalance: a lot of CPU compute power (2x Xeon 5xxx, right?) and little GPU
compute power which will lead to the CPU idling while waiting for the GPU.

Cheers,

--
Szilárd


On Thu, Feb 28, 2013 at 4:52 PM, George Patargias wrote:

> Hello
>
> We are trying to install the GPU version of GROMACS 4.6 on our own
> MacOS cluster. So for the cluster nodes that have the NVIDIA Quadro 4000
> cards:
>
> - We have downloaded and install the MAC OS  X CUDA 5.0 Production Release
> from here: https://developer.nvidia.com/cuda-downloads
>
> placing the libraries contained in this download in /usr/local/cuda/lib
>
> - We have managed to compile GROMACS 4.6 linking it statically with these
> CUDA libraries and the MPI libraries (with BUILD_SHARED_LIBS=OFF and
> GMX_PREFER_STATIC_LIBS=ON)
>
> Unfortunately, when we tried to run a test job with the generated
> mdrun_mpi, GROMACS reported that it cannot detect any CUDA-enabled
> devices. It also reports 0.0 version for CUDA driver and runtime.
>
> Is the actual CUDA driver missing from the MAC OS  X CUDA 5.0 Production
> Release that we installed? Do we need to install it from here:
>
> http://www.nvidia.com/object/cuda-mac-driver.html
>
> Or is something else that we need to do?
>
> Many thanks in advance.
> George
>
>
> Dr. George Patargias
> Postdoctoral Researcher
> Biomedical Research Foundation
> Academy of Athens
> 4, Soranou Ephessiou
> 115 27
> Athens
> Greece
>
> Office: +302106597568
>
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] GPU version of GROMACS 4.6 in MacOS cluster

[Szilárd Páll]

Hi George,

As I said before, that just means that most probably the GPU driver is not
compatible with the CUDA runtime (libcudart) that you installed with the
CUDA toolkit. I've no clue about the Mac OS installers and releases, you'll
have to do the research on that. Let us know if you have further
(GROMACS-related) issues.

Cheers,

--
Szilárd


On Fri, Mar 1, 2013 at 2:48 PM, George Patargias  wrote:

> Hi Szilαrd
>
> Thanks for your reply. I have run the deviceQuery utility and what I got
> back is
>
> /deviceQuery Starting...
>
>  CUDA Device Query (Runtime API) version (CUDART static linking)
>
> cudaGetDeviceCount returned 38
> -> no CUDA-capable device is detected
>
> Should I understand from this that the CUDA driver was not installed from
> the MAC OS  X CUDA 5.0 Production Release?
>
> George
>
>
> > HI,
> >
> > That looks like the driver does not work or is incompatible with the
> > runtime. Please get the SDK, compile a simple program, e.g. deviceQuery
> > and
> > see if that works (I suspect that it won't).
> >
> > Regarding your machines, just FYI, the Quadro 4000 is a pretty slow card
> > (somewhat slower than a GTX 460) so you'll hava a quite strong resource
> > imbalance: a lot of CPU compute power (2x Xeon 5xxx, right?) and little
> > GPU
> > compute power which will lead to the CPU idling while waiting for the
> GPU.
> >
> > Cheers,
> >
> > --
> > Szilαrd
> >
> >
> > On Thu, Feb 28, 2013 at 4:52 PM, George Patargias
> > wrote:
> >
> >> Hello
> >>
> >> We are trying to install the GPU version of GROMACS 4.6 on our own
> >> MacOS cluster. So for the cluster nodes that have the NVIDIA Quadro 4000
> >> cards:
> >>
> >> - We have downloaded and install the MAC OS  X CUDA 5.0 Production
> >> Release
> >> from here: https://developer.nvidia.com/cuda-downloads
> >>
> >> placing the libraries contained in this download in /usr/local/cuda/lib
> >>
> >> - We have managed to compile GROMACS 4.6 linking it statically with
> >> these
> >> CUDA libraries and the MPI libraries (with BUILD_SHARED_LIBS=OFF and
> >> GMX_PREFER_STATIC_LIBS=ON)
> >>
> >> Unfortunately, when we tried to run a test job with the generated
> >> mdrun_mpi, GROMACS reported that it cannot detect any CUDA-enabled
> >> devices. It also reports 0.0 version for CUDA driver and runtime.
> >>
> >> Is the actual CUDA driver missing from the MAC OS  X CUDA 5.0 Production
> >> Release that we installed? Do we need to install it from here:
> >>
> >> http://www.nvidia.com/object/cuda-mac-driver.html
> >>
> >> Or is something else that we need to do?
> >>
> >> Many thanks in advance.
> >> George
> >>
> >>
> >> Dr. George Patargias
> >> Postdoctoral Researcher
> >> Biomedical Research Foundation
> >> Academy of Athens
> >> 4, Soranou Ephessiou
> >> 115 27
> >> Athens
> >> Greece
> >>
> >> Office: +302106597568
> >>
> >>
> >> --
> >> gmx-users mailing listgmx-users@gromacs.org
> >> http://lists.gromacs.org/mailman/listinfo/gmx-users
> >> * Please search the archive at
> >> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> >> * Please don't post (un)subscribe requests to the list. Use the
> >> www interface or send it to gmx-users-requ...@gromacs.org.
> >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >>
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
>
>
> Dr. George Patargias
> Postdoctoral Researcher
> Biomedical Research Foundation
> Academy of Athens
> 4, Soranou Ephessiou
> 115 27
> Athens
> Greece
>
> Office: +302106597568
>
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Thread affinity setting failed

[Szilárd Páll]

Hi,

There are some clarifications needed and as this might help you and other
understand what's going on, I'll take the time to explain things.

Affinity setting is a low-, operating system-level, operation and "locks"
(="pins") threads to physical cores of the CPU preventing the OS from
moving them which can cause performance drop - especially when using
OpenMP-multithreading on multi-socket and NUMA machines.

Now, mdrun will by default *try* to set affinity if you use all cores
detected (i.e if mdrun can be sure that it is the only application running
on the machine), but will by default *not* set thread affinities if the
number of thread/processes per compute node is less than the number of
cores detected. Hence, when you decrease -ntmpi to 7, you implicitly end up
turning off thread pinning, that's why the warnings don't show up.

The fact that affinity setting fails on your machine suggests that either
the system libraries don't support this or the mdrun code is not fully
compatible with your OS, the type of CPUs AFAIK don't matter at all. What
OS are you using? Is it an old installation?

If you are not using OpenMP - which btw you probably should with the Verlet
scheme if you are running running on a single node or at high
parallelization -, the performance will not be affected very much by the
lack of thread pinning. While the warnings themselves can often be safely
ignored, if only some of the threads/processes can't set affinities, this
might indicate a problem. I your case, if you were really seeing only 5
cores being used with 3 warnings, this might suggest that while the
affinity setting failed, three threads are using already "busy" cores
overlapping with others which will cause severe performance drop.

What you can do to avoid the performance drop is to turn of pinning by
passing "-pin off" to mdrun. Without OpenMP this will typically not cause a
large performance drop compared to having correct pinning and it will avoid
the bad overlapping threads/processes case.

I suspect that your machines might be running an old OS which could be
causing the failed affinity setting. If that is the case, you should talk
to your sysadmins and have them figure out the issue. If you have a
moderately new OS, you should not be seeing such issues, so I suggest that
you file a bug report with details like: OS + version + kernel version,
pthread library version, standard C library version.

Cheers,

--
Szilárd


On Mon, Mar 4, 2013 at 1:45 PM, Mark Abraham wrote:

> On Mon, Mar 4, 2013 at 6:02 AM, Reid Van Lehn  wrote:
>
> > Hello users,
> >
> > I ran into a bug I do not understand today upon upgrading from v. 4.5.5
> to
> > v 4.6. I'm using older 8 core Intel Xeon E5430 machines, and when I
> > submitted a job for 8 cores to one of the nodes I received the following
> > error:
> >
> > NOTE: In thread-MPI thread #3: Affinity setting failed.
> >   This can cause performance degradation!
> >
> > NOTE: In thread-MPI thread #2: Affinity setting failed.
> >   This can cause performance degradation!
> >
> > NOTE: In thread-MPI thread #1: Affinity setting failed.
> >   This can cause performance degradation!
> >
> > I ran mdrun simply with the flags:
> >
> > mdrun -v -ntmpi 8 -deffnm em
> >
> > Using the top command, I confirmed that no other programs were running
> and
> > that mdrun was in fact only using 5 cores. Reducing -ntmpi to 7, however,
> > resulted in no error (only a warning about not using all of the logical
> > cores) and mdrun used 7 cores correctly. Since it warned about thread
> > affinity settings, I tried setting -pin on -pinoffset 0 even though I was
> > using all the cores on the machine. This resulted in the same error.
> > However, turning pinning off explicitly with -pin off (rather than -pin
> > auto) did correctly give me the all 8 cores again.
> >
> > While I figured out a solution in this particular instance, my question
> is
> > whether I should be have known from my hardware/settings that pinning
> > should be turned off (for future reference), or if this is a bug?
> >
>
> I'm not sure - those are 2007-era processors, so there may be some
> limitations in what they could do (or how well the kernel and system
> libraries support it). So investing time into working out the real problem
> is not really worthwhile. Thanks for reporting your work-around, however,
> others might benefit from it. If you plan on doing lengthy simulations, you
> might like to verify that you get linear scaling with increasing -ntmpi,
> and/or compare performance with the MPI version on the same hardware.
>
> Mark
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Ma

Re: [gmx-users] GROMACS 4.6.1 released

[Szilárd Páll]

On Tue, Mar 5, 2013 at 8:14 PM, Mark Abraham wrote:

> *Hi GROMACS users,
>
> GROMACS 4.6.1 is officially released. It contains numerous bug fixes, some
> simulation performance enhancements and some documentation updates. We
> encourage all users to upgrade their installations from 4.6.
>
> You can find the code, manual, release notes, installation instructions and
> test
> suite at the links below.
>
> ftp://ftp.gromacs.org/pub/gromacs/gromacs-4.6.1.tar.gz
> ftp://ftp.gromacs.org/pub/manual/manual-4.6.1.pdf
> http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.1.x


Link incorrect. Here's the correct one:
http://www.gromacs.org/About_Gromacs/Release_Notes/Versions_4.6.x


>
> http://www.gromacs.org/Documentation/Installation_Instructions
> http://gromacs.googlecode.com/files/regressiontests-4.6.1.tar.gz
>
> Happy simulating!
>
> The GROMACS development team*
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
-- 
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the 
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Re: [gmx-developers] Gromacs 4.6.1 ATLAS/CUDA detection problems...

[Szilárd Páll]

On Thu, Mar 7, 2013 at 2:02 PM, Berk Hess  wrote:

>
> Hi,
>
> This was only a note, not a fix.
> I was just trying to say that what linear algebra library you use for
> Gromacs is irrelevant in more than 99% of the cases.
> But having said that, the choice of library should not complicate the
> configure stage of Gromacs.
>

I guess Evren assumed that GROMACS has an explicit dependency on an
external linear algebra library.

To be concrete, you can simply use the GROMACS internal linear algebra code
by setting -DGMX_EXTERNAL_BLAS=OFF -DGMX_EXTERNAL_LAPACK=OFF.

--
Szilárd

PS: This issue sounds like a valid reason to consider switching external
BLAS/LAPACK off by default.



> Cheers,
>
> Berk
>
> 
> > Date: Thu, 7 Mar 2013 14:54:39 +0200
> > From: eyurt...@abo.fi
> > To: gmx-users@gromacs.org
> > Subject: RE: [gmx-users] Re: [gmx-developers] Gromacs 4.6.1 ATLAS/CUDA
> detection problems...
> >
> >
> >
> > On Thu, 7 Mar 2013, Berk Hess wrote:
> >
> > >
> > > Hi,
> > >
> > > Note that the linear algebra in Gromacs is limited to eigenvalue
> solving in two analysis tools,
> > > the MD engine does not use any (large) linear algebra.
> > > So linear algebra library performance is irrelevant, except for the
> cases where users
> > > would want to repeated uses one of the two tools with large systems.
> > > Even then, our built-in code is not bad.
> > >
> > > Cheers,
> > >
> > > Berk
> >
> > Hello,
> >
> > I am not sure what you are suggesting as a fix to the problem? Do you
> mean
> > the support for ATLAS should/will be removed as a fix because it is not
> > very useful performancewise (and non-functional) anyway?
> >
> > Thanks,
> > Evren
> > --
> > gmx-users mailing list gmx-users@gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>   --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

Re: [gmx-users] Thread affinity setting failed

[Szilárd Páll]

Hi Reid,

Just saw your bug report and realized that you have an ancient kernel which
could be causing the issue. Let's move the discussion to the bug page (
http://redmine.gromacs.org/issues/1184), hopefully we can narrow the issue
down and then post the conclusions to the list later.

Cheers,

--
Szilárd


On Thu, Mar 7, 2013 at 7:06 AM, Roland Schulz  wrote:

> Hi Raid,
>
> I just tested Gromacs 4.6.1 compiled with ICC 13 and GCC 4.1.2 on CentOS
> 5.6 and I don't have any problems with pinning. So it might be useful to
> open a bug and provide more details, because it should work for CentOS 5.x.
>
> Yes, for pure water the group kernels are faster than Verlet.
>
> Roland
>
>
> On Wed, Mar 6, 2013 at 10:17 PM, Reid Van Lehn  wrote:
>
> > Hi Szilárd,
> >
> > Thank you very much for the detailed write up. To answer your question,
> > yes, I am using an old Linux distro, specifically CentOS 5.4, though
> > upgrading to 5.9 still had the same problem. I have another few machines
> > with different hardware CentOS 6.3 which does not have this issue so it
> is
> > likely an operating system issue based on your description. As I'm
> > (unfortunately...) also the sysadmin on this cluster I'm unlikely to find
> > the time to upgrade all the nodes, so I'll probably stick with the "-pin
> > off" workaround for now. Hopefully this thread might help out other
> users!
> >
> > As an aside, I found that the OpenMP + Verlet combination was slower for
> > this particular system, but I suspect that it's because it's almost
> > entirely water and hence probably benefits from the Group scheme
> > optimizations for water described on the Gromacs website.
> >
> > Thanks again for the explanation,
> > Reid
> >
> > On Mon, Mar 4, 2013 at 3:45 PM, Szilárd Páll 
> > wrote:
> >
> > > Hi,
> > >
> > > There are some clarifications needed and as this might help you and
> other
> > > understand what's going on, I'll take the time to explain things.
> > >
> > > Affinity setting is a low-, operating system-level, operation and
> "locks"
> > > (="pins") threads to physical cores of the CPU preventing the OS from
> > > moving them which can cause performance drop - especially when using
> > > OpenMP-multithreading on multi-socket and NUMA machines.
> > >
> > > Now, mdrun will by default *try* to set affinity if you use all cores
> > > detected (i.e if mdrun can be sure that it is the only application
> > running
> > > on the machine), but will by default *not* set thread affinities if the
> > > number of thread/processes per compute node is less than the number of
> > > cores detected. Hence, when you decrease -ntmpi to 7, you implicitly
> end
> > up
> > > turning off thread pinning, that's why the warnings don't show up.
> > >
> > > The fact that affinity setting fails on your machine suggests that
> either
> > > the system libraries don't support this or the mdrun code is not fully
> > > compatible with your OS, the type of CPUs AFAIK don't matter at all.
> What
> > > OS are you using? Is it an old installation?
> > >
> > > If you are not using OpenMP - which btw you probably should with the
> > Verlet
> > > scheme if you are running running on a single node or at high
> > > parallelization -, the performance will not be affected very much by
> the
> > > lack of thread pinning. While the warnings themselves can often be
> safely
> > > ignored, if only some of the threads/processes can't set affinities,
> this
> > > might indicate a problem. I your case, if you were really seeing only 5
> > > cores being used with 3 warnings, this might suggest that while the
> > > affinity setting failed, three threads are using already "busy" cores
> > > overlapping with others which will cause severe performance drop.
> > >
> > > What you can do to avoid the performance drop is to turn of pinning by
> > > passing "-pin off" to mdrun. Without OpenMP this will typically not
> > cause a
> > > large performance drop compared to having correct pinning and it will
> > avoid
> > > the bad overlapping threads/processes case.
> > >
> > > I suspect that your machines might be running an old OS which could be
> > > causing the failed affinity setting. If that is the case, you should
> talk
> > > to your sysadmins and have them figure out the issue. If you 

Re: [gmx-users] Performance of 4.6.1 vs. 4.5.5

[Szilárd Páll]

As Mark said, we need concrete details to answer the question:
- log files (all four of them: 1/2 nodes, 4.5/4.6)
- hardware (CPUs, network)
- compilers
The 4.6 log files contain much of the second and third point except the
network.

Note that you can compare the performance summary table's entries one by
one and see what has changed.

I suspect that the answer is simply load imbalance, but we'll have to see
the numbers to know.


--
Szilárd


On Sat, Mar 9, 2013 at 3:00 PM, Mark Abraham wrote:

> On Sat, Mar 9, 2013 at 6:53 AM, Christopher Neale <
> chris.ne...@mail.utoronto.ca> wrote:
>
> > Dear users:
> >
> > I am seeing a 140% performance boost when moving from gromacs 4.5.5 to
> > 4.6.1 when I run a simulation on a single node. However, I am "only"
> seeing
> > a 110% performance boost when running on multiple nodes. Does anyone else
> > see this? Note that I am not using the verlet cutoff scheme.
> >
>
> What's the processor and network for those runs?
>
> I'm not sure that this is a problem, but I was surprised to see how big the
> > difference was between 1 and 2 nodes, while for 2-10 nodes I saw a
> reliable
> > 10% performance boost.
> >
>
> Not sure what you mean by "reliable 10% performance boost." Reporting
> actual ns/day rates would be clearer. Is a "140% performance boost" a
> factor of 1.4 more ns/day or a factor of 2.4 more ns/day?
>
> Please note that, while I compiled the fftw (with sse2) and gromacs 4.6.1,
> > I did not compile the 4.5.5 version that I am comparing to (or its fftw)
> so
> > the difference might be in compilation options.
>
>
> Indeed.
>
>
> > Still, I wonder why the benefits of 4.6.1 are so fantastic on 1 node but
> > fall off to good-but-not-amazing on > 1 node.
> >
>
> Finding the answer would start by examining the changes in the timing
> breakdowns in your .log files. Switching from using in-memory MPI to
> network MPI is a significant cost on busy/weak networks.
>
> The system is about 43K atoms. I have not tested this with other systems or
> > cutoffs.
> >
> > My mdp file follows. Thank you for any advice.
> >
>
> Your system is probably not calculating energies very much. 4.6 uses
> force-only kernels if that's all you need from it.
>
> Mark
>
> Chris.
> >
> > constraints = all-bonds
> > lincs-iter =  1
> > lincs-order =  6
> > constraint_algorithm =  lincs
> > integrator = sd
> > dt = 0.002
> > tinit = 0
> > nsteps = 25
> > nstcomm = 1
> > nstxout = 25
> > nstvout = 25
> > nstfout = 25
> > nstxtcout = 5
> > nstenergy = 5
> > nstlist = 10
> > nstlog=0
> > ns_type = grid
> > vdwtype = switch
> > rlist = 1.0
> > rlistlong = 1.6
> > rvdw = 1.5
> > rvdw-switch = 1.4
> > rcoulomb = 1.0
> > coulombtype = PME
> > ewald-rtol = 1e-5
> > optimize_fft = yes
> > fourierspacing = 0.12
> > fourier_nx = 0
> > fourier_ny = 0
> > fourier_nz = 0
> > pme_order = 4
> > tc_grps =  System
> > tau_t   =  1.0
> > ld_seed =  -1
> > ref_t = 310
> > gen_temp = 310
> > gen_vel = yes
> > unconstrained_start = no
> > gen_seed = -1
> > Pcoupl = berendsen
> > pcoupltype = semiisotropic
> > tau_p = 4 4
> > compressibility = 4.5e-5 4.5e-5
> > ref_p = 1.0 1.0
> > dispcorr = EnerPres
> >
> > --
> > gmx-users mailing listgmx-users@gromacs.org
> > http://lists.gromacs.org/mailman/listinfo/gmx-users
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> > * Please don't post (un)subscribe requests to the list. Use the
> > www interface or send it to gmx-users-requ...@gromacs.org.
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> --
> gmx-users mailing listgmx-users@gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> * Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-users-requ...@gromacs.org.
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
--
gmx-users mailing listgmx-users@gromacs.org
http://lists.gromacs.org/mailman/listinfo/gmx-users
* Please search the archive at 
http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
* Please don't post (un)subscribe requests to the list. Use the
www interface or send it to gmx-users-requ...@gromacs.org.
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists