Re: [OMPI users] resolution of MPI_Wtime

2016-04-11 Thread Dave Love
George Bosilca writes: > MPI_Wtick is not about the precision but about the resolution of the > underlying timer (aka. the best you can hope to get). What's the distinction here? (clock_getres(2) says "resolution (precision)".) My point (like JH's?) is that it doesn't generally return the inte

Re: [OMPI users] resolution of MPI_Wtime

2016-04-11 Thread Dave Love
Jeff Hammond writes: > George: > > Indeed, MPI_Wtick is not always a good measure of the precision of > MPI_Wtime. The way I would measure resolution is to call MPI_Wtime a few > million times. Is there typically a problem with just looping until the result changes a few times? Obviously if th

Re: [OMPI users] resolution of MPI_Wtime

2016-04-08 Thread George Bosilca
MPI_Wtick is not about the precision but about the resolution of the underlying timer (aka. the best you can hope to get). Thus, the measured time will certainly be larger, but, and this is almost a certainty, it will hardly be smaller. As a result, I am doubtful that an MPI implementation will pro

Re: [OMPI users] resolution of MPI_Wtime

2016-04-08 Thread Jeff Hammond
George: Indeed, MPI_Wtick is not always a good measure of the precision of MPI_Wtime. The way I would measure resolution is to call MPI_Wtime a few million times. For example, on Blue Gene/Q, MPI_Wtime was ~220 cycles per call. I don't remember what MPI_Wtick returned, but I guess it was 1./1.6

Re: [OMPI users] resolution of MPI_Wtime

2016-04-08 Thread Gilles Gouaillardet
Dave, gettimeofday() uses (seconds, microseconds) to represent the time, and hence, the resolution is hardcoded to 1 microsecond clock_gettine() uses (seconds, nanoseconds) and hence the resolution is hard coded to 1 nanosecond. this is the max resolution, and it could be lower than that depending

Re: [OMPI users] resolution of MPI_Wtime

2016-04-08 Thread Dave Love
George Bosilca writes: >> Other implementations of MPI have very accurate counters. >> > > This might be a discutable topic. A quick survey of some of the MPI > libraries available on a Linux cluster give the following precision for > MPI_Wtime implementation : > > mpich-3.1.4: wtick = 1.00e-

Re: [OMPI users] resolution of MPI_Wtime

2016-04-08 Thread George Bosilca
On Thu, Apr 7, 2016 at 4:19 PM, Jeff Hammond wrote: > > > On Thu, Apr 7, 2016 at 9:28 AM, Dave Love wrote: > >> ... >> Anyhow, the lesson seems to be that you shouldn't use mpi_wtime if you >> need decent precision or realistic mpi_wtick across implementations. >> >> I certainly hope that this i

Re: [OMPI users] resolution of MPI_Wtime

2016-04-07 Thread Jeff Hammond
On Thu, Apr 7, 2016 at 9:28 AM, Dave Love wrote: > ... > Anyhow, the lesson seems to be that you shouldn't use mpi_wtime if you > need decent precision or realistic mpi_wtick across implementations. > > I certainly hope that this isn't the lesson anyone learns from this. It is extremely importan

Re: [OMPI users] resolution of MPI_Wtime

2016-04-07 Thread Dave Love
Gilles Gouaillardet writes: > clock_gettime() has a higher precision than gettimeofday(), though it > likely has > a lower precision and higher overhead than opal_sys_timer_get_cycles() Do you mean OMPI actually uses cycles for timing somewhere? The precision of that would be irrelevant, when i

Re: [OMPI users] resolution of MPI_Wtime

2016-04-06 Thread Gilles Gouaillardet
Dave, i checked again, and yes, v1.10 uses gettimeofday() in MPI_Wtime() on linux. native timers are only used in aix and solaris $ grep -R OPAL_TIMER_USEC_NATIVE opal/mca/timer opal/mca/timer/aix/timer_aix.h:#define OPAL_TIMER_USEC_NATIVE 1 opal/mca/timer/altix/timer_altix.h:#define OPAL_TIMER

Re: [OMPI users] resolution of MPI_Wtime

2016-04-06 Thread Dave Love
Gilles Gouaillardet writes: > Dave, > > fwiw, on v1.10, we likely use the number of cycles / cpu freq. That would be a horribly broken means of timing. gettimeofday is actually called under mpi_wtime, as ompi_info claims. > see opal_sys_timer_get_cycles in > https://github.com/open-mpi/ompi-re

Re: [OMPI users] resolution of MPI_Wtime

2016-04-06 Thread George Bosilca
git blame reports that the gettime patch is in master for the last 1 1/2 years (Nov 2014). Here is an untested patch (cherrypicked 00300f464d plus few changes). george. On Wed, Apr 6, 2016 at 9:32 AM, Gilles Gouaillardet < gilles.gouaillar...@gmail.com> wrote: > Dave, > > fwiw, on v1.10, w

Re: [OMPI users] resolution of MPI_Wtime

2016-04-06 Thread Gilles Gouaillardet
Dave, fwiw, on v1.10, we likely use the number of cycles / cpu freq. see opal_sys_timer_get_cycles in https://github.com/open-mpi/ompi-release/blob/v1.10/opal/include/opal/sys/amd64/timer.h I cannot remember whether this is a monotonic timer. (e.g. MPI_Wtime() invoked on a given cpu is always lo

Re: [OMPI users] resolution of MPI_Wtime

2016-04-06 Thread Dave Love
I wrote: > Are you talking about different source? On further investigation it looks so -- the development source does feature clock_gettime, but the release doesn't. Back to LD_PRELOAD...

Re: [OMPI users] resolution of MPI_Wtime

2016-04-06 Thread Dave Love
George Bosilca writes: > In opal/mca/linux/timer_linux_component.c. The timer is a special component > which is statically included during the build process (via > the MCA_timer_IMPLEMENTATION_HEADER define in opal/mca/timer/base/base.h > line 48). Thus the symbol should appear directly in the li

Re: [OMPI users] resolution of MPI_Wtime

2016-04-06 Thread Dave Love
Nathan Hjelm writes: > I wouldn't be concerned about the usage of gettimeofday in scif. Those > calls are disabled by default and are for internal measurements > only. Nothing to do with Wtime. That was the point -- clock_gettime doesn't anywhere relevant.

Re: [OMPI users] resolution of MPI_Wtime

2016-04-05 Thread George Bosilca
In opal/mca/linux/timer_linux_component.c. The timer is a special component which is statically included during the build process (via the MCA_timer_IMPLEMENTATION_HEADER define in opal/mca/timer/base/base.h line 48). Thus the symbol should appear directly in the libmpi.a george. On Tue, Apr 5

Re: [OMPI users] resolution of MPI_Wtime

2016-04-05 Thread Nathan Hjelm
I wouldn't be concerned about the usage of gettimeofday in scif. Those calls are disabled by default and are for internal measurements only. Nothing to do with Wtime. -Nathan On Tue, Apr 05, 2016 at 05:31:38PM +0100, Dave Love wrote: > Aurélien Bouteiller writes: > > > Open MPI uses clock_gett

Re: [OMPI users] resolution of MPI_Wtime

2016-04-05 Thread Dave Love
Aurélien Bouteiller writes: > Open MPI uses clock_gettime when it is available, and defaults to > gettimeofday only when this better option can't be found. Check that > your system has clock_gettime and the resolution of this timer. That's what I thought after I raised this originally, but where

Re: [OMPI users] resolution of MPI_Wtime

2016-04-05 Thread Mark Santcroos
> On 05 Apr 2016, at 16:46 , Aurélien Bouteiller wrote: > Open MPI uses clock_gettime when it is available, and defaults to > gettimeofday only when this better option can't be found. Check that your > system has clock_gettime and the resolution of this timer. Depending on what you mean, I do

Re: [OMPI users] resolution of MPI_Wtime

2016-04-05 Thread Aurélien Bouteiller
Open MPI uses clock_gettime when it is available, and defaults to gettimeofday only when this better option can't be found. Check that your system has clock_gettime and the resolution of this timer. Aurélien -- Aurélien Bouteiller, Ph.D. ~~ https://icl.cs.utk.edu/~bouteill/ > Le 5 avr. 2016 à

[OMPI users] resolution of MPI_Wtime

2016-04-05 Thread Dave Love
I can't immediately find a reference, but I thought it had been agreed some time ago that MPI_Wtime should use high resolution clocks where they're available. However, 1.10 is still using gettimeofday on GNU/Linux. Is there a good reason for that?