Hi, What was the timer used previously for benchmarks ?
Your page focuses on accuracy while I think an important point is already the availability/functionality of clock_gettime(CLOCK_PROCESS/THREAD_CPUTIME_ID, ...) on ARM, which measures process/thread execution time vs "wall time" for most other timer APIs. Good to know it looks accurate (and Internet seems to say that API is available on Android) Generally your program is alone to be benchmarked so wall time converges to exec time but as you suggest, it would be more correct to use this clock. On our side, team was using basic timer computing "wall time" + scheduler trace from perf or systemtap (to remove periods of non-execution in a pos-processing tool) in order for example to profile/debug a specific sequence in an application. We could get rid of post-processing tool now. Regards Fred Frederic Turgis OMAP Platform Business Unit - OMAP System Engineering - Platform Enablement > Texas Instruments France SA, 821 Avenue Jack Kilby, 06270 Villeneuve Loubet. 036 420 040 R.C.S Antibes. Capital de EUR 753.920 -----Original Message----- > From: linaro-dev-boun...@lists.linaro.org > [mailto:linaro-dev-boun...@lists.linaro.org] On Behalf Of Michael Hope > Sent: Tuesday, August 30, 2011 3:47 AM > To: Linaro Dev > Subject: Choosing the timer to use in benchmarking > > Being able to accurately and consistently measure the elapsed > CPU time is important for toolchain benchmarking. I ran a few > experiments today and wrote up the results at: > > https://wiki.linaro.org/WorkingGroups/ToolChain/Benchmarks/Tim > erAccuracy > > The original is available at: > > http://bazaar.launchpad.net/~michaelh1/linaro-toolchain-benchm > arks/trunk/view/head:/reports/timer-accuracy.rst > > Short story: > * Use clock_gettime(CLOCK_PROCESS_CPUTIME_ID, ...) > * The mean is unaffected by CPU load > * I/O load has a significant effect > * Use nice -n -20 to reduce the variance > > For a CPU bound, non-VFP, L1 bound, 5 s long program: > * The variation coefficient is < 0.01 % so we can reliably measure > 0.1 % changes > * The accuracy is around 50 us > > I've changed eembc-linaro and will change denbench-linaro > next. I recommend anyone else measuring time on core based > benchmarks to do the same. > > -- Michael > > _______________________________________________ > linaro-dev mailing list > linaro-dev@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/linaro-dev > _______________________________________________ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
