@all, Is there some way to disable garbage collection during such benchmarks?
cheers -ben On Sun, Jun 18, 2017 at 3:51 AM, Steven Costiou <steven.cost...@kloum.io> wrote: > I spent my day testing and comparing execution speed between classes and > anonymous subclasses when i tried to compare two sets of values between > pharo classes: they also differ. > > In fact it seems that every instruction has a different execution speed if > we run it enough times. So it seems impossible to precisely compare > execution time between anon classes and pharo classes, at least i don't > know how to do it. I computed confidence intervals of various measures, the > differences in execution time that exist between classes and anonymous > classes can be between -2% to + 9%. But comparing sets of speed between > pharo classes, i also have similar intervals (but smaller, from -2% speed > to +5% speed). > > > > So maybe they have similar performances, at least sometimes they do and > sometimes one is slightly faster than the other, but in the end it is not > possible to tell. But maybe i'm looking for something which does not even > exist, maybe anonymous classes are designed to be as fast as regular > classes ? Or maybe it is common to have such variability when benchmarking > code ? > > > > Does it exist any instruction in pharo with a constant execution time ? > (or it could be possible with one million years to compute enough speed > tests and do an accurate mean...) > > > > Steven. > > > > Le 2017-06-16 18:14, Steven Costiou a écrit : > > You can find the code below. I just change the call to m by m1 to test the > two methods. > > I started again in a fresh pharo 6 image and now the results seem all > similar for the following code. I will do all my tests again to see if it > was my fault but it takes a lot of time (2 hours for each full test). > > What sould be the expected results ? Should an instance of an anonymous > class be as fast as a regular instance when calling the same code, being > defined in the anon class or its super class ? > > > > A implements m ^100 printString and B subclass of A implements m1 ^100 > printString > > a := B new. > > > class := A newAnonymousSubclass > addSlot: (InstanceVariableSlot named: #y); compile: 'm1 > ^100 printString'; yourself. > c := A new becomeForward: class new. > > res := Dictionary new. > col1 := OrderedCollection new. > col2 := OrderedCollection new. > > > 10000 timesRepeat: [ > Smalltalk garbageCollect . > col1 add: [1000000 timesRepeat: [ a m ]] timeToRun]. > > > 10000 timesRepeat: [ > Smalltalk garbageCollect . > col2 add: [1000000 timesRepeat: [ c m ]] timeToRun]. > > > res at: 'a' put: col1. > res at: 'c' put: col2. > res inspect > >
