Hi, i'm having trouble doing benchmarks in Pharo, at least i don't understand the results (maybe i'm missing something). I want to compare anonymous subclasses lookup speed with standard pharo classes lookup speed.
I have a method m and a method m2 that execute the following code: ^ 100 printString. I have implemented these 2 methods in one class then in one anonymous subclass. I thought that measuring the execution speed of the same code in these two different cases would give me different times if the lookup speed was not the same. So I did the following: |o results repetitions | o := theClass new. results := OrderedCollection new. repetitions := 100. repetitions timesRepeat:[ results add: [1000000 timesRepeat:[o m]] timeToRun]. I do the same for m2, and then i compute an average of all the values measured in results. What i don't understand is that, for example, for an average on 100 measurements, m is 1% slower and m2 is 2% faster in the Pharo class case than with anonymous subclasses. But for 1 000 measurements, m is 11% faster but m2 is 3% slower. Results continue to vary as i change the number of measurements, but they do not increase with it (seems not to be linear). I don't have enough knowledge about how to benchmark code, or what will make a difference in Pharo. For now the only explanations i have is that maybe the results are too slow to be significant and then they can vary, or i have done something wrong in how i measure it. How would you measure the lookup speed of a method ? Steven.
