Hi,
> I would like to see the time for each run (if 10 runs, then 10 columns > i.e., xx_run1, xx_run2, ...), rather than only average of them. > > Collect as much as insights we could, maybe we find some pattern or > something that might help us (not sure what though). Since I have a > feeling, we might get better performance in one run and not so good in > another. > Thats true, that’s what troubling me the most. Before testing I read on stack overflow that sys+user time is always correct, but the most disturbing factor was that same port gives results significant differences on multiple runs. Like in case of gettext + reps, sometimes I get time as high as 13 mins and sometimes 11:30. Differences are also noticeable in sys and user times. Also I always made these tests on a completely new installation of the base. > > I notice that some ports take less time with modification and some > ports take more time. Just want to make sure it happens in multiple > runs. > I have made some more improvements since the last time, This time will make tests on every port for 10 times and upload on spreadsheet. > > In an ideal scenario, I would prefer a clean environment (can be a > fresh install of OS, bare minimum apps running, reboot before each > run, etc.) to run each test (containers maybe? or some CI like travis) > which might not be affected by other processes running in background. > > I didn’t reboot the system last time when I tested, but I had all my apps closed and only 2 tabs open in terminal, 1 for modified base and 1 for original. Also I didn’t make any runs simultaneously. This time I will reboot after each run, but testing like that for sure Is gonna take some days for a good number of ports. Fresh install of OS probably would be hard for me, can’t it be like a guest user. Thanks for the help, Mihir
