Hi, On Tue, 23 Nov 2021 at 14:39, Jacob Hrbek <krey...@rixotstudio.cz> wrote:
>> This approximation would not even be accurate enough for the same >> machine. For instance, the test suite of the julia package runs >> mainly sequential using one thread... > > I am aware of this scenario and I adapted the equasion for it, but I > recognize that this exponentially increases the inaccuracy with more > threads and I don't believe that there is a mathematical way with the > provided values to handle that scenario so we would have to adjust the > calculation for those packages. What I am trying to explain is that the model cannot work to be predictable enough with what I consider a meaningful accuracy. Obviously, relaxing the precision, it is easy to infer a rule of thumb; a simple cross-multiplication fits the job. ;-) The “pokémon-battle” model is a simple linear model (cross-multiplication); using Jacob’s “notation”: - HP: time to build on machine A - DPS = nthread * cpufreq : “power” of machine Then it is expected to evaluate ’a’ and ’b’ on average such that: HP = a * DPS + b based on some experiments. Last, on machine B, knowing both nthread' and cpufreq' for that machine B, you are expecting to evaluate HP' for that machine B applying the formula: HP' = a * nthread' * cpufreq' + b Jacob, do I correctly understand the model? In any case, that’s what LFS is doing, instead HP is named SBU. And instead DPS, they use a reference package. And this normalization is better, IMHO. Other said, for one specific package considered as reference, they compute HP1 (resp. HP2) for machine A (resp. B), then for machine A, they know HP for another package and they deduce, HP' = HP2/HP1 * HP All this is trivial. :-) The key is the accuracy, i.e., the error between the prediction HP' and the real time. Here, the issue is that HP1 and HP2 capture for one specific package the overall time; which depends on hidden parameters as nthread, cpufreq, IO, and other parameters from hardware. But that a strong assumption when considering these hidden parameters (evaluated for one specific package) are equally the same for any other package. It is a strong assumption because the hidden parameters depends on hardware specifications (nthread, cpufreq, etc.) *and* how the package itself exploits them. Therefore, the difference between the prediction and the real time is highly variable, and thus personally I am not convince the effort is worth; for local build. That’s another story. ;-) LSF is well-aware of the issue and it is documented [1,2]. The root of the issue is the model based on a strong assumption; both (model and assumption) do not fit how the reality concrete works, IMHO. One straightforward way — requiring some work though – for improving the accuracy is to use statistical regressions. We cannot do really better to capture the hardware specification – noticing that the machine stress (what the machine is currently doing when the build happens) introduces a variability hard to estimate beforehand. However, it is possible to do better when dealing with packages. Other said, exploit the data from the build farms. Well, I stop here because it rings a bell: model could be discussed at length if it is never applied to concrete numbers. :-) Let keep it pragmatic! :-) Using the simple LFS model and SBU, what would be the typical error? For instance, I propose that we collectively send the timings of packages: bash, gmsh, julia, emacs, vim; or any other 5 packages for x86_64 architecture. Then we can compare typical errors between prediction and real, i.e., evaluate “accuracy“ for SBU and then decide if it is acceptable or not. :-) Cheers, simon 1: <https://www.linuxfromscratch.org/lfs/view/stable/chapter04/aboutsbus.html> 2: <https://www.linuxfromscratch.org/~bdubbs/about.html>
