Hi Simon, > We are far from OpenBLAS. :-)
That's fine with me. The more distance between me and OpenBLAS, the happier I am ;-) > On Wed, 16 Feb 2022 at 14:04, Konrad Hinsen <konrad.hin...@fastmail.net> > wrote: > >> Making scientific computations bit-for-bit reproducible is the moral >> equivalent of keeping a detailed lab notebook: doing your best to tell >> others exactly what you did. > > A detailed lab notebook implies transparency and full control of > variability, not bit-for-bit reproducibility. That's why I said "moral" equivalent. Computations are different from experiments. Typical mistakes are different, and technical possibilities are different. 1. You can't have the equivalent of bit-for-bit reproducibility with experiment. You can with computers, and with good tool support (Guix!) it can become a routine task that takes little effort. So... why *not* do it? 2. A computation involves many more details than any typical experiment. Just writing down what you did is *not* enough for documenting a computation, as experience has shown. So you need more than the lab notebook. If your computation is bit-for-bit reproducible, you know that you have documented every last detail. Inversely, if you cannot reproduce to the bit level, you know that *something* is out of your control. In the end, my argument is more pragmatic than philosophical. If bit-for-bit reproducibility is (1) useful for resolving issues in the future, and (2) cheap to get with good tool support, then we should go for it. The main reason why people argue against it is lack of tool support in their work environments. They conclude that it's a difficult goal to achieve, and then start to reason that it's not strictly necessary for the scientific method. Which is true. But... it's still very useful. >> And that's the role of bit-for-bit reproducibility. > > From my understanding, the validation of a reproduction depends on > trust: what is the confidence about this or that? Well, bit-for-bit > reproducibility is one criteria for establishing such trust. However, > IMHO, such criteria is not the unique one, and defeating it can be > compensated by other criteria used by many experimental sciences. Definitely. But in many cases, bit-for-bit reproducibility is the cheapest way to build trust, given good tool support. In other cases, e.g. HPC or exotic hardware, it's expensive, and then you look for something else. Cheers, Konrad.
