Hi Konrad, We agree on the main points in the scope of Guix. :-) We probably disagree on some specific points about epistemology or epistemic justification; I am not sure to understand enough these terms to put them here. :-)
We are far from OpenBLAS. :-) On Wed, 16 Feb 2022 at 14:04, Konrad Hinsen <konrad.hin...@fastmail.net> wrote: > Making scientific computations bit-for-bit reproducible is the moral > equivalent of keeping a detailed lab notebook: doing your best to tell > others exactly what you did. A detailed lab notebook implies transparency and full control of variability, not bit-for-bit reproducibility. If my detailed lab notebook tracks my experiment to test gravity and pendulum, as detailed and ideal (moral?) as it would be, i.e., providing the capacity to build and re-build two exact same benches, then two experiences would not provide the bit-for-bit numbers in a table measuring the oscillations. Because, for instance, it would depend on the two locations, on the touch of the experimenter, etc. In many fields, the experimental reproduction depends on the variability of the inputs or of the instruments and therefore the scientific community, field by field, somehow defines what “same” means, depending on their common variability from their field. For one, I do not see why it would be different for the computational processing part of the experiment. And two, asking bit-for-bit reproducibility for one part of the experiment is asking far more than for the others non-computational part of the same experiment. Because I use daily computers and am deeply interested in what a computation means, for sure, I advocate for bit-to-bit reproducibility. But then, I discuss with my colleagues biologist or MD and somehow my views are biased, i.e, I am trying to apply my own criteria defining “same” from my “field” to their “field” where the same “same” must be applied to the all chain, computational processing included. Or at least they have to define what is acceptable for each part. Do not take me wrong, such computational part must be transparent where the variability must also be controlled, but no strictly more or totally less than the other parts. > When the forensics are called in, then... > >> Thus far, "show me the code" is the usual way to ask someone >> what they did, and guix makes is possible to answer in great >> detail. > > ... "show me the code" is not sufficient. You must also be sure that the > code you look at is really the code that was run. I agree. It is “show me ALL the code” and e.g., “guix graph python-scipy” points it is a long read. :-) Therefore, being able to build, run, re-build and re-run are weak requirements to establish trust. > And that's the role of > bit-for-bit reproducibility. >From my understanding, the validation of a reproduction depends on trust: what is the confidence about this or that? Well, bit-for-bit reproducibility is one criteria for establishing such trust. However, IMHO, such criteria is not the unique one, and defeating it can be compensated by other criteria used by many experimental sciences. Bah for what my opinion is worth on this topic. :-) In any cases, thanks Konrad for the materials you provide about this topic. For the interested French reader: :-) - https://www.societe-informatique-de-france.fr/wp-content/uploads/2021/11/1024_18_2021_11.html - https://webcast.in2p3.fr/video/les-enjeux-et-defis-de-la-recherche-reproductible - https://www.fun-mooc.fr/en/courses/reproducible-research-methodological-principles-transparent-scie/ Cheers, simon
