Hi Pjotr,
Pjotr Prins <pjotr.publi...@thebird.nl> writes:
> and he gave me a new insight which rang immediately true. He said:
> what is the point of running tests everywhere? If two people test the
> same thing, what is the added value of that? (I paraphrase)
>
> With Guix a reproducibly building package generates the same Hash on
> all dependencies. Running the same tests every time on that makes no
> sense.
I appreciate your thoughts on this, but I respectfully disagree.
> I know there are two 'inputs' I am not accounting for: (1) hardware
> variants and (2) the Linux kernel. But, honestly, I do not think we
> are in the business of testing those. We can assume these work.
No, we can't. For example, I recently discovered that GNU Tar fails one
of its tests on my GuixSD system based on Btrfs. It turned out to be a
real bug in GNU Tar that could lead to data loss when creating an
archive of recently written files, with --sparse enabled. I fixed it in
commit 45413064c9db1712c845e5a1065aa81f66667abe on core-updates.
I would not have discovered this bug if I had simply assumed that since
GNU Tar passes its tests on ext4fs, it surely must also pass its tests
on every other file system.
> If not, any issues will be found in other ways (typically a segfault
> ;).
The GNU Tar bug on Btrfs would never produce a segfault. The only way
the bug could be observed is by noticing that data was lost. I don't
think that's a good way to discover a bug. I'd much rather discover the
bug by a failing test suite.
Tests on different hardware/kernel/kernel-config/file-system
combinations are quite useful for those who care about reliability of
their systems. I, for one, would like to keep running test suites on my
own systems.
Regards,
Mark
