Am 20.09.2024 um 14:44 schrieb Greg Troxel: > Roland Illig <roland.il...@gmx.de> writes: > >> The first commit demonstrates the buggy state by having a test that >> technically succeeds but has lots of FIXME comments in all the places >> that are wrong, already stating what is expected after the bugfix. >> >> The second commit fixes one of the bugs and updates the tests, removing >> several of the FIXME comments from the first commit. Further commits fix >> more of the problems, one by one. > > Why do you propose a test that passes with a FIXME, rather than xfail? > Isn't that what xfail is for?
Yes, you're right. I'm just not used to using xfail, and I never tried to integrate xfail into the test suites of usr.bin/make or usr.bin/xlint. And since the FIXME comments usually have a short half-life, it didn't make much of a difference to me, in practical terms. >> The benefit of this approach is that each commit shows the changes to >> the main code together with their effects on the tests, and it provides >> a direct before-after comparison. > > Doesn't xfail do that too? No. With xfail, the results before the bugfix are only in the test output, but not under version control. I expected the old test output to be deleted after a few weeks, until I saw that I can still view the test results from 2011, which, given the amount of test runs, is impressive. Still, in general, having the before-after comparison in the code itself feels more straight-forward to me, to have a single source to rely on. > I don't follow "to see what the _change_ was". There will be a bugfix > commit, with a comment. How is this hard, and how is it different from > what we usually do? The hard thing is the extra step of looking up the old results in a different source, instead of having all information bundled in the commit. Plus, your current case involves formatted floating point output, so the test output using %g may not be as accurate as an equality comparison in the source code, as the %g conversion specifier rounds the output. Roland
