Jean Abou Samra <j...@abou-samra.fr> writes:
> Hi Paulo, > > > Le 14/02/2022 à 18:50, Paulo Matos a écrit : >> Hi, >> >> I am attempting to understand what are the failures in >> https://gitlab.com/lilypond/lilypond/-/merge_requests/875 >> >> I decided to have a baseline of test results from HEAD before applying >> !875. For this I configured git HEAD with ./configure >> GUILE_FLAVOUR=guile-2.2. > > > > I guess you mean 'git master' (HEAD is the top commit of whatever branch > you happen to be on, if you are on a branch at all). > > Oh dear - of course I did that. It's not the first time and it's embarrassing cause I work with git daily but there you go. My brain keeps writing `git HEAD` to mean the HEAD of the default branch. > >> After building I ran the tests with `make CPU_COUNT=12 -j12 test` >> >> I get (log tail): >> Running texi2pdf on file /tmp/tmp8dctbdf6.texi to detect default page >> settings. >> >> Dissecting... >> Writing snippets... >> Processing... >> Processing >> /home/pmatos/dev/lilypond/build/out/lybook-testdb/snippet-names-cc1d2dc88d557a856e363b03eeaff6cc.ly >> Command '/home/pmatos/dev/lilypond/build/out/bin/lilypond \ >> -I /home/pmatos/dev/lilypond/input/regression/abc2ly >> -dseparate-page-formats=ps -djob-count=12 -dseparate-log-files >> -dinclude-eps-fonts -dgs-load-fonts --header=texidoc -I >> /home/pmatos/dev/lilypond/Documentation/included/ >> -dcheck-internal-types -ddump-signatures -danti-alias-factor=1 -I >> "/home/pmatos/dev/lilypond/input/regression/abc2ly" -I >> "/home/pmatos/dev/lilypond/build/input/regression/abc2ly" >> -daux-files -dread-file-list -dno-strip-output-dir >> -dtall-page-formats=eps >> "/home/pmatos/dev/lilypond/build/out/lybook-testdb/snippet-names-cc1d2dc88d557a856e363b03eeaff6cc.ly"' >> returned non-zero exit status 1. >> make[2]: *** [/home/pmatos/dev/lilypond/build/.././make/ly-rules.make:36: >> out-test/collated-files.texi] Error 1 >> make[1]: *** >> [/home/pmatos/dev/lilypond/build/.././make/lysdoc-targets.make:19: >> lysdoc-test] Error 2 >> make: *** [/home/pmatos/dev/lilypond/build/../GNUmakefile.in:316: test] >> Error 2 > > > > Phooey. Whatever the better methods to obtain what you > want, this will need figuring out since it is important > that new contributors don't get stuck in cryptic build > failures. > > Interestingly enough after removing the build dir and redoing it all over, it doesn't fail. I thought it was related to configuring for guile 1.8 then configuring for guile 2.2, building and testing but I cannot reproduce the issue now. I can actually successfully build and run the tests. > >> Running that command with -V does not help much: >> >> [...] >> Processing `73/lily-d7b70d6a.ly' >> >> This returns with exit code 1. >> >> Has anyone seen this before? > > > > That command is not going to work in the source tree. > You need to run it in out/lybook-testdb. Furthermore, > it has -dseparate-log-files which is the source of the > non-verbosity: the logging output is redirected to separate > .log files for each .ly files. You should find information > in out/lybook-testdb/73/lily-d7b70d6a.log. > That's useful to know. I certainly was not aware of that. > Also, the build system gave you advice on how to find > failures at the beginning of the 'make test' run. Try > these commands. > Yes, I can see that now. Scrolling up is not my forte. It would certainly be more useful to have the notice at the end such that a tail command of the `make test` could show it, but now that I know it, it won't bother me. :) > >> Or is there a better way to obtain a test >> baseline so I can compare it with the results from !875 so that I have a >> chance to understand if I can move the MR forward? > > > Note that 'make test' does not give you a test baseline but > just checks that all regression tests compile. To get > a test baseline, you want 'make test-baseline'. On the other > hand, if all you want is to see the result of a 'make check' > run for the MR, you can just click on the middle green > mark on the MR page (there are three of them on the right > of 'Detached merge request pipeline #... passed') and > scroll to the end of the log to find a link to the > result of the make check run done automatically in the CI > system (it will be > https://vpetzel.gitlab.io/-/lilypond/-/jobs/1500499681/artifacts/test-results/index.html). > As you will find, this is the same as what has been posted > in the form of screenshots on the MR. > I successfully reproduced the failure in the MR. So `make test` runs the regression tests to `out`, `make test-baseline` creates a current baseline in `out-baseline` and `make check` runs the tests and creates a comparison with baseline. :) I hope I got it. > Remember, however, that existing tests passing is a necessary > condition for a merge request to be accepted, but definitely > not a sufficient one. > Makes sense. I will keep you posted on my developments in this regard. Thanks for your time helping a new wanna-be contributor. -- Paulo Matos
