On Fri, Feb 7, 2020 at 9:48 PM Dan Eble <d...@faithful.be> wrote: > On Feb 7, 2020, at 15:21, Han-Wen Nienhuys <hanw...@gmail.com> wrote: > >>> * use a headless browser to take a image snapshot of the top of > regtest > >>> result page. > >>> > >> Sounds convoluted. Why not attach the difference images directly? > > > > Those are potentially 1372 images to attach if you made a change with > global impact. > > Why not attach the N images with the greatest differences directly? > > More generally, I'd want a digest of the results (not all of which are > visual) that is as useful as possible for the size we are willing to post > to the review. We control output-distance.py, so we could generate > something new that fits this case. > > More work , and I'm lazy :)
but yes, you are right. We could potentially do somehting more clever here. > >> Are full logs and test results retained, or does a developer need to > reproduce the test locally to get them? > > > > You'd retain the full logs and results as part of the docker image. > Currently my checkout is about 1.8G of data, and a lilypond docker image > itself would be close to that too. > > This approach is new to me. I'm used to CI systems that are configured to > archive particular files from the workspace (e.g., the regtest output tree, > the final docs) and full build log for a limited time (days to weeks). I > think it balances the types of things you can investigate without > reproducing the build yourself against retaining a huge amount of data. > IIRC, the regtest output tree is also fairly large. Can you expand on the purpose of saving the full Docker image--which is not > just the LilyPond workspace but the OS too, correct? Are you thinking that > someone would prefer to download it and debug in a container rather than > reproduce the build in their usual development environment? > You would save the output of the binary build, because it's the input to the regtest. The OS would be in a different layer, so we wouldn't shipping around OS images, but my hunch is that it will be a lot of data to ship around. I haven't measured though. > — > Dan > > -- Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen
