I believe pre-merge testing would be very useful. The thing which I would find very useful (let's call it a feature request) is to be able to get some sort of an overview/dashboard of all the runs on this infrastructure and their results. It doesn't have to be super detailed -- the question which I want an answer to is "which builds have an lldb test failing", and the reason is to check for possible flakyness.
The reason I am bringing this up is because lldb tests tend to be more flaky than most llvm tests (for various reasons). I've been trying to hunt down all the causes of this, and I believe we're in a pretty good shape right now (I haven't seen a flaky build on linux for at least three weeks), but it is not uncommon for new test setups to uncover new sources of flakyness. In particular, I am interested in the behavior of tests exercising the hardware debug features of cpus (e.g. hardware watchpoints), as I know that a lot of virtualization environments don't virtualize these properly/reliably (and IIUC, this infrastructure uses at least one level of virtualization). If these tests are not reliably there, we should avoid running them on this infrastructure. I don't think this needs to block anything, but I want to make sure everyone is aware of the possible issues. pl On 17/05/2020 03:08, Eric Christopher via lldb-dev wrote: > > > On Sat, May 16, 2020 at 12:18 PM Greg Clayton <clayb...@gmail.com > <mailto:clayb...@gmail.com>> wrote: > > > >> On May 15, 2020, at 7:04 PM, Eric Christopher via lldb-dev >> <lldb-dev@lists.llvm.org <mailto:lldb-dev@lists.llvm.org>> wrote: >> >> Hi All, >> >> We've been testing[1] a number of patches upstream by default via >> some pre-merge checks in phabricator. I was thinking of turning >> them on for lldb as well. Mostly it well just help people know >> whether or not they've broken lldb before they commit something, >> but won't stop committing or do anything else that direction. > > I am all for it! > >> Let me know what you think and otherwise I'd like to turn it on in >> a week or so. This will also help keep the test suite a little >> cleaner on linux FWIW. > > Please do. > >> There are a few additional links down below and if you have any >> questions send them my way. > > Will the lldb tests be run automagically if and only if lldb code is > modified in the patch? > > > I don't think our dependencies in cmake are that good for tests ... > especially since lldb uses a largeish chunk of clang and llvm anyhow :) > > -eric > > > >> Thanks! >> >> -eric >> >> >> [1] >> >> https://github.com/google/llvm-premerge-checks/blob/master/docs/user_doc.md >> [2] https://reviews.llvm.org/project/members/78/ >> [3] https://github.com/google/llvm-premerge-checks/issues >> _______________________________________________ >> lldb-dev mailing list >> lldb-dev@lists.llvm.org <mailto:lldb-dev@lists.llvm.org> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > > > _______________________________________________ > lldb-dev mailing list > lldb-dev@lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
