Sure multiple run lines can be commands being executed that depend on each other, but it's very common for a test to have multiple check labels as well
On Wed, Apr 25, 2018 at 9:19 AM Adrian Prantl <apra...@apple.com> wrote: > > On Apr 25, 2018, at 9:08 AM, Zachary Turner <ztur...@google.com> wrote: > > But is there a reason why that is more valuable with LLDB than it is with > LLVM? In LLVM when a test fails it stops and doesn't run subsequent run > lines. So you have the same issue there. > > > That's not a good analogy. Multiple RUN lines don't necessarily mean > separate tests, they mean separate commands being executed that may depend > on each other. As Pavel pointed out before, this would be closer to how > gtests behave. > > -- adrian > > The way this is handled in LLVM is that if you think tests are > sufficiently different that they could be broken by different types of > changes, you split them up into different files. > > Do you feel LLDB is fundamentally different in this regard? > > On Wed, Apr 25, 2018 at 8:27 AM Adrian Prantl <apra...@apple.com> wrote: > >> Disregarding the added complexity I believe that being able to >> communicate the number of tests filed/passed inside a single file would be >> very valuable. For example when one test gets broken by upstream clang >> changes and it takes a few days to fix it properly, we don't want to loose >> signal on the other tests in the same file during that period. I think it's >> worth it. >> >> -- adrian >> >> >> On Apr 25, 2018, at 8:13 AM, Zachary Turner <ztur...@google.com> wrote: >> >> Well let’s see what Davide and Adrian think. I’m more of an outsider >> these days so consider my perspective an llvm-centric one, which would >> sometimes (but not always) be the best for lldb >> On Wed, Apr 25, 2018 at 12:31 AM Pavel Labath via Phabricator < >> revi...@reviews.llvm.org> wrote: >> >>> labath added a comment. >>> >>> In https://reviews.llvm.org/D46005#1077016, @zturner wrote: >>> >>> > Note that there's currently no precedent (that i'm aware of anwyay) in >>> LLVM or any of its subprojects for splitting the running of a single file >>> into multiple parallel jobs. All of LLVM's other projects parallelize at >>> file granularity, and so it's up to to the person writing tests to ensure >>> that the file granularity matches that with which they want tests to be >>> parallelized at. >>> >>> >>> There's always gtest. Regardless of whether you consider a .cpp file or >>> the built exectable to be a "test file", it will generally contain a number >>> of tests. And arguably, our test suite is more similar to the gtest suite >>> than the traditional lit (ShTest) suites (for one, it hijacks a third party >>> unit testing library, and then tries to make it run under lit). And we run >>> all gtest tests individually (I've often wondered what kind of performance >>> impact that has, but they seem to be running fast enough anyway...). >>> >>> For what it's worth, paralelization is not my motivation here. There >>> some tests which run disproportionately long, and this will speed them up, >>> but that may be offset by the fact we need to start more work this way (if >>> anyone is interested I can try to do some measurements). My main reason for >>> doing this is so that we can have better mapping of test result states. As >>> it stands now, we only have two possible results for a test: passed or >>> failed. Lit has more results than that, and they roughly match the existing >>> dotest states (the different treatment of XFAILs is the biggest >>> difference), so it should be possible to represent them with more fidelity. >>> However, right now it's not possible to translate them reasonably, as a >>> "single test" can result in any number of fails/xfails/skips/... (or no >>> results at all). >>> >>> > That doesn't mean it's not possible (as you've shown here), but it >>> adds some additional complexity and I'm not sure it's worth it. >>> >>> It adds complexity, but not much imho. I was actually pleasantly >>> surprised at how easy it was to pull this off. The entire implementation is >>> 78 LOC right now. The changes to the test format will probably push it over >>> a 100, but not by much. >>> >>> That said, I am not interested in forcing this onto everyone. I did it >>> because it seemed like a nice thing to have and I was curious if it is >>> doable (easily). However, if there's no interest for it, then I can just >>> abandon it.. >>> >>> >>> https://reviews.llvm.org/D46005 >>> >>> >>> >>> >>
_______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
