That seems fair.  It would be great to have some higher level mechanism to 
generate runs of the tests that only differ by how the target programs are 
built.  For instance, if some day we ever get serious about optimized code 
debugging (suppresses giggle) it would be good to run the test suite on 
optimized binaries as well...  So if you are inclined to work on that, that 
would be excellent, and then we can revisit how to do clean etc. at that time.

Jim

> On Aug 26, 2015, at 11:10 AM, Zachary Turner via lldb-dev 
> <lldb-dev@lists.llvm.org> wrote:
> 
> I think I'm goign to abandon this idea for now.  Mostly because I've found a 
> workaround which is a) one line of code and b) only affects windows.  So the 
> impact of this workaround is narrower, no risk of messing up any tests which 
> depend on cleaning, and gives more time to come up with a more comprehensive 
> solution, such as something like I and Pavel proposed earlier.
> 
> On Wed, Aug 26, 2015 at 10:24 AM Todd Fiala <todd.fi...@gmail.com> wrote:
> Not sure if this is relevant, but I seem to recall the remote test execution 
> would spin off each test method run (test case level, not test suite level) 
> into a new directory.  I don't think that would be inherently broken by a 
> no-clean scenario but we'd want to make sure it doesn't break.
> 
> -Todd
> 
> On Wed, Aug 26, 2015 at 7:52 AM, Zachary Turner via lldb-dev 
> <lldb-dev@lists.llvm.org> wrote:
> I thought of this too and I started prototyping it.
> 
> The issue that I ran into is that dsym and dwarf tests can all be xfailed, 
> skipped, etc for different reasons, so if there is one method body, you need 
> a way to still define the set of conditions under which dsym and dwarf tests 
> should run, skip, xfail, timeout, etc.  
> 
> Do you want to start writing @skipIfDwarfAndOsIsLinuxButCompilerIsNotClang?  
> Because I know I don't want to deal with the combinatorial explosion of 
> decorators that would result :)
> 
> I have some ideas here as well, for example I think we only actually need 1 
> decorator that we can configure via keyword arguments that can handle 
> arbitrarily complex scenarios of xfailing, debug infos, etc.  e.g. 
> @lldb_test(debug_types="dwo,dwarf", xfail = {...}, skip = {...}).  But it's 
> all unrelated to the original problem I'm trying to solve.  So I think it 
> would be good to design a solution for that, but to do it separately.
> 
> The nice thing about just changing the default from clean to not clean is 
> that it's about a 3 line change, has potentially large speed improvements 
> across the board, and also fixes bugs.
> 
> 
> 
> On Wed, Aug 26, 2015 at 2:01 AM Pavel Labath <lab...@google.com> wrote:
> On 26 August 2015 at 06:14, Zachary Turner via lldb-dev
> <lldb-dev@lists.llvm.org> wrote:
> >
> > I'll wait and see if anyone can remember which tests rebuild binaries on
> > purpose.  Otherwise I will try to look through them and see if I can figure
> > it out.
> 
> TestInferiorChanged is one that I remember.
> 
> I think this is a good thing to do, but it will need to be done with a
> steady hand.
> 
> 
> 
> I was also thinking about the dsym/dwo tests.. Instead of basically
> having a copy of each test for dwarf and dsym (and soon also dwo), how
> about having just one test, and have some higher level logic (the test
> runner) know that it needs to execute each test multiple times. The
> tests would then just do a buildDefault() (or something) and on the
> first run it would build normal dwarf, on the second one dsym, etc. If
> we need to run a test only for some combination of debug infos, we
> could have @skipIfDsym annotations, like we do for the rest of stuff.
> I think this will make what Zachary is proposing easier to do, and it
> will make the test writing less awkard.
> 
> 
> What do you think? I'm ready to chip in on this if we agree to go down
> this way...
> 
> pl
> 
> _______________________________________________
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> 
> 
> 
> 
> -- 
> -Todd
> _______________________________________________
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

_______________________________________________
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Reply via email to