Verification tests parked (i.e. disabled) here: r255134 I decided to leave them in the repo so it is faster/easier to do this in the future.
-Todd On Wed, Dec 9, 2015 at 10:26 AM, Todd Fiala <todd.fi...@gmail.com> wrote: > The reports look good at the test level: > > > http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9294 > > I'd say the buildbot reflection script missed the ERROR, so that is > something maybe Ying can look at (the summary line in the build run), but > that is unrelated AFAICT. > > I'm going to move aside the failures. > > On Wed, Dec 9, 2015 at 10:13 AM, Todd Fiala <todd.fi...@gmail.com> wrote: > >> I am going to stop the current build on that builder. There was one >> change in it, and it will be another 20 minutes before it completes. I >> don't want the repo in a known broken state that long. >> >> On Wed, Dec 9, 2015 at 10:07 AM, Todd Fiala <todd.fi...@gmail.com> wrote: >> >>> I forced a build on the ubuntu 14.04 cmake builder. The build _after_ >>> 9292 will contain the two changes (and we will expect failures on it). >>> >>> On Wed, Dec 9, 2015 at 10:05 AM, Todd Fiala <todd.fi...@gmail.com> >>> wrote: >>> >>>> These went in as: >>>> >>>> r255130 - turn it on by default >>>> r255131 - create known issues. This one is to be reverted if all 3 >>>> types show up properly. >>>> >>>> On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala <todd.fi...@gmail.com> >>>> wrote: >>>> >>>>> It is a small change. >>>>> >>>>> I almost have all the trial tests ready, so I'll just commit both >>>>> changes at the same time (the flip on, and the trial balloon issues). >>>>> >>>>> If all goes well and the three types of issue show up, then the last >>>>> of the two will get reverted (the one with the failures). >>>>> >>>>> If none (or only some) of the issues show up, they'll both get >>>>> reverted. >>>>> >>>>> -Todd >>>>> >>>>> On Wed, Dec 9, 2015 at 9:39 AM, Pavel Labath <lab...@google.com> >>>>> wrote: >>>>> >>>>>> If it's not too much work, I think the extra bit of noise will not be >>>>>> a problem. But I don't think it is really necessary either. >>>>>> >>>>>> I assume the actual flip will be a small change that we can back out >>>>>> easily if we notice troubles... After a sufficient grace period we can >>>>>> remove the old formatter altogether and hopefully simplify the code >>>>>> somewhat. >>>>>> >>>>>> pl >>>>>> >>>>>> On 9 December 2015 at 17:08, Todd Fiala <todd.fi...@gmail.com> wrote: >>>>>> > Here's what I can do. >>>>>> > >>>>>> > Put in the change (setting the default to use the new format). >>>>>> > >>>>>> > Separately, put in a trial balloon commit with one failing test, one >>>>>> > exceptional exit test, and one timeout test, and watch the ubuntu >>>>>> 14.04 >>>>>> > buildbot catch it and fail. Then reverse this out. That should >>>>>> show beyond >>>>>> > a reasonable doubt whether the buildbot catches new failures and >>>>>> errors. (I >>>>>> > think this is a noisy way to accomplish this, but it certainly would >>>>>> > validate if its working). >>>>>> > >>>>>> > -Todd >>>>>> > >>>>>> > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala <todd.fi...@gmail.com> >>>>>> wrote: >>>>>> >> >>>>>> >> Specifically, the markers for issue details are: >>>>>> >> >>>>>> >> FAIL >>>>>> >> ERROR >>>>>> >> UNEXPECTED SUCCESS >>>>>> >> TIMEOUT >>>>>> >> >>>>>> >> (These are the fourth field in the array entries (lines 275 - 290) >>>>>> of >>>>>> >> packages/Python/lldbsuite/test/basic_results_formatter.py). >>>>>> >> >>>>>> >> -Todd >>>>>> >> >>>>>> >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala <todd.fi...@gmail.com> >>>>>> wrote: >>>>>> >>> >>>>>> >>> That's a good point, Tamas. >>>>>> >>> >>>>>> >>> I use (so I claim) the same all upper-case markers for the test >>>>>> result >>>>>> >>> details. Including, not using XPASS but rather UNEXPECTED >>>>>> SUCCESS for >>>>>> >>> unexpected successes. (The former would trigger the lit script >>>>>> IIRC to >>>>>> >>> parse that as a failing-style result). >>>>>> >>> >>>>>> >>> The intent is this is a no-op on the test runner. >>>>>> >>> >>>>>> >>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer < >>>>>> tbergham...@google.com> >>>>>> >>> wrote: >>>>>> >>>> >>>>>> >>>> +Ying Chen >>>>>> >>>> >>>>>> >>>> Ying, what do we have to do on the build bot side to support a >>>>>> change in >>>>>> >>>> the default test result summary formatter? >>>>>> >>>> >>>>>> >>>> On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev >>>>>> >>>> <lldb-dev@lists.llvm.org> wrote: >>>>>> >>>>> >>>>>> >>>>> Hi all, >>>>>> >>>>> >>>>>> >>>>> Per a previous thread on this, I've made all the changes I >>>>>> intended to >>>>>> >>>>> make last night to get the intended replacement of test run >>>>>> results meet or >>>>>> >>>>> exceed current requirements. >>>>>> >>>>> >>>>>> >>>>> I'd like to switch over to that by default. I'm depending on >>>>>> the test >>>>>> >>>>> event system to be able to handle test method reruns in test >>>>>> results >>>>>> >>>>> accounting. >>>>>> >>>>> >>>>>> >>>>> The primary thing missing before was that timeouts were not >>>>>> routed >>>>>> >>>>> through the test events system, nor were exception process >>>>>> exits (i.e. test >>>>>> >>>>> inferiors exiting with a signal on POSIX systems). Those were >>>>>> added last >>>>>> >>>>> night so that test events are generated for those, and the >>>>>> >>>>> BasicResultsFormatter presents that information properly. >>>>>> >>>>> >>>>>> >>>>> I will switch it over to being the default output in a bit here. >>>>>> >>>>> Please let me know if you have any concerns once I flip it on >>>>>> by default. >>>>>> >>>>> >>>>>> >>>>> Thanks! >>>>>> >>>>> -- >>>>>> >>>>> -Todd >>>>>> >>>>> _______________________________________________ >>>>>> >>>>> lldb-dev mailing list >>>>>> >>>>> lldb-dev@lists.llvm.org >>>>>> >>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> -- >>>>>> >>> -Todd >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> -- >>>>>> >> -Todd >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > -- >>>>>> > -Todd >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> -Todd >>>>> >>>> >>>> >>>> >>>> -- >>>> -Todd >>>> >>> >>> >>> >>> -- >>> -Todd >>> >> >> >> >> -- >> -Todd >> > > > > -- > -Todd > -- -Todd
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
