I get that it is technically fixable, but if we can't seem to accomplish it as a group I categorize that as not fixable. I think this feature requires more shepherding. "Hey Phil, your test is broken. Do you need pointers for fixing it?" Think of Zwoop when clang-format is busted or a new Coverity issue is found. Maybe it is my fault for not paying better attention, but I didn't even know there was a separate repo for this.
FWIW, I am trying to be constructive and I will try to be more active in fixing tsqa if I break it, but I will probably ask for help like that email that I sent to the list not to long ago. On Mon, Sep 21, 2015 at 3:15 PM Thomas Jackson <jacksontj...@gmail.com> wrote: > I'll repeat again, lint is fixable-- people just need to fix their test > code to pass those tests. I'm thinking we should just remove the broader > list, and just email the people that break the tests going forward. This > way we can still (hopefully) clean them up without spamming everyone to > death. > > > TSQA has docs both within the tests themselves (examples) as well as docs > in the trafficserver-qa repository (with examples). I'd be more than happy > to make changes/additions to documentation if something is missing :) > > On Mon, Sep 21, 2015 at 8:09 AM, Phil Sorber <sor...@apache.org> wrote: > > > On Mon, Sep 21, 2015 at 9:04 AM Thomas Jackson <jacksontj...@gmail.com> > > wrote: > > > > > Tsqa-lint is intended to pass, but if no one ever looks at the mails > then > > > they never get better. > > > On Sep 18, 2015 6:34 PM, "Leif Hedstrom" <zw...@apache.org> wrote: > > > > > > > > > > > > On Sep 18, 2015, at 3:32 PM, Thomas Jackson <jackso...@apache.org> > > > > wrote: > > > > > > > > > > *TL;DR: if you make a commit, and the tests fail-- there is a high > > > > > probability that your code change broke something. Please take the > > time > > > > to > > > > > figure out if the bug was you or not, and if it isn't let someone > > > know-- > > > > > the community can help. TSQA is back on, but tsqa-lint is off for > > now.* > > > > > > > > > > I can understand some of the complaints about tsqa-lint, but the > test > > > is > > > > > simply a lint check of the test code. If we want to remove that I'm > > > okay > > > > > with it, but the intent is to make sure your tests are well > formatted > > > (we > > > > > already do similar things for the rest of the codebase, so we have > > > > > precedent). > > > > > > > > > > > > Btw, if tsqa-lint is expected to fail, we should remove it IMO. > > > > > > > > — Leif > > > > > > > > > > > > > > > Thanks for fixing it. I'm +1 on removing lint if it's not fixable. If it > > was never right, it shouldn't have been mailing people. Also, it appears > > that was my commit that broke it, but I had no idea. I guess it just > seems > > to me like it was broken in the beginning and so I ignored the static of > > the emails and so if it was ever working correctly I didn't notice. > > > > I'm all for more testing. I think we should make sure that everyone knows > > how to run it and fix it though. We did something similar when we put > > clang-analyzer into Jenkins. It didn't break the build until we had > > everything cleared up. Then we made it a dependency and so emails mean > > something. > > > > So if we can leave lint emails off until it is passing (or removed) and > we > > can depend on TSQA not being noise then I am good with it being a proper > > build dependency going forward. > > > > Are there any docs on TSQA? Perhaps we should create a wiki page for it? > > > > Thanks. > > >
