*TL;DR: if you make a commit, and the tests fail-- there is a high probability that your code change broke something. Please take the time to figure out if the bug was you or not, and if it isn't let someone know-- the community can help. TSQA is back on, but tsqa-lint is off for now.*
I can understand some of the complaints about tsqa-lint, but the test is simply a lint check of the test code. If we want to remove that I'm okay with it, but the intent is to make sure your tests are well formatted (we already do similar things for the rest of the codebase, so we have precedent). As for the main tsqa tests I'm a little disappointed that we'd turn them off without really looking into what the issue is. I have had a few different people ping me on IRC about "stability" problems with tsqa in the past, and to-date all of them have been issues with code changes breaking tests (since tsqa replaced the old one). I took some time to look into this today, and there were 2 problems (which collectively took 30m to fix). First, the builds have been failing since Jun 15th. This gets a bit murky because Leif had actually made a change which broke all of tsqa (backwards incompatible change to records.config option the framework uses). Once that was fixed, the tests were mostly passing-- but a single test failed-- which is still failing. That one test tests which tests proxy.config.log.squid_log_is_ascii. This feature was removed June 16th ( https://issues.apache.org/jira/browse/TS-1985) which explains why all builds since then have failed-- because a backwards incompatible change was made and a test caught it! I'm going to assume that this change was made on a major version release-- which would be okay, but we need to keep the tests up-to-date with these sorts of changes (and hopefully update CHANGES)-- not just turn them off. After figuring this out locally, I went to run the tests in jenkins and they were failing with some crazy error message: > ERROR: Error fetching remote repo 'origin' > ERROR <http://stacktrace.jenkins-ci.org/search?query=ERROR>: Error > fetching remote repo 'origin' Turns out there is a bug in the git plugin to jenkins that the error message is bad... ( https://cloudbees.zendesk.com/hc/en-us/articles/205023034-Error-cloning-remote-repo-origin). After checking the repo out by logging into the box it seems that someone was on box modifying jenkins' workspace without fixing it: [root@qa3 src]# git clean -dfx > fatal: Not a git repository: > /Users/mitsuhiko/Development/flask/.git/modules/docs/_themes After blowing away the local checkout on the QA box-- its working just fine. If you log onto QA boxes to manually do stuff, please make sure to clean up after yourself (or maybe not make changes to the QA boxes' workspaces ;) ). On Fri, Sep 11, 2015 at 7:55 AM, Phil Sorber <sor...@apache.org> wrote: > +1 > > On Fri, Sep 11, 2015 at 8:50 AM Leif Hedstrom <zw...@apache.org> wrote: > > > Hi, > > > > To restore sanity in the email traffic from Jenkins, I have disabled both > > the tsqa and tsqa-lint jobs. Once they are shown to function without > errors > > / warnings again, we can enable them. > > > > — Leif > > > > >
