[
https://issues.apache.org/jira/browse/SOLR-5775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13913149#comment-13913149
]
Michael McCandless commented on SOLR-5775:
------------------------------------------
{quote}
bq. Precious few people can look at a Solr test failure and "know" that this
failure is OK and that other one is not, nor that it's OK to release / it
isn't, etc.
Disabling tests does nothing to address that. It's like closing your eyes and
saying "everythings okay". If those tests didn't run, it would be silly to
release. We could have no confidence in the system.
{quote}
Well, at release time, the RM could run all the bad apple tests, have you look
at those failures, and then you decide if they are the "OK bad apple failures"
or not?
Again, this is a poor way to run an open source project (that only Mark can say
that failure is OK and that other one is not), but it would at least still be
progress over what we now have.
bq. If someone is concerned, they should help with the tests.
If someone is concerned, they should help out in whatever way they are able,
including what Rob did here: it is progress if we all can trust Solr's test
suite. Then, if a failure does occur, we know it's a "real" failure and not a
"bad apple failure that only Mark can look at to decide if it's real or not".
> Disable constantly failing solr tests
> -------------------------------------
>
> Key: SOLR-5775
> URL: https://issues.apache.org/jira/browse/SOLR-5775
> Project: Solr
> Issue Type: Bug
> Components: Build
> Reporter: Robert Muir
>
> Currently, solr tests are failing 90%+ of the time. We've been through this
> before many times, the argument is always that someone is looking at the
> failures and knows which ones are bad.
> This argument is a lie. Nobody is watching these failures, or
> DistributedQueryComponentOptimizationTest would not have failed repeatedly
> for two straight days when the fix was trivial (I fixed this last night:
> http://svn.apache.org/r1571930)
> Its frustrating to me as a committer, solr tests *NEVER* pass on my machine,
> no matter how many times I try. How can i possibly commit something without
> knowing i am making the situation even worse?
> This is all a big problem for developers, release managers, even users of the
> project. The test suite should pass.
> The old argument that "solr tests are allowed to fail" is no longer valid. I
> will disable all constantly failing tests.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
