FWIW, this test almost always fails on my local Jenkins machine: org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr (Failed 16 times in the last 21 runs. Stability: 23 %)
I've also seen it fail 2 or 3 times on my primary dev machine in the last couple days. - Mark On Tue, May 26, 2015 at 9:05 AM Ishan Chattopadhyaya < [email protected]> wrote: > > The thing with randomized testing and test harness is that it's > > supposed to make your life easier -- to uncover things you wouldn't > > think about > > I am not sure randomized testing is of any help here. Isolated runs of > this test always passes for every seed. During full suite runs, the test > sometimes passes and sometimes fails. Hence, I've not been able to set my > debugger on the test and reproduce. I've added another patch at SOLR-7468, > which Anshum and I are testing right now. Hopefully that fixes it. > > > > This does not mean the code is correct, only that you were lucky :) > I'm beginning to lose faith in hadoop-minikdc and hence we're also testing > the same thing using an external KDC, to make sure there's no code issue. > > > > On Tue, May 26, 2015 at 2:15 PM, Dawid Weiss <[email protected] > > wrote: > >> Ah, ok. Yes, I didn't track the context that much, I know Mark's been >> trying to straighten out those tests but I don't follow that closely >> -- too much going on in my own field. >> >> Dawid >> >> On Tue, May 26, 2015 at 10:36 AM, Anshum Gupta <[email protected]> >> wrote: >> > I think you misunderstood me there. I'm not talking about not using the >> test >> > framework at all, but parts of it. e.g. how the test using >> > MiniSolrCloudCluster follows a different approach as compared to other >> > SolrCloud tests. I forgot to update here but I've finally figure why it >> > never failed for me (I had a default realm set in my /etc/krb5.conf >> file on >> > my machine). >> > So yes, I'm just trying to find a way to test this part in the correct >> > manner, and it may just involve an approach that is different from what >> most >> > tests currently use. I hope that makes sense. >> > >> > On Tue, May 26, 2015 at 12:07 AM, Dawid Weiss < >> [email protected]> >> > wrote: >> >> >> >> > Right, but I've had about 10 successful runs even since my last >> checkin. >> >> >> >> This does not mean the code is correct, only that you were lucky :) >> >> And the fact it still failed in spite of your efforts is not something >> >> to be ashamed of -- it's a sign you did a lot and there's *still* >> >> something wrong. >> >> >> >> The thing with randomized testing and test harness is that it's >> >> supposed to make your life easier -- to uncover things you wouldn't >> >> think about (or wouldn't have a chance to test, as is the case with >> >> filesystem emulation layers). Resigning from all this infrastructure >> >> and writing tests in plain JUnit runner would be dodging the problem, >> >> not solving it. Sure, it's not easy. And sure, it's a pain in the >> >> arse. But it's also gratifying to know you nailed the problem once you >> >> find it. >> >> >> >> Dawid >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: [email protected] >> >> For additional commands, e-mail: [email protected] >> >> >> > >> > >> > >> > -- >> > Anshum Gupta >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> >
