Hi Folks, I want to resurrect this process and start to fix tests. Currently approx. 60-80 tests are failed each run on master.
As one from the first point I suggest to use TC investigations for brief research of test failures. If you can help with research please create and assign investigation on TC (on test fail select dropdown menu, then 'Investigate / Mute' then 'Investigated by: me' & 'Resolve: auto/manually'; Runs https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunAll&branch_IgniteTests24Java8=%3Cdefault%3E ). Also help with existing fixes review/merge would be very appreciated. Issue creation is still desired, but investigation will help us to identify tests failures which are under research and failures not researched for now. Sincerely, Dmitriy Pavlov вт, 7 нояб. 2017 г. в 0:06, Dmitry Pavlov <dpavlov....@gmail.com>: > "Ignored" means that the test is disabled from the code, using annotation > @Ignore. Not sure it works for our JUnit3 style Java tests. It is used for > .NET. > "Muted" test is disabled in the interface TeamCity, but test code is > actually executed (which is why hanging tests needs to be failed in > addition to mute). > > The fact that the run all now became green means all the tests that fall > at this point are flaky. If it is about 200 non-muted flaky tests having > the probability of passing 98%, then the whole chain will pass only with > the probability 0.98 ^ 200 = 1.7% > > I suggest continue to identify, fix, or mute such flaky tests to get > stable green RunAll for correct code. > > пн, 6 нояб. 2017 г. в 22:57, Denis Magda <dma...@apache.org>: > >> Dmitriy, >> >> Now it’s time to fix a pretty decent number of muted and ignored tests. >> BTW, what’s the difference between “ignored” and “muted”? I’ve seen the >> former before. >> >> — >> Denis >> >> > On Nov 5, 2017, at 9:58 AM, Dmitry Pavlov <dpavlov....@gmail.com> >> wrote: >> > >> > Hi Igniters, >> > >> > I am happy to report that the activity of monitoring and fixing the >> tests >> > brought the first significant victory! >> > >> > Today tests launch running all Ignite tests (Ignite 2.0 Tests->RunAll) >> is >> > completely green: >> https://ci.ignite.apache.org/viewLog.html?buildId=930953 >> > >> > This is an reason for being pride, but not a reason to stop this >> activity. >> > Unfortunately, there are still a number of tests with unpredictable >> > failures (flaky, 234 tests), and a number of tests have been muted. Full >> > project scope is available in the article >> > >> https://cwiki.apache.org/confluence/display/IGNITE/Make+Teamcity+Green+Again >> > >> > In any case, thank to all of you who helped and continues to help the >> > project to correct the tests and bugs. >> > >> > Sincerely, >> > Pavlov Dmitry >> > >> > >> > пт, 21 июл. 2017 г. в 14:29, Pavel Tupitsyn <ptupit...@apache.org>: >> > >> >>> Green Again >> >>> Again >> >> As if it ever was green :) >> >> >> >> Of course +1 on this, let me know if you see any .NET-related failures. >> >> >> >> On Fri, Jul 21, 2017 at 1:06 PM, Николай Ижиков < >> nizhikov....@gmail.com> >> >> wrote: >> >> >> >>> Hello, Igniters. >> >>> >> >>> Also ready to help to #MakeTeamcityGreenAgain ! >> >>> >> >>> 21 июля 2017 г. 12:56 PM пользователь "Vyacheslav Daradur" < >> >>> daradu...@gmail.com> написал: >> >>> >> >>>> Hi guys. >> >>>> >> >>>> I vote for #MakeTeamcityGreenAgain. :-) >> >>>> >> >>>> FYI: it had been described and supported previously[1] >> >>>> >> >>>> After the completion of my current task I will try to help with this >> >>>> activity. >> >>>> >> >>>> [1] >> >>>> http://apache-ignite-developers.2346864.n4.nabble. >> >>>> com/Test-failures-td14353.html >> >>>> >> >>>> 2017-07-21 12:39 GMT+03:00 Anton Vinogradov <a...@apache.org>: >> >>>> >> >>>>> Nikolay, >> >>>>> >> >>>>> That's also a big problem for me, as reviewer, to accept changes and >> >>>> merge >> >>>>> them to master. >> >>>>> >> >>>>> Dmitriy, >> >>>>> >> >>>>> Currently we have some contributions *from community* blocked by red >> >>>>> Teamcity. >> >>>>> >> >>>>> On Fri, Jul 21, 2017 at 12:47 AM, Nikolay Izhikov < >> >>>> nizhikov....@gmail.com> >> >>>>> wrote: >> >>>>> >> >>>>>> +1 to Dmitry Pavlov from me. >> >>>>>> >> >>>>>> Green team city would be very helpful. >> >>>>>> >> >>>>>> >> >>>>>>> If you are a newcomer, please share your feeling: are you >> >> confused >> >>> by >> >>>>>> test >> >>>>>>> failures on Teamcity? >> >>>>>> >> >>>>>> I'm working on my very first ignite issue >> >>>>>> and it's not easy to understand TC build results. >> >>>>>> >> >>>>>> >> >>>>>> 21.07.2017 00:30, Dmitriy Setrakyan пишет: >> >>>>>> >> >>>>>> I think the right question to ask is why do we have RED tests to >> >>> begin >> >>>>>>> with. In my view, TC should be green. If there are tests that do >> >> not >> >>>>> pass, >> >>>>>>> we should disable them and file blocker tickets for the next >> >>> releases. >> >>>>>>> >> >>>>>>> D. >> >>>>>>> >> >>>>>>> On Thu, Jul 20, 2017 at 1:20 PM, Dmitry Pavlov < >> >>> dpavlov....@gmail.com >> >>>>> >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>> Hi Igniters, >> >>>>>>>> >> >>>>>>>> We all know that there are a lot of open tasks in Ignite project. >> >>> To >> >>>> be >> >>>>>>>> able to handle all these tasks, we need to grow our community. >> >> And >> >>> to >> >>>>>>>> achieve that, we should pay special attention to new community >> >>>> members. >> >>>>>>>> >> >>>>>>>> As recent newcomer I’ve faced the following problem: There is no >> >>> way >> >>>> to >> >>>>>>>> find out by yourself whether your pull request changes are >> >> correct >> >>> or >> >>>>>>>> not. >> >>>>>>>> Contributor can not distinguish pull request test failures and >> >>> master >> >>>>>>>> test >> >>>>>>>> failures. Experienced Ignite contributor can estimate the damage, >> >>> but >> >>>>> how >> >>>>>>>> it can be done by new community member? >> >>>>>>>> >> >>>>>>>> Once failing test is introduced to master it is multiplexed to >> >> all >> >>>>> later >> >>>>>>>> PRs. As a result, we have tons of failures and suites timeouts, >> >>> which >> >>>>>>>> waste >> >>>>>>>> Teamcity agents time. >> >>>>>>>> >> >>>>>>>> I suggest to start new activity - MakeTeamcityGreenAgain. I’m >> >> sure >> >>>> that >> >>>>>>>> there are a lot of experienced developers in the community who >> >>>>> understand >> >>>>>>>> issues with tests very quickly. Please contact me if you are >> >> ready >> >>> to >> >>>>>>>> help >> >>>>>>>> the community to fix current test issues. >> >>>>>>>> >> >>>>>>>> I would appreciate any ideas and sharing your vision how we can >> >>>> achieve >> >>>>>>>> green teamcity. >> >>>>>>>> >> >>>>>>>> If you are a newcomer, please share your feeling: are you >> >> confused >> >>> by >> >>>>>>>> test >> >>>>>>>> failures on Teamcity? >> >>>>>>>> >> >>>>>>>> Sincerely, >> >>>>>>>> >> >>>>>>>> Dmitriy Pavlov >> >>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Best Regards, Vyacheslav D. >> >>>> >> >>> >> >> >> >>
