Yep, it's working. Thank you. 2018-09-13 19:09 GMT+03:00 Dmitriy Pavlov <dpavlov....@gmail.com>:
> Hi Dmitriy, > > Sure, I agree with extra permissions assignment. Done. > > Could you please check if you can manage your builds now? > > Sincerely, > Dmitriy Pavlov > > чт, 13 сент. 2018 г. в 16:05, Dmitrii Ryabov <somefire...@gmail.com>: > > > Hi, Dmitriy, > > > > Can you give me rights to stop my builds on TeamCity? Working on the > TCH, I > > run a lot of builds, and I don't like to ask other people to stop builds > > too often. > > > > 2018-09-10 23:53 GMT+03:00 Dmitrii Ryabov <somefire...@gmail.com>: > > > > > Hi, Dmitriy, > > > > > > I made PRs in my fork for test purposes. Real PRs were made to the > GitHub > > > mirror, and one of them is already merged by D. Govorukhin. PR with > > GitHub > > > statuses [1] is ready for review. PR with JIRA comment will be ready > in a > > > few days. > > > > > > [1] https://github.com/apache/ignite-teamcity-bot/pull/5 > > > > > > 2018-09-10 23:00 GMT+03:00 Dmitriy Pavlov <dpavlov....@gmail.com>: > > > > > >> Hi Dmitrii, > > >> > > >> I deployed change with blockers summary of failures at the top of PR > > >> result > > >> page. The Bot is migrating entries now, I hope it will be done in 1-4 > > >> hours. > > >> > > >> I noticed your PRs are created in your own fork, not in > > >> https://github.com/apache/ignite-teamcity-bot > > >> Could you please create unmerged PR in GitHub mirror so it could be > > merged > > >> after reviewing by the apply-pr script. > > >> > > >> Sincerely, > > >> Dmitriy Pavlov > > >> > > >> пн, 3 сент. 2018 г. в 18:20, Dmitrii Ryabov <somefire...@gmail.com>: > > >> > > >> > Hi, Dmitriy, > > >> > > > >> > I created a table with possible blockers [1] for the page with the > > >> latest > > >> > build analysis. Anton K. has reviewed it. Can you check and merge > it? > > >> > > > >> > Also, I created a button to notify PR about analisis results [2]. I > > used > > >> > GitHub statuses (example [3]), but to work it needs a token with > push > > >> > permission. As Apache PMC, can you acquire such token? I think > > statuses > > >> are > > >> > much more notable than comments. > > >> > > > >> > [1] https://issues.apache.org/jira/browse/IGNITE-9376 > > >> > [2] https://issues.apache.org/jira/browse/IGNITE-9377 > > >> > [3] https://github.com/SomeFire/ignite-teamcity-bot/pulls > > >> > > > >> > 2018-08-25 1:04 GMT+03:00 Dmitriy Pavlov <dpavlov....@gmail.com>: > > >> > > > >> > > Not sure I understood what bot statistic reduction means. > > >> > > > > >> > > As far as I understood, you also mean locating failure > automatically > > >> with > > >> > > re-runs on specific git revisions (hashes). I found it will be a > > good > > >> > > contribution to TC Bot for both flaky and non-flaky failures > > >> detection. > > >> > E.g > > >> > > failure occurred after 10 commits merged to master. How to find > out > > >> bad > > >> > > commit? Automatic location (analog of bisecting, but using TC > test) > > >> will > > >> > be > > >> > > really helpful. But it is not a simple contribution. > > >> > > > > >> > > пт, 24 авг. 2018 г. в 18:35, Dmitrii Ryabov < > somefire...@gmail.com > > >: > > >> > > > > >> > > > Hi, Dmitriy, > > >> > > > > > >> > > > Good, I'll create tickets for this steps. > > >> > > > > > >> > > > Stable suites are a good idea, but also we need to do some > actions > > >> > when a > > >> > > > flaky test will appear in stable suite of master branch run. To > > find > > >> > out > > >> > > a > > >> > > > guilty commit, we should run previous commits on empty > TeamCity's > > >> > queue. > > >> > > > This runs will reduce bot's statistics for the latest commits. > > >> > > > > > >> > > > 2018-08-24 16:32 GMT+03:00 Dmitriy Pavlov < > dpavlov....@gmail.com > > >: > > >> > > > > > >> > > > > Hi Dmitriy, > > >> > > > > > > >> > > > > Sounds like a plan ;) I totally agree. > > >> > > > > Just one proposal: I would like to avoid hiding any test > > >> failures. 2 > > >> > > > > separate tables named 'Possible Blockers' and 'Other failures' > > >> should > > >> > > be > > >> > > > > completely clear. Comment to PR can contain only first > category. > > >> > > > > > > >> > > > > We had a private discussion with Anton K. and he proposed a > very > > >> > > > > interesting thing, I would like to bring it here. We can add > > >> > > > configuration > > >> > > > > into TC bot and mark some tests and some suites as 'Known > > Stable' > > >> It > > >> > > > means > > >> > > > > that suite or test failure should be considered as a blocker > for > > >> > merge > > >> > > > > every time it fails, even if fail rate is non-zero. Very first > > >> list > > >> > of > > >> > > > such > > >> > > > > suites are > > >> > > > > - Build Apache Ignite > > >> > > > > - License check > > >> > > > > And tests: > > >> > > > > - .Net API Parity check > > >> > > > > Please share your vision. > > >> > > > > > > >> > > > > Meanwhile, I've updated the TC bot. > > >> > > > > - Underlying Apache Ignite DB was refactored to use lower > > >> partitions > > >> > > > count, > > >> > > > > so restart should be faster. > > >> > > > > - Now 100 builds are saved as our failure rate statistics > basis. > > >> > > > Flakiness > > >> > > > > border was not changed, so more test now will be considered as > > >> flaky. > > >> > > > > - Now 4 builds required to be failed or timed out in a row > > before > > >> > > > > notification is sent. > > >> > > > > > > >> > > > > Sincerely, > > >> > > > > Dmitriy Pavlov > > >> > > > > > > >> > > > > чт, 23 авг. 2018 г. в 17:09, Dmitrii Ryabov < > > >> somefire...@gmail.com>: > > >> > > > > > > >> > > > > > Hi, Dmitriy, > > >> > > > > > > > >> > > > > > I propose the next steps: > > >> > > > > > > > >> > > > > > 1. Show current 0% tests in a separate table at the top of > the > > >> > > analysis > > >> > > > > > results page. Thus, we'll see most suspicious (new or very > > >> flaky) > > >> > > tests > > >> > > > > > firstly. Or we can hide other tests under "More >>" button, > > like > > >> > top > > >> > > > long > > >> > > > > > running tests. > > >> > > > > > 2. Create a button by clicking on which the info about 0% > > tests > > >> > will > > >> > > be > > >> > > > > > written in the PR. > > >> > > > > > 3. Replace button by webhook for TeamCity (for Run All), > which > > >> will > > >> > > > start > > >> > > > > > analysis on TCH and write results in the PR. > > >> > > > > > 4. ... > > >> > > > > > > > >> > > > > > What do you think? > > >> > > > > > > > >> > > > > > > > >> > > > > > 2018-08-22 8:55 GMT+03:00 Dmitrii Ryabov < > > somefire...@gmail.com > > >> >: > > >> > > > > > > > >> > > > > > > I think we should check not N last runs, but all runs in > > last > > >> N > > >> > > days. > > >> > > > > > > > > >> > > > > > > A simple rule to detect flaky fails by hands - get test > > >> history > > >> > > > ordered > > >> > > > > > > by TEST_STATUS_DESC and check its date. As I see, we can > get > > >> list > > >> > > of > > >> > > > > > > failures from TC. We don't need to check successfull runs, > > >> > because > > >> > > > they > > >> > > > > > are > > >> > > > > > > uninformative for our needs. > > >> > > > > > > > > >> > > > > > > 2018-08-21 20:24 GMT+03:00 Dmitriy Pavlov < > > >> dpavlov....@gmail.com > > >> > >: > > >> > > > > > > > > >> > > > > > >> Hi Dmitriy, > > >> > > > > > >> > > >> > > > > > >> The Bot is able to detect a frequent change of test > status, > > >> but > > >> > > > > > currently > > >> > > > > > >> only 50 last runs count. Same is true for the failure > rate. > > >> > > > > > >> > > >> > > > > > >> This value can be easily changed to 70 or 100, moreover, > > the > > >> > auto > > >> > > > > > >> trigger feature gives us much more builds. > > >> > > > > > >> > > >> > > > > > >> We can improve these rules. We can add not only status > > >> change, > > >> > but > > >> > > > > > status > > >> > > > > > >> change without any code changes. We can somehow save this > > >> data > > >> > in > > >> > > > > > RunStat > > >> > > > > > >> class. Let's create a better rule, and later we can code > > it. > > >> > > > > > >> > > >> > > > > > >> Sincerely, > > >> > > > > > >> Dmitriy Pavlov > > >> > > > > > >> > > >> > > > > > >> вт, 21 авг. 2018 г. в 19:22, Dmitrii Ryabov < > > >> > > somefire...@gmail.com > > >> > > > >: > > >> > > > > > >> > > >> > > > > > >> > I think plugin will be more pretty looking, but > comments > > >> can > > >> > > > contain > > >> > > > > > any > > >> > > > > > >> > information, so they can be more usefull. I agree with > > your > > >> > idea > > >> > > > to > > >> > > > > > >> create > > >> > > > > > >> > bot instead of plugin. > > >> > > > > > >> > > > >> > > > > > >> > As for fail rate - I'm not sure it is working as you > > >> describe. > > >> > > > > > >> > I'm looking on my runAll [1]. There is > > >> > > > > > >> > `IgniteCacheGroupsTest.testCacheApiTxReplicated` > > >> > > > > > >> > in `Cache 3` suite with fail rate = 0.0%. But it is > flaky > > >> and > > >> > > > fails > > >> > > > > in > > >> > > > > > >> > master branch [2]. > > >> > > > > > >> > > > >> > > > > > >> > [1] > > >> > > > > > >> > > > >> > > > > > >> > https://mtcga.gridgain.com/pr. > > >> html?serverId=apache&suiteId=I > > >> > > > > > >> > > >> > > > gniteTests24Java8_RunAll&branchForTc=pull%2F4519%2Fhead& > > >> action=Latest > > >> > > > > > >> > [2] > > >> > > > > > >> > > > >> > > > > > >> > https://ci.ignite.apache.org/p > > >> roject.html?projectId=IgniteTe > > >> > > > > > >> sts24Java8&buildTypeId=&tab= > testDetails&testNameId=5628470 > > >> > > > > > >> 782089555961&order=TEST_STATUS_DESC&branch_IgniteTests > > >> > > > > > >> 24Java8=__all_branches__&itemsCount=50 > > >> > > > > > >> > > > >> > > > > > >> > 2018-08-21 18:00 GMT+03:00 Dmitriy Pavlov < > > >> > > dpavlov....@gmail.com > > >> > > > >: > > >> > > > > > >> > > > >> > > > > > >> > > Hi Dmitrii, > > >> > > > > > >> > > > > >> > > > > > >> > > I'm not sure we're able to install Github apps to > > Apache > > >> > > > mirrors. > > >> > > > > > >> > > > > >> > > > > > >> > > The simplest solution, what can be as efficient as a > > >> plugin, > > >> > > is > > >> > > > > fake > > >> > > > > > >> > MTCGA > > >> > > > > > >> > > bot account in Github, which will provide PR comments > > >> using > > >> > > > Github > > >> > > > > > >> > program > > >> > > > > > >> > > interface. What do you think? > > >> > > > > > >> > > > > >> > > > > > >> > > A new test failure can be identified by the Ignite TC > > >> Bot by > > >> > > > > master > > >> > > > > > >> > recent > > >> > > > > > >> > > fail rate = 0.0%. The same rule can be applied to > timed > > >> out > > >> > > > > suites. > > >> > > > > > >> > > > > >> > > > > > >> > > Sincerely, > > >> > > > > > >> > > Dmitriy Pavlov > > >> > > > > > >> > > > > >> > > > > > >> > > вт, 21 авг. 2018 г. в 16:16, Dmitrii Ryabov < > > >> > > > > somefire...@gmail.com > > >> > > > > > >: > > >> > > > > > >> > > > > >> > > > > > >> > > > Hello, Igniters! > > >> > > > > > >> > > > > > >> > > > > > >> > > > I want to suggest improvement for TeamCity Helper > [1] > > >> – we > > >> > > > need > > >> > > > > an > > >> > > > > > >> easy > > >> > > > > > >> > > way > > >> > > > > > >> > > > to get list of failed tests that don’t fall in the > > >> master > > >> > > > > branch. > > >> > > > > > >> These > > >> > > > > > >> > > > tests should: > > >> > > > > > >> > > > * fail in the PR > > >> > > > > > >> > > > * not fail in the master > > >> > > > > > >> > > > * not be flaky. > > >> > > > > > >> > > > > > >> > > > > > >> > > > Also, I want to suggest to create a GitHub plugin, > > >> which > > >> > > will > > >> > > > > > >> notify PR > > >> > > > > > >> > > if > > >> > > > > > >> > > > it has such tests. PR will have a marker, which > > >> > > > allows/prohibits > > >> > > > > > >> merge. > > >> > > > > > >> > > > This marker will be shown near PR conflicts. > > >> > > > > > >> > > > > > >> > > > > > >> > > > Allowing marker will be shown in case: > > >> > > > > > >> > > > * no new fails. > > >> > > > > > >> > > > > > >> > > > > > >> > > > Prohibiting marker will be shown in cases: > > >> > > > > > >> > > > * new fails – tests must be fixed. > > >> > > > > > >> > > > * new timed out test suite – suite should be > > restarted > > >> or > > >> > > > tests > > >> > > > > > >> must be > > >> > > > > > >> > > > fixed. > > >> > > > > > >> > > > * runAll wasn’t launched – tests must be launched. > > >> > > > > > >> > > > > > >> > > > > > >> > > > This will make test checks much faster and easier. > > >> Also, > > >> > > this > > >> > > > > will > > >> > > > > > >> > > decrease > > >> > > > > > >> > > > the number of merges with new failed tests made by > > >> > > inattention > > >> > > > > to > > >> > > > > > >> the > > >> > > > > > >> > > > tests. > > >> > > > > > >> > > > > > >> > > > > > >> > > > Further, we can expand the plugin by adding new > > checks, > > >> > > > showing > > >> > > > > PR > > >> > > > > > >> > > quality. > > >> > > > > > >> > > > > > >> > > > > > >> > > > [1] https://github.com/apache/ignite-teamcity-bot > > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > >
