Re: Proposal: keeping post-commit tests green

Hi everyone, I've summarized things discussed in this design doc into a Beam site page: https://beam.apache.org/contribute/postcommits-policies/ Regards, --Mikhail Have feedback ? On Thu, Jun 14, 2018 at 9:13 AM Mikhail Gryzykhin wrote: > It is one-time action. Aft

Re: Proposal: keeping post-commit tests green

It is one-time action. Afterwards, we treat flakes as failing tests that require investigation and fixes. --Mikhail Have feedback ? On Wed, Jun 13, 2018 at 5:06 PM Ahmet Altay wrote: > > > On Wed, Jun 13, 2018 at 3:52 PM, Mikhail Gryzykhin > wrote: > >> Hi Ahmet, >

Re: Proposal: keeping post-commit tests green

On Wed, Jun 13, 2018 at 3:52 PM, Mikhail Gryzykhin wrote: > Hi Ahmet, > > I've checked on tests status and most of other tests are green 98% of the > time. So I feel that we do not need any explicit actions for those tests. > Is it going to be a one time action to fix existing flaky tests? Or is

Re: Proposal: keeping post-commit tests green

Hi Ahmet, I've checked on tests status and most of other tests are green 98% of the time. So I feel that we do not need any explicit actions for those tests. However java tests seem to have most of the problems. So I moved it to requirements explicitly. I do not bring in fixing failing tests as

Re: Proposal: keeping post-commit tests green

On Wed, Jun 13, 2018 at 3:45 PM, Mikhail Gryzykhin wrote: > Hello everybody, > > Thanks everyone. I didn't receive any more feedback on the design proposal > document [1] and I believe we've reached consensus. I've added > implementation tasks in JIRA (BEAM-4559 [2]) and will start coding soon.

Re: Proposal: keeping post-commit tests green

Hello everybody, Thanks everyone. I didn't receive any more feedback on the design proposal document [1] and I believe we've reached consensus. I've added implementation tasks in JIRA (BEAM-4559 [2]) and will start coding soon. As a recap, the high-level plan is: - Split existing post-commit

Re: Proposal: keeping post-commit tests green

Hello everyone, Most of the comments on my last draft addressed technical details of automation implementation of specific processes proposed. No major process changes were suggested. If you have not yet, please review this document. Highlights from last change: * Bumped splitting tests jobs aft

Re: Proposal: keeping post-commit tests green

Thanks for taking this initiative. As the number of contributors grows, so does the cost of broken builds. I'm also in favor of locking master merges until related issues are fixed (short term pain for long term gain). It would penalize a few for the benefit of many. On that note, recently we also

Re: Proposal: keeping post-commit tests green

I've taken another pass over the doc, and it looks good to me. Thanks for driving this effort! On Mon, Jun 4, 2018 at 9:08 AM Mikhail Gryzykhin wrote: > Hello everyone, > > I have addressed comments on the proposal doc and updated it accordingly. > I have also added section on metrics that we wa

Re: Proposal: keeping post-commit tests green

Hello everyone, I have addressed comments on the proposal doc and updated it accordingly. I have also added section on metrics that we want to track for pre-commit tests and contents for dashboard. Please, take a second look at the document. Highlights: * Sections that I feel require more discus

Re: Proposal: keeping post-commit tests green

Thanks for the thoughtful proposal Mikhail. I've left some comments in the doc. I encourage others to take a look: the proposal adds some strong policies about dealing with post-commit failures (rollback policy, locking master). Currently our post-commits are frequently red, and we're missing out

Re: Proposal: keeping post-commit tests green

Hi Everyone, I've updated design doc according to comments. https://docs.google.com/document/d/1sczGwnCvdHiboVajGVdnZL0rfnr7ViXXAebBAf_uQME In general, ideas proposed seem to be appreciated. Still, some of sections require more discussion. Changes highlight: * Added roll-back first policy to bes

Re: Proposal: keeping post-commit tests green

Blocking commits to master on test flaps seems critical here. The test flaps won't get the attention they deserve as long as people are just spamming their PRs with 'Run Java Precommit' until they turn green. I'm guilty of this behavior and I know it masks new flaky tests. I added a comment to you

Re: Proposal: keeping post-commit tests green

Love it. I would pull out from the doc also the key point: make the postcommit status constantly visible to everyone. Kenn On Fri, May 18, 2018 at 10:17 AM Mikhail Gryzykhin wrote: > Hi everyone, > > I'm Mikhail and started working on Google Dataflow several months ago. I'm > really excited to

Proposal: keeping post-commit tests green

Hi everyone, I'm Mikhail and started working on Google Dataflow several months ago. I'm really excited to work with Beam opensource community. I have a proposal to improve contributor experience by keeping post-commit tests green. I'm looking to get community consensus and approval about the pro