Re: [DISCUSS] Make `Pulsar CI / CI - Unit - Brokers - Flaky` test group to be required

2022-08-24 Thread Qiang Huang
Failure flaky tests reduce the passion of contributors. I suggest making stable tests as required. I have 3 points. 1. Find the existing failure flaky tests and fix them. I believe that there is already a project to record them. 2. Move the failure flaky tests out of the flaky test group. 3. Add th

Re: [DISCUSS] Make `Pulsar CI / CI - Unit - Brokers - Flaky` test group to be required

2022-08-22 Thread PengHui Li
I agree with moving the tests out of the flaky test group. I just checking some new PRs https://github.com/apache/pulsar/pull/17195 https://github.com/apache/pulsar/pull/17201 https://github.com/apache/pulsar/pull/17193 https://github.com/apache/pulsar/pull/17204 The `Pulsar CI / CI - Unit - Brok

Re: [DISCUSS] Make `Pulsar CI / CI - Unit - Brokers - Flaky` test group to be required

2022-08-22 Thread tison
While agree to require more tests to pass, here are my two coins: 1. This group is named "flaky tests" so I regard it as flaky tests literally. NOT require these tests to pass could be by design. Besides, IIRC some developers keep investigating tests in the flaky test group, try to make them stabl

[DISCUSS] Make `Pulsar CI / CI - Unit - Brokers - Flaky` test group to be required

2022-08-22 Thread mattison chao
Hi all Recently, some tests in the `broker-flaky` test group always failed, but since it doesn't block CI, no one cared for a long time. This behaviour causes some test scenarios to go unchecked and risk introducing some regressions, and I think we need to make this test group set required. e.g.