Dear community, Here's a report of the flaky tests in Pulsar CI during the observation period of 2022-11-26 - 2022-12-02
https://docs.google.com/spreadsheets/d/165FHpHjs5fHccSsmQM4beeg6brn-zfUjcrXf6xAu4yQ/edit#gid=3884523 Top 4 flaky tests: org.apache.pulsar.metadata.ZKSessionTest.testReacquireLeadershipAfterSessionLost 15 Detailed report <https://github.com/nicoloboschi/pulsar-flakes/tree/master/2022-10-07-to-2022-10-14/org.apache.pulsar.metadata.ZKSessionTest.testReacquireLeadershipAfterSessionLost.md> Search issues <https://github.com/apache/pulsar/issues?q=ZKSessionTest.testReacquireLeadershipAfterSessionLost+testReacquireLeadershipAfterSessionLost> org.apache.pulsar.client.impl.KeySharedSubscriptionTest.testCanRecoverConsumptionWhenLiftMaxUnAckedMessagesRestriction 8 Detailed report <https://github.com/nicoloboschi/pulsar-flakes/tree/master/2022-10-07-to-2022-10-14/org.apache.pulsar.client.impl.KeySharedSubscriptionTest.testCanRecoverConsumptionWhenLiftMaxUnAckedMessagesRestriction.md> Search issues <https://github.com/apache/pulsar/issues?q=KeySharedSubscriptionTest.testCanRecoverConsumptionWhenLiftMaxUnAckedMessagesRestriction+testCanRecoverConsumptionWhenLiftMaxUnAckedMessagesRestriction> org.apache.pulsar.metadata.ZKSessionTest.testReacquireLocksAfterSessionLost 7 Detailed report <https://github.com/nicoloboschi/pulsar-flakes/tree/master/2022-10-07-to-2022-10-14/org.apache.pulsar.metadata.ZKSessionTest.testReacquireLocksAfterSessionLost.md> Search issues <https://github.com/apache/pulsar/issues?q=ZKSessionTest.testReacquireLocksAfterSessionLost+testReacquireLocksAfterSessionLost> org.apache.pulsar.tests.integration.io.sources.debezium.PulsarDebeziumOracleSourceTest.testDebeziumOracleDbSource 6 Detailed report <https://github.com/nicoloboschi/pulsar-flakes/tree/master/2022-10-07-to-2022-10-14/org.apache.pulsar.tests.integration.io.sources.debezium.PulsarDebeziumOracleSourceTest.testDebeziumOracleDbSource.md> Search issues <https://github.com/apache/pulsar/issues?q=PulsarDebeziumOracleSourceTest.testDebeziumOracleDbSource+testDebeziumOracleDbSource> At the moment the CI seems pretty stable compared to just a couple of months ago. I'd like to thank every contributor that worked hard to fix and track the CI flakyness! Markdown formatted summary reports for each test class can be accessed at https://github.com/nicoloboschi/pulsar-flakes/tree/master/2022-11-26-to-2022-12-02 <https://github.com/nicoloboschi/pulsar-flakes/tree/master/2022-10-07-to-2022-10-14> More flaky test issues: https://github.com/apache/pulsar/issues?q=flaky+sort%3Aupdated-desc+is%3Aopen+is:issue To coordinate the work, 1) please search for an existing issues or search for all flaky issues with "flaky" or the test class name (without package) in the search: https://github.com/apache/pulsar/issues?q=is%3Aopen+flaky+sort%3Aupdated-desc 2) If there isn't an issue for a particular flaky test failure that you'd like to fix, please create an issue using the "Flaky test" template at https://github.com/apache/pulsar/issues/new/choose 3) Please comment on the issue that you are working on. I'm looking forward to more contributors joining the efforts. Please join the #testing channel on Slack if you'd like to ask questions and tips about reproducing flaky tests locally and how to fix them. Sharing stories about fixing flaky tests is also helpful for sharing the knowledge about how flaky tests can be fixed. That's also a valuable way to contribute. Some flaky tests might be actual real production code bugs. Fixing the flaky test might result in fixing a real production code bug. More contributions are welcome! Please keep up the good work! Thanks, Nicolò Boschi
