+1. For disabling, one could create a new TestNG group which just doesn't get run at all (or get runs without a failure blocking PR merge). Perhaps we could call this group "quarantine" or "quarantined"?
-Lari On Sat, 3 Apr 2021, 12:06 Enrico Olivelli, <eolive...@gmail.com> wrote: > I propose to disable those tests and unblock CI and create blocker issues > for 2.8.0 release. > Issue are probably already present and I see a few PRs about tentative > fixes. > > Does this sound good? > > Enrico > > Il Ven 2 Apr 2021, 20:32 Enrico Olivelli <eolive...@gmail.com> ha scritto: > > > Jonathan > > I am not aware of commits that may have introduced instability > > > > Il Ven 2 Apr 2021, 20:17 Ali Ahmed <ahmal...@gmail.com> ha scritto: > > > >> If it's a few tests you can isolate it by running if after the group by > >> excluding it from it. > >> > >> or simply disable them for now. > >> > > > > Ali > > I am +1 to this idea. > > > > I cannot send a PR now > > > > > > Other ideas from the community? > > > > Enrico > > > > > >> On Fri, Apr 2, 2021 at 10:56 AM Enrico Olivelli <eolive...@gmail.com> > >> wrote: > >> > >> > Il Ven 2 Apr 2021, 18:47 Ali Ahmed <ahmal...@gmail.com> ha scritto: > >> > > >> > > You can rebuild the commit and force push this results in the job > >> being > >> > > queued on different machines. It ends helping the test pass. > >> > > > >> > > >> > I see these error on many PRs, and today I have restarted at least 20 > >> jobs. > >> > Today (my timezone) we merged only 1pr, due to this problem > >> > > >> > > >> > Enrico > >> > > >> > > >> > > -Ali > >> > > > >> > > On Fri, Apr 2, 2021 at 9:09 AM Jonathan Ellis <jbel...@gmail.com> > >> wrote: > >> > > > >> > > > Is there a legit regression causing elevated failures? > >> > > > > >> > > > On Fri, Apr 2, 2021 at 10:30 AM Enrico Olivelli < > >> eolive...@gmail.com> > >> > > > wrote: > >> > > > > >> > > > > Hello, > >> > > > > today I see these tests (below) that are failing most of the PR > >> > > > > validation jobs that I am tracking. > >> > > > > it it literally the whole day that I am rerunning CI jobs, > without > >> > > > success > >> > > > > > >> > > > > What can we do ? Perhaps I am too pessimistic but it looks like > >> the > >> > > > > whole PR validation pipeline is stuck. > >> > > > > > >> > > > > Enrico > >> > > > > > >> > > > > Error: > >> > > > > > >> > > > > >> > > > >> > > >> > testOrderingWithConsumerListener(org.apache.pulsar.client.api.KeySharedSubscriptionTest) > >> > > > > Time elapsed: 1.912 s <<< FAILURE! > >> > > > > 925 > >> > > > > java.lang.AssertionError: expected [true] but found [false] > >> > > > > > >> > > > > Error: > >> > > > > > >> > > > > >> > > > >> > > >> > testBrokerConnectionStats(org.apache.pulsar.broker.service.PersistentTopicE2ETest) > >> > > > > Time elapsed: 0.275 s <<< FAILURE! > >> > > > > java.lang.AssertionError: expected [0] but found [1] > >> > > > > > >> > > > > Error: > >> > > > > > >> > > > > >> > > > >> > > >> > testRedeliveryFailOverConsumer(org.apache.pulsar.broker.service.persistent.SimpleProducerConsumerTestStreamingDispatcherTest) > >> > > > > Time elapsed: 2.213 s <<< FAILURE! > >> > > > > java.lang.AssertionError: expected [12] but found [11] > >> > > > > > >> > > > > Error: > >> > > > > > >> > > > > >> > > > >> > > >> > testDeadLetterTopicByCustomTopicName(org.apache.pulsar.client.api.DeadLetterTopicTest) > >> > > > > Time elapsed: 6.152 s <<< FAILURE! > >> > > > > java.lang.AssertionError: expected [null] but found > >> > > > > [org.apache.pulsar.client.impl.MessageImpl@7feabb64] > >> > > > > > >> > > > > >> > > > > >> > > > -- > >> > > > Jonathan Ellis > >> > > > co-founder, http://www.datastax.com > >> > > > @spyced > >> > > > > >> > > > >> > > > >> > > -- > >> > > -Ali > >> > > > >> > > >> > >> > >> -- > >> -Ali > >> > > >
