Hi Dave, Please don't add broken tests to extended tests.
See https://gerrit.fd.io/r/c/vpp/+/24567/12/test/test_bfd.py Thanks! On Fri, Mar 13, 2020 at 1:33 PM Dave Wallace <dwallac...@gmail.com> wrote: > Matt, > > I will keep an eye on this gerrit and merge it once the verify jobs have > completed. > If there are other tests which fail, are you ok if I add them to this > patch and turn it into a generic 'disable failing tests' gerrit change? > > The other possibility is that this is due to the recent disabling of the > Naginator retry plugin. > > I'm going to investigate if this issue may have been masked by Naginator... > > Thanks for your help on keeping the CI operational! > -daw- > > On 3/12/2020 12:09 PM, Matthew Smith via Lists.Fd.Io wrote: > > > Change submitted - https://gerrit.fd.io/r/c/vpp/+/25834. Verification > jobs are running. Hopefully they won't fail :) > > -Matt > > > On Thu, Mar 12, 2020 at 10:22 AM Matthew Smith via Lists.Fd.Io <mgsmith= > netgate....@lists.fd.io> wrote: > >> >> I don't have a solution yet, but one observation has popped up quickly.... >> >> In the 2 failed jobs Ray sent links for, one of them had a test fail >> which was not related to VRRP. There is a BFD6 test failure for the NAT >> change https://gerrit.fd.io/r/c/vpp/+/25462: >> >> >> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2678/archives/ >> >> Looking back through a couple of recent failed runs of that job, there is >> also a DHCP6 PD test failure for rdma change >> https://gerrit.fd.io/r/c/vpp/+/25823: >> >> >> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2682/archives/ >> >> The most obvious common thread between BFD6, DHCP6 and VRRP to me seems >> to be that they all maintain state which is dependent on timers. There >> could be a more general issue with timing-sensitive tests. I am going to >> submit a change which will prevent the VRRP tests from running temporarily >> while I can figure out a proper solution. Based on the above, other tests >> may need the same treatment. >> >> -Matt >> >> >> >> >> On Thu, Mar 12, 2020 at 8:57 AM Matthew Smith <mgsm...@netgate.com> >> wrote: >> >>> Hi Ray, >>> >>> Thanks for bringing it to my attention. I'll look into it. >>> >>> -Matt >>> >>> >>> On Thu, Mar 12, 2020 at 8:24 AM Ray Kinsella <m...@ashroe.eu> wrote: >>> >>>> Anyone else noticing seeming spurious failures related to the VRRP >>>> plugin's unit tests. >>>> Some examples from un-related commits. >>>> >>>> Ray K >>>> >>>> nat: timed out session scavenging upgrade ( >>>> https://gerrit.fd.io/r/c/vpp/+/25462) >>>> >>>> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2678/console.log.gz >>>> >>>> >>>> ============================================================================== >>>> TEST RESULTS: >>>> Scheduled tests: 1138 >>>> Executed tests: 1138 >>>> Passed tests: 1021 >>>> Skipped tests: 112 >>>> Failures: 3 >>>> Errors: 2 >>>> FAILURES AND ERRORS IN TESTS: >>>> Testcase name: IPv4 VRRP Test Case >>>> FAILURE: IPv4 Master VR does not reply for VIP w/ accept mode off >>>> [test_vrrp.TestVRRP4.test_vrrp4_accept_mode_disabled] >>>> FAILURE: IPv4 Master VR preempted by higher priority backup >>>> [test_vrrp.TestVRRP4.test_vrrp4_master_preempted] >>>> Testcase name: IPv6 VRRP Test Case >>>> FAILURE: IPv6 Master VR preempted by higher priority backup >>>> [test_vrrp.TestVRRP6.test_vrrp6_master_preempted] >>>> ERROR: IPv6 Backup VR preempts lower priority master >>>> [test_vrrp.TestVRRP6.test_vrrp6_backup_preempts] >>>> Testcase name: Bidirectional Forwarding Detection (BFD) (IPv6) >>>> ERROR: echo function [test_bfd.BFD6TestCase.test_echo] >>>> >>>> ============================================================================== >>>> >>>> vlib: startup multi-arch variant configuration ( >>>> https://gerrit.fd.io/r/c/vpp/+/25798_ >>>> >>>> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2675/console.log.gz >>>> >>>> >>>> ============================================================================== >>>> TEST RESULTS: >>>> Scheduled tests: 22 >>>> Executed tests: 22 >>>> Passed tests: 21 >>>> Failures: 1 >>>> FAILURES AND ERRORS IN TESTS: >>>> Testcase name: IPv4 VRRP Test Case >>>> FAILURE: IPv4 Master VR preempted by higher priority backup >>>> [test_vrrp.TestVRRP4.test_vrrp4_master_preempted] >>>> >>>> ============================================================================== >>>> >>>> >>>> >>>> >> > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15782): https://lists.fd.io/g/vpp-dev/message/15782 Mute This Topic: https://lists.fd.io/mt/71901798/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
