Thanks everyone and thanks Dave for keeping us in the loop!! Cheers, Florin
> On Jul 20, 2021, at 3:08 PM, Dave Wallace <dwallac...@gmail.com> wrote: > > Folks, > > After troubleshooting latency issues in the datapath between the Jenkins > openstack instance and the Nomad cluster, the host of the Ingress instance > appeared to be the source of the problem and the Ingress instance was live > migrated to another host. In addition, the primary Nomad server was > rebooted. These changes resolved the 'Java Connection Closed Exception' > issues. > > However, at this time, the vpp-device job is still failing due to a known > issue which will be resolved when Peter Mikus comes online tomorrow morning > CET. Once the vpp-device job failures have been resolved, I will be issuing > 'recheck' on open VPP gerrit changes which are failing due to the vpp-device > job. Please feel free to 'recheck' your gerrit changes if you would like to > verify that the rest of the CI jobs complete successfully. > > I'd like to thank Mohammed Naser, Vanessa Valderrama, Anton Baranov, Peter > Mikus & Maciek Konstantynowicz for their coordinated efforts in resolving > this outage. > > Thanks again for your patience during this CI outage. > -daw- > > On 7/19/2021 10:51 PM, Dave Wallace via lists.fd.io wrote: >> Folks, >> >> Vanessa performed a Jenkins reset at my request to see if that would resolve >> this problem. Unfortunately the Jenkins reset did not resolve the >> connection resets. A recheck of gerrit change after the Jenkins restart >> failed with multiple job failures due to TCP connection resets: >> >> https://gerrit.fd.io/r/c/vpp/+/32858/6#message-c77806c2fd58c3c00935e1b5589a402e4b670f9f >> >> <https://gerrit.fd.io/r/c/vpp/+/32858/6#message-c77806c2fd58c3c00935e1b5589a402e4b670f9f> >> >> There has also been no correlation with Ping Monitor events, Nomad cluster >> events, Nomad host, subnet, or docker image. >> >> Investigation continues in the datapath between the Jenkins openstack >> instance and the Nomad cluster. >> >> Thanks again for your patience. >> -daw- >> >> On 7/19/2021 11:29 AM, Dave Wallace via lists.fd.io wrote: >>> Folks, >>> >>> There have been large numbers CI job failures due to 'Java Connection >>> Closed Exception' that appear to have started occurring on July 17. >>> >>> I have opened a ticket with Vexxhost and am actively diagnosing the problem >>> with them. >>> >>> Thank you for your patience while the issue is being resolved. >>> -daw- >>> >>> >> >> >> >> >> > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#19841): https://lists.fd.io/g/vpp-dev/message/19841 Mute This Topic: https://lists.fd.io/mt/84344520/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
