On 25/02/2019 16:52, Ian Jackson wrote: > We have about a 10% failure rate of a problem where the symptoms are > that the test box fails to get some thing from security.debian.org. > > The apt-cacher-ng logs show the relevant test box's ip address > fetching the file that it is supposed to. But, it is possible that > there are different timeouts, so that does not mean the problem is > inside the colo. > > Fetching from other apt sources, notably the main Debian archive and > snapshot.d.o, do not seem to be affected. > > Specifically, I searched the logs for the last 1000 host install steps, > and looked for the failures, with the following rune: > > select flight,job,logfile,started from (select *, (select val from runvars r > where r.job=steps.job and r.flight=steps.flight and r.name='host') from > steps where testid like 'host-install%' and flight>130000 order by finished > desc limit 1000) sub where status='broken'; > > I then used these runes to correlate that with the syslogs from the > installer: > > perl <~/t -ne 'use strict; s/^ *//; my ($flight,$job,$logf) = split / +\| > +/; next unless $flight =~ m/^\d+$/; my $f = > "$flight/$job/?.ts-syslog-server.log"; my @y = glob $f; print $_, "\n" for > @y;' >~/u > > xargs <~/junk/u egrep -L 'Failed to fetch > http://security\.debian\.org.*Connection failed' > > The only logs which did not mention that error message were three > failed jobs on the same host, joubertin1, which seems not to be > rebooting reliably. > > So I think this is a problem with the security.debian.org CDN. > > For now, disable security updates entirely. We don't really care > about the security patch status of test boxes anyway. Hopefully this > will cause the system to become reliable again. > > CC: Juergen Gross <jgr...@suse.com> > Signed-off-by: Ian Jackson <ian.jack...@eu.citrix.com>
Release-acked-by: Juergen Gross <jgr...@suse.com> Juergen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
