Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-15 Thread Jan Beulich
>>> On 15.06.15 at 10:57, wrote: > So 3/20 = 15% failure rate (fiano0: 1/11=9%; fiano1: 2/9=22%). Which is > better than the ~50% seen at the start of this thread, so it is worth > applying the ucode update I think (and it would have been regardless the > right thing to do), > > I do think a 15-

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-15 Thread Ian Campbell
On Thu, 2015-06-11 at 09:45 +0100, Ian Campbell wrote: > On Thu, 2015-06-11 at 08:02 +0100, Jan Beulich wrote: > > >>> On 10.06.15 at 16:08, wrote: > > > On Wed, 2015-06-10 at 14:45 +0100, Jan Beulich wrote: > > >> So if we're going to approach Intel with this - will you or should I? > > > > > >

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-11 Thread Ian Campbell
On Thu, 2015-06-11 at 08:02 +0100, Jan Beulich wrote: > >>> On 10.06.15 at 16:08, wrote: > > On Wed, 2015-06-10 at 14:45 +0100, Jan Beulich wrote: > >> So if we're going to approach Intel with this - will you or should I? > > > > I think it'd be best coming from you. > > Just have sent it off; i

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-11 Thread Jan Beulich
>>> On 10.06.15 at 16:08, wrote: > On Wed, 2015-06-10 at 14:45 +0100, Jan Beulich wrote: >> So if we're going to approach Intel with this - will you or should I? > > I think it'd be best coming from you. Just have sent it off; in putting together the technical details it became clear that elblin

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Ian Campbell
On Wed, 2015-06-10 at 16:59 +0100, Jan Beulich wrote: > >>> On 10.06.15 at 16:34, wrote: > > For Intel I'm less sure, I've got microcode-20150121.tgz containing > > microcode.dat. Is that just to be placed at > > kernel/x86/microcode/GenuineIntel.bin and done, or is there some > > processing neede

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Don Slutz
On 06/10/15 11:59, Jan Beulich wrote: On 10.06.15 at 16:34, wrote: >> For Intel I'm less sure, I've got microcode-20150121.tgz containing >> microcode.dat. Is that just to be placed at >> kernel/x86/microcode/GenuineIntel.bin and done, or is there some >> processing needed? > > The full bl

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Jan Beulich
>>> On 10.06.15 at 16:34, wrote: > For Intel I'm less sure, I've got microcode-20150121.tgz containing > microcode.dat. Is that just to be placed at > kernel/x86/microcode/GenuineIntel.bin and done, or is there some > processing needed? The full blob (albeit usually named microcode.bin; microcode

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Ian Campbell
On Wed, 2015-06-10 at 13:56 +0100, Ian Campbell wrote: > > > Arranging to do microcode updates looks like it is going to be a bit > > > non-trivial from the osstest side. > > OK. I think this is something which is worth doing So for AMD I think things are pretty clear, cat linux-firmware.git/amd

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Ian Campbell
On Wed, 2015-06-10 at 14:45 +0100, Jan Beulich wrote: > >>> On 10.06.15 at 14:56, wrote: > > On Wed, 2015-06-10 at 12:48 +0100, Jan Beulich wrote: > >> Sure we can; I just generally prefer not to bother people with > >> problems they already solved, but maybe that's the wrong approach > >> a case

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Jan Beulich
>>> On 10.06.15 at 14:56, wrote: > On Wed, 2015-06-10 at 12:48 +0100, Jan Beulich wrote: >> Sure we can; I just generally prefer not to bother people with >> problems they already solved, but maybe that's the wrong approach >> a case like this. > > Is the list of errata fixed by a given ucode upd

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Jan Beulich
>>> On 10.06.15 at 14:56, wrote: > On Wed, 2015-06-10 at 12:48 +0100, Jan Beulich wrote: >> But I guess you could at least check >> what microcode the box has in use - if there's nothing newer available, >> then trying to get the microcode updating working isn't of immediate >> importance anymore

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Ian Campbell
On Wed, 2015-06-10 at 12:48 +0100, Jan Beulich wrote: > >>> On 10.06.15 at 13:01, wrote: > > On Wed, 2015-06-10 at 10:36 +0100, Jan Beulich wrote: > >> Indeed. Leaving us with the slight hope that there is a microcode > >> update available that's newer than what the BIOS of those boxes > >> loads.

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Jan Beulich
>>> On 10.06.15 at 13:01, wrote: > On Wed, 2015-06-10 at 10:36 +0100, Jan Beulich wrote: >> Indeed. Leaving us with the slight hope that there is a microcode >> update available that's newer than what the BIOS of those boxes >> loads. Could we perhaps afford un-blessing the two systems for >> the

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Ian Campbell
On Wed, 2015-06-10 at 10:36 +0100, Jan Beulich wrote: > Indeed. Leaving us with the slight hope that there is a microcode > update available that's newer than what the BIOS of those boxes > loads. Could we perhaps afford un-blessing the two systems for > the time being? And maybe get Intel involved

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Jan Beulich
>>> On 10.06.15 at 10:50, wrote: > On Tue, 2015-06-09 at 10:29 +0100, Jan Beulich wrote: >> >>> On 09.06.15 at 10:26, wrote: >> > On Mon, 2015-06-08 at 13:16 +0100, Ian Campbell wrote: >> > >> >> The adhoc run passed, but that's not statistically significant. >> > >> > I ran a bunch more in thi

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-10 Thread Ian Campbell
On Tue, 2015-06-09 at 10:29 +0100, Jan Beulich wrote: > >>> On 09.06.15 at 10:26, wrote: > > On Mon, 2015-06-08 at 13:16 +0100, Ian Campbell wrote: > > > >> The adhoc run passed, but that's not statistically significant. > > > > I ran a bunch more in this no-apicv configuration, the logs are at

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-09 Thread Jan Beulich
>>> On 09.06.15 at 10:26, wrote: > On Mon, 2015-06-08 at 13:16 +0100, Ian Campbell wrote: > >> The adhoc run passed, but that's not statistically significant. > > I ran a bunch more in this no-apicv configuration, the logs are at > http://logs.test-lab.xenproject.org/osstest/logs/: > > Flight

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-09 Thread Ian Campbell
On Mon, 2015-06-08 at 13:16 +0100, Ian Campbell wrote: > The adhoc run passed, but that's not statistically significant. I ran a bunch more in this no-apicv configuration, the logs are at http://logs.test-lab.xenproject.org/osstest/logs/: Flight HostFailed at 58190 fiano0 ts-guest-stop 5

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Ian Campbell
On Mon, 2015-06-08 at 11:21 -0400, Konrad Rzeszutek Wilk wrote: > On Mon, Jun 08, 2015 at 03:47:22PM +0100, Ian Jackson wrote: > > Konrad Rzeszutek Wilk writes ("Re: [Xen-devel] [xen-unstable test] 57852: > > regressions - FAIL"): > > > Could it be an missing mic

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Konrad Rzeszutek Wilk
On Mon, Jun 08, 2015 at 03:47:22PM +0100, Ian Jackson wrote: > Konrad Rzeszutek Wilk writes ("Re: [Xen-devel] [xen-unstable test] 57852: > regressions - FAIL"): > > Could it be an missing microcode update? I don't know if the OSSTest does > > the ucode=scan or up

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Ian Jackson
Konrad Rzeszutek Wilk writes ("Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL"): > Could it be an missing microcode update? I don't know if the OSSTest does > the ucode=scan or updates the microcode later? I think osstest's machines don't get micro

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Ian Jackson
Jan Beulich writes ("Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL"): > On 08.06.15 at 11:27, wrote: > > I don't know much about the hardware in the pool other than what can be > > gathered from the serial and dmesg logs. > > Right - this

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Ian Campbell
On Mon, 2015-06-08 at 09:50 -0400, Konrad Rzeszutek Wilk wrote: > On Mon, Jun 08, 2015 at 10:27:32AM +0100, Ian Campbell wrote: > > On Mon, 2015-06-08 at 10:15 +0100, Jan Beulich wrote: > > > >>> On 08.06.15 at 10:53, wrote: > > > > That's 6/14 (43%) failure rate on fiano0 and 2/10 (20%) on fiano1

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Konrad Rzeszutek Wilk
On Mon, Jun 08, 2015 at 10:27:32AM +0100, Ian Campbell wrote: > On Mon, 2015-06-08 at 10:15 +0100, Jan Beulich wrote: > > >>> On 08.06.15 at 10:53, wrote: > > > That's 6/14 (43%) failure rate on fiano0 and 2/10 (20%) on fiano1. Which > > > differs form the apparent xen-unstable failure rate. But I

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Jan Beulich
>>> On 08.06.15 at 14:16, wrote: > On Mon, 2015-06-08 at 10:27 +0100, Ian Campbell wrote: >> On Mon, 2015-06-08 at 10:15 +0100, Jan Beulich wrote: >> > >>> On 08.06.15 at 10:53, wrote: >> > > That's 6/14 (43%) failure rate on fiano0 and 2/10 (20%) on fiano1. Which >> > > differs form the apparent

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Andrew Cooper
On 08/06/15 13:16, Ian Campbell wrote: > On Mon, 2015-06-08 at 10:27 +0100, Ian Campbell wrote: >> On Mon, 2015-06-08 at 10:15 +0100, Jan Beulich wrote: >> On 08.06.15 at 10:53, wrote: That's 6/14 (43%) failure rate on fiano0 and 2/10 (20%) on fiano1. Which differs form the apparent

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Ian Campbell
On Mon, 2015-06-08 at 10:27 +0100, Ian Campbell wrote: > On Mon, 2015-06-08 at 10:15 +0100, Jan Beulich wrote: > > >>> On 08.06.15 at 10:53, wrote: > > > That's 6/14 (43%) failure rate on fiano0 and 2/10 (20%) on fiano1. Which > > > differs form the apparent xen-unstable failure rate. But I wouldn

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Jan Beulich
>>> On 08.06.15 at 11:27, wrote: > I don't know much about the hardware in the pool other than what can be > gathered from the serial and dmesg logs. Right - this is useful for learning details of an individual system, but isn't really helpful when wanting to compare all system kinds that are in

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Ian Campbell
On Mon, 2015-06-08 at 10:15 +0100, Jan Beulich wrote: > (I'm trying to make myself a picture of what debugging options we > have.) In the meantime I've kicked off an adhoc job using no-apicv as suggested by Andy (on IIRC last week IIRC). Assuming that my tweak takes effect in practice I'll run a b

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Ian Campbell
On Mon, 2015-06-08 at 10:15 +0100, Jan Beulich wrote: > >>> On 08.06.15 at 10:53, wrote: > > That's 6/14 (43%) failure rate on fiano0 and 2/10 (20%) on fiano1. Which > > differs form the apparent xen-unstable failure rate. But I wouldn't take > > this as evidence that the two systems differ signif

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Jan Beulich
>>> On 08.06.15 at 10:53, wrote: > That's 6/14 (43%) failure rate on fiano0 and 2/10 (20%) on fiano1. Which > differs form the apparent xen-unstable failure rate. But I wouldn't take > this as evidence that the two systems differ significantly, despite how > the unstable results looked at first gl

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Ian Campbell
On Mon, 2015-06-08 at 09:07 +0100, Jan Beulich wrote: > >>> On 05.06.15 at 12:48, wrote: > > On Fri, 2015-06-05 at 10:18 +0100, Jan Beulich wrote: > >> >>> On 05.06.15 at 11:07, wrote: > >> > WRT the move to the colo, flights in 5 are in the new one, while > >> > 3 are in the old one, > >

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-08 Thread Jan Beulich
>>> On 05.06.15 at 12:48, wrote: > On Fri, 2015-06-05 at 10:18 +0100, Jan Beulich wrote: >> >>> On 05.06.15 at 11:07, wrote: >> > WRT the move to the colo, flights in 5 are in the new one, while >> > 3 are in the old one, >> > http://logs.test-lab.xenproject.org/osstest/results/history.te

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-05 Thread Ian Campbell
On Fri, 2015-06-05 at 11:48 +0100, Ian Campbell wrote: > All the flights in the new colo seem to have been on fiano[01]. > > But having looked at the page again the early success was all on fiano0 > while the later failures were all on fiano1. > > fiano[01] are supposedly identical hardware. > >

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-05 Thread Ian Campbell
On Fri, 2015-06-05 at 10:18 +0100, Jan Beulich wrote: > >>> On 05.06.15 at 11:07, wrote: > > From: > > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64 > > > > -xl-qemuu-win7-amd64.xen-4.4-testing.html > > http://logs.test-lab.xenproject.org/osstest/results/history.te

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-05 Thread Jan Beulich
>>> On 05.06.15 at 11:07, wrote: > From: > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64 > -xl-qemuu-win7-amd64.xen-4.4-testing.html > http://logs.test-lab.xenproject.org/osstest/results/history.test-amd64-amd64 > -xl-qemuu-win7-amd64.xen-4.5-testing.html > it look

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-05 Thread Ian Campbell
On Fri, 2015-06-05 at 10:00 +0100, Jan Beulich wrote: > >>> On 05.06.15 at 10:45, wrote: > > On Thu, 2015-06-04 at 12:01 +, osstest service user wrote: > >> flight 57852 xen-unstable real [real] > >> http://logs.test-lab.xenproject.org/osstest/logs/57852/ > >> > >> Regressions :-( > >> > >>

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-05 Thread Jan Beulich
>>> On 05.06.15 at 10:45, wrote: > On Thu, 2015-06-04 at 12:01 +, osstest service user wrote: >> flight 57852 xen-unstable real [real] >> http://logs.test-lab.xenproject.org/osstest/logs/57852/ >> >> Regressions :-( >> >> Tests which did not succeed and are blocking, >> including tests whic

Re: [Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-05 Thread Ian Campbell
On Thu, 2015-06-04 at 12:01 +, osstest service user wrote: > flight 57852 xen-unstable real [real] > http://logs.test-lab.xenproject.org/osstest/logs/57852/ > > Regressions :-( > > Tests which did not succeed and are blocking, > including tests which could not be run: > test-amd64-amd64-xl-q

[Xen-devel] [xen-unstable test] 57852: regressions - FAIL

2015-06-04 Thread osstest service user
flight 57852 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/57852/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-xl-qemuu-win7-amd64 9 windows-install fail REGR. vs. 57419 Regressions which ar