[PATCH] x86emul: replace UB shifts

2020-07-31 Thread Jan Beulich
Displacement values can be negative, hence we shouldn't left-shift them. While auditing shifts, I noticed a pair of missing parentheses, which also get added right here. Reported-by: Andrew Cooper Signed-off-by: Jan Beulich --- a/xen/arch/x86/x86_emulate/x86_emulate.c +++ b/xen/arch/x86/x86_em

Re: [PATCH 3/5] xen/memory: Fix compat XENMEM_acquire_resource for size requests

2020-07-31 Thread Jan Beulich
On 30.07.2020 21:12, Andrew Cooper wrote: > On 29/07/2020 21:09, Jan Beulich wrote: >> On 28.07.2020 13:37, Andrew Cooper wrote: >>> Copy the nr_frames from the correct structure, so the caller doesn't >>> unconditionally receive 0. >> >> Well, no - it does get copied from the correct structure. It

Re: [PATCH 1/4] x86: replace __ASM_{CL,ST}AC

2020-07-31 Thread Jan Beulich
On 28.07.2020 15:55, Andrew Cooper wrote: > On 15/07/2020 11:48, Jan Beulich wrote: >> --- a/xen/arch/x86/arch.mk >> +++ b/xen/arch/x86/arch.mk >> @@ -20,6 +20,7 @@ $(call as-option-add,CFLAGS,CC,"rdrand % >> $(call as-option-add,CFLAGS,CC,"rdfsbase %rax",-DHAVE_AS_FSGSBASE) >> $(call as-option-a

Re: [PATCH 1/4] x86: replace __ASM_{CL,ST}AC

2020-07-31 Thread Jan Beulich
On 28.07.2020 11:06, Roger Pau Monné wrote: > On Mon, Jul 27, 2020 at 09:47:52PM +0200, Jan Beulich wrote: >> On 27.07.2020 16:55, Roger Pau Monné wrote: >>> On Wed, Jul 15, 2020 at 12:48:14PM +0200, Jan Beulich wrote: --- /dev/null +++ b/xen/include/asm-x86/asm-defns.h >>> >>> Maybe this

[qemu-mainline test] 152309: regressions - FAIL

2020-07-31 Thread osstest service owner
flight 152309 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/152309/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-qemuu-nested-intel 14 xen-boot/l1 fail REGR. vs. 151065 test-amd64-i386-x

Re: [PATCH 1/4] x86: replace __ASM_{CL,ST}AC

2020-07-31 Thread Roger Pau Monné
On Fri, Jul 31, 2020 at 10:05:07AM +0200, Jan Beulich wrote: > On 28.07.2020 11:06, Roger Pau Monné wrote: > > On Mon, Jul 27, 2020 at 09:47:52PM +0200, Jan Beulich wrote: > >> On 27.07.2020 16:55, Roger Pau Monné wrote: > >>> On Wed, Jul 15, 2020 at 12:48:14PM +0200, Jan Beulich wrote: > ---

Re: 4.14.0 FTBFS for Debian unstable, libxlu_pci.c (�s°□°)�s�� 擤ォ

2020-07-31 Thread Michael Young
On Fri, 31 Jul 2020, Hans van Kranenburg wrote: Hi! News from the Debian Xen team (well, that's still only Ian and me). We still have Xen 4.11 in Debian unstable and stable (Buster) now, but at this point I really want to start working on the preparations for the next Debian release which will

[PATCH RFCv1 5/5] hv_balloon:: try to merge "System RAM" resources

2020-07-31 Thread David Hildenbrand
Let's reuse the new mechanism to merge "System RAM" resources below the root. We are the only one hotplugging "System RAM" and DIMMs don't apply, so this is safe to use. Cc: Andrew Morton Cc: Michal Hocko Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Wei Liu Signed-off-b

[PATCH RFCv1 0/5] mm/memory_hotplug: selective merging of memory resources

2020-07-31 Thread David Hildenbrand
Some add_memory*() users add memory in small, contiguous memory blocks. Examples include virtio-mem, hyper-v balloon, and the XEN balloon. This can quickly result in a lot of memory resources, whereby the actual resource boundaries are not of interest (e.g., it might be relevant for DIMMs, exposed

[PATCH RFCv1 4/5] xen/balloon: try to merge "System RAM" resources

2020-07-31 Thread David Hildenbrand
Let's reuse the new mechanism to merge "System RAM" resources below the root. We are the only one hotplugging "System RAM" and DIMMs don't apply, so this is safe to use. Cc: Andrew Morton Cc: Michal Hocko Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Roger Pau Monné Cc: Ju

[PATCH RFCv1 2/5] kernel/resource: merge_child_mem_resources() to merge memory resources after adding succeeded

2020-07-31 Thread David Hildenbrand
Some add_memory*() users add memory in small, contiguous memory blocks. Examples include virtio-mem, hyper-v balloon, and the XEN balloon. This can quickly result in a lot of memory resources, whereby the actual resource boundaries are not of interest (e.g., it might be relevant for DIMMs, exposed

[PATCH RFCv1 1/5] kernel/resource: make release_mem_region_adjustable() never fail

2020-07-31 Thread David Hildenbrand
Let's make sure splitting a resource on memory hotunplug will never fail. This will become more relevant once we merge selected system ram resources - then, we'll trigger that case more often un memory unplug. In general, this function is already unlikely to fail. When we remove memory, we free up

[PATCH RFCv1 3/5] virtio-mem: try to merge "System RAM (virtio_mem)" resources

2020-07-31 Thread David Hildenbrand
virtio-mem adds memory in memory block granularity, to be able to remove it in the same granularity again later, and to grow slowly on demand. This, however, results in quite a lot of resources when adding a lot of memory. Resources are effectively stored in a list-based tree. Having a lot of resou

Re: [PATCH] x86/vhpet: Fix type size in timer_int_route_valid

2020-07-31 Thread Jan Beulich
On 31.07.2020 10:38, Eslam Elnikety wrote: > On 28.07.20 19:51, Jan Beulich wrote: >> On 28.07.2020 11:26, Andrew Cooper wrote: >>> --- a/xen/include/asm-x86/hvm/vpt.h >>> +++ b/xen/include/asm-x86/hvm/vpt.h >>> @@ -73,7 +73,13 @@ struct hpet_registers { >>>   uint64_t isr;   /* int

Re: [PATCH v2] xen/arm: Convert runstate address during hypcall

2020-07-31 Thread Julien Grall
Hi Jan, On 31/07/2020 07:39, Jan Beulich wrote: We're fixing other issues without breaking the ABI. Where's the problem of backporting the kernel side change (which I anticipate to not be overly involved)? This means you can't take advantage of the runstage on existing Linux without any modific

Re: [PATCH v2] xen/arm: Convert runstate address during hypcall

2020-07-31 Thread Jan Beulich
On 31.07.2020 12:12, Julien Grall wrote: > On 31/07/2020 07:39, Jan Beulich wrote: >> We're fixing other issues without breaking the ABI. Where's the >> problem of backporting the kernel side change (which I anticipate >> to not be overly involved)? > This means you can't take advantage of the runs

Re: [OSSTEST PATCH 14/14] duration_estimator: Move duration query loop into database

2020-07-31 Thread Ian Jackson
George Dunlap writes ("Re: [OSSTEST PATCH 14/14] duration_estimator: Move duration query loop into database"): > > On Jul 21, 2020, at 7:42 PM, Ian Jackson wrote: ... > > Example queries before (from the debugging output): > > > > Query A part I: > > > >SELECT f.flight AS flight, >

Re: [PATCH RFCv1 3/5] virtio-mem: try to merge "System RAM (virtio_mem)" resources

2020-07-31 Thread David Hildenbrand
On 31.07.20 11:18, David Hildenbrand wrote: Grml, forgot to add cc: list for this patch, ccing the right people. > virtio-mem adds memory in memory block granularity, to be able to > remove it in the same granularity again later, and to grow slowly on > demand. This, however, results in quite a l

Re: [OSSTEST PATCH 06/14] sg-report-flight: Use WITH clause to use index for $anypassq

2020-07-31 Thread Ian Jackson
George Dunlap writes ("Re: [OSSTEST PATCH 06/14] sg-report-flight: Use WITH clause to use index for $anypassq"): > > On Jul 21, 2020, at 7:41 PM, Ian Jackson wrote: > > +# In psql 9.6 this WITH clause makes postgresql do the steps query > > +# first. This is good because if this test nev

Re: [OSSTEST PATCH 14/14] duration_estimator: Move duration query loop into database

2020-07-31 Thread George Dunlap
> On Jul 31, 2020, at 11:39 AM, Ian Jackson wrote: > > George Dunlap writes ("Re: [OSSTEST PATCH 14/14] duration_estimator: Move > duration query loop into database"): >>> On Jul 21, 2020, at 7:42 PM, Ian Jackson wrote: > ... >>> Example queries before (from the debugging output): >>> >>> Qu

[PATCH] x86/hvm: set 'ipat' in EPT for special pages

2020-07-31 Thread Paul Durrant
From: Paul Durrant All non-MMIO ranges (i.e those not mapping real device MMIO regions) that map valid MFNs are normally marked MTRR_TYPE_WRBACK and 'ipat' is set. Hence when PV drivers running in a guest populate the BAR space of the Xen Platform PCI Device with pages such as the Shared Info pag

Re: [PATCH] x86emul: replace UB shifts

2020-07-31 Thread Andrew Cooper
On 31/07/2020 08:06, Jan Beulich wrote: > Displacement values can be negative, hence we shouldn't left-shift them. > > While auditing shifts, I noticed a pair of missing parentheses, which > also get added right here. > > Reported-by: Andrew Cooper > Signed-off-by: Jan Beulich I'd suggest puttin

Re: [PATCH 2/2] libgnttab: Add support for Linux dma-buf offset

2020-07-31 Thread Oleksandr Andrushchenko
Hello, Ian, Wei! Initially I have sent this patch as a part of the series for the display protocol changes, but later decided to split. At the moment the protocol changes are already accepted, but this part is still missing feedback and is still wanted. I would really appreciate if you could h

Re: [PATCH] x86/hvm: set 'ipat' in EPT for special pages

2020-07-31 Thread Jan Beulich
On 31.07.2020 12:46, Paul Durrant wrote: > From: Paul Durrant > > All non-MMIO ranges (i.e those not mapping real device MMIO regions) that > map valid MFNs are normally marked MTRR_TYPE_WRBACK and 'ipat' is set. Hence > when PV drivers running in a guest populate the BAR space of the Xen Platfor

Re: [PATCH] x86/hvm: set 'ipat' in EPT for special pages

2020-07-31 Thread Andrew Cooper
On 31/07/2020 11:46, Paul Durrant wrote: > From: Paul Durrant > > All non-MMIO ranges (i.e those not mapping real device MMIO regions) that > map valid MFNs are normally marked MTRR_TYPE_WRBACK and 'ipat' is set. Hence > when PV drivers running in a guest populate the BAR space of the Xen Platform

[libvirt test] 152317: regressions - FAIL

2020-07-31 Thread osstest service owner
flight 152317 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/152317/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-i386-libvirt6 libvirt-buildfail REGR. vs. 151777 build-arm64-libvirt

RE: [PATCH] x86/hvm: set 'ipat' in EPT for special pages

2020-07-31 Thread Paul Durrant
> -Original Message- > From: Jan Beulich > Sent: 31 July 2020 12:15 > To: Paul Durrant > Cc: xen-devel@lists.xenproject.org; Paul Durrant ; > Andrew Cooper > ; Wei Liu ; Roger Pau Monné > > Subject: Re: [PATCH] x86/hvm: set 'ipat' in EPT for special pages > > On 31.07.2020 12:46, Paul

RE: [PATCH] x86/hvm: set 'ipat' in EPT for special pages

2020-07-31 Thread Paul Durrant
> -Original Message- > From: Andrew Cooper > Sent: 31 July 2020 12:21 > To: Paul Durrant ; xen-devel@lists.xenproject.org > Cc: Paul Durrant ; Jan Beulich ; Wei > Liu ; Roger > Pau Monné > Subject: Re: [PATCH] x86/hvm: set 'ipat' in EPT for special pages > > On 31/07/2020 11:46, Paul Du

Re: kernel-doc and xen.git

2020-07-31 Thread Jan Beulich
On 30.07.2020 03:27, Stefano Stabellini wrote: > Hi all, > > I would like to ask for your feedback on the adoption of the kernel-doc > format for in-code comments. > > In the FuSa SIG we have started looking into FuSa documents for Xen. One > of the things we are investigating are ways to link th

Re: [RESEND][PATCH v2 5/7] xen: include xen/guest_access.h rather than asm/guest_access.h

2020-07-31 Thread Jan Beulich
On 30.07.2020 20:18, Julien Grall wrote: > From: Julien Grall > > Only a few places are actually including asm/guest_access.h. While this > is fine today, a follow-up patch will want to move most of the helpers > from asm/guest_access.h to xen/guest_access.h. > > To prepare the move, everyone sh

[OSSTEST PATCH v2 01/41] Add cperl-indent-level to .dir-locals.el

2020-07-31 Thread Ian Jackson
My personal config on my laptop has this set to 2 and that makes editing osstest, which uses 4, quite annoying. Signed-off-by: Ian Jackson --- New in v2. --- .dir-locals.el | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.dir-locals.el b/.dir-locals.el index d87916f7..ad4fa

[OSSTEST PATCH v2 00/41] Performance work

2020-07-31 Thread Ian Jackson
This is a combination of two series and some new work: * [OSSTEST PATCH 00/14] Flight report performance improvements * [OSSTEST PATCH 00/11] Improve performance of sg-report-host-history * New work to improve the performance of cs-bisection-step * Fixes to usage of SQL LIKE Thanks to George f

[OSSTEST PATCH v2 03/41] SQL: Fix incorrect LIKE pattern syntax (literals)

2020-07-31 Thread Ian Jackson
LIKE takes a weird SQLish glob pattern, where % is like a glob * and (relevantly, here) _ is like a glob ?. Every _ in one of these LIKE patterns needs to be escaped with \. Do that for all the literal LIKE patterns. This fixes bugs. Generally, bugs where the wrong rows might be returned (excep

[OSSTEST PATCH v2 05/41] sg-report-flight: Add a comment re same-flight search narrowing

2020-07-31 Thread Ian Jackson
In afe851ca1771e5da6395b596afa69e509dbbc278 sg-report-flight: When justifying, disregard out-of-flight build jobs we narrowed sg-report-flight's search algorith. An extensive justification is in the commit message. I think much of this information belongs in-tree, so c&p it (with slight edits)

[OSSTEST PATCH v2 06/41] sg-report-flight: Sort failures by job name as last resort

2020-07-31 Thread Ian Jackson
This removes some nondeterminism from the output. Signed-off-by: Ian Jackson --- sg-report-flight | 1 + 1 file changed, 1 insertion(+) diff --git a/sg-report-flight b/sg-report-flight index fc439495..7f2790ce 100755 --- a/sg-report-flight +++ b/sg-report-flight @@ -813,6 +813,7 @@ END

[OSSTEST PATCH v2 02/41] SQL: Use "LIKE" rather than "like", etc.

2020-07-31 Thread Ian Jackson
This is more like the rest of the style. It will also make it easier to find instances of the mistaken LIKE syntax. I found these with "git grep" and manually edited them. I have checked the before-and-after result of find * -type f | xargs perl -i~ -pe 's/\bLIKE\b/like/g' and it has only the

[OSSTEST PATCH v2 07/41] schema: Provide indices for sg-report-flight

2020-07-31 Thread Ian Jackson
These indexes allow very fast lookup of "relevant" flights eg when trying to justify failures. In my ad-hoc test case, these indices (along with the subsequent changes to sg-report-flight and Executive.pm, reduce the runtime of sg-report-flight from 2-3ks (unacceptably long!) to as little as 5-7s

[OSSTEST PATCH v2 04/41] SQL: Fix incorrect LIKE pattern syntax (program variables)

2020-07-31 Thread Ian Jackson
In two places the pattern for LIKE is constructed programmatically. In this case, too, we need to escape % and _. We pass the actual pattern (or pattern fragment) via ?, so we do not need to worry about '. Signed-off-by: Ian Jackson --- New in v2. --- Osstest.pm | 8 +++- Os

[OSSTEST PATCH v2 09/41] sg-report-flight: Use WITH to use best index use for $flightsq

2020-07-31 Thread Ian Jackson
While we're here, convert this EXISTS subquery to a JOIN. Perf: runtime of my test case now ~200-300s. Example query before (from the Perl DBI trace): SELECT * FROM ( SELECT DISTINCT flight, blessing FROM flights JOIN runvars r1 USING (flight)

[OSSTEST PATCH v2 08/41] sg-report-flight: Ask the db for flights of interest

2020-07-31 Thread Ian Jackson
Specifically, we narrow the initial query to flights which have at least some job with the built_revision_foo we are looking for. This condition is strictly broader than that implemented inside the flight search loop, so there is no functional change. Perf: runtime of my test case now ~300s-500s.

Re: [RESEND][PATCH v2 7/7] xen/guest_access: Fix coding style in xen/guest_access.h

2020-07-31 Thread Jan Beulich
On 30.07.2020 20:18, Julien Grall wrote: > From: Julien Grall > > * Add space before and after operator > * Align \ > * Format comments How about also * remove/replace leading underscores ? Jan

Re: [RESEND][PATCH v2 6/7] xen/guest_access: Consolidate guest access helpers in xen/guest_access.h

2020-07-31 Thread Jan Beulich
On 30.07.2020 20:18, Julien Grall wrote: > From: Julien Grall > > Most of the helpers to access guest memory are implemented the same way > on Arm and x86. The only differences are: > - guest_handle_{from, to}_param(): while on x86 XEN_GUEST_HANDLE() > and XEN_GUEST_HANDLE_PARAM() are t

[OSSTEST PATCH v2 30/41] sg-report-host-history: Fork

2020-07-31 Thread Ian Jackson
Run each host's report in a separate child. This is considerably faster. Signed-off-by: Ian Jackson --- sg-report-host-history | 47 +++--- 1 file changed, 40 insertions(+), 7 deletions(-) diff --git a/sg-report-host-history b/sg-report-host-history index f4

[OSSTEST PATCH v2 27/41] sg-report-host-history: Rerganisation: Read old logs later

2020-07-31 Thread Ian Jackson
Perhaps at one point something read from these logs influenced the db query for thye flights range, but that is no longer the case and it doesn't seem likely to need to come back. We want to move the per-host stuff together. No functional change. Signed-off-by: Ian Jackson --- sg-report-host-h

[OSSTEST PATCH v2 20/41] schema: Add index for quick lookup by host

2020-07-31 Thread Ian Jackson
Signed-off-by: Ian Jackson --- v2: Use proper \ escaping for underscores in LIKE --- schema/runvars-host-index.sql | 8 1 file changed, 8 insertions(+) create mode 100644 schema/runvars-host-index.sql diff --git a/schema/runvars-host-index.sql b/schema/runvars-host-index.sql new file m

[OSSTEST PATCH v2 21/41] sg-report-host-history: Find flight limit by flight start date

2020-07-31 Thread Ian Jackson
By default we look for anything in (roughly) the last year. This query is in fact quite fast because the flights table is small. There is still the per-host limit of $limit (2000) recent runs. Signed-off-by: Ian Jackson --- sg-report-host-history | 56 --

[OSSTEST PATCH v2 19/41] Executive: Drop redundant AND clause

2020-07-31 Thread Ian Jackson
In "Executive: Use index for report__find_test" we changed an EXISTS subquery into a JOIN. Now, the condition r.flight=f.flight is redundant because this is the join column (from USING). No functional change. CC: George Dunlap Signed-off-by: Ian Jackson --- Osstest/Executive.pm | 1 - 1 file

[OSSTEST PATCH v2 12/41] Executive: Use index for report__find_test

2020-07-31 Thread Ian Jackson
After we refactor this query then we can enable the index use. (Both of these things together in this commit because I haven't perf tested the version with just the refactoring.) (We have provided an index that can answer this question really quickly if a version is specified. But the query plann

[OSSTEST PATCH v2 37/41] cs-bisection-step: temporary table: Insert only rows we care about

2020-07-31 Thread Ian Jackson
Every use of this table has a WHERE or ON which invokes at least one of these conditions. So put only those rows into the table. This provides a significant speedup (which I haven't properly measured). No overall functional change. Signed-off-by: Ian Jackson --- v2: New patch. --- cs-bisectio

[OSSTEST PATCH v2 15/41] duration_estimator: Explicitly provide null in general host q

2020-07-31 Thread Ian Jackson
Our spec. says we return nulls for started and status if we don't find a job matching the host spec. The way this works right now is that we look up the nonexistent entries in $refs->[0]. This is not really brilliant and is going to be troublesome as we continue to refactor. Provide these values

[OSSTEST PATCH v2 25/41] sg-report-host-history: Do the main query per host

2020-07-31 Thread Ian Jackson
In f6001d628c3b3fd42b10cd15351981a04bc02572 we combined these queries into one: sg-report-host-history: Aggregate runvars query for all hosts Now that we have an index, there is a faster way for the db to do this query: via that index. But it doesn't like to do that if be aggregate the queries.

[OSSTEST PATCH v2 41/41] duration_estimator: Clarify recentflights query a bit

2020-07-31 Thread Ian Jackson
The condition on r.job is more naturally thought of as a join condition than a where condition. (This is an inner join, so the semantics are identical.) Also, for clarity, swap the flight and job conditions round, so that the ON clause is a series of r.thing = otherthing. No functional change.

[OSSTEST PATCH v2 38/41] SQL: Change LIKE E'...\\_...' to LIKE '...\_...'

2020-07-31 Thread Ian Jackson
E'...' means to interpret \-escapes. But we don't want them: without E, we can avoid some toothpick-doubling. No functional change. Signed-off-by: Ian Jackson --- v2: New patch. --- cs-bisection-step | 8 sg-report-job-history | 4 ++-- 2 files changed, 6 insertions(+), 6 deletion

[OSSTEST PATCH v2 16/41] duration_estimator: Return job column in first query

2020-07-31 Thread Ian Jackson
Right now this is pointless since the Perl code doesn't need it. But this row is going to be part of a WITH clause soon. No functional change. Diffs to two example queries (from the Perl DBI trace): SELECT f.flight AS flight, + j.job AS job, f.st

[OSSTEST PATCH v2 13/41] duration_estimator: Ignore truncated jobs unless we know the step

2020-07-31 Thread Ian Jackson
If we are looking for a particular step then we will ignore jobs without that step, so any job which was truncated before it will be ignored. Otherwise we are looking for the whole job duration and a truncated job is not a good representative. This is a bugfix (to duration estimation), not a perf

[OSSTEST PATCH v2 39/41] cs-bisection-step: Add a debug print when we run dot(1)

2020-07-31 Thread Ian Jackson
Amongst other things this was useful for perf investigation. Signed-off-by: Ian Jackson --- v2: New patch. --- cs-bisection-step | 1 + 1 file changed, 1 insertion(+) diff --git a/cs-bisection-step b/cs-bisection-step index a82cbfb8..027032a1 100755 --- a/cs-bisection-step +++ b/cs-bisection-st

[OSSTEST PATCH v2 14/41] duration_estimator: Introduce some _qtxt variables

2020-07-31 Thread Ian Jackson
No functional change. Signed-off-by: Ian Jackson --- Osstest/Executive.pm | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm index f528edd0..4cb22cc9 100644 --- a/Osstest/Executive.pm +++ b/Osstest/Executive.pm @@ -1146,7 +1146,7

[OSSTEST PATCH v2 31/41] schema: Add index to help cs-bisection-step

2020-07-31 Thread Ian Jackson
cs-bisection step basis search involves looking for recent flights that weren't broken. A flight is broken if it has broken steps. Make an index for this to save it scanning the steps table. Signed-off-by: Ian Jackson --- schema/steps-broken-index.sql | 7 +++ 1 file changed, 7 insertions(+

[OSSTEST PATCH v2 40/41] cs-bisection-step: Lay out the revision tuple graph once

2020-07-31 Thread Ian Jackson
The graph layout algorithm is not very fast, particularly if the revision graph is big. In my test case this saves about 10s. Signed-off-by: Ian Jackson --- v2: New patch. --- cs-bisection-step | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/cs-bisection-step b/c

[OSSTEST PATCH v2 24/41] sg-report-host-history: Add a debug print after sorting jobs

2020-07-31 Thread Ian Jackson
This helps rule this sort out as a source of slowness. Signed-off-by: Ian Jackson --- sg-report-host-history | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sg-report-host-history b/sg-report-host-history index a34458e0..1c2d19ae 100755 --- a/sg-report-host-history +++ b/sg-report-host-his

[OSSTEST PATCH v2 11/41] sg-report-flight: Use the job row from the intitial query

2020-07-31 Thread Ian Jackson
$jcheckq is redundant: we looked this up right at the start. This is not expected to speed things up very much, but it makes things somewhat cleaner and clearer. Signed-off-by: Ian Jackson --- sg-report-flight | 11 +++ 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/sg-re

[OSSTEST PATCH v2 35/41] cs-bisection-step: Break out qtxt_common_ok

2020-07-31 Thread Ian Jackson
Make this bit of query into a subref which takes a $table argument. No change to the generated query. Signed-off-by: Ian Jackson --- v2: New patch. --- cs-bisection-step | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/cs-bisection-step b/cs-bisection-step index f1172

[OSSTEST PATCH v2 17/41] duration_estimator: Move $uptincl_testid to separate @x_params

2020-07-31 Thread Ian Jackson
This is going to be useful soon. No functional change. Signed-off-by: Ian Jackson --- Osstest/Executive.pm | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm index 359120c0..fb975dac 100644 --- a/Osstest/Executive.pm +++ b/Osstest

[OSSTEST PATCH v2 32/41] adhoc-revtuple-generator: Fix an undef warning in a debug print

2020-07-31 Thread Ian Jackson
$parents might be undef here. Signed-off-by: Ian Jackson --- New in v2. --- adhoc-revtuple-generator | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/adhoc-revtuple-generator b/adhoc-revtuple-generator index c8d6f4ad..ec33305a 100755 --- a/adhoc-revtuple-generator +++ b/adhoc-

[OSSTEST PATCH v2 33/41] cs-bisection-step: Generalise qtxt_common_rev_ok

2020-07-31 Thread Ian Jackson
* Make it into a subref which takes a $table argument. * Change the two references into function calls using the @{...} syntax * Move the definition earlier in the file No change to the generated query. Signed-off-by: Ian Jackson --- v2: New patch. --- cs-bisection-step | 16 ++-- 1

[OSSTEST PATCH v2 26/41] sg-report-host-history: Rerganisation: Make mainquery per-host

2020-07-31 Thread Ian Jackson
This moves the loop over hosts into the main program. We are working our way to a new code structure. No functional change. Signed-off-by: Ian Jackson --- sg-report-host-history | 19 +++ 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/sg-report-host-history b/sg

Re: [PATCH v1] tools/xen-cpuid: show enqcmd

2020-07-31 Thread Jan Beulich
On 30.07.2020 18:34, Olaf Hering wrote: > Translate <29> into a feature string. > > Signed-off-by: Olaf Hering Acked-by: Jan Beulich Albeit I'm pretty sure there are more missing than just this lone one. Jan > --- a/tools/misc/xen-cpuid.c > +++ b/tools/misc/xen-cpuid.c > @@ -133,7 +133,7 @@

[xen-unstable test] 152311: tolerable FAIL - PUSHED

2020-07-31 Thread osstest service owner
flight 152311 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/152311/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-armhf-armhf-libvirt 14 saverestore-support-checkfail like 152293 test-amd64-amd64-xl-qemuu-win7-amd64

Re: [PATCH v1] tools/xen-cpuid: show enqcmd

2020-07-31 Thread Andrew Cooper
On 31/07/2020 13:03, Jan Beulich wrote: > On 30.07.2020 18:34, Olaf Hering wrote: >> Translate <29> into a feature string. >> >> Signed-off-by: Olaf Hering > Acked-by: Jan Beulich > > Albeit I'm pretty sure there are more missing than just this lone one. And in particular, probably missing from

[OSSTEST PATCH v2 23/41] Executive: Export opendb_tests

2020-07-31 Thread Ian Jackson
sg-report-host-history is going to want this in a moment Signed-off-by: Ian Jackson --- Osstest/Executive.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm index 2f81e89d..8e4c5b9a 100644 --- a/Osstest/Executive.pm +++ b/Osstest/Ex

[OSSTEST PATCH v2 28/41] sg-report-host-history: Rerganisation: Change loops

2020-07-31 Thread Ian Jackson
Move the per-host code all into the same per-host loop. One effect is to transpose the db_retry and host loops for mainquery. No functional change. Signed-off-by: Ian Jackson --- sg-report-host-history | 12 ++-- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/sg-report-

[OSSTEST PATCH v2 29/41] sg-report-host-history: Drop a redundznt AND clause

2020-07-31 Thread Ian Jackson
This condition is the same as $flightcond. (This has no effect on the db performance since the query planner figures it out, but it is confusing.) Signed-off-by: Ian Jackson --- sg-report-host-history | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/sg-report-host-history b

[OSSTEST PATCH v2 22/41] sg-report-host-history: Drop per-job debug etc.

2020-07-31 Thread Ian Jackson
This printing has a significant effect on the performance of this program, at least after we optimise various other things. Signed-off-by: Ian Jackson --- sg-report-host-history | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/sg-report-host-history b/sg-report-

[OSSTEST PATCH v2 34/41] cs-bisection-step: Move an AND

2020-07-31 Thread Ian Jackson
This obviously-fine change makes the next commit easier to review. Signed-off-by: Ian Jackson --- v2: New patch. --- cs-bisection-step | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cs-bisection-step b/cs-bisection-step index 5d4e179e..f11726aa 100755 --- a/cs-bisection-

[OSSTEST PATCH v2 18/41] duration_estimator: Move duration query loop into database

2020-07-31 Thread Ian Jackson
Stuff the two queries together: we use the firsty query as a WITH clause. This is significantly faster, perhaps because the query optimiser does a better job but probably just because it saves on round trips. No functional change. Perf: subjectively this seemed to help when the cache was cold.

[OSSTEST PATCH v2 10/41] sg-report-flight: Use WITH clause to use index for $anypassq

2020-07-31 Thread Ian Jackson
Perf: runtime of my test case now ~11s Example query before (from the Perl DBI trace): SELECT * FROM flights JOIN steps USING (flight) WHERE (branch='xen-unstable') AND job=? and testid=? and status='pass' AND ( (TRUE AND flight <= 151903) AND (bles

[OSSTEST PATCH v2 36/41] cs-bisection-step: Use db_prepare a few times instead of ->do

2020-07-31 Thread Ian Jackson
With $dbh_tests->do(...), we can only get a debug trace of the queries by using DBI_TRACE which produces voluminous output. Using our own db_prepare invokes our own debugging. No functional change. Signed-off-by: Ian Jackson --- v2: New patch. --- cs-bisection-step | 6 +++--- 1 file changed,

Re: [PATCH v1] tools/xen-cpuid: show enqcmd

2020-07-31 Thread Olaf Hering
Am Fri, 31 Jul 2020 13:04:35 +0100 schrieb Andrew Cooper : > And in particular, probably missing from libxl_cpuid.c, which I was > meaning to check when I've got a free moment. Will a ever domU see this flag? I just spotted the <29> when comparing 'xen-cpuid' output between recent Xen releases.

Re: [PATCH v3] xen/arm: Convert runstate address during hypcall

2020-07-31 Thread Jan Beulich
On 30.07.2020 22:50, Julien Grall wrote: > On 30/07/2020 11:24, Bertrand Marquis wrote: >> At the moment on Arm, a Linux guest running with KTPI enabled will >> cause the following error when a context switch happens in user mode: >> (XEN) p2m.c:1890: d1v0: Failed to walk page-table va 0xff837e

Re: [PATCH v1] tools/xen-cpuid: show enqcmd

2020-07-31 Thread Jan Beulich
On 31.07.2020 14:04, Andrew Cooper wrote: > On 31/07/2020 13:03, Jan Beulich wrote: >> On 30.07.2020 18:34, Olaf Hering wrote: >>> Translate <29> into a feature string. >>> >>> Signed-off-by: Olaf Hering >> Acked-by: Jan Beulich >> >> Albeit I'm pretty sure there are more missing than just this l

Re: [PATCH v1] tools/xen-cpuid: show enqcmd

2020-07-31 Thread Andrew Cooper
On 31/07/2020 13:15, Olaf Hering wrote: > Am Fri, 31 Jul 2020 13:04:35 +0100 > schrieb Andrew Cooper : > >> And in particular, probably missing from libxl_cpuid.c, which I was >> meaning to check when I've got a free moment. > Will a ever domU see this flag? I just spotted the <29> when comparing

Re: RESCHEDULED Call for agenda items for Community Call, August 13 @ 15:00 UTC

2020-07-31 Thread Jan Beulich
On 30.07.2020 17:41, George Dunlap wrote: >> On Jul 30, 2020, at 4:17 PM, George Dunlap wrote: >> >> Hey all, >> >> The community call is scheduled for next week, 6 August. I, however, will >> be on PTO that week; I propose rescheduling it for the following week, 13 >> August, at the same time.

Re: RESCHEDULED Call for agenda items for Community Call, August 13 @ 15:00 UTC

2020-07-31 Thread George Dunlap
> On Jul 31, 2020, at 1:25 PM, Jan Beulich wrote: > > On 30.07.2020 17:41, George Dunlap wrote: >>> On Jul 30, 2020, at 4:17 PM, George Dunlap wrote: >>> >>> Hey all, >>> >>> The community call is scheduled for next week, 6 August. I, however, will >>> be on PTO that week; I propose resche

Re: [PATCH] x86/vhpet: Fix type size in timer_int_route_valid

2020-07-31 Thread Julien Grall
Hi Jan, On 31/07/2020 10:53, Jan Beulich wrote: On 31.07.2020 10:38, Eslam Elnikety wrote: On 28.07.20 19:51, Jan Beulich wrote: On 28.07.2020 11:26, Andrew Cooper wrote: --- a/xen/include/asm-x86/hvm/vpt.h +++ b/xen/include/asm-x86/hvm/vpt.h @@ -73,7 +73,13 @@ struct hpet_registers {  

Re: RESCHEDULED Call for agenda items for Community Call, August 13 @ 15:00 UTC

2020-07-31 Thread Jan Beulich
On 31.07.2020 14:27, George Dunlap wrote: >> On Jul 31, 2020, at 1:25 PM, Jan Beulich wrote: >> On 30.07.2020 17:41, George Dunlap wrote: On Jul 30, 2020, at 4:17 PM, George Dunlap wrote: Hey all, The community call is scheduled for next week, 6 August. I, however,

Re: [PATCH] x86/vhpet: Fix type size in timer_int_route_valid

2020-07-31 Thread Jan Beulich
On 31.07.2020 14:35, Julien Grall wrote: > Hi Jan, > > On 31/07/2020 10:53, Jan Beulich wrote: >> On 31.07.2020 10:38, Eslam Elnikety wrote: >>> On 28.07.20 19:51, Jan Beulich wrote: On 28.07.2020 11:26, Andrew Cooper wrote: > --- a/xen/include/asm-x86/hvm/vpt.h > +++ b/xen/include/as

[PATCH v2 2/2] x86/hvm: simplify 'mmio_direct' check in epte_get_entry_emt()

2020-07-31 Thread Paul Durrant
From: Paul Durrant Re-factor the code to take advantage of the fact that the APIC access page is a 'special' page. Suggested-by: Jan Beulich Signed-off-by: Paul Durrant --- Cc: Jan Beulich Cc: Andrew Cooper Cc: Wei Liu Cc: "Roger Pau Monné" --- xen/arch/x86/hvm/mtrr.c | 15 ---

[PATCH v2 0/2] epte_get_entry_emt() modifications

2020-07-31 Thread Paul Durrant
From: Paul Durrant This series was originally a singleton (of patch #1) Paul Durrant (2): x86/hvm: set 'ipat' in EPT for special pages x86/hvm: simplify 'mmio_direct' check in epte_get_entry_emt() xen/arch/x86/hvm/mtrr.c | 16 +--- 1 file changed, 5 insertions(+), 11 deletions(

[PATCH v2 1/2] x86/hvm: set 'ipat' in EPT for special pages

2020-07-31 Thread Paul Durrant
From: Paul Durrant All non-MMIO ranges (i.e those not mapping real device MMIO regions) that map valid MFNs are normally marked MTRR_TYPE_WRBACK and 'ipat' is set. Hence when PV drivers running in a guest populate the BAR space of the Xen Platform PCI Device with pages such as the Shared Info pag

Re: [RESEND][PATCH v2 5/7] xen: include xen/guest_access.h rather than asm/guest_access.h

2020-07-31 Thread Bertrand Marquis
> On 30 Jul 2020, at 20:18, Julien Grall wrote: > > From: Julien Grall > > Only a few places are actually including asm/guest_access.h. While this > is fine today, a follow-up patch will want to move most of the helpers > from asm/guest_access.h to xen/guest_access.h. > > To prepare the move,

Re: [RESEND][PATCH v2 5/7] xen: include xen/guest_access.h rather than asm/guest_access.h

2020-07-31 Thread Bertrand Marquis
> On 30 Jul 2020, at 20:18, Julien Grall wrote: > > From: Julien Grall > > Only a few places are actually including asm/guest_access.h. While this > is fine today, a follow-up patch will want to move most of the helpers > from asm/guest_access.h to xen/guest_access.h. > > To prepare the mov

Re: [RESEND][PATCH v2 7/7] xen/guest_access: Fix coding style in xen/guest_access.h

2020-07-31 Thread Bertrand Marquis
> On 30 Jul 2020, at 20:18, Julien Grall wrote: > > From: Julien Grall > >* Add space before and after operator >* Align \ >* Format comments > > No functional changes expected. > > Signed-off-by: Julien Grall Reviewed-by: Bertrand Marquis > --- > xen/include/xen/guest_acces

Re: kernel-doc and xen.git

2020-07-31 Thread Bertrand Marquis
> On 31 Jul 2020, at 13:29, Jan Beulich wrote: > > On 30.07.2020 03:27, Stefano Stabellini wrote: >> Hi all, >> >> I would like to ask for your feedback on the adoption of the kernel-doc >> format for in-code comments. >> >> In the FuSa SIG we have started looking into FuSa documents for Xen

Re: kernel-doc and xen.git

2020-07-31 Thread George Dunlap
> On Jul 31, 2020, at 12:29 PM, Jan Beulich wrote: > > On 30.07.2020 03:27, Stefano Stabellini wrote: >> Hi all, >> >> I would like to ask for your feedback on the adoption of the kernel-doc >> format for in-code comments. >> >> In the FuSa SIG we have started looking into FuSa documents for

[PATCH 0/6] Fixes and improvements for Xen pvdrm

2020-07-31 Thread Oleksandr Andrushchenko
From: Oleksandr Andrushchenko Hello, This series contains an assorted set of fixes and improvements for the Xen para-virtualized display driver and grant device driver which I have collected over the last couple of months: 1. Minor fixes to grant device driver and drm/xen-front. 2. New format

[PATCH 2/6] drm/xen-front: Fix misused IS_ERR_OR_NULL checks

2020-07-31 Thread Oleksandr Andrushchenko
From: Oleksandr Andrushchenko The patch c575b7eeb89f: "drm/xen-front: Add support for Xen PV display frontend" from Apr 3, 2018, leads to the following static checker warning: drivers/gpu/drm/xen/xen_drm_front_gem.c:140 xen_drm_front_gem_create() warn: passing zero to 'ERR_CAST'

Re: [PATCH] xen/arm: cmpxchg: Add missing memory barriers in __cmpxchg_mb_timeout()

2020-07-31 Thread Bertrand Marquis
> On 30 Jul 2020, at 19:07, Julien Grall wrote: > > From: Julien Grall > > The function __cmpxchg_mb_timeout() was intended to have the same > semantics as __cmpxchg_mb(). Unfortunately, the memory barriers were > not added when first implemented. > > There is no known issue with the existi

[PATCH 4/6] xen: Sync up with the canonical protocol definition in Xen

2020-07-31 Thread Oleksandr Andrushchenko
From: Oleksandr Andrushchenko This is the sync up with the canonical definition of the display protocol in Xen. 1. Add protocol version as an integer Version string, which is in fact an integer, is hard to handle in the code that supports different protocol versions. To simplify that also add t

[PATCH 1/6] xen/gntdev: Fix dmabuf import with non-zero sgt offset

2020-07-31 Thread Oleksandr Andrushchenko
From: Oleksandr Andrushchenko It is possible that the scatter-gather table during dmabuf import has non-zero offset of the data, but user-space doesn't expect that. Fix this by failing the import, so user-space doesn't access wrong data. Fixes: 37ccb44d0b00 ("xen/gntdev: Implement dma-buf import

[PATCH 5/6] drm/xen-front: Pass dumb buffer data offset to the backend

2020-07-31 Thread Oleksandr Andrushchenko
From: Oleksandr Andrushchenko While importing a dmabuf it is possible that the data of the buffer is put with offset which is indicated by the SGT offset. Respect the offset value and forward it to the backend. Signed-off-by: Oleksandr Andrushchenko --- drivers/gpu/drm/xen/xen_drm_front.c

  1   2   >