Xen 4.14 RC5

[Paul Durrant]

Hi all,

Xen 4.14 RC5 is tagged. You can check that out from xen.git:

git://xenbits.xen.org/xen.git 4.14.0-rc5

For your convenience there is also a tarball at:
https://downloads.xenproject.org/release/xen/4.14.0-rc5/xen-4.14.0-rc5.tar.gz

And the signature is at:
https://downloads.xenproject.org/release/xen/4.14.0-rc5/xen-4.14.0-rc5.tar.gz.sig

Please send bug reports and test reports to xen-devel@lists.xenproject.org.
When sending bug reports, please CC relevant maintainers and me (p...@xen.org).

As a reminder, there will be a Xen Test Day. Please see the test day schedule at
https://wiki.xenproject.org/wiki/Xen_Project_Test_Days and test instructions at
https://wiki.xenproject.org/wiki/Xen_4.14_RC_test_instructions.

  Paul Durrant

Re: [BUG] Xen build for RCAR failing

[Oleksandr Andrushchenko]

On 7/7/20 10:58 AM, Manikandan Chockalingam (RBEI/ECF3) wrote:
>
> Hello Team,
>
> I am trying to build xen hypervisor for RCAR and following the 
> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Salvator-X
>  steps.
>
> But am facing the following issues
>
> 1.SRC_URI="http://v3.sk/~lkundrak/dev86/archive/Dev86src-${PV}.tar.gz in 
> recipes-extended/dev86/dev86_0.16.20.bb is not accesible
>
> *Modification 
> done:*SRC_URI=https://src.fedoraproject.org/lookaside/extras/dev86/Dev86src-0.16.20.tar.gz/567cf460d132f9d8775dd95f9208e49a/Dev86src-${PV}.tar.gz
>  
> 
>
You can try what we use [1]. And the issue you are facing is rather Yocto 
related, not R-Car specific, IMO

[1] 
https://github.com/xen-troops/meta-xt-prod-devel/blob/master/recipes-domd/domd-image-weston/files/meta-xt-prod-extra/recipes-extended/dev86/dev86_%25.bbappend

Re: [PATCH v4 03/10] tools/libxl: add vmtrace_pt_size parameter

[Julien Grall]

Hi,

On 06/07/2020 09:46, Jan Beulich wrote:

On 04.07.2020 19:23, Julien Grall wrote:

Hi,

On 03/07/2020 11:11, Roger Pau Monné wrote:

On Fri, Jul 03, 2020 at 11:56:38AM +0200, Jan Beulich wrote:

On 03.07.2020 11:44, Roger Pau Monné wrote:

On Thu, Jul 02, 2020 at 06:23:28PM +0200, Michał Leszczyński wrote:

- 2 lip 2020 o 11:00, Roger Pau Monné roger@citrix.com napisał(a):


On Tue, Jun 30, 2020 at 02:33:46PM +0200, Michał Leszczyński wrote:

diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 59bdc28c89..7b8289d436 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -92,6 +92,7 @@ struct xen_domctl_createdomain {
   uint32_t max_evtchn_port;
   int32_t max_grant_frames;
   int32_t max_maptrack_frames;
+uint8_t vmtrace_pt_order;


I've been thinking about this, and even though this is a domctl (so
not a stable interface) we might want to consider using a size (or a
number of pages) here rather than an order. IPT also supports
TOPA mode (kind of a linked list of buffers) that would allow for
sizes not rounded to order boundaries to be used, since then only each
item in the linked list needs to be rounded to an order boundary, so
you could for example use three 4K pages in TOPA mode AFAICT.

Roger.


In previous versions it was "size" but it was requested to change it
to "order" in order to shrink the variable size from uint64_t to
uint8_t, because there is limited space for xen_domctl_createdomain
structure.


It's likely I'm missing something here, but I wasn't aware
xen_domctl_createdomain had any constrains regarding it's size. It's
currently 48bytes which seems fairly small.


Additionally I would guess a uint32_t could do here, if the value
passed was "number of pages" rather than "number of bytes"?

Looking at the rest of the code, the toolstack accepts a 64-bit value.
So this would lead to truncation of the buffer if it is bigger than 2^44
bytes.

I agree such buffer is unlikely, yet I still think we want to harden the
code whenever we can. So the solution is to either prevent check
truncation in libxl or directly use 64-bit in the domctl.

My preference is the latter.



That could work, not sure if it needs to state however that those will
be 4K pages, since Arm can have a different minimum page size IIRC?
(or that's already the assumption for all number of frames fields)
vmtrace_nr_frames seems fine to me.


The hypercalls interface is using the same page granularity as the
hypervisor (i.e 4KB).

While we already support guest using 64KB page granularity, it is
impossible to have a 64KB Arm hypervisor in the current state. You are
going to either break existing guest (if you switch to 64KB page
granularity for the hypercall ABI) or render them insecure (the mimimum
mapping in the P2M would be 64KB).

DOMCTLs are not stable yet, so using a number of pages is OK. However, I
would strongly suggest to use a number of bytes for any xl/libxl/stable
libraries interfaces as this avoids confusion and also make more
futureproof.


If we can't settle on what "page size" means in the public interface
(which imo is embarrassing), then how about going with number of kb,
like other memory libxl controls do? (I guess using Mb, in line with
other config file controls, may end up being too coarse here.) This
would likely still allow for a 32-bit field to be wide enough.


A 32-bit field would definitely not be able to cover a full address 
space. So do you mind to explain what is the upper bound you expect here?


Cheers,

--
Julien Grall

[qemu-mainline test] 151685: regressions - FAIL

[osstest service owner]

flight 151685 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151685/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail REGR. vs. 151065
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 
151065
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail REGR. vs. 
151065
 test-amd64-amd64-xl-qemuu-win7-amd64 10 windows-install  fail REGR. vs. 151065
 test-amd64-amd64-libvirt-vhd 10 debian-di-installfail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-win7-amd64 10 windows-install   fail REGR. vs. 151065
 test-amd64-amd64-qemuu-nested-intel 10 debian-hvm-install fail REGR. vs. 151065
 test-amd64-i386-libvirt-xsm  12 guest-start  fail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 10 debian-hvm-install fail REGR. 
vs. 151065
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail REGR. vs. 151065
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-install  fail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 151065
 test-amd64-amd64-qemuu-nested-amd 10 debian-hvm-install  fail REGR. vs. 151065
 test-amd64-amd64-xl-qcow210 debian-di-installfail REGR. vs. 151065
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 10 debian-hvm-install fail REGR. 
vs. 151065
 test-amd64-i386-qemuu-rhel6hvm-amd 10 redhat-install fail REGR. vs. 151065
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-i386-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail REGR. vs. 
151065
 test-amd64-i386-xl-qemuu-ws16-amd64 10 windows-install   fail REGR. vs. 151065
 test-amd64-i386-libvirt-pair 21 guest-start/debian   fail REGR. vs. 151065
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-i386-freebsd10-i386 11 guest-startfail REGR. vs. 151065
 test-amd64-i386-qemuu-rhel6hvm-intel 10 redhat-install   fail REGR. vs. 151065
 test-amd64-amd64-libvirt 12 guest-start  fail REGR. vs. 151065
 test-amd64-amd64-libvirt-xsm 12 guest-start  fail REGR. vs. 151065
 test-amd64-i386-libvirt  12 guest-start  fail REGR. vs. 151065
 test-amd64-i386-freebsd10-amd64 11 guest-start   fail REGR. vs. 151065
 test-armhf-armhf-xl-vhd  10 debian-di-installfail REGR. vs. 151065
 test-amd64-amd64-libvirt-pair 21 guest-start/debian  fail REGR. vs. 151065
 test-armhf-armhf-libvirt 12 guest-start  fail REGR. vs. 151065
 test-arm64-arm64-libvirt-xsm 12 guest-start  fail REGR. vs. 151065
 test-armhf-armhf-libvirt-raw 10 debian-di-installfail REGR. vs. 151065

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-rtds 12 guest-startfail pass in 151669
 test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10 fail pass in 151669

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds13 migrate-support-check fail in 151669 never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-check fail in 151669 never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail

Re: [BUG] Xen build for RCAR failing

[Bertrand Marquis]

> On 7 Jul 2020, at 08:58, Manikandan Chockalingam (RBEI/ECF3) 
>  wrote:
> 
> Hello Team,
>  
> I am trying to build xen hypervisor for RCAR and following the 
> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Salvator-X
>  steps.
>  
> But am facing the following issues
> 1.  SRC_URI="http://v3.sk/~lkundrak/dev86/archive/Dev86src-${PV}.tar.gz 
> in recipes-extended/dev86/dev86_0.16.20.bb is not accesible
> Modification done: 
> SRC_URI=https://src.fedoraproject.org/lookaside/extras/dev86/Dev86src-0.16.20.tar.gz/567cf460d132f9d8775dd95f9208e49a/Dev86src-${PV}.tar.gz
> 2.  LIC_FILES_CHKSUM changed in recipes-extended/xen/xen.inc
> 3.  QA Issue: xen: Files/directories were installed but not shipped in 
> any package:
> /usr/bin/vchan-socket-proxy
>   /usr/sbin/xenmon
>   /usr/sbin/xenhypfs
>   /usr/lib/libxenfsimage.so.4.14.0
>   /usr/lib/libxenhypfs.so.1
>   /usr/lib/libxenfsimage.so
>   /usr/lib/libxenhypfs.so.1.0
>   /usr/lib/libxenfsimage.so.4.14
>   /usr/lib/libxenhypfs.so
>   /usr/lib/pkgconfig
>   /usr/lib/xen/bin/depriv-fd-checker
>   /usr/lib/pkgconfig/xenlight.pc
>   /usr/lib/pkgconfig/xenguest.pc
>   /usr/lib/pkgconfig/xenhypfs.pc
>   /usr/lib/pkgconfig/xlutil.pc
>   /usr/lib/pkgconfig/xentoolcore.pc
>   /usr/lib/pkgconfig/xentoollog.pc
>   /usr/lib/pkgconfig/xenstore.pc
>   /usr/lib/pkgconfig/xencall.pc
>   /usr/lib/pkgconfig/xencontrol.pc
>   /usr/lib/pkgconfig/xendevicemodel.pc
>   /usr/lib/pkgconfig/xenstat.pc
>   /usr/lib/pkgconfig/xengnttab.pc
>   /usr/lib/pkgconfig/xenevtchn.pc
>   /usr/lib/pkgconfig/xenvchan.pc
>   /usr/lib/pkgconfig/xenforeignmemory.pc
>   /usr/lib/xenfsimage/fat/fsimage.so
>   /usr/lib/xenfsimage/iso9660/fsimage.so
>   /usr/lib/xenfsimage/reiserfs/fsimage.so
>   /usr/lib/xenfsimage/ufs/fsimage.so
>   /usr/lib/xenfsimage/ext2fs-lib/fsimage.so
>   /usr/lib/xenfsimage/zfs/fsimage.so
> Please set FILES such that these items are packaged. Alternatively if they 
> are unneeded, avoid installing them or delete them within do_install.
> xen: 32 installed and not shipped files. [installed-vs-shipped]
> ERROR: xen-unstable+gitAUTOINC+be63d9d47f-r0 do_package: Fatal QA errors 
> found, failing task.
> ERROR: xen-unstable+gitAUTOINC+be63d9d47f-r0 do_package: Function failed: 
> do_package
> ERROR: Logfile of failure stored in: 
> /home/manikandan/yocto_2.19/build/build/tmp/work/aarch64-poky-linux/xen/unstable+gitAUTOINC+be63d9d47f-r0/temp/log.do_package.17889
> ERROR: Task 13 
> (/home/manikandan/yocto_2.19/build/meta-virtualization/recipes-extended/xen/xen_git.bb,
>  do_package) failed with exit code '1'

The configuration from the page is using a rather old release of Yocto.
I would suggest to switch to dunfell which will use xen 4.12.

Current master status of Yocto is not building at the moment.
Christopher Clark has done some work on it here [1] in meta-virtualization but 
this is not merged yet.

If you are trying to build Xen master by modifying a recipe you will have 
issues as some things have been added like hypfs which are creating the issues 
you see when Yocto is checking that everything was installed.

Bertrand

[1] https://lists.yoctoproject.org/g/meta-virtualization/message/5495

Re: [PATCH v4 03/10] tools/libxl: add vmtrace_pt_size parameter

[Jan Beulich]

On 07.07.2020 10:44, Julien Grall wrote:
> Hi,
> 
> On 06/07/2020 09:46, Jan Beulich wrote:
>> On 04.07.2020 19:23, Julien Grall wrote:
>>> Hi,
>>>
>>> On 03/07/2020 11:11, Roger Pau Monné wrote:
 On Fri, Jul 03, 2020 at 11:56:38AM +0200, Jan Beulich wrote:
> On 03.07.2020 11:44, Roger Pau Monné wrote:
>> On Thu, Jul 02, 2020 at 06:23:28PM +0200, Michał Leszczyński wrote:
>>> - 2 lip 2020 o 11:00, Roger Pau Monné roger@citrix.com 
>>> napisał(a):
>>>
 On Tue, Jun 30, 2020 at 02:33:46PM +0200, Michał Leszczyński wrote:
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 59bdc28c89..7b8289d436 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -92,6 +92,7 @@ struct xen_domctl_createdomain {
>uint32_t max_evtchn_port;
>int32_t max_grant_frames;
>int32_t max_maptrack_frames;
> +uint8_t vmtrace_pt_order;

 I've been thinking about this, and even though this is a domctl (so
 not a stable interface) we might want to consider using a size (or a
 number of pages) here rather than an order. IPT also supports
 TOPA mode (kind of a linked list of buffers) that would allow for
 sizes not rounded to order boundaries to be used, since then only each
 item in the linked list needs to be rounded to an order boundary, so
 you could for example use three 4K pages in TOPA mode AFAICT.

 Roger.
>>>
>>> In previous versions it was "size" but it was requested to change it
>>> to "order" in order to shrink the variable size from uint64_t to
>>> uint8_t, because there is limited space for xen_domctl_createdomain
>>> structure.
>>
>> It's likely I'm missing something here, but I wasn't aware
>> xen_domctl_createdomain had any constrains regarding it's size. It's
>> currently 48bytes which seems fairly small.
>
> Additionally I would guess a uint32_t could do here, if the value
> passed was "number of pages" rather than "number of bytes"?
>>> Looking at the rest of the code, the toolstack accepts a 64-bit value.
>>> So this would lead to truncation of the buffer if it is bigger than 2^44
>>> bytes.
>>>
>>> I agree such buffer is unlikely, yet I still think we want to harden the
>>> code whenever we can. So the solution is to either prevent check
>>> truncation in libxl or directly use 64-bit in the domctl.
>>>
>>> My preference is the latter.
>>>

 That could work, not sure if it needs to state however that those will
 be 4K pages, since Arm can have a different minimum page size IIRC?
 (or that's already the assumption for all number of frames fields)
 vmtrace_nr_frames seems fine to me.
>>>
>>> The hypercalls interface is using the same page granularity as the
>>> hypervisor (i.e 4KB).
>>>
>>> While we already support guest using 64KB page granularity, it is
>>> impossible to have a 64KB Arm hypervisor in the current state. You are
>>> going to either break existing guest (if you switch to 64KB page
>>> granularity for the hypercall ABI) or render them insecure (the mimimum
>>> mapping in the P2M would be 64KB).
>>>
>>> DOMCTLs are not stable yet, so using a number of pages is OK. However, I
>>> would strongly suggest to use a number of bytes for any xl/libxl/stable
>>> libraries interfaces as this avoids confusion and also make more
>>> futureproof.
>>
>> If we can't settle on what "page size" means in the public interface
>> (which imo is embarrassing), then how about going with number of kb,
>> like other memory libxl controls do? (I guess using Mb, in line with
>> other config file controls, may end up being too coarse here.) This
>> would likely still allow for a 32-bit field to be wide enough.
> 
> A 32-bit field would definitely not be able to cover a full address 
> space. So do you mind to explain what is the upper bound you expect here?

Do you foresee a need for buffer sizes of 4Tb and up?

Jan

Re: [PATCH v4 03/10] tools/libxl: add vmtrace_pt_size parameter

[Julien Grall]

On 07/07/2020 10:10, Jan Beulich wrote:

On 07.07.2020 10:44, Julien Grall wrote:

Hi,

On 06/07/2020 09:46, Jan Beulich wrote:

On 04.07.2020 19:23, Julien Grall wrote:

Hi,

On 03/07/2020 11:11, Roger Pau Monné wrote:

On Fri, Jul 03, 2020 at 11:56:38AM +0200, Jan Beulich wrote:

On 03.07.2020 11:44, Roger Pau Monné wrote:

On Thu, Jul 02, 2020 at 06:23:28PM +0200, Michał Leszczyński wrote:

- 2 lip 2020 o 11:00, Roger Pau Monné roger@citrix.com napisał(a):


On Tue, Jun 30, 2020 at 02:33:46PM +0200, Michał Leszczyński wrote:

diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 59bdc28c89..7b8289d436 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -92,6 +92,7 @@ struct xen_domctl_createdomain {
uint32_t max_evtchn_port;
int32_t max_grant_frames;
int32_t max_maptrack_frames;
+uint8_t vmtrace_pt_order;


I've been thinking about this, and even though this is a domctl (so
not a stable interface) we might want to consider using a size (or a
number of pages) here rather than an order. IPT also supports
TOPA mode (kind of a linked list of buffers) that would allow for
sizes not rounded to order boundaries to be used, since then only each
item in the linked list needs to be rounded to an order boundary, so
you could for example use three 4K pages in TOPA mode AFAICT.

Roger.


In previous versions it was "size" but it was requested to change it
to "order" in order to shrink the variable size from uint64_t to
uint8_t, because there is limited space for xen_domctl_createdomain
structure.


It's likely I'm missing something here, but I wasn't aware
xen_domctl_createdomain had any constrains regarding it's size. It's
currently 48bytes which seems fairly small.


Additionally I would guess a uint32_t could do here, if the value
passed was "number of pages" rather than "number of bytes"?

Looking at the rest of the code, the toolstack accepts a 64-bit value.
So this would lead to truncation of the buffer if it is bigger than 2^44
bytes.

I agree such buffer is unlikely, yet I still think we want to harden the
code whenever we can. So the solution is to either prevent check
truncation in libxl or directly use 64-bit in the domctl.

My preference is the latter.



That could work, not sure if it needs to state however that those will
be 4K pages, since Arm can have a different minimum page size IIRC?
(or that's already the assumption for all number of frames fields)
vmtrace_nr_frames seems fine to me.


The hypercalls interface is using the same page granularity as the
hypervisor (i.e 4KB).

While we already support guest using 64KB page granularity, it is
impossible to have a 64KB Arm hypervisor in the current state. You are
going to either break existing guest (if you switch to 64KB page
granularity for the hypercall ABI) or render them insecure (the mimimum
mapping in the P2M would be 64KB).

DOMCTLs are not stable yet, so using a number of pages is OK. However, I
would strongly suggest to use a number of bytes for any xl/libxl/stable
libraries interfaces as this avoids confusion and also make more
futureproof.


If we can't settle on what "page size" means in the public interface
(which imo is embarrassing), then how about going with number of kb,
like other memory libxl controls do? (I guess using Mb, in line with
other config file controls, may end up being too coarse here.) This
would likely still allow for a 32-bit field to be wide enough.


A 32-bit field would definitely not be able to cover a full address
space. So do you mind to explain what is the upper bound you expect here?


Do you foresee a need for buffer sizes of 4Tb and up?


Not I am aware of... However, I think the question was worth it given 
that "wide enough" can mean anything.


Cheers,

--
Julien Grall

[linux-linus test] 151690: regressions - FAIL

[osstest service owner]

flight 151690 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151690/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-xsm 16 guest-start/debian.repeat fail REGR. vs. 151214
 build-i386-pvops  6 kernel-build fail REGR. vs. 151214
 test-armhf-armhf-xl-arndale   7 xen-boot fail REGR. vs. 151214

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 17 guest-start.2fail REGR. vs. 151214

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-examine   1 build-check(1)   blocked  n/a
 test-amd64-coresched-i386-xl  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked 
n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 151214
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 151214
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 151214
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 151214
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 151214
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 151214
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 tes

Re: [PATCH] trivial: Remove trailing whitespaces

[David Gibson]

On Mon, Jul 06, 2020 at 06:23:00PM +0200, Christophe de Dinechin wrote:
> There are a number of unnecessary trailing whitespaces that have
> accumulated over time in the source code. They cause stray changes
> in patches if you use tools that automatically remove them.
> 
> Tested by doing a `git diff -w` after the change.
> 
> This could probably be turned into a pre-commit hook.
> 
> Signed-off-by: Christophe de Dinechin 

ppc parts

Acked-by: David Gibson 

> ---
>  block/iscsi.c |   2 +-
>  disas/cris.c  |   2 +-
>  disas/microblaze.c|  80 +++---
>  disas/nios2.c | 256 +-
>  hmp-commands.hx   |   2 +-
>  hw/alpha/typhoon.c|   6 +-
>  hw/arm/gumstix.c  |   6 +-
>  hw/arm/omap1.c|   2 +-
>  hw/arm/stellaris.c|   2 +-
>  hw/char/etraxfs_ser.c |   2 +-
>  hw/core/ptimer.c  |   2 +-
>  hw/cris/axis_dev88.c  |   2 +-
>  hw/cris/boot.c|   2 +-
>  hw/display/qxl.c  |   2 +-
>  hw/dma/etraxfs_dma.c  |  18 +-
>  hw/dma/i82374.c   |   2 +-
>  hw/i2c/bitbang_i2c.c  |   2 +-
>  hw/input/tsc2005.c|   2 +-
>  hw/input/tsc210x.c|   2 +-
>  hw/intc/etraxfs_pic.c |   8 +-
>  hw/intc/sh_intc.c |  10 +-
>  hw/intc/xilinx_intc.c |   2 +-
>  hw/misc/imx25_ccm.c   |   6 +-
>  hw/misc/imx31_ccm.c   |   2 +-
>  hw/net/vmxnet3.h  |   2 +-
>  hw/net/xilinx_ethlite.c   |   2 +-
>  hw/pci/pcie.c |   2 +-
>  hw/sd/omap_mmc.c  |   2 +-
>  hw/sh4/shix.c |   2 +-
>  hw/sparc64/sun4u.c|   2 +-
>  hw/timer/etraxfs_timer.c  |   2 +-
>  hw/timer/xilinx_timer.c   |   4 +-
>  hw/usb/hcd-musb.c |  10 +-
>  hw/usb/hcd-ohci.c |   6 +-
>  hw/usb/hcd-uhci.c |   2 +-
>  hw/virtio/virtio-pci.c|   2 +-
>  include/hw/cris/etraxfs_dma.h |   4 +-
>  include/hw/net/lance.h|   2 +-
>  include/hw/ppc/spapr.h|   2 +-
>  include/hw/xen/interface/io/ring.h|  34 +--
>  include/qemu/log.h|   2 +-
>  include/qom/object.h  |   4 +-
>  linux-user/cris/cpu_loop.c|  16 +-
>  linux-user/microblaze/cpu_loop.c  |  16 +-
>  linux-user/mmap.c |   8 +-
>  linux-user/sparc/signal.c |   4 +-
>  linux-user/syscall.c  |  24 +-
>  linux-user/syscall_defs.h |   2 +-
>  linux-user/uaccess.c  |   2 +-
>  os-posix.c|   2 +-
>  qapi/qapi-util.c  |   2 +-
>  qemu-img.c|   2 +-
>  qemu-options.hx   |  26 +-
>  qom/object.c  |   2 +-
>  target/cris/translate.c   |  28 +-
>  target/cris/translate_v10.inc.c   |   6 +-
>  target/i386/hvf/hvf.c |   4 +-
>  target/i386/hvf/x86.c |   4 +-
>  target/i386/hvf/x86_decode.c  |  20 +-
>  target/i386/hvf/x86_decode.h  |   4 +-
>  target/i386/hvf/x86_descr.c   |   2 +-
>  target/i386/hvf/x86_emu.c |   2 +-
>  target/i386/hvf/x86_mmu.c |   6 +-
>  target/i386/hvf/x86_task.c|   2 +-
>  target/i386/hvf/x86hvf.c  |  42 +--
>  target/i386/translate.c   |   8 +-
>  target/microblaze/mmu.c   |   2 +-
>  target/microblaze/translate.c |   2 +-
>  target/sh4/op_helper.c|   4 +-
>  target/xtensa/core-de212/core-isa.h   |   6 +-
>  .../xtensa/core-sample_controller/core-isa.h  |   6 +-
>  target/xtensa/core-test_kc705_be/core-isa.h   |   2 +-
>  tcg/sparc/tcg-target.inc.c|   2 +-
>  tcg/tcg.c |  32 +--
>  tests/tcg/multiarch/test-mmap.c   |  72 ++---
>  ui/curses.c   |   4 +-
>  ui/curses_keys.h  |   4 +-
>  util/cutils.c

Re: [BUG] Xen build for RCAR failing

[Bertrand Marquis]

Hi,

> On 7 Jul 2020, at 10:28, Manikandan Chockalingam (RBEI/ECF3) 
>  wrote:
> 
> Hello Bertrand,
> 
> Thanks for your quick response. I tired switching to dunfell branch and build 
> gives parse error as below.
> 
> bitbake core-image-weston
> ERROR: ParseError in 
> /home/manikandan/yocto_2.19/build/meta-virtualization/classes/: not a BitBake 
> file
> 
> Is there any additional changes required here?

I do not have this on my side when building using dunfell.
You might need to restart from a fresh build and checkout (you need dunfell 
branch on all layers).

Bertrand

Re: [PATCH v2 1/3] xen/privcmd: Corrected error handling path

[Jürgen Groß]

On 06.07.20 20:16, Souptick Joarder wrote:

Previously, if lock_pages() end up partially mapping pages, it used
to return -ERRNO due to which unlock_pages() have to go through
each pages[i] till *nr_pages* to validate them. This can be avoided
by passing correct number of partially mapped pages & -ERRNO separately,
while returning from lock_pages() due to error.

With this fix unlock_pages() doesn't need to validate pages[i] till
*nr_pages* for error scenario and few condition checks can be ignored.

Signed-off-by: Souptick Joarder 
Cc: John Hubbard 
Cc: Boris Ostrovsky 
Cc: Paul Durrant 
---
  drivers/xen/privcmd.c | 31 +++
  1 file changed, 15 insertions(+), 16 deletions(-)

diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index a250d11..33677ea 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -580,13 +580,13 @@ static long privcmd_ioctl_mmap_batch(
  
  static int lock_pages(

struct privcmd_dm_op_buf kbufs[], unsigned int num,
-   struct page *pages[], unsigned int nr_pages)
+   struct page *pages[], unsigned int nr_pages, unsigned int *pinned)
  {
unsigned int i;
+   int page_count = 0;


Initial value shouldn't be needed, and ...

  
  	for (i = 0; i < num; i++) {

unsigned int requested;
-   int pinned;


... you could move the declaration here.

With that done you can add my

Reviewed-by: Juergen Gross 


Juergen

[PATCH] x86emul: avoid assembler warning about .type not taking effect in test harness

[Jan Beulich]

gcc 9.3 started to re-order top level blocks by default when optimizing.
This re-ordering results in all our .type directives to get emitted to
the assembly file first, followed by gcc's. The assembler warns about
attempts to change the type of a symbol when it was already set (and
when there's no intervening setting to "notype").

Signed-off-by: Jan Beulich 

--- a/tools/tests/x86_emulator/Makefile
+++ b/tools/tests/x86_emulator/Makefile
@@ -295,4 +295,9 @@ x86-emulate.o cpuid.o test_x86_emulator.
 x86-emulate.o: x86_emulate/x86_emulate.c
 x86-emulate.o: HOSTCFLAGS += -D__XEN_TOOLS__
 
+# In order for our custom .type assembler directives to reliably land after
+# gcc's, we need to keep it from re-ordering top-level constructs.
+$(call cc-option-add,HOSTCFLAGS-toplevel,HOSTCC,-fno-toplevel-reorder)
+test_x86_emulator.o: HOSTCFLAGS += $(HOSTCFLAGS-toplevel)
+
 test_x86_emulator.o: $(addsuffix .h,$(TESTCASES)) $(addsuffix 
-opmask.h,$(OPMASK))

Re: [PATCH v2 2/3] xen/privcmd: Mark pages as dirty

[Jürgen Groß]

On 06.07.20 20:16, Souptick Joarder wrote:

pages need to be marked as dirty before unpinned it in
unlock_pages() which was oversight. This is fixed now.

Signed-off-by: Souptick Joarder 
Suggested-by: John Hubbard 
Cc: John Hubbard 
Cc: Boris Ostrovsky 
Cc: Paul Durrant 
---
  drivers/xen/privcmd.c | 5 -
  1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 33677ea..f6c1543 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -612,8 +612,11 @@ static void unlock_pages(struct page *pages[], unsigned 
int nr_pages)
  {
unsigned int i;
  
-	for (i = 0; i < nr_pages; i++)

+   for (i = 0; i < nr_pages; i++) {
+   if (!PageDirty(pages[i]))
+   set_page_dirty_lock(pages[i]);


With put_page() directly following I think you should be able to use
set_page_dirty() instead, as there is obviously a reference to the page
existing.


put_page(pages[i]);
+   }
  }
  
  static long privcmd_ioctl_dm_op(struct file *file, void __user *udata)




Juergen

Re: [PATCH] trivial: Remove trailing whitespaces

[Laurent Vivier]

Le 06/07/2020 à 18:23, Christophe de Dinechin a écrit :
> There are a number of unnecessary trailing whitespaces that have
> accumulated over time in the source code. They cause stray changes
> in patches if you use tools that automatically remove them.
> 
> Tested by doing a `git diff -w` after the change.
> 
> This could probably be turned into a pre-commit hook.
> 
> Signed-off-by: Christophe de Dinechin 
> ---
>  block/iscsi.c |   2 +-
>  disas/cris.c  |   2 +-
>  disas/microblaze.c|  80 +++---
>  disas/nios2.c | 256 +-
>  hmp-commands.hx   |   2 +-
>  hw/alpha/typhoon.c|   6 +-
>  hw/arm/gumstix.c  |   6 +-
>  hw/arm/omap1.c|   2 +-
>  hw/arm/stellaris.c|   2 +-
>  hw/char/etraxfs_ser.c |   2 +-
>  hw/core/ptimer.c  |   2 +-
>  hw/cris/axis_dev88.c  |   2 +-
>  hw/cris/boot.c|   2 +-
>  hw/display/qxl.c  |   2 +-
>  hw/dma/etraxfs_dma.c  |  18 +-
>  hw/dma/i82374.c   |   2 +-
>  hw/i2c/bitbang_i2c.c  |   2 +-
>  hw/input/tsc2005.c|   2 +-
>  hw/input/tsc210x.c|   2 +-
>  hw/intc/etraxfs_pic.c |   8 +-
>  hw/intc/sh_intc.c |  10 +-
>  hw/intc/xilinx_intc.c |   2 +-
>  hw/misc/imx25_ccm.c   |   6 +-
>  hw/misc/imx31_ccm.c   |   2 +-
>  hw/net/vmxnet3.h  |   2 +-
>  hw/net/xilinx_ethlite.c   |   2 +-
>  hw/pci/pcie.c |   2 +-
>  hw/sd/omap_mmc.c  |   2 +-
>  hw/sh4/shix.c |   2 +-
>  hw/sparc64/sun4u.c|   2 +-
>  hw/timer/etraxfs_timer.c  |   2 +-
>  hw/timer/xilinx_timer.c   |   4 +-
>  hw/usb/hcd-musb.c |  10 +-
>  hw/usb/hcd-ohci.c |   6 +-
>  hw/usb/hcd-uhci.c |   2 +-
>  hw/virtio/virtio-pci.c|   2 +-
>  include/hw/cris/etraxfs_dma.h |   4 +-
>  include/hw/net/lance.h|   2 +-
>  include/hw/ppc/spapr.h|   2 +-
>  include/hw/xen/interface/io/ring.h|  34 +--
>  include/qemu/log.h|   2 +-
>  include/qom/object.h  |   4 +-
>  linux-user/cris/cpu_loop.c|  16 +-
>  linux-user/microblaze/cpu_loop.c  |  16 +-
>  linux-user/mmap.c |   8 +-
>  linux-user/sparc/signal.c |   4 +-
>  linux-user/syscall.c  |  24 +-
>  linux-user/syscall_defs.h |   2 +-
>  linux-user/uaccess.c  |   2 +-
>  os-posix.c|   2 +-
>  qapi/qapi-util.c  |   2 +-
>  qemu-img.c|   2 +-
>  qemu-options.hx   |  26 +-
>  qom/object.c  |   2 +-
>  target/cris/translate.c   |  28 +-
>  target/cris/translate_v10.inc.c   |   6 +-
>  target/i386/hvf/hvf.c |   4 +-
>  target/i386/hvf/x86.c |   4 +-
>  target/i386/hvf/x86_decode.c  |  20 +-
>  target/i386/hvf/x86_decode.h  |   4 +-
>  target/i386/hvf/x86_descr.c   |   2 +-
>  target/i386/hvf/x86_emu.c |   2 +-
>  target/i386/hvf/x86_mmu.c |   6 +-
>  target/i386/hvf/x86_task.c|   2 +-
>  target/i386/hvf/x86hvf.c  |  42 +--
>  target/i386/translate.c   |   8 +-
>  target/microblaze/mmu.c   |   2 +-
>  target/microblaze/translate.c |   2 +-
>  target/sh4/op_helper.c|   4 +-
>  target/xtensa/core-de212/core-isa.h   |   6 +-
>  .../xtensa/core-sample_controller/core-isa.h  |   6 +-
>  target/xtensa/core-test_kc705_be/core-isa.h   |   2 +-
>  tcg/sparc/tcg-target.inc.c|   2 +-
>  tcg/tcg.c |  32 +--
>  tests/tcg/multiarch/test-mmap.c   |  72 ++---
>  ui/curses.c   |   4 +-
>  ui/curses_keys.h  |   4 +-
>  util/cutils.c |   2 +-
>  78 files changed, 440 insert

Re: [PATCH] trivial: Remove trailing whitespaces

[Daniel P . Berrangé]

On Mon, Jul 06, 2020 at 06:23:00PM +0200, Christophe de Dinechin wrote:
> There are a number of unnecessary trailing whitespaces that have
> accumulated over time in the source code. They cause stray changes
> in patches if you use tools that automatically remove them.
> 
> Tested by doing a `git diff -w` after the change.
> 
> This could probably be turned into a pre-commit hook.

scripts/checkpatch.pl ought to be made to check it.

> 
> Signed-off-by: Christophe de Dinechin 
> ---
>  block/iscsi.c |   2 +-
>  disas/cris.c  |   2 +-
>  disas/microblaze.c|  80 +++---
>  disas/nios2.c | 256 +-
>  hmp-commands.hx   |   2 +-
>  hw/alpha/typhoon.c|   6 +-
>  hw/arm/gumstix.c  |   6 +-
>  hw/arm/omap1.c|   2 +-
>  hw/arm/stellaris.c|   2 +-
>  hw/char/etraxfs_ser.c |   2 +-
>  hw/core/ptimer.c  |   2 +-
>  hw/cris/axis_dev88.c  |   2 +-
>  hw/cris/boot.c|   2 +-
>  hw/display/qxl.c  |   2 +-
>  hw/dma/etraxfs_dma.c  |  18 +-
>  hw/dma/i82374.c   |   2 +-
>  hw/i2c/bitbang_i2c.c  |   2 +-
>  hw/input/tsc2005.c|   2 +-
>  hw/input/tsc210x.c|   2 +-
>  hw/intc/etraxfs_pic.c |   8 +-
>  hw/intc/sh_intc.c |  10 +-
>  hw/intc/xilinx_intc.c |   2 +-
>  hw/misc/imx25_ccm.c   |   6 +-
>  hw/misc/imx31_ccm.c   |   2 +-
>  hw/net/vmxnet3.h  |   2 +-
>  hw/net/xilinx_ethlite.c   |   2 +-
>  hw/pci/pcie.c |   2 +-
>  hw/sd/omap_mmc.c  |   2 +-
>  hw/sh4/shix.c |   2 +-
>  hw/sparc64/sun4u.c|   2 +-
>  hw/timer/etraxfs_timer.c  |   2 +-
>  hw/timer/xilinx_timer.c   |   4 +-
>  hw/usb/hcd-musb.c |  10 +-
>  hw/usb/hcd-ohci.c |   6 +-
>  hw/usb/hcd-uhci.c |   2 +-
>  hw/virtio/virtio-pci.c|   2 +-
>  include/hw/cris/etraxfs_dma.h |   4 +-
>  include/hw/net/lance.h|   2 +-
>  include/hw/ppc/spapr.h|   2 +-
>  include/hw/xen/interface/io/ring.h|  34 +--
>  include/qemu/log.h|   2 +-
>  include/qom/object.h  |   4 +-
>  linux-user/cris/cpu_loop.c|  16 +-
>  linux-user/microblaze/cpu_loop.c  |  16 +-
>  linux-user/mmap.c |   8 +-
>  linux-user/sparc/signal.c |   4 +-
>  linux-user/syscall.c  |  24 +-
>  linux-user/syscall_defs.h |   2 +-
>  linux-user/uaccess.c  |   2 +-
>  os-posix.c|   2 +-
>  qapi/qapi-util.c  |   2 +-
>  qemu-img.c|   2 +-
>  qemu-options.hx   |  26 +-
>  qom/object.c  |   2 +-
>  target/cris/translate.c   |  28 +-
>  target/cris/translate_v10.inc.c   |   6 +-
>  target/i386/hvf/hvf.c |   4 +-
>  target/i386/hvf/x86.c |   4 +-
>  target/i386/hvf/x86_decode.c  |  20 +-
>  target/i386/hvf/x86_decode.h  |   4 +-
>  target/i386/hvf/x86_descr.c   |   2 +-
>  target/i386/hvf/x86_emu.c |   2 +-
>  target/i386/hvf/x86_mmu.c |   6 +-
>  target/i386/hvf/x86_task.c|   2 +-
>  target/i386/hvf/x86hvf.c  |  42 +--
>  target/i386/translate.c   |   8 +-
>  target/microblaze/mmu.c   |   2 +-
>  target/microblaze/translate.c |   2 +-
>  target/sh4/op_helper.c|   4 +-
>  target/xtensa/core-de212/core-isa.h   |   6 +-
>  .../xtensa/core-sample_controller/core-isa.h  |   6 +-
>  target/xtensa/core-test_kc705_be/core-isa.h   |   2 +-
>  tcg/sparc/tcg-target.inc.c|   2 +-
>  tcg/tcg.c |  32 +--
>  tests/tcg/multiarch/test-mmap.c   |  72 ++---
>  ui/curses.c   |   4 +-
>  ui/curses_keys.h  |   4 +-
>  util/cutils.c   

Re: [PATCH v4 03/10] tools/libxl: add vmtrace_pt_size parameter

[Michał Leszczyński]

- 7 lip 2020 o 11:16, Julien Grall jul...@xen.org napisał(a):

> On 07/07/2020 10:10, Jan Beulich wrote:
>> On 07.07.2020 10:44, Julien Grall wrote:
>>> Hi,
>>>
>>> On 06/07/2020 09:46, Jan Beulich wrote:
 On 04.07.2020 19:23, Julien Grall wrote:
> Hi,
>
> On 03/07/2020 11:11, Roger Pau Monné wrote:
>> On Fri, Jul 03, 2020 at 11:56:38AM +0200, Jan Beulich wrote:
>>> On 03.07.2020 11:44, Roger Pau Monné wrote:
 On Thu, Jul 02, 2020 at 06:23:28PM +0200, Michał Leszczyński wrote:
> In previous versions it was "size" but it was requested to change it
> to "order" in order to shrink the variable size from uint64_t to
> uint8_t, because there is limited space for xen_domctl_createdomain
> structure.

 It's likely I'm missing something here, but I wasn't aware
 xen_domctl_createdomain had any constrains regarding it's size. It's
 currently 48bytes which seems fairly small.
>>>
>>> Additionally I would guess a uint32_t could do here, if the value
>>> passed was "number of pages" rather than "number of bytes"?
> Looking at the rest of the code, the toolstack accepts a 64-bit value.
> So this would lead to truncation of the buffer if it is bigger than 2^44
> bytes.
>
> I agree such buffer is unlikely, yet I still think we want to harden the
> code whenever we can. So the solution is to either prevent check
> truncation in libxl or directly use 64-bit in the domctl.
>
> My preference is the latter.
>
>>
>> That could work, not sure if it needs to state however that those will
>> be 4K pages, since Arm can have a different minimum page size IIRC?
>> (or that's already the assumption for all number of frames fields)
>> vmtrace_nr_frames seems fine to me.
>
> The hypercalls interface is using the same page granularity as the
> hypervisor (i.e 4KB).
>
> While we already support guest using 64KB page granularity, it is
> impossible to have a 64KB Arm hypervisor in the current state. You are
> going to either break existing guest (if you switch to 64KB page
> granularity for the hypercall ABI) or render them insecure (the mimimum
> mapping in the P2M would be 64KB).
>
> DOMCTLs are not stable yet, so using a number of pages is OK. However, I
> would strongly suggest to use a number of bytes for any xl/libxl/stable
> libraries interfaces as this avoids confusion and also make more
> futureproof.

 If we can't settle on what "page size" means in the public interface
 (which imo is embarrassing), then how about going with number of kb,
 like other memory libxl controls do? (I guess using Mb, in line with
 other config file controls, may end up being too coarse here.) This
 would likely still allow for a 32-bit field to be wide enough.
>>>
>>> A 32-bit field would definitely not be able to cover a full address
>>> space. So do you mind to explain what is the upper bound you expect here?
>> 
>> Do you foresee a need for buffer sizes of 4Tb and up?
> 
> Not I am aware of... However, I think the question was worth it given
> that "wide enough" can mean anything.
> 
> Cheers,
> 
> --
> Julien Grall


So would it be OK to use uint32_t everywhere and to store the trace buffer
size as number of kB? I think this is the most straightforward option.

I would also stick with the name "processor_trace_buf_size"
everywhere, both in the hypervisor, ABI and the toolstack, with the
respective comments that the size is in kB.


Best regards,
Michał Leszczyński
CERT Polska

Re: [PATCH v4 03/10] tools/libxl: add vmtrace_pt_size parameter

[Jan Beulich]

On 07.07.2020 13:17, Michał Leszczyński wrote:
> So would it be OK to use uint32_t everywhere and to store the trace buffer
> size as number of kB? I think this is the most straightforward option.
> 
> I would also stick with the name "processor_trace_buf_size"
> everywhere, both in the hypervisor, ABI and the toolstack, with the
> respective comments that the size is in kB.

Perhaps even more clearly "processor_trace_buf_kb" then?

Jan

Re: [PATCH v2 2/3] xen/privcmd: Mark pages as dirty

[Souptick Joarder]

On Tue, Jul 7, 2020 at 3:08 PM Jürgen Groß  wrote:
>
> On 06.07.20 20:16, Souptick Joarder wrote:
> > pages need to be marked as dirty before unpinned it in
> > unlock_pages() which was oversight. This is fixed now.
> >
> > Signed-off-by: Souptick Joarder 
> > Suggested-by: John Hubbard 
> > Cc: John Hubbard 
> > Cc: Boris Ostrovsky 
> > Cc: Paul Durrant 
> > ---
> >   drivers/xen/privcmd.c | 5 -
> >   1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
> > index 33677ea..f6c1543 100644
> > --- a/drivers/xen/privcmd.c
> > +++ b/drivers/xen/privcmd.c
> > @@ -612,8 +612,11 @@ static void unlock_pages(struct page *pages[], 
> > unsigned int nr_pages)
> >   {
> >   unsigned int i;
> >
> > - for (i = 0; i < nr_pages; i++)
> > + for (i = 0; i < nr_pages; i++) {
> > + if (!PageDirty(pages[i]))
> > + set_page_dirty_lock(pages[i]);
>
> With put_page() directly following I think you should be able to use
> set_page_dirty() instead, as there is obviously a reference to the page
> existing.

Patch [3/3] will convert above codes to use unpin_user_pages_dirty_lock()
which internally do the same check. So I thought to keep linux-stable and
linux-next code in sync. John had a similar concern [1] and later agreed to keep
this check.

Shall I keep this check ?  No ?

[1] 
https://lore.kernel.org/xen-devel/a750e5e5-fd5d-663b-c5fd-261d7c939...@nvidia.com/

>
> >   put_page(pages[i]);
> > + }
> >   }
> >
> >   static long privcmd_ioctl_dm_op(struct file *file, void __user *udata)
> >
>
> Juergen

Re: [PATCH v4 03/10] tools/libxl: add vmtrace_pt_size parameter

[Michał Leszczyński]

- 7 lip 2020 o 13:21, Jan Beulich jbeul...@suse.com napisał(a):

> On 07.07.2020 13:17, Michał Leszczyński wrote:
>> So would it be OK to use uint32_t everywhere and to store the trace buffer
>> size as number of kB? I think this is the most straightforward option.
>> 
>> I would also stick with the name "processor_trace_buf_size"
>> everywhere, both in the hypervisor, ABI and the toolstack, with the
>> respective comments that the size is in kB.
> 
> Perhaps even more clearly "processor_trace_buf_kb" then?
> 
> Jan


Ok.

Best regards,
Michał Leszczyński
CERT Polska

Re: [PATCH v2 1/3] xen/privcmd: Corrected error handling path

[Souptick Joarder]

On Tue, Jul 7, 2020 at 3:05 PM Jürgen Groß  wrote:
>
> On 06.07.20 20:16, Souptick Joarder wrote:
> > Previously, if lock_pages() end up partially mapping pages, it used
> > to return -ERRNO due to which unlock_pages() have to go through
> > each pages[i] till *nr_pages* to validate them. This can be avoided
> > by passing correct number of partially mapped pages & -ERRNO separately,
> > while returning from lock_pages() due to error.
> >
> > With this fix unlock_pages() doesn't need to validate pages[i] till
> > *nr_pages* for error scenario and few condition checks can be ignored.
> >
> > Signed-off-by: Souptick Joarder 
> > Cc: John Hubbard 
> > Cc: Boris Ostrovsky 
> > Cc: Paul Durrant 
> > ---
> >   drivers/xen/privcmd.c | 31 +++
> >   1 file changed, 15 insertions(+), 16 deletions(-)
> >
> > diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
> > index a250d11..33677ea 100644
> > --- a/drivers/xen/privcmd.c
> > +++ b/drivers/xen/privcmd.c
> > @@ -580,13 +580,13 @@ static long privcmd_ioctl_mmap_batch(
> >
> >   static int lock_pages(
> >   struct privcmd_dm_op_buf kbufs[], unsigned int num,
> > - struct page *pages[], unsigned int nr_pages)
> > + struct page *pages[], unsigned int nr_pages, unsigned int *pinned)
> >   {
> >   unsigned int i;
> > + int page_count = 0;
>
> Initial value shouldn't be needed, and ...
>
> >
> >   for (i = 0; i < num; i++) {
> >   unsigned int requested;
> > - int pinned;
>
> ... you could move the declaration here.
>
> With that done you can add my
>
> Reviewed-by: Juergen Gross 

Ok. But does it going make any difference other than limiting scope ?

>
>
> Juergen

Re: [PATCH v2 2/3] xen/privcmd: Mark pages as dirty

[Jürgen Groß]

On 07.07.20 13:30, Souptick Joarder wrote:

On Tue, Jul 7, 2020 at 3:08 PM Jürgen Groß  wrote:


On 06.07.20 20:16, Souptick Joarder wrote:

pages need to be marked as dirty before unpinned it in
unlock_pages() which was oversight. This is fixed now.

Signed-off-by: Souptick Joarder 
Suggested-by: John Hubbard 
Cc: John Hubbard 
Cc: Boris Ostrovsky 
Cc: Paul Durrant 
---
   drivers/xen/privcmd.c | 5 -
   1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 33677ea..f6c1543 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -612,8 +612,11 @@ static void unlock_pages(struct page *pages[], unsigned 
int nr_pages)
   {
   unsigned int i;

- for (i = 0; i < nr_pages; i++)
+ for (i = 0; i < nr_pages; i++) {
+ if (!PageDirty(pages[i]))
+ set_page_dirty_lock(pages[i]);


With put_page() directly following I think you should be able to use
set_page_dirty() instead, as there is obviously a reference to the page
existing.


Patch [3/3] will convert above codes to use unpin_user_pages_dirty_lock()
which internally do the same check. So I thought to keep linux-stable and
linux-next code in sync. John had a similar concern [1] and later agreed to keep
this check.

Shall I keep this check ?  No ?

[1] 
https://lore.kernel.org/xen-devel/a750e5e5-fd5d-663b-c5fd-261d7c939...@nvidia.com/


I wasn't referring to checking PageDirty(), but to the use of
set_page_dirty_lock().

Looking at the comment just before the implementation of
set_page_dirty_lock() suggests that it is fine to use set_page_dirty()
instead (so not calling lock_page()).

Only the transition from get_user_pages_fast() to pin_user_pages_fast()
requires to use the locked version IMO.


Juergen

Re: [PATCH v2 1/3] xen/privcmd: Corrected error handling path

[Jürgen Groß]

On 07.07.20 13:40, Souptick Joarder wrote:

On Tue, Jul 7, 2020 at 3:05 PM Jürgen Groß  wrote:


On 06.07.20 20:16, Souptick Joarder wrote:

Previously, if lock_pages() end up partially mapping pages, it used
to return -ERRNO due to which unlock_pages() have to go through
each pages[i] till *nr_pages* to validate them. This can be avoided
by passing correct number of partially mapped pages & -ERRNO separately,
while returning from lock_pages() due to error.

With this fix unlock_pages() doesn't need to validate pages[i] till
*nr_pages* for error scenario and few condition checks can be ignored.

Signed-off-by: Souptick Joarder 
Cc: John Hubbard 
Cc: Boris Ostrovsky 
Cc: Paul Durrant 
---
   drivers/xen/privcmd.c | 31 +++
   1 file changed, 15 insertions(+), 16 deletions(-)

diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index a250d11..33677ea 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -580,13 +580,13 @@ static long privcmd_ioctl_mmap_batch(

   static int lock_pages(
   struct privcmd_dm_op_buf kbufs[], unsigned int num,
- struct page *pages[], unsigned int nr_pages)
+ struct page *pages[], unsigned int nr_pages, unsigned int *pinned)
   {
   unsigned int i;
+ int page_count = 0;


Initial value shouldn't be needed, and ...



   for (i = 0; i < num; i++) {
   unsigned int requested;
- int pinned;


... you could move the declaration here.

With that done you can add my

Reviewed-by: Juergen Gross 


Ok. But does it going make any difference other than limiting scope ?


Dropping the initializer surely does, and in the end page_count just
replaces the former pinned variable, so why would we want to widen the
scope with this patch?


Juergen

Re: [BUG] Xen build for RCAR failing

[Bertrand Marquis]

> On 7 Jul 2020, at 11:18, Manikandan Chockalingam (RBEI/ECF3) 
>  wrote:
> 
> Hello Bertrand,
> 
> Thank you. I will try a fresh build with dunfell branch. All layers in the 
> sense [poky, meta-openembedded, meta-linaro, meta-rensas, 
> meta-virtualisation, meta-selinux, xen-troops] right?

right

> 
> Also, Can I use the same proprietary drivers which I used for 
> yocto2.19[R-Car_Gen3_Series_Evaluation_Software_Package_for_Linux-20170427.zip]
>  for this branch?

I have no idea what is in that but i would guess it will probably not work that 
easily.
You might need to get in contact with Renesas to get more up-to-date 
instructions on how to build that.

Bertrand

Re: [PATCH v2 3/3] xen/privcmd: Convert get_user_pages*() to pin_user_pages*()

[Jürgen Groß]

On 06.07.20 20:16, Souptick Joarder wrote:

In 2019, we introduced pin_user_pages*() and now we are converting
get_user_pages*() to the new API as appropriate. [1] & [2] could
be referred for more information. This is case 5 as per document [1].

[1] Documentation/core-api/pin_user_pages.rst

[2] "Explicit pinning of user-space pages":
 https://lwn.net/Articles/807108/

Signed-off-by: Souptick Joarder 
Cc: John Hubbard 
Cc: Boris Ostrovsky 
Cc: Paul Durrant 


Reviewed-by: Juergen Gross 


Juergen

[BUG] Xen build for RCAR failing

[Manikandan Chockalingam (RBEI/ECF3)]

Hello Team,

I am trying to build xen hypervisor for RCAR and following the 
https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Salvator-X
 steps.

But am facing the following issues

1.  SRC_URI="http://v3.sk/~lkundrak/dev86/archive/Dev86src-${PV}.tar.gz in 
recipes-extended/dev86/dev86_0.16.20.bb is not accesible

Modification done: 
SRC_URI=https://src.fedoraproject.org/lookaside/extras/dev86/Dev86src-0.16.20.tar.gz/567cf460d132f9d8775dd95f9208e49a/Dev86src-${PV}.tar.gz

2.  LIC_FILES_CHKSUM changed in recipes-extended/xen/xen.inc

3.  QA Issue: xen: Files/directories were installed but not shipped in any 
package:

/usr/bin/vchan-socket-proxy

  /usr/sbin/xenmon

  /usr/sbin/xenhypfs

  /usr/lib/libxenfsimage.so.4.14.0

  /usr/lib/libxenhypfs.so.1

  /usr/lib/libxenfsimage.so

  /usr/lib/libxenhypfs.so.1.0

  /usr/lib/libxenfsimage.so.4.14

  /usr/lib/libxenhypfs.so

  /usr/lib/pkgconfig

  /usr/lib/xen/bin/depriv-fd-checker

  /usr/lib/pkgconfig/xenlight.pc

  /usr/lib/pkgconfig/xenguest.pc

  /usr/lib/pkgconfig/xenhypfs.pc

  /usr/lib/pkgconfig/xlutil.pc

  /usr/lib/pkgconfig/xentoolcore.pc

  /usr/lib/pkgconfig/xentoollog.pc

  /usr/lib/pkgconfig/xenstore.pc

  /usr/lib/pkgconfig/xencall.pc

  /usr/lib/pkgconfig/xencontrol.pc

  /usr/lib/pkgconfig/xendevicemodel.pc

  /usr/lib/pkgconfig/xenstat.pc

  /usr/lib/pkgconfig/xengnttab.pc

  /usr/lib/pkgconfig/xenevtchn.pc

  /usr/lib/pkgconfig/xenvchan.pc

  /usr/lib/pkgconfig/xenforeignmemory.pc

  /usr/lib/xenfsimage/fat/fsimage.so

  /usr/lib/xenfsimage/iso9660/fsimage.so

  /usr/lib/xenfsimage/reiserfs/fsimage.so

  /usr/lib/xenfsimage/ufs/fsimage.so

  /usr/lib/xenfsimage/ext2fs-lib/fsimage.so

  /usr/lib/xenfsimage/zfs/fsimage.so

Please set FILES such that these items are packaged. Alternatively if they are 
unneeded, avoid installing them or delete them within do_install.

xen: 32 installed and not shipped files. [installed-vs-shipped]

ERROR: xen-unstable+gitAUTOINC+be63d9d47f-r0 do_package: Fatal QA errors found, 
failing task.

ERROR: xen-unstable+gitAUTOINC+be63d9d47f-r0 do_package: Function failed: 
do_package

ERROR: Logfile of failure stored in: 
/home/manikandan/yocto_2.19/build/build/tmp/work/aarch64-poky-linux/xen/unstable+gitAUTOINC+be63d9d47f-r0/temp/log.do_package.17889

ERROR: Task 13 
(/home/manikandan/yocto_2.19/build/meta-virtualization/recipes-extended/xen/xen_git.bb,
 do_package) failed with exit code '1'

Can you please let me know is there any update needs to be done in the build 
steps mentioned in the link? If not, can you please let me know how I can 
overcome this issues and build a hypervisor image for RCAR?

Thanks in Advance. My build configuration  is as follows.

Build Configuration:
BB_VERSION= "1.30.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS= "aarch64-poky-linux"
MACHINE   = "salvator-x"
DISTRO= "poky"
DISTRO_VERSION= "2.1.2"
TUNE_FEATURES = "aarch64 cortexa57-cortexa53"
TARGET_FPU= ""
SOC_FAMILY= "rcar-gen3:r8a7795"
meta
meta-poky
meta-yocto-bsp= "tmp:cca8dd15c8096626052f6d8d25ff1e9a606104a3"
meta-rcar-gen3= "tmp:95cb48ba09bc7e55fd549817e3e26723409e68d5"
meta-linaro-toolchain
meta-optee= "tmp:2f51d38048599d9878f149d6d15539fb97603f8f"
meta-oe   = "tmp:55c8a76da5dc099a7bc3838495c672140cedb78e"
meta-virtualization = "morty:6249631f59ad6ee3dc93762de49fc4b443d99abc"
meta-selinux  = "jethro:4c75d9cbcf1d75043c7c5ab315aa383d9b227510"
meta-networking
meta-python   = "tmp:55c8a76da5dc099a7bc3838495c672140cedb78e"
meta-rcar-gen3-xen = "master:60699c631d541aeeaebaeec9a087efed9385ee42"


Mit freundlichen Grüßen / Best regards

Chockalingam Manikandan

ES-CM Core fn,ADIT (RBEI/ECF3)
Robert Bosch GmbH | Postfach 10 60 50 | 70049 Stuttgart | GERMANY | 
www.bosch.com
Tel. +91 80 6136-4452 | Fax +91 80 6617-0711 | 
manikandan.chockalin...@in.bosch.com

Registered Office: Stuttgart, Registration Court: Amtsgericht Stuttgart, HRB 
14000;
Chairman of the Supervisory Board: Franz Fehrenbach; Managing Directors: Dr. 
Volkmar Denner,
Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Christian Fischer, 
Dr. Stefan Hartung,
Dr. Markus Heyn, Harald Kröger, Christoph Kübel, Rolf Najork, Uwe Raschke, 
Peter Tyroller
​

Xen Security Advisory 317 v3 (CVE-2020-15566) - Incorrect error handling in event channel port allocation

[Xen . org security team]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Xen Security Advisory CVE-2020-15566 / XSA-317
   version 3

   Incorrect error handling in event channel port allocation

UPDATES IN VERSION 3


Public release.

ISSUE DESCRIPTION
=

The allocation of an event channel port may fail for multiple reasons:
1) Port is already in use
2) The memory allocation failed
3) The port we try to allocate is higher than what is supported by
   the ABI (e.g 2L or FIFO) used by the guest or the limit set by an
   administrator ('max_event_channels' in xl cfg).

Due to the missing error checks, only 1) will be considered as an error.  All
the other cases will provide a "valid" port and will result to a crash when
trying to access the event channel.

IMPACT
==

When the administrator configured a guest to allow more than 1023
event channels, that guest may be able to crash the host.

When Xen is out-of-memory, allocation of new event channels will
result in crashing the host rather than reporting an error.

VULNERABLE SYSTEMS
==

Xen versions 4.10 and later are affected.  (The special Xen 4.8
"Comet" branch for XSA-254 contains changes similar to those which led
to this vulnerability; so it is likely to be affected, but - like
mainline Xen 4.8 - that branch is longer security-supported.)

Older Xen versions are unaffected.

All architectures are affected.

The default configuration, when guests are created with xl/libxl, is
not vulnerable, because of the default event channel limit (see
Mitigation, below).

MITIGATION
==

The problem can be avoided by reducing the number of event channels
available to the guest no more than 1023.  For example, setting
"max_event_channels=1023" in the xl domain configuration, or deleting
any existing setting (since 1023 is the default for xl/libxl).

For ARM systems, any limit no more than 4095 is safe.

For 64-bit x86 PV guests, any limit no more than 4095 is likewise safe
if the host configuration prevents the guest administrator from
substituting and running a 32-bit kernel (and thereby putting the
guest into 32-bit PV mode).

CREDITS
===

This issue was discovered by Amazon.

RESOLUTION
==

Applying the attached patch resolves this issue.

Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball.  Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.

xsa317.patch   Xen 4.10 - xen-unstable

$ sha256sum xsa317*
11e77dd8644cee40cee609d02e27d70655f3999005cae8c24fb2801980ebb4f2  xsa317.meta
17908035e2da07f6070fa8de345db68c96ed9bd78f8b114e43ba0194c1be3f15  xsa317.patch
$

DEPLOYMENT DURING EMBARGO
=

Deployment of the *patch* described above (or others which are
substantially similar) is permitted during the embargo, even on
public-facing systems with untrusted guest users and administrators.

But: Distribution of updated software is prohibited (except to other
members of the predisclosure list).


And: deployment of the event channel limit reduction mitigation is NOT
permitted (except where all the affected systems and VMs are
administered and used only by organisations which are members of the
Xen Project Security Issues Predisclosure List).  Specifically,
deployment on public cloud systems is NOT permitted.

This is because such a change can be visible to the guest, so it would
leak the preconditions for the vulnerability and maybe lead to
rediscovery.

Deployment of this, or similar mitigations, is permitted only AFTER
the embargo ends.


Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.

(Note: this during-embargo deployment notice is retained in
post-embargo publicly released Xen Project advisories, even though it
is then no longer applicable.  This is to enable the community to have
oversight of the Xen Project Security Team's decisionmaking.)

For more information about permissible uses of embargoed information,
consult the Xen Project community's agreed Security Policy:
  http://www.xenproject.org/security-policy.html
-BEGIN PGP SIGNATURE-

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAl8EZ/gMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZQUwIAK8W8bZ0xml2bzAu4vsXi8QqhDX4VrpkgADYZS+M
BD8hpllQ+O/CiM5ZMECj7zaWYTt7+VrGrqK4jtf2REBs/sOWcO+k7KdEury4XCKf
jIG4CzCBHC46RVEKftiqQNTX2ebVBDwoj+1fGeIvm7OhcZ7f6KdhYPHvE2bU8D45
ghr2jw33HZHoG7IsPQvJn8u6wqd6l+7h0BxhgzO5U8pI+w3ZXRM4XAno+ERzs8LO
N5ffv8UeaMIpkHoYEdsKOK/ItjhoCASoWTFvbE90u7f2WbimFnBG3oCPEVPt89kv
Y/o0+0jBk+WjXbPChMmMu5WuQuKVFDelMXLLE6mjfhGAvnI=
=vEgE
-END PGP SIGNATURE-


xsa317.meta
Description: Binary data


xsa317.patch
Description: Binary data

Xen Security Advisory 319 v3 (CVE-2020-15563) - inverted code paths in x86 dirty VRAM tracking

[Xen . org security team]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Xen Security Advisory CVE-2020-15563 / XSA-319
   version 3

inverted code paths in x86 dirty VRAM tracking

UPDATES IN VERSION 3


Public release.

ISSUE DESCRIPTION
=

An inverted conditional in x86 HVM guests' dirty video RAM tracking
code allows such guests to make Xen de-reference a pointer guaranteed
to point at unmapped space.

IMPACT
==

A malicious or buggy HVM guest may cause the hypervisor to crash,
resulting in Denial of Service (DoS) affecting the entire host.

VULNERABLE SYSTEMS
==

Xen versions from 4.8 onwards are affected.  Xen versions 4.7 and
earlier are not affected.

Only x86 systems are affected.  Arm systems are not affected.

Only x86 HVM guests using shadow paging can leverage the vulnerability.
In addition there needs to be an entity actively monitoring a guest's
video frame buffer (typically for display purposes) in order for such a
guest to be able to leverage the vulnerability.  x86 PV guests as well
as x86 HVM guest using hardware assisted paging (HAP) cannot leverage
the vulnerability.

MITIGATION
==

Running only PV guests will avoid the vulnerability.

For HVM guest explicitly configured to use shadow paging (e.g. via the
`hap=0' xl domain configuration file parameter), changing to HAP (e.g.
by setting `hap=1') will avoid exposing the vulnerability to those
guests.  HAP is the default (in upstream Xen), where the hardware
supports it; so this mitigation is only applicable if HAP has been
disabled by configuration.

CREDITS
===

This issue was discovered by Jan Beulich of SUSE.

RESOLUTION
==

Applying the attached patch resolves this issue.

Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball.  Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.

xsa319.patch   xen-unstable, 4.13 - 4.9

$ sha256sum xsa319*
1fe0dc2e274776b8e1275f85129280f280f94ca4eabe6a8166113283dad93ed8  xsa319.meta
c145f394f8ac7d8838c376a97e1850c4125c12e478fc66ebe025ae397b27e6ea  xsa319.patch
$

DEPLOYMENT DURING EMBARGO
=

Deployment of the patch described above (or others which are
substantially similar) is permitted during the embargo, even on
public-facing systems with untrusted guest users and administrators.

HOWEVER deployment of the "use HAP mode" mitigation described above is
NOT permitted (except where all the affected systems and VMs are
administered and used only by organisations which are members of the Xen
Project Security Issues Predisclosure List).  Specifically, deployment
on public cloud systems is NOT permitted.

This is because in that case the configuration change can be observed
by guests, which could lead to the rediscovery of the vulnerability.

But: Distribution of updated software is prohibited (except to other
members of the predisclosure list).

Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.


(Note: this during-embargo deployment notice is retained in
post-embargo publicly released Xen Project advisories, even though it
is then no longer applicable.  This is to enable the community to have
oversight of the Xen Project Security Team's decisionmaking.)

For more information about permissible uses of embargoed information,
consult the Xen Project community's agreed Security Policy:
  http://www.xenproject.org/security-policy.html
-BEGIN PGP SIGNATURE-

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAl8EZ/sMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZ75YH/jX/sAs0icOgBtHkwVZHg318OBExxt9x+ehk/pxb
i+1ZlS/IrJ8eJdHJYq8HYvAlxmtmFP1I0t+C9vmwbP4QMcR++RmKgdJI4+/sqCsB
AMEnK+cVJSbHxD7y7eW2CPuU3h0cKx0H24JgtzA2ONse7dVz7RN+oa97D5IKryTL
cBW8WroMn2InbKMCUy/5zj89NLAlbSuWSVZzQidDwzTITukzhZZ7Xw0+Q2yh1nkK
S4kcmz7Bzzd5Mc1gFr1Eh1FxfmVVl5RxwDE//3a5VbmfPVo/f0kMOIWjXVd1R1dj
x78SPrPojOAZbb8+f1LYqHmqzCgzvpa4EFbsOnsB7CBmP2Q=
=bDFh
-END PGP SIGNATURE-


xsa319.meta
Description: Binary data


xsa319.patch
Description: Binary data

Xen Security Advisory 328 v3 (CVE-2020-15567) - non-atomic modification of live EPT PTE

[Xen . org security team]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Xen Security Advisory CVE-2020-15567 / XSA-328
   version 3

non-atomic modification of live EPT PTE

UPDATES IN VERSION 3


Public release.

ISSUE DESCRIPTION
=

When mapping guest EPT (nested paging) tables, Xen would in some
circumstances use a series of non-atomic bitfield writes.

Depending on the compiler version and optimisation flags, Xen might
expose a dangerous partially-written PTE to the hardware, which an
attacker might be able to race to exploit.

IMPACT
==

A guest administrator or perhaps even unprivileged guest user might
be able to cause denial of service, data corruption, or privilege
escalation.

VULNERABLE SYSTEMS
==

Only systems using Intel CPUs are vulnerable.  Sytems using AMD CPUs,
and Arm systems, are not vulnerable.

Only systems using nested paging ("hap", aka nested paging, aka in
this case Intel EPT) are vulnerable.

Only HVM and PVH guests can exploit the vulnerability.

The presence and scope of the vulnerability depends on the precise
optimisations performed by the compiler used to build Xen.  If the
compiler generates (a) a single 64-bit write, or (b) a series of
read-modify-write operations which are in the same order as the source
code, the hypervisor is not vulnerable.

For example, in one test build with gcc 8.3 with normal settings, the
compiler generated multiple (unlocked) read-modify-write operations in
source code order, which did *not* constitute a vulnerability.

We have not been able to survey compilers; consequently we cannot say
which compiler(s) might produce vulnerable code (with which code
generation options).  The code clearly violates the C rules.  So we
have chosen to issue this advisory.

MITIGATION
==

Running only PV guests will avoid this vulnerability.

Switching to shadow paging (e.g. using the "hap=0" xl domain domain
configuration file parameter) will avoid exposing the vulnerability to
those guests.

Manual inspection of the generated assembly code might allow a
suitably qualified person to say that a particular build is not
vulnerable.

There is no less broad mitigation.

CREDITS
===

This issue was discovered by Jan Beulich of SUSE.

For patch 1:
Reviewed-by: Roger Pau Monné 

For patch 2:
From: Roger Pau Monné 
Reported-by: Jan Beulich 
Signed-off-by: Roger Pau Monné 

RESOLUTION
==

Applying the appropriate pair of attached patches resolves this issue.

Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball.  Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.

xsa328/xsa328-?.patchxen-unstable
xsa328/xsa328-4.13-?.patch   Xen 4.13.x
xsa328/xsa328-4.12-?.patch   Xen 4.12.x
xsa328/xsa328-4.11-?.patch   Xen 4.11.x, Xen 4.10.x
xsa328/xsa328-4.9-?.patchXen 4.9.x

$ sha256sum xsa328* xsa328*/*
61ceb3d039c3ebb06f480a17593b367b01e7c1e5cc3669d77caecb704fbc7071  xsa328.meta
cae53f7e6c46fe245790036279bc50eaa10e4271790e871ad8a7d446629b2e12  
xsa328/xsa328-1.patch
d61354a992869451cd7a3c92254672b5e253d1a994135cf9b4a5c784be0a07ef  
xsa328/xsa328-2.patch
018412fba6f153c1d6b03fc2fa6f3ac381060efe6a8651404462028d24c830a8  
xsa328/xsa328-4.9-1.patch
f3deb26e0ce27c385ab16065a0ba67b86a228afd949c0a6a78b9d48366fc2554  
xsa328/xsa328-4.9-2.patch
a600ecef784485e8608cd4549f756ffa24705747a4d876147f9ba64fff118580  
xsa328/xsa328-4.11-1.patch
f3deb26e0ce27c385ab16065a0ba67b86a228afd949c0a6a78b9d48366fc2554  
xsa328/xsa328-4.11-2.patch
d608921359e561f9c594c9f8f7ee02432518a229ecea638d472ab91227d705ec  
xsa328/xsa328-4.12-1.patch
a51162c019e7e6ed394faa7d40c932456059b7b76a784dc7886dd0a47c43da0b  
xsa328/xsa328-4.12-2.patch
51a41fae885aed40839887da473e0c8ab4c4d897a121f5fac2cc3c6c0188d6d2  
xsa328/xsa328-4.13-1.patch
a51162c019e7e6ed394faa7d40c932456059b7b76a784dc7886dd0a47c43da0b  
xsa328/xsa328-4.13-2.patch
$

DEPLOYMENT DURING EMBARGO
=

Deployment of the patches and/or mitigations described above (or
others which are substantially similar) is permitted during the
embargo, even on public-facing systems with untrusted guest users and
administrators.

But: Distribution of updated software is prohibited (except to other
members of the predisclosure list).

Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.

(Note: this during-embargo deployment notice is retained in
post-embargo publicly released Xen Project advisories, even though it
is then no longer applicable.  This is to enable the community to have
oversight of the Xen Project Security Team's decisionmaking.)

For more information about permissible uses of embargoed information,
consult the Xen Project community's agreed Security Policy:
  http://www.xenproject.org/s

Xen Security Advisory 327 v3 (CVE-2020-15564) - Missing alignment check in VCPUOP_register_vcpu_info

[Xen . org security team]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Xen Security Advisory CVE-2020-15564 / XSA-327
   version 3

 Missing alignment check in VCPUOP_register_vcpu_info

UPDATES IN VERSION 3


Public release.

ISSUE DESCRIPTION
=

The hypercall VCPUOP_register_vcpu_info is used by a guest to register
a shared region with the hypervisor. The region will be mapped into Xen address
space so it can be directly accessed.

On Arm, the region is accessed with instructions which require a specific
alignment. Unfortunately, there is no check that the address provided by
the guest will be correctly aligned.

As a result, a malicious guest could cause a hypervisor crash by passing
a misaligned address.

IMPACT
==

A malicious guest administrator may cause a hypervisor crash, resulting in a
Denial of Service (DoS).

VULNERABLE SYSTEMS
==

All Xen versions are vulnerable.

Only Arm systems are vulnerable.  x86 systems are not affected.

MITIGATION
==

There is no mitigation.

CREDITS
===

This issue was discovered by Julien Grall of Amazon.

RESOLUTION
==

Applying the attached patch resolves this issue.

Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball.  Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.

xsa327.patch   Xen 4.9 - xen-unstable

$ sha256sum xsa327*
f046eefcc1368708bd1fafc88e063d3dbc5c4cdb593d68b3b04917c6cdb7bcb5  xsa327.meta
1d057695d5b74ce2857204103e943caeaf773bc4fb9d91ea78016e01a9147ed7  xsa327.patch
$

DEPLOYMENT DURING EMBARGO
=

Deployment of the patch and/or mitigations described above (or
others which are substantially similar) is permitted during the
embargo, even on public-facing systems with untrusted guest users and
administrators.

But: Distribution of updated software is prohibited (except to other
members of the predisclosure list).

Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.

(Note: this during-embargo deployment notice is retained in
post-embargo publicly released Xen Project advisories, even though it
is then no longer applicable.  This is to enable the community to have
oversight of the Xen Project Security Team's decisionmaking.)

For more information about permissible uses of embargoed information,
consult the Xen Project community's agreed Security Policy:
  http://www.xenproject.org/security-policy.html
-BEGIN PGP SIGNATURE-

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAl8EaVAMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZcqIIAKpb992pMq1jFStIGPhk6HsaIhxVEGep67eJHq9d
TMaFiyBix125djY0zV8KaznmZmRpM2pNKVsIkGe1XHgtEMcWgMAYARejJLRC4UnW
xHhpunI7rJMQc1vL5ZGxAFbVYF6U/PX0rwESwQb2/Rt0eLBTAmH4m25TQiSEnrkM
3C4Dbk3puCbaeB7VGiyccK07hh6qQhEO8s1FhZTNVTaqqcNWZYqy/SbmRYHiT/in
2dK6XOiBgRhHnjsDDoXj5abSMb00KnJ9PkWu8RC2b7+BVZJUii1557T8zpDo9Fyl
CJ3YXrekd+gQSFxgwCts00BbLr2NUf3uqEtpY1EEV7UKmvQ=
=fPiG
-END PGP SIGNATURE-


xsa327.meta
Description: Binary data


xsa327.patch
Description: Binary data

[xen-unstable test] 151696: tolerable FAIL - PUSHED

[osstest service owner]

flight 151696 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151696/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 151661
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 151661
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 151661
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 151661
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 151661
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 151661
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 151661
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 151661
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 151661
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 151661
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f97f99c8d88ebc108f6adc3ba74e87d53ba57c70
baseline version:
 xen  be63d9d47f571a60d70f8fb630c03871312d9655

Last test of basis   151661  2020-07-06 01:54:10 Z1 days
Failing since151684  2020-07-06 16:36:21 Z0 days2 attempts
Testing same since   151696  2020-07-07 03:11:56 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Anthony PERARD 
  Jan Beulich 
  Wei Liu 

jobs:
 build-amd64-xsm  pass
 

[qemu-mainline bisection] complete test-amd64-amd64-xl-qemuu-debianhvm-amd64

[osstest service owner]

branch xen-unstable
xenbranch xen-unstable
job test-amd64-amd64-xl-qemuu-debianhvm-amd64
testid debian-hvm-install

Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://git.qemu.org/qemu.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  qemuu git://git.qemu.org/qemu.git
  Bug introduced:  81cb05732efb36971901c515b007869cc1d3a532
  Bug not present: d6b78ac8ecf94f56dbfbecc23fb4365d8772a41a
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/151709/


  commit 81cb05732efb36971901c515b007869cc1d3a532
  Author: Markus Armbruster 
  Date:   Tue Jun 9 14:23:37 2020 +0200
  
  qdev: Assert devices are plugged into a bus that can take them
  
  This would have caught some of the bugs I just fixed.
  
  Signed-off-by: Markus Armbruster 
  Reviewed-by: Mark Cave-Ayland 
  Message-Id: <20200609122339.937862-23-arm...@redhat.com>


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/qemu-mainline/test-amd64-amd64-xl-qemuu-debianhvm-amd64.debian-hvm-install.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/qemu-mainline/test-amd64-amd64-xl-qemuu-debianhvm-amd64.debian-hvm-install
 --summary-out=tmp/151709.bisection-summary --basis-template=151065 
--blessings=real,real-bisect qemu-mainline 
test-amd64-amd64-xl-qemuu-debianhvm-amd64 debian-hvm-install
Searching for failure / basis pass:
 151685 fail [host=huxelrebe0] / 151149 [host=albana1] 151101 [host=elbling0] 
151065 [host=godello1] 151047 [host=chardonnay1] 150970 [host=elbling1] 150930 
[host=godello0] 150916 [host=chardonnay0] 150909 [host=fiano1] 150899 
[host=fiano0] 150895 [host=debina1] 150831 [host=huxelrebe1] 150694 
[host=pinot1] 150631 [host=italia0] 150608 [host=albana0] 150593 ok.
Failure / basis pass flights: 151685 / 150593
(tree with no url: minios)
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://git.qemu.org/qemu.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git
Latest c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
627d1d6693b0594d257dbe1a3363a8d4bd4d8307 
3c659044118e34603161457db9934a34f816d78b 
eb6490f544388dd24c0d054a96dd304bc7284450 
88ab0c15525ced2eefe39220742efe4769089ad8 
be63d9d47f571a60d70f8fb630c03871312d9655
Basis pass c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
568eee7cf319fa95183c8d3b5e8dcf6e078ab8b3 
3c659044118e34603161457db9934a34f816d78b 
4ec2a1f53e8aaa22924614b64dde97321126943e 
2e3de6253422112ae43e608661ba94ea6b345694 
1497e78068421d83956f8e82fb6e1bf1fc3b1199
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/linux-pvops.git#c3038e718a19fc596f7b1baba0f83d5146dc7784-c3038e718a19fc596f7b1baba0f83d5146dc7784
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/osstest/ovmf.git#568eee7cf319fa95183c8d3b5e8dcf6e078ab8b3-627d1d6693b0594d257dbe1a3363a8d4bd4d8307
 git://xenbits.xen.org/qemu-xen-traditional.git#3c659044118e34603161457db99\
 34a34f816d78b-3c659044118e34603161457db9934a34f816d78b 
git://git.qemu.org/qemu.git#4ec2a1f53e8aaa22924614b64dde97321126943e-eb6490f544388dd24c0d054a96dd304bc7284450
 
git://xenbits.xen.org/osstest/seabios.git#2e3de6253422112ae43e608661ba94ea6b345694-88ab0c15525ced2eefe39220742efe4769089ad8
 
git://xenbits.xen.org/xen.git#1497e78068421d83956f8e82fb6e1bf1fc3b1199-be63d9d47f571a60d70f8fb630c03871312d9655
>From git://cache:9419/git://xenbits.xen.org/xen
   be63d9d47f..f97f99c8d8  master -> origin/master
   f97f99c8d8..3fdc211b01  staging-> origin/staging
Use of uninitialized value $parents in array dereference at 
./adhoc-revtuple-generator line 465.
Use of uninitialized value in concatenation (.) or string at 
./adhoc-revtuple-generator line 465.
Use of uninitialized value $parents in array dereference at 
./adhoc-revtuple-generator line 465.
Use of uninitialized value in concatenation (.) or string at 
./adhoc-revtuple-generator line 465.
Use of uninitialized value $parents in array dereference at 
./adhoc-revtuple-generator line 465.
Use of uninitialized value in concatenation (.) or string at 
./adhoc-revtuple-generator line 465.
Loaded 43179 nodes in revision graph
Searching for test results:
 150585 [host=elbling0]
 1

RE: [BUG] Xen build for RCAR failing

[Manikandan Chockalingam (RBEI/ECF3)]

Hello Oleksandr Andrushchenko,

Thanks for your quick response. I am using the yocto version[yocto_2.19] 
mentioned in the link. Still I face the issue.

Mit freundlichen Grüßen / Best regards

 Chockalingam Manikandan

ES-CM Core fn,ADIT (RBEI/ECF3)
Robert Bosch GmbH | Postfach 10 60 50 | 70049 Stuttgart | GERMANY | 
www.bosch.com
Tel. +91 80 6136-4452 | Fax +91 80 6617-0711 | 
manikandan.chockalin...@in.bosch.com

Registered Office: Stuttgart, Registration Court: Amtsgericht Stuttgart, HRB 
14000;
Chairman of the Supervisory Board: Franz Fehrenbach; Managing Directors: Dr. 
Volkmar Denner, 
Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Christian Fischer, 
Dr. Stefan Hartung,
Dr. Markus Heyn, Harald Kröger, Christoph Kübel, Rolf Najork, Uwe Raschke, 
Peter Tyroller

-Original Message-
From: Oleksandr Andrushchenko  
Sent: Tuesday, July 7, 2020 1:42 PM
To: Manikandan Chockalingam (RBEI/ECF3) ; 
xen-de...@lists.xen.org
Subject: Re: [BUG] Xen build for RCAR failing


On 7/7/20 10:58 AM, Manikandan Chockalingam (RBEI/ECF3) wrote:
>
> Hello Team,
>
> I am trying to build xen hypervisor for RCAR and following the 
> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Salvator-X
>  steps.
>
> But am facing the following issues
>
> 1.SRC_URI="http://v3.sk/~lkundrak/dev86/archive/Dev86src-${PV}.tar.gz in 
> recipes-extended/dev86/dev86_0.16.20.bb is not accesible
>
> *Modification 
> done:*SRC_URI=https://src.fedoraproject.org/lookaside/extras/dev86/Dev86src-0.16.20.tar.gz/567cf460d132f9d8775dd95f9208e49a/Dev86src-${PV}.tar.gz
>  
> 
>
You can try what we use [1]. And the issue you are facing is rather Yocto 
related, not R-Car specific, IMO

[1] 
https://github.com/xen-troops/meta-xt-prod-devel/blob/master/recipes-domd/domd-image-weston/files/meta-xt-prod-extra/recipes-extended/dev86/dev86_%25.bbappend

RE: [BUG] Xen build for RCAR failing

[Manikandan Chockalingam (RBEI/ECF3)]

Hello Bertrand,

Thanks for your quick response. I tired switching to dunfell branch and build 
gives parse error as below.

bitbake core-image-weston
ERROR: ParseError in 
/home/manikandan/yocto_2.19/build/meta-virtualization/classes/: not a BitBake 
file

Is there any additional changes required here?

Mit freundlichen Grüßen / Best regards

 Chockalingam Manikandan

ES-CM Core fn,ADIT (RBEI/ECF3)
Robert Bosch GmbH | Postfach 10 60 50 | 70049 Stuttgart | GERMANY | 
www.bosch.com
Tel. +91 80 6136-4452 | Fax +91 80 6617-0711 | 
manikandan.chockalin...@in.bosch.com

Registered Office: Stuttgart, Registration Court: Amtsgericht Stuttgart, HRB 
14000;
Chairman of the Supervisory Board: Franz Fehrenbach; Managing Directors: Dr. 
Volkmar Denner, 
Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Christian Fischer, 
Dr. Stefan Hartung,
Dr. Markus Heyn, Harald Kröger, Christoph Kübel, Rolf Najork, Uwe Raschke, 
Peter Tyroller


-Original Message-
From: Bertrand Marquis  
Sent: Tuesday, July 7, 2020 2:27 PM
To: Manikandan Chockalingam (RBEI/ECF3) 
Cc: xen-de...@lists.xen.org; nd 
Subject: Re: [BUG] Xen build for RCAR failing



> On 7 Jul 2020, at 08:58, Manikandan Chockalingam (RBEI/ECF3) 
>  wrote:
> 
> Hello Team,
>  
> I am trying to build xen hypervisor for RCAR and following the 
> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Salvator-X
>  steps.
>  
> But am facing the following issues
> 1.  SRC_URI="http://v3.sk/~lkundrak/dev86/archive/Dev86src-${PV}.tar.gz 
> in recipes-extended/dev86/dev86_0.16.20.bb is not accesible
> Modification done: 
> SRC_URI=https://src.fedoraproject.org/lookaside/extras/dev86/Dev86src-0.16.20.tar.gz/567cf460d132f9d8775dd95f9208e49a/Dev86src-${PV}.tar.gz
> 2.  LIC_FILES_CHKSUM changed in recipes-extended/xen/xen.inc
> 3.  QA Issue: xen: Files/directories were installed but not shipped in 
> any package:
> /usr/bin/vchan-socket-proxy
>   /usr/sbin/xenmon
>   /usr/sbin/xenhypfs
>   /usr/lib/libxenfsimage.so.4.14.0
>   /usr/lib/libxenhypfs.so.1
>   /usr/lib/libxenfsimage.so
>   /usr/lib/libxenhypfs.so.1.0
>   /usr/lib/libxenfsimage.so.4.14
>   /usr/lib/libxenhypfs.so
>   /usr/lib/pkgconfig
>   /usr/lib/xen/bin/depriv-fd-checker
>   /usr/lib/pkgconfig/xenlight.pc
>   /usr/lib/pkgconfig/xenguest.pc
>   /usr/lib/pkgconfig/xenhypfs.pc
>   /usr/lib/pkgconfig/xlutil.pc
>   /usr/lib/pkgconfig/xentoolcore.pc
>   /usr/lib/pkgconfig/xentoollog.pc
>   /usr/lib/pkgconfig/xenstore.pc
>   /usr/lib/pkgconfig/xencall.pc
>   /usr/lib/pkgconfig/xencontrol.pc
>   /usr/lib/pkgconfig/xendevicemodel.pc
>   /usr/lib/pkgconfig/xenstat.pc
>   /usr/lib/pkgconfig/xengnttab.pc
>   /usr/lib/pkgconfig/xenevtchn.pc
>   /usr/lib/pkgconfig/xenvchan.pc
>   /usr/lib/pkgconfig/xenforeignmemory.pc
>   /usr/lib/xenfsimage/fat/fsimage.so
>   /usr/lib/xenfsimage/iso9660/fsimage.so
>   /usr/lib/xenfsimage/reiserfs/fsimage.so
>   /usr/lib/xenfsimage/ufs/fsimage.so
>   /usr/lib/xenfsimage/ext2fs-lib/fsimage.so
>   /usr/lib/xenfsimage/zfs/fsimage.so
> Please set FILES such that these items are packaged. Alternatively if they 
> are unneeded, avoid installing them or delete them within do_install.
> xen: 32 installed and not shipped files. [installed-vs-shipped]
> ERROR: xen-unstable+gitAUTOINC+be63d9d47f-r0 do_package: Fatal QA errors 
> found, failing task.
> ERROR: xen-unstable+gitAUTOINC+be63d9d47f-r0 do_package: Function failed: 
> do_package
> ERROR: Logfile of failure stored in: 
> /home/manikandan/yocto_2.19/build/build/tmp/work/aarch64-poky-linux/xen/unstable+gitAUTOINC+be63d9d47f-r0/temp/log.do_package.17889
> ERROR: Task 13 
> (/home/manikandan/yocto_2.19/build/meta-virtualization/recipes-extended/xen/xen_git.bb,
>  do_package) failed with exit code '1'

The configuration from the page is using a rather old release of Yocto.
I would suggest to switch to dunfell which will use xen 4.12.

Current master status of Yocto is not building at the moment.
Christopher Clark has done some work on it here [1] in meta-virtualization but 
this is not merged yet.

If you are trying to build Xen master by modifying a recipe you will have 
issues as some things have been added like hypfs which are creating the issues 
you see when Yocto is checking that everything was installed.

Bertrand

[1] https://lists.yoctoproject.org/g/meta-virtualization/message/5495

[libvirt test] 151698: regressions - FAIL

[osstest service owner]

flight 151698 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151698/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-libvirt-pair 22 guest-migrate/src_host/dst_host fail REGR. 
vs. 151496
 test-amd64-i386-libvirt-pair 22 guest-migrate/src_host/dst_host fail REGR. vs. 
151496

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 151496
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 151496
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-libvirt-qcow2 12 migrate-support-checkfail never pass
 test-arm64-arm64-libvirt-qcow2 13 saverestore-support-checkfail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt 14 saverestore-support-checkfail   never pass

version targeted for testing:
 libvirt  852ee1950aee5f31c9656b30c5fe9124f734c38c
baseline version:
 libvirt  e7998ebeaf15e4e8825be0dd97aa1316f194f00d

Last test of basis   151496  2020-07-01 04:23:43 Z6 days
Failing since151527  2020-07-02 04:29:15 Z5 days6 attempts
Testing same since   151665  2020-07-06 04:18:46 Z1 days2 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Daniel Henrique Barboza 
  Daniel P. Berrangé 
  Daniel Veillard 
  Laine Stump 
  Michal Privoznik 
  Yanqiu Zhang 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-arm64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-arm64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-arm64-arm64-libvirt-xsm pass
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-arm64-arm64-libvirt pass
 test-armhf-armhf-libvirt pass
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairfail
 test-amd64-i386-libvirt-pair fail
 test-arm64-arm64-libvirt-qcow2   pass
 test-armhf-armhf-libvirt-raw pass
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness 

Re: [PATCH] x86emul: avoid assembler warning about .type not taking effect in test harness

[Jan Beulich]

On 07.07.2020 11:35, Jan Beulich wrote:
> gcc 9.3 started to re-order top level blocks by default when optimizing.
> This re-ordering results in all our .type directives to get emitted to
> the assembly file first, followed by gcc's. The assembler warns about
> attempts to change the type of a symbol when it was already set (and
> when there's no intervening setting to "notype").

Turns out this wasn't a gcc change - the problem had been there all the
time, it just went through silently. It was the newer gas that I built
gcc 9.3 with that caused to issue to become visible. I've slightly
updated the description to account for this.

Jan

RE: [BUG] Xen build for RCAR failing

[Manikandan Chockalingam (RBEI/ECF3)]

Hello Bertrand,

Thank you. I will try a fresh build with dunfell branch. All layers in the 
sense [poky, meta-openembedded, meta-linaro, meta-rensas, meta-virtualisation, 
meta-selinux, xen-troops] right?

Also, Can I use the same proprietary drivers which I used for 
yocto2.19[R-Car_Gen3_Series_Evaluation_Software_Package_for_Linux-20170427.zip] 
for this branch?

Mit freundlichen Grüßen / Best regards

 Chockalingam Manikandan

ES-CM Core fn,ADIT (RBEI/ECF3)
Robert Bosch GmbH | Postfach 10 60 50 | 70049 Stuttgart | GERMANY | 
www.bosch.com
Tel. +91 80 6136-4452 | Fax +91 80 6617-0711 | 
manikandan.chockalin...@in.bosch.com

Registered Office: Stuttgart, Registration Court: Amtsgericht Stuttgart, HRB 
14000;
Chairman of the Supervisory Board: Franz Fehrenbach; Managing Directors: Dr. 
Volkmar Denner, 
Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Christian Fischer, 
Dr. Stefan Hartung,
Dr. Markus Heyn, Harald Kröger, Christoph Kübel, Rolf Najork, Uwe Raschke, 
Peter Tyroller


-Original Message-
From: Bertrand Marquis  
Sent: Tuesday, July 7, 2020 3:03 PM
To: Manikandan Chockalingam (RBEI/ECF3) 
Cc: xen-de...@lists.xen.org; nd 
Subject: Re: [BUG] Xen build for RCAR failing

Hi,

> On 7 Jul 2020, at 10:28, Manikandan Chockalingam (RBEI/ECF3) 
>  wrote:
> 
> Hello Bertrand,
> 
> Thanks for your quick response. I tired switching to dunfell branch and build 
> gives parse error as below.
> 
> bitbake core-image-weston
> ERROR: ParseError in 
> /home/manikandan/yocto_2.19/build/meta-virtualization/classes/: not a BitBake 
> file
> 
> Is there any additional changes required here?

I do not have this on my side when building using dunfell.
You might need to restart from a fresh build and checkout (you need dunfell 
branch on all layers).

Bertrand

[xtf test] 151707: all pass - PUSHED

[osstest service owner]

flight 151707 xtf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151707/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 xtf  2dd14fbcf9d03fdc300491939aeac75d3eb9e05f
baseline version:
 xtf  2a8859e87761a0efc119778e094f203dc2ea487a

Last test of basis   150870  2020-06-05 20:09:44 Z   31 days
Testing same since   151707  2020-07-07 10:39:37 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 

jobs:
 build-amd64-xtf  pass
 build-amd64  pass
 build-amd64-pvopspass
 test-xtf-amd64-amd64-1   pass
 test-xtf-amd64-amd64-2   pass
 test-xtf-amd64-amd64-3   pass
 test-xtf-amd64-amd64-4   pass
 test-xtf-amd64-amd64-5   pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xtf.git
   2a8859e..2dd14fb  2dd14fbcf9d03fdc300491939aeac75d3eb9e05f -> 
xen-tested-master

[xen-unstable-smoke test] 151711: tolerable all pass - PUSHED

[osstest service owner]

flight 151711 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151711/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  3fdc211b01b29f252166937238efe02d15cb5780
baseline version:
 xen  f97f99c8d88ebc108f6adc3ba74e87d53ba57c70

Last test of basis   151687  2020-07-06 19:01:28 Z0 days
Testing same since   151711  2020-07-07 13:13:16 Z0 days1 attempts


People who touched revisions under test:
  Jan Beulich 
  Julien Grall 
  Roger Pau Monné 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f97f99c8d8..3fdc211b01  3fdc211b01b29f252166937238efe02d15cb5780 -> smoke

[PATCH v12 0/8] error: auto propagated local_err part I

[Markus Armbruster]

To speed things up, I'm taking the liberty to respin Vladimir's series
with my documentation amendments.

After my documentation work, I'm very much inclined to rename
ERRP_AUTO_PROPAGATE() to ERRP_GUARD().  The fact that it propagates
below the hood is detail.  What matters to its users is that it lets
them use @errp more freely.  Thoughts?

Based-on: Message-Id: <20200707160613.848843-1-arm...@redhat.com>

Available from my public repository https://repo.or.cz/qemu/armbru.git
on branch error-auto.

v12: (based on "[PATCH v4 00/45] Less clumsy error checking")
01: Comments merged properly with recent commit "error: Document Error
API usage rules", and edited for clarity.  Put ERRP_AUTO_PROPAGATE()
before its helpers, and touch up style.
01-08: Commit messages tweaked

Vladimir's cover letter for v11:

v11: (based-on "[PATCH v2 00/44] Less clumsy error checking")
01: minor rebase of documentation, keep r-bs
02: - minor comment tweaks [Markus]
- use explicit file name in MAINTAINERS instead of pattern
- add Markus's r-b
03,07,08: rabase changes, drop r-bs


v11 is available at
 https://src.openvz.org/scm/~vsementsov/qemu.git #tag 
up-auto-local-err-partI-v11
v10 is available at
 https://src.openvz.org/scm/~vsementsov/qemu.git #tag 
up-auto-local-err-partI-v10

In these series, there is no commit-per-subsystem script, each generated
commit is generated in separate.

Still, generating commands are very similar, and looks like

sed -n '/^$/,/^$/{s/^F: //p}' MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80

Note, that in each generated commit, generation command is the only
text, indented by 8 spaces in 'git log -1' output, so, to regenerate all
commits (for example, after rebase, or change in coccinelle script), you
may use the following command:

git rebase -x "sh -c \"git show --pretty= --name-only | xargs git checkout 
HEAD^ -- ; git reset; git log -1 | grep '^' | sh\"" HEAD~6

Which will start automated interactive rebase for generated patches,
which will stop if generated patch changed
(you may do git commit --amend to apply updated generated changes).

Note:
  git show --pretty= --name-only   - lists files, changed in HEAD
  git log -1 | grep '^' | sh   - rerun generation command of HEAD


Check for compilation of changed .c files
git rebase -x "sh -c \"git show --pretty= --name-only | sed -n 's/\.c$/.o/p' | 
xargs make -j9\"" HEAD~6

Vladimir Sementsov-Ogievskiy (8):
  error: New macro ERRP_AUTO_PROPAGATE()
  scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
  sd: Use ERRP_AUTO_PROPAGATE()
  pflash: Use ERRP_AUTO_PROPAGATE()
  fw_cfg: Use ERRP_AUTO_PROPAGATE()
  virtio-9p: Use ERRP_AUTO_PROPAGATE()
  nbd: Use ERRP_AUTO_PROPAGATE()
  xen: Use ERRP_AUTO_PROPAGATE()

 scripts/coccinelle/auto-propagated-errp.cocci | 337 ++
 include/block/nbd.h   |   1 +
 include/qapi/error.h  | 163 -
 block/nbd.c   |   7 +-
 hw/9pfs/9p-local.c|  12 +-
 hw/9pfs/9p.c  |   1 +
 hw/block/dataplane/xen-block.c|  17 +-
 hw/block/pflash_cfi01.c   |   7 +-
 hw/block/pflash_cfi02.c   |   7 +-
 hw/block/xen-block.c  | 102 +++---
 hw/nvram/fw_cfg.c |  14 +-
 hw/pci-host/xen_igd_pt.c  |   7 +-
 hw/sd/sdhci-pci.c |   7 +-
 hw/sd/sdhci.c |  21 +-
 hw/sd/ssi-sd.c|  10 +-
 hw/xen/xen-backend.c  |   7 +-
 hw/xen/xen-bus.c  |  92 ++---
 hw/xen/xen-host-pci-device.c  |  27 +-
 hw/xen/xen_pt.c   |  25 +-
 hw/xen/xen_pt_config_init.c   |  17 +-
 nbd/client.c  |   5 +
 nbd/server.c  |   5 +
 MAINTAINERS   |   1 +
 23 files changed, 659 insertions(+), 233 deletions(-)
 create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

-- 
2.26.2

[PATCH v12 5/8] fw_cfg: Use ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

From: Vladimir Sementsov-Ogievskiy 

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  Fix such cases.

If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

This commit is generated by command

sed -n '/^Firmware configuration (fw_cfg)$/,/^$/{s/^F: //p}' \
MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf 
Reported-by: Greg Kurz 
Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Philippe Mathieu-Daudé 
[Commit message tweaked]
Signed-off-by: Markus Armbruster 
---
 hw/nvram/fw_cfg.c | 14 ++
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/hw/nvram/fw_cfg.c b/hw/nvram/fw_cfg.c
index 0408a31f8e..d5386c3235 100644
--- a/hw/nvram/fw_cfg.c
+++ b/hw/nvram/fw_cfg.c
@@ -1231,12 +1231,11 @@ static Property fw_cfg_io_properties[] = {
 
 static void fw_cfg_io_realize(DeviceState *dev, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 FWCfgIoState *s = FW_CFG_IO(dev);
-Error *local_err = NULL;
 
-fw_cfg_file_slots_allocate(FW_CFG(s), &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+fw_cfg_file_slots_allocate(FW_CFG(s), errp);
+if (*errp) {
 return;
 }
 
@@ -1282,14 +1281,13 @@ static Property fw_cfg_mem_properties[] = {
 
 static void fw_cfg_mem_realize(DeviceState *dev, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 FWCfgMemState *s = FW_CFG_MEM(dev);
 SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
 const MemoryRegionOps *data_ops = &fw_cfg_data_mem_ops;
-Error *local_err = NULL;
 
-fw_cfg_file_slots_allocate(FW_CFG(s), &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+fw_cfg_file_slots_allocate(FW_CFG(s), errp);
+if (*errp) {
 return;
 }
 
-- 
2.26.2

[PATCH v12 6/8] virtio-9p: Use ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

From: Vladimir Sementsov-Ogievskiy 

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  Fix such cases.

If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

This commit is generated by command

sed -n '/^virtio-9p$/,/^$/{s/^F: //p}' MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf 
Reported-by: Greg Kurz 
Signed-off-by: Vladimir Sementsov-Ogievskiy 
Acked-by: Greg Kurz 
Reviewed-by: Christian Schoenebeck 
[Commit message tweaked]
Signed-off-by: Markus Armbruster 
---
 hw/9pfs/9p-local.c | 12 +---
 hw/9pfs/9p.c   |  1 +
 2 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/hw/9pfs/9p-local.c b/hw/9pfs/9p-local.c
index 54e012e5b4..0361e0c0b4 100644
--- a/hw/9pfs/9p-local.c
+++ b/hw/9pfs/9p-local.c
@@ -1479,10 +1479,10 @@ static void error_append_security_model_hint(Error 
*const *errp)
 
 static int local_parse_opts(QemuOpts *opts, FsDriverEntry *fse, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 const char *sec_model = qemu_opt_get(opts, "security_model");
 const char *path = qemu_opt_get(opts, "path");
 const char *multidevs = qemu_opt_get(opts, "multidevs");
-Error *local_err = NULL;
 
 if (!sec_model) {
 error_setg(errp, "security_model property not set");
@@ -1516,11 +1516,10 @@ static int local_parse_opts(QemuOpts *opts, 
FsDriverEntry *fse, Error **errp)
 fse->export_flags &= ~V9FS_FORBID_MULTIDEVS;
 fse->export_flags &= ~V9FS_REMAP_INODES;
 } else {
-error_setg(&local_err, "invalid multidevs property '%s'",
+error_setg(errp, "invalid multidevs property '%s'",
multidevs);
-error_append_hint(&local_err, "Valid options are: multidevs="
+error_append_hint(errp, "Valid options are: multidevs="
   "[remap|forbid|warn]\n");
-error_propagate(errp, local_err);
 return -1;
 }
 }
@@ -1530,9 +1529,8 @@ static int local_parse_opts(QemuOpts *opts, FsDriverEntry 
*fse, Error **errp)
 return -1;
 }
 
-if (fsdev_throttle_parse_opts(opts, &fse->fst, &local_err)) {
-error_propagate_prepend(errp, local_err,
-"invalid throttle configuration: ");
+if (fsdev_throttle_parse_opts(opts, &fse->fst, errp)) {
+error_prepend(errp, "invalid throttle configuration: ");
 return -1;
 }
 
diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
index 9755fba9a9..bdb1360482 100644
--- a/hw/9pfs/9p.c
+++ b/hw/9pfs/9p.c
@@ -4011,6 +4011,7 @@ void pdu_submit(V9fsPDU *pdu, P9MsgHeader *hdr)
 int v9fs_device_realize_common(V9fsState *s, const V9fsTransport *t,
Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 int i, len;
 struct stat stat;
 FsDriverEntry *fse;
-- 
2.26.2

[PATCH v12 1/8] error: New macro ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

From: Vladimir Sementsov-Ogievskiy 

Introduce a new ERRP_AUTO_PROPAGATE macro, to be used at start of
functions with an errp OUT parameter.

It has three goals:

1. Fix issue with error_fatal and error_prepend/error_append_hint: user
can't see this additional information, because exit() happens in
error_setg earlier than information is added. [Reported by Greg Kurz]

2. Fix issue with error_abort and error_propagate: when we wrap
error_abort by local_err+error_propagate, the resulting coredump will
refer to error_propagate and not to the place where error happened.
(the macro itself doesn't fix the issue, but it allows us to [3.] drop
the local_err+error_propagate pattern, which will definitely fix the
issue) [Reported by Kevin Wolf]

3. Drop local_err+error_propagate pattern, which is used to workaround
void functions with errp parameter, when caller wants to know resulting
status. (Note: actually these functions could be merely updated to
return int error code).

To achieve these goals, later patches will add invocations
of this macro at the start of functions with either use
error_prepend/error_append_hint (solving 1) or which use
local_err+error_propagate to check errors, switching those
functions to use *errp instead (solving 2 and 3).

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Paul Durrant 
Reviewed-by: Greg Kurz 
Reviewed-by: Eric Blake 
[Comments merged properly with recent commit "error: Document Error
API usage rules", and edited for clarity.  Put ERRP_AUTO_PROPAGATE()
before its helpers, and touch up style.  Commit message tweaked.]
Signed-off-by: Markus Armbruster 
---
 include/qapi/error.h | 160 ++-
 1 file changed, 141 insertions(+), 19 deletions(-)

diff --git a/include/qapi/error.h b/include/qapi/error.h
index 3fed49747d..c865a7d2f1 100644
--- a/include/qapi/error.h
+++ b/include/qapi/error.h
@@ -30,6 +30,10 @@
  *   job.  Since the value of @errp is about handling the error, the
  *   function should not examine it.
  *
+ * - The function may pass @errp to functions it calls to pass on
+ *   their errors to its caller.  If it dereferences @errp to check
+ *   for errors, it must use ERRP_AUTO_PROPAGATE().
+ *
  * - On success, the function should not touch *errp.  On failure, it
  *   should set a new error, e.g. with error_setg(errp, ...), or
  *   propagate an existing one, e.g. with error_propagate(errp, ...).
@@ -45,15 +49,17 @@
  * = Creating errors =
  *
  * Create an error:
- * error_setg(&err, "situation normal, all fouled up");
+ * error_setg(errp, "situation normal, all fouled up");
+ * where @errp points to the location to receive the error.
  *
  * Create an error and add additional explanation:
- * error_setg(&err, "invalid quark");
- * error_append_hint(&err, "Valid quarks are up, down, strange, "
+ * error_setg(errp, "invalid quark");
+ * error_append_hint(errp, "Valid quarks are up, down, strange, "
  *   "charm, top, bottom.\n");
+ * This may require use of ERRP_AUTO_PROPAGATE(); more on that below.
  *
  * Do *not* contract this to
- * error_setg(&err, "invalid quark\n" // WRONG!
+ * error_setg(errp, "invalid quark\n" // WRONG!
  *"Valid quarks are up, down, strange, charm, top, bottom.");
  *
  * = Reporting and destroying errors =
@@ -107,18 +113,6 @@
  * Errors get passed to the caller through the conventional @errp
  * parameter.
  *
- * Pass an existing error to the caller:
- * error_propagate(errp, err);
- * where Error **errp is a parameter, by convention the last one.
- *
- * Pass an existing error to the caller with the message modified:
- * error_propagate_prepend(errp, err,
- * "Could not frobnicate '%s': ", name);
- * This is more concise than
- * error_propagate(errp, err); // don't do this
- * error_prepend(errp, "Could not frobnicate '%s': ", name);
- * and works even when @errp is &error_fatal.
- *
  * Create a new error and pass it to the caller:
  * error_setg(errp, "situation normal, all fouled up");
  *
@@ -128,18 +122,26 @@
  * handle the error...
  * }
  * when it doesn't, say a void function:
+ * ERRP_AUTO_PROPAGATE();
+ * foo(arg, errp);
+ * if (*errp) {
+ * handle the error...
+ * }
+ * More on ERRP_AUTO_PROPAGATE() below.
+ *
+ * Code predating ERRP_AUTO_PROPAGATE() still exits, and looks like this:
  * Error *err = NULL;
  * foo(arg, &err);
  * if (err) {
  * handle the error...
- * error_propagate(errp, err);
+ * error_propagate(errp, err); // deprecated
  * }
- * Do *not* "optimize" this to
+ * Avoid in new code.  Do *not* "optimize" it to
  * foo(arg, errp);
  * if (*errp) { // WRONG!
  * handle the error...
  * }
- * because errp may be NULL!
+ * because errp may be NULL!  Guard with ERRP_AUTO_PROPAGATE().
  *
  * But when all you do with the error is pass it on, please use
  *

[PATCH v12 2/8] scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

From: Vladimir Sementsov-Ogievskiy 

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
does corresponding changes in code (look for details in
include/qapi/error.h)

Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
 --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
 --max-width 80 FILES...

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Markus Armbruster 
Signed-off-by: Markus Armbruster 
---
 scripts/coccinelle/auto-propagated-errp.cocci | 337 ++
 include/qapi/error.h  |   3 +
 MAINTAINERS   |   1 +
 3 files changed, 341 insertions(+)
 create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

diff --git a/scripts/coccinelle/auto-propagated-errp.cocci 
b/scripts/coccinelle/auto-propagated-errp.cocci
new file mode 100644
index 00..c29f695adf
--- /dev/null
+++ b/scripts/coccinelle/auto-propagated-errp.cocci
@@ -0,0 +1,337 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or
+// modify it under the terms of the GNU General Public License as
+// published by the Free Software Foundation; either version 2 of the
+// License, or (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see
+// .
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place \
+//  --no-show-diff --max-width 80 FILES...
+//
+// Note: --max-width 80 is needed because coccinelle default is less
+// than 80, and without this parameter coccinelle may reindent some
+// lines which fit into 80 characters but not to coccinelle default,
+// which in turn produces extra patch hunks for no reason.
+
+// Switch unusual Error ** parameter names to errp
+// (this is necessary to use ERRP_AUTO_PROPAGATE).
+//
+// Disable optional_qualifier to skip functions with
+// "Error *const *errp" parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, because
+// that signals unusual semantics, and the parameter name may well
+// serve a purpose. (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate() and
+// error_propagate_prepend().
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+-   Error **_errp
++   Error **errp
+,...)
+ {
+(
+ ... when != assert(_errp && *_errp)
+&
+ <...
+-_errp
++errp
+ ...>
+)
+ }
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where
+// necessary
+//
+// Note, that without "when any" the final "..." does not mach
+// something matched by previous pattern, i.e. the rule will not match
+// double error_prepend in control flow like in
+// vfio_set_irq_signaling().
+//
+// Note, "exists" says that we want apply rule even if it does not
+// match on all possible control flows (otherwise, it will not match
+// standard pattern when error_propagate() call is in if branch).
+@ disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+
+ fn(..., Error **errp, ...)
+ {
++   ERRP_AUTO_PROPAGATE();
+...  when != ERRP_AUTO_PROPAGATE();
+(
+(
+error_append_hint(errp, ...);
+|
+error_prepend(errp, ...);
+|
+error_vprepend(errp, ...);
+)
+... when any
+|
+Error *local_err = NULL;
+...
+(
+error_propagate_prepend(errp, local_err, ...);
+|
+error_propagate(errp, local_err);
+)
+...
+)
+ }
+
+// Warn when several Error * definitions are in the control flow.
+// This rule is not chained to rule1 and less restrictive, to cover more
+// functions to warn (even those we are not going to convert).
+//
+// Note, that even with one (or zero) Error * definition in the each
+// control flow we may have several (in total) Error * definitions in
+// the function. This case deserves attention too, but I don't see
+// simple way to match with help of coccinelle.
+@check1 disable optional_qualifier exists@
+identifier fn, _errp, local_err, local_err2;
+position p1, p2;
+@@
+
+ fn(..., Error **_errp, ...)
+ {
+ ...
+ Error *local_err = NULL;@p1
+ ... when any
+ Error *local_err2 = NULL;@p2
+ ... when any
+ }
+
+@ script:python @
+fn << check1.fn;
+p1 << check1.p1;
+p2 << check1.p2;
+@@
+
+print('Warning: function {} has several definitions of '
+  'Error * local variable: at {}:{} and then at {}:{}'.format(
+  fn, p1[0].file, p1[0].line, 

[PATCH v12 3/8] sd: Use ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

From: Vladimir Sementsov-Ogievskiy 

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  Fix such cases.

If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

This commit is generated by command

sed -n '/^SD (Secure Card)$/,/^$/{s/^F: //p}' \
MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf 
Reported-by: Greg Kurz 
Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Philippe Mathieu-Daudé 
[Commit message tweaked]
Signed-off-by: Markus Armbruster 
---
 hw/sd/sdhci-pci.c |  7 +++
 hw/sd/sdhci.c | 21 +
 hw/sd/ssi-sd.c| 10 +-
 3 files changed, 17 insertions(+), 21 deletions(-)

diff --git a/hw/sd/sdhci-pci.c b/hw/sd/sdhci-pci.c
index 4f5977d487..38ec572fc6 100644
--- a/hw/sd/sdhci-pci.c
+++ b/hw/sd/sdhci-pci.c
@@ -29,13 +29,12 @@ static Property sdhci_pci_properties[] = {
 
 static void sdhci_pci_realize(PCIDevice *dev, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 SDHCIState *s = PCI_SDHCI(dev);
-Error *local_err = NULL;
 
 sdhci_initfn(s);
-sdhci_common_realize(s, &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+sdhci_common_realize(s, errp);
+if (*errp) {
 return;
 }
 
diff --git a/hw/sd/sdhci.c b/hw/sd/sdhci.c
index eb2be6529e..be1928784d 100644
--- a/hw/sd/sdhci.c
+++ b/hw/sd/sdhci.c
@@ -1288,7 +1288,7 @@ static const MemoryRegionOps sdhci_mmio_ops = {
 
 static void sdhci_init_readonly_registers(SDHCIState *s, Error **errp)
 {
-Error *local_err = NULL;
+ERRP_AUTO_PROPAGATE();
 
 switch (s->sd_spec_version) {
 case 2 ... 3:
@@ -1299,9 +1299,8 @@ static void sdhci_init_readonly_registers(SDHCIState *s, 
Error **errp)
 }
 s->version = (SDHC_HCVER_VENDOR << 8) | (s->sd_spec_version - 1);
 
-sdhci_check_capareg(s, &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+sdhci_check_capareg(s, errp);
+if (*errp) {
 return;
 }
 }
@@ -1332,11 +1331,10 @@ void sdhci_uninitfn(SDHCIState *s)
 
 void sdhci_common_realize(SDHCIState *s, Error **errp)
 {
-Error *local_err = NULL;
+ERRP_AUTO_PROPAGATE();
 
-sdhci_init_readonly_registers(s, &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+sdhci_init_readonly_registers(s, errp);
+if (*errp) {
 return;
 }
 s->buf_maxsz = sdhci_get_fifolen(s);
@@ -1456,13 +1454,12 @@ static void sdhci_sysbus_finalize(Object *obj)
 
 static void sdhci_sysbus_realize(DeviceState *dev, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 SDHCIState *s = SYSBUS_SDHCI(dev);
 SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
-Error *local_err = NULL;
 
-sdhci_common_realize(s, &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+sdhci_common_realize(s, errp);
+if (*errp) {
 return;
 }
 
diff --git a/hw/sd/ssi-sd.c b/hw/sd/ssi-sd.c
index 3717b2e721..adb7fa9c24 100644
--- a/hw/sd/ssi-sd.c
+++ b/hw/sd/ssi-sd.c
@@ -241,10 +241,10 @@ static const VMStateDescription vmstate_ssi_sd = {
 
 static void ssi_sd_realize(SSISlave *d, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 ssi_sd_state *s = SSI_SD(d);
 DeviceState *carddev;
 DriveInfo *dinfo;
-Error *err = NULL;
 
 qbus_create_inplace(&s->sdbus, sizeof(s->sdbus), TYPE_SD_BUS,
 DEVICE(d), "sd-bus");
@@ -255,23 +255,23 @@ static void ssi_sd_realize(SSISlave *d, Error **errp)
 carddev = qdev_new(TYPE_SD_CARD);
 if (dinfo) {
 if (!qdev_prop_set_drive_err(carddev, "drive",
- blk_by_legacy_dinfo(dinfo), &err)) {
+ blk_by_legacy_dinfo(dinfo), errp)) {
 goto fail;
 }
 }
 
-if (!object_property_set_bool(OBJECT(carddev), "spi", true, &err)) {
+if (!object_property_set_bool(OBJECT(carddev), "spi", true, errp)) {
 goto fail;
 }
 
-if (!qdev_realize_and_unref(carddev, BUS(&s->sdbus), &err)) {
+if (!qdev_realize_and_unref(carddev, BUS(&s->sdbus), errp)) {
 goto fail;
 }
 
 return;
 
 fail:
-error_propagate_prepend(errp, err, "failed to init SD 

[PATCH v12 4/8] pflash: Use ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

From: Vladimir Sementsov-Ogievskiy 

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  Fix such cases.

If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

This commit is generated by command

sed -n '/^Parallel NOR Flash devices$/,/^$/{s/^F: //p}' \
MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf 
Reported-by: Greg Kurz 
Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Philippe Mathieu-Daudé 
[Commit message tweaked]
Signed-off-by: Markus Armbruster 
---
 hw/block/pflash_cfi01.c | 7 +++
 hw/block/pflash_cfi02.c | 7 +++
 2 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/hw/block/pflash_cfi01.c b/hw/block/pflash_cfi01.c
index cddc3a5a0c..859cfeae14 100644
--- a/hw/block/pflash_cfi01.c
+++ b/hw/block/pflash_cfi01.c
@@ -696,12 +696,12 @@ static const MemoryRegionOps pflash_cfi01_ops = {
 
 static void pflash_cfi01_realize(DeviceState *dev, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 PFlashCFI01 *pfl = PFLASH_CFI01(dev);
 uint64_t total_len;
 int ret;
 uint64_t blocks_per_device, sector_len_per_device, device_len;
 int num_devices;
-Error *local_err = NULL;
 
 if (pfl->sector_len == 0) {
 error_setg(errp, "attribute \"sector-length\" not specified or zero.");
@@ -735,9 +735,8 @@ static void pflash_cfi01_realize(DeviceState *dev, Error 
**errp)
 &pfl->mem, OBJECT(dev),
 &pflash_cfi01_ops,
 pfl,
-pfl->name, total_len, &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+pfl->name, total_len, errp);
+if (*errp) {
 return;
 }
 
diff --git a/hw/block/pflash_cfi02.c b/hw/block/pflash_cfi02.c
index b40ce2335a..15035ee5ef 100644
--- a/hw/block/pflash_cfi02.c
+++ b/hw/block/pflash_cfi02.c
@@ -724,9 +724,9 @@ static const MemoryRegionOps pflash_cfi02_ops = {
 
 static void pflash_cfi02_realize(DeviceState *dev, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 PFlashCFI02 *pfl = PFLASH_CFI02(dev);
 int ret;
-Error *local_err = NULL;
 
 if (pfl->uniform_sector_len == 0 && pfl->sector_len[0] == 0) {
 error_setg(errp, "attribute \"sector-length\" not specified or zero.");
@@ -792,9 +792,8 @@ static void pflash_cfi02_realize(DeviceState *dev, Error 
**errp)
 
 memory_region_init_rom_device(&pfl->orig_mem, OBJECT(pfl),
   &pflash_cfi02_ops, pfl, pfl->name,
-  pfl->chip_len, &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+  pfl->chip_len, errp);
+if (*errp) {
 return;
 }
 
-- 
2.26.2

[PATCH v12 8/8] xen: Use ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

From: Vladimir Sementsov-Ogievskiy 

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  Fix such cases.

If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

This commit is generated by command

sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf 
Reported-by: Greg Kurz 
Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Philippe Mathieu-Daudé 
[Commit message tweaked]
Signed-off-by: Markus Armbruster 
---
 hw/block/dataplane/xen-block.c |  17 +++---
 hw/block/xen-block.c   | 102 ++---
 hw/pci-host/xen_igd_pt.c   |   7 +--
 hw/xen/xen-backend.c   |   7 +--
 hw/xen/xen-bus.c   |  92 +
 hw/xen/xen-host-pci-device.c   |  27 +
 hw/xen/xen_pt.c|  25 
 hw/xen/xen_pt_config_init.c|  17 +++---
 8 files changed, 128 insertions(+), 166 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 5f8f15778b..1a077cc05f 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -723,8 +723,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
unsigned int protocol,
Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 XenDevice *xendev = dataplane->xendev;
-Error *local_err = NULL;
 unsigned int ring_size;
 unsigned int i;
 
@@ -760,9 +760,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
 }
 
 xen_device_set_max_grant_refs(xendev, dataplane->nr_ring_ref,
-  &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+  errp);
+if (*errp) {
 goto stop;
 }
 
@@ -770,9 +769,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
   dataplane->ring_ref,
   dataplane->nr_ring_ref,
   PROT_READ | PROT_WRITE,
-  &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+  errp);
+if (*errp) {
 goto stop;
 }
 
@@ -805,9 +803,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
 dataplane->event_channel =
 xen_device_bind_event_channel(xendev, event_channel,
   xen_block_dataplane_event, dataplane,
-  &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+  errp);
+if (*errp) {
 goto stop;
 }
 
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index a775fba7c0..623ae5b8e0 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -195,6 +195,7 @@ static const BlockDevOps xen_block_dev_ops = {
 
 static void xen_block_realize(XenDevice *xendev, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
 XenBlockDeviceClass *blockdev_class =
 XEN_BLOCK_DEVICE_GET_CLASS(xendev);
@@ -202,7 +203,6 @@ static void xen_block_realize(XenDevice *xendev, Error 
**errp)
 XenBlockVdev *vdev = &blockdev->props.vdev;
 BlockConf *conf = &blockdev->props.conf;
 BlockBackend *blk = conf->blk;
-Error *local_err = NULL;
 
 if (vdev->type == XEN_BLOCK_VDEV_TYPE_INVALID) {
 error_setg(errp, "vdev property not set");
@@ -212,9 +212,8 @@ static void xen_block_realize(XenDevice *xendev, Error 
**errp)
 trace_xen_block_realize(type, vdev->disk, vdev->partition);
 
 if (blockdev_class->realize) {
-blockdev_class->realize(blockdev, &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+blockdev_class->realize(blockdev, errp);
+if (*errp) {
 return;
 }
 }
@@ -280,8 +279,8 @@ static void xen_block_frontend_changed(XenDevice *xendev,
enum xenbus_state f

[PATCH v12 7/8] nbd: Use ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

From: Vladimir Sementsov-Ogievskiy 

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  Fix such cases.

If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
   &error_fatal, this means that we don't break error_abort
   (we'll abort on error_set, not on error_propagate)

This commit is generated by command

sed -n '/^Network Block Device (NBD)$/,/^$/{s/^F: //p}' \
MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf 
Reported-by: Greg Kurz 
Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Markus Armbruster 
[Commit message tweaked]
Signed-off-by: Markus Armbruster 
---
 include/block/nbd.h | 1 +
 block/nbd.c | 7 +++
 nbd/client.c| 5 +
 nbd/server.c| 5 +
 4 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/include/block/nbd.h b/include/block/nbd.h
index 20363280ae..f7d87636d3 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -361,6 +361,7 @@ void nbd_server_start_options(NbdServerOptions *arg, Error 
**errp);
 static inline int nbd_read(QIOChannel *ioc, void *buffer, size_t size,
const char *desc, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 int ret = qio_channel_read_all(ioc, buffer, size, errp) < 0 ? -EIO : 0;
 
 if (ret < 0) {
diff --git a/block/nbd.c b/block/nbd.c
index 6876da04a7..b7cea0f650 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -1408,16 +1408,15 @@ static void nbd_client_close(BlockDriverState *bs)
 static QIOChannelSocket *nbd_establish_connection(SocketAddress *saddr,
   Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 QIOChannelSocket *sioc;
-Error *local_err = NULL;
 
 sioc = qio_channel_socket_new();
 qio_channel_set_name(QIO_CHANNEL(sioc), "nbd-client");
 
-qio_channel_socket_connect_sync(sioc, saddr, &local_err);
-if (local_err) {
+qio_channel_socket_connect_sync(sioc, saddr, errp);
+if (*errp) {
 object_unref(OBJECT(sioc));
-error_propagate(errp, local_err);
 return NULL;
 }
 
diff --git a/nbd/client.c b/nbd/client.c
index ba173108ba..e258ef3f7e 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -68,6 +68,7 @@ static int nbd_send_option_request(QIOChannel *ioc, uint32_t 
opt,
uint32_t len, const char *data,
Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 NBDOption req;
 QEMU_BUILD_BUG_ON(sizeof(req) != 16);
 
@@ -153,6 +154,7 @@ static int nbd_receive_option_reply(QIOChannel *ioc, 
uint32_t opt,
 static int nbd_handle_reply_err(QIOChannel *ioc, NBDOptionReply *reply,
 bool strict, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 g_autofree char *msg = NULL;
 
 if (!(reply->type & (1 << 31))) {
@@ -337,6 +339,7 @@ static int nbd_receive_list(QIOChannel *ioc, char **name, 
char **description,
 static int nbd_opt_info_or_go(QIOChannel *ioc, uint32_t opt,
   NBDExportInfo *info, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 NBDOptionReply reply;
 uint32_t len = strlen(info->name);
 uint16_t type;
@@ -882,6 +885,7 @@ static int nbd_start_negotiate(AioContext *aio_context, 
QIOChannel *ioc,
bool structured_reply, bool *zeroes,
Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 uint64_t magic;
 
 trace_nbd_start_negotiate(tlscreds, hostname ? hostname : "");
@@ -1017,6 +1021,7 @@ int nbd_receive_negotiate(AioContext *aio_context, 
QIOChannel *ioc,
   const char *hostname, QIOChannel **outioc,
   NBDExportInfo *info, Error **errp)
 {
+ERRP_AUTO_PROPAGATE();
 int result;
 bool zeroes;
 bool base_allocation = info->base_allocation;
diff --git a/nbd/server.c b/nbd/server.c
index 20754e9ebc..8a12e586d7 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -211,6 +211,7 @@ static int GCC_FMT_ATTR(4, 0)
 nbd_negotiate_send_rep_verr(NBDClient *client, uint32_t type,
 Error **errp, const char *fmt, va_list va)
 {
+ERRP_AUTO_PROPAGATE();
 g_autofree char *msg = NULL;
 int ret;
 size_t len;
@@ -382,6 +383,7 @@ static int nbd_opt_read_name(NBDClient *client, char 
*

[qemu-mainline test] 151704: regressions - FAIL

[osstest service owner]

flight 151704 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151704/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail REGR. vs. 151065
 test-amd64-amd64-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 
151065
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail REGR. vs. 
151065
 test-amd64-amd64-xl-qemuu-win7-amd64 10 windows-install  fail REGR. vs. 151065
 test-amd64-amd64-libvirt-vhd 10 debian-di-installfail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-win7-amd64 10 windows-install   fail REGR. vs. 151065
 test-amd64-amd64-qemuu-nested-intel 10 debian-hvm-install fail REGR. vs. 151065
 test-amd64-i386-libvirt-xsm  12 guest-start  fail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 10 debian-hvm-install fail REGR. 
vs. 151065
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-install  fail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 151065
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail REGR. vs. 151065
 test-amd64-amd64-qemuu-nested-amd 10 debian-hvm-install  fail REGR. vs. 151065
 test-amd64-amd64-xl-qcow210 debian-di-installfail REGR. vs. 151065
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 10 debian-hvm-install fail REGR. 
vs. 151065
 test-amd64-i386-qemuu-rhel6hvm-amd 10 redhat-install fail REGR. vs. 151065
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-i386-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail REGR. vs. 
151065
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-i386-xl-qemuu-ws16-amd64 10 windows-install   fail REGR. vs. 151065
 test-amd64-i386-libvirt-pair 21 guest-start/debian   fail REGR. vs. 151065
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail 
REGR. vs. 151065
 test-amd64-i386-freebsd10-i386 11 guest-startfail REGR. vs. 151065
 test-amd64-i386-qemuu-rhel6hvm-intel 10 redhat-install   fail REGR. vs. 151065
 test-amd64-amd64-libvirt 12 guest-start  fail REGR. vs. 151065
 test-amd64-amd64-libvirt-xsm 12 guest-start  fail REGR. vs. 151065
 test-amd64-i386-libvirt  12 guest-start  fail REGR. vs. 151065
 test-amd64-i386-freebsd10-amd64 11 guest-start   fail REGR. vs. 151065
 test-armhf-armhf-xl-vhd  10 debian-di-installfail REGR. vs. 151065
 test-amd64-amd64-libvirt-pair 21 guest-start/debian  fail REGR. vs. 151065
 test-armhf-armhf-libvirt 12 guest-start  fail REGR. vs. 151065
 test-arm64-arm64-libvirt-xsm 12 guest-start  fail REGR. vs. 151065
 test-armhf-armhf-libvirt-raw 10 debian-di-installfail REGR. vs. 151065

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail  like 151065
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saveresto

Re: [PATCH v12 0/8] error: auto propagated local_err part I

[Vladimir Sementsov-Ogievskiy]

07.07.2020 19:50, Markus Armbruster wrote:

To speed things up, I'm taking the liberty to respin Vladimir's series
with my documentation amendments.


Thank you!



After my documentation work, I'm very much inclined to rename
ERRP_AUTO_PROPAGATE() to ERRP_GUARD().  The fact that it propagates
below the hood is detail.  What matters to its users is that it lets
them use @errp more freely.  Thoughts?


No objections, if we are making error-propagation to be internal implementation 
detail, no reason to shout about it in the macro name.


--
Best regards,
Vladimir

Re: [PATCH v12 0/8] error: auto propagated local_err part I

[Eric Blake]

On 7/7/20 11:50 AM, Markus Armbruster wrote:

To speed things up, I'm taking the liberty to respin Vladimir's series
with my documentation amendments.

After my documentation work, I'm very much inclined to rename
ERRP_AUTO_PROPAGATE() to ERRP_GUARD().  The fact that it propagates
below the hood is detail.  What matters to its users is that it lets
them use @errp more freely.  Thoughts?


I like it - the shorter name is easier to type.

(The rename is a mechanical change, so if we agree to it, we should do 
it up front to minimize the churn to all the functions where we add use 
of the macro)




Based-on: Message-Id: <20200707160613.848843-1-arm...@redhat.com>

Available from my public repository https://repo.or.cz/qemu/armbru.git
on branch error-auto.

v12: (based on "[PATCH v4 00/45] Less clumsy error checking")
01: Comments merged properly with recent commit "error: Document Error
API usage rules", and edited for clarity.  Put ERRP_AUTO_PROPAGATE()
before its helpers, and touch up style.
01-08: Commit messages tweaked



--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH v12 1/8] error: New macro ERRP_AUTO_PROPAGATE()

[Eric Blake]

On 7/7/20 11:50 AM, Markus Armbruster wrote:

From: Vladimir Sementsov-Ogievskiy 

Introduce a new ERRP_AUTO_PROPAGATE macro, to be used at start of
functions with an errp OUT parameter.

It has three goals:

1. Fix issue with error_fatal and error_prepend/error_append_hint: user


the user


can't see this additional information, because exit() happens in
error_setg earlier than information is added. [Reported by Greg Kurz]

2. Fix issue with error_abort and error_propagate: when we wrap
error_abort by local_err+error_propagate, the resulting coredump will
refer to error_propagate and not to the place where error happened.
(the macro itself doesn't fix the issue, but it allows us to [3.] drop
the local_err+error_propagate pattern, which will definitely fix the
issue) [Reported by Kevin Wolf]

3. Drop local_err+error_propagate pattern, which is used to workaround
void functions with errp parameter, when caller wants to know resulting
status. (Note: actually these functions could be merely updated to
return int error code).

To achieve these goals, later patches will add invocations
of this macro at the start of functions with either use
error_prepend/error_append_hint (solving 1) or which use
local_err+error_propagate to check errors, switching those
functions to use *errp instead (solving 2 and 3).

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Paul Durrant 
Reviewed-by: Greg Kurz 
Reviewed-by: Eric Blake 
[Comments merged properly with recent commit "error: Document Error
API usage rules", and edited for clarity.  Put ERRP_AUTO_PROPAGATE()
before its helpers, and touch up style.  Commit message tweaked.]
Signed-off-by: Markus Armbruster 
---
  include/qapi/error.h | 160 ++-
  1 file changed, 141 insertions(+), 19 deletions(-)

diff --git a/include/qapi/error.h b/include/qapi/error.h
index 3fed49747d..c865a7d2f1 100644
--- a/include/qapi/error.h
+++ b/include/qapi/error.h



@@ -128,18 +122,26 @@
   * handle the error...
   * }
   * when it doesn't, say a void function:
+ * ERRP_AUTO_PROPAGATE();
+ * foo(arg, errp);
+ * if (*errp) {
+ * handle the error...
+ * }
+ * More on ERRP_AUTO_PROPAGATE() below.
+ *
+ * Code predating ERRP_AUTO_PROPAGATE() still exits, and looks like this:


exists


   * Error *err = NULL;
   * foo(arg, &err);
   * if (err) {
   * handle the error...
- * error_propagate(errp, err);
+ * error_propagate(errp, err); // deprecated
   * }
- * Do *not* "optimize" this to
+ * Avoid in new code.  Do *not* "optimize" it to
   * foo(arg, errp);
   * if (*errp) { // WRONG!
   * handle the error...
   * }
- * because errp may be NULL!
+ * because errp may be NULL!  Guard with ERRP_AUTO_PROPAGATE().


maybe:

because errp may be NULL without the ERRP_AUTO_PROPAGATE() guard.


   *
   * But when all you do with the error is pass it on, please use
   * foo(arg, errp);
@@ -158,6 +160,19 @@
   * handle the error...
   * }
   *
+ * Pass an existing error to the caller:



+ * = Converting to ERRP_AUTO_PROPAGATE() =
+ *
+ * To convert a function to use ERRP_AUTO_PROPAGATE():
+ *
+ * 0. If the Error ** parameter is not named @errp, rename it to
+ *@errp.
+ *
+ * 1. Add an ERRP_AUTO_PROPAGATE() invocation, by convention right at
+ *the beginning of the function.  This makes @errp safe to use.
+ *
+ * 2. Replace &err by errp, and err by *errp.  Delete local variable
+ *@err.
+ *
+ * 3. Delete error_propagate(errp, *errp), replace
+ *error_propagate_prepend(errp, *errp, ...) by error_prepend(errp, ...),
+ *


Why a comma here?


+ * 4. Ensure @errp is valid at return: when you destroy *errp, set
+ *errp = NULL.
+ *
+ * Example:
+ *
+ * bool fn(..., Error **errp)
+ * {
+ * Error *err = NULL;
+ *
+ * foo(arg, &err);
+ * if (err) {
+ * handle the error...
+ * error_propagate(errp, err);
+ * return false;
+ * }
+ * ...
+ * }
+ *
+ * becomes
+ *
+ * bool fn(..., Error **errp)
+ * {
+ * ERRP_AUTO_PROPAGATE();
+ *
+ * foo(arg, errp);
+ * if (*errp) {
+ * handle the error...
+ * return false;
+ * }
+ * ...
+ * }


Do we want the example to show the use of error_free and *errp = NULL?

Otherwise, this is looking good to me.  It will need a tweak if we go 
with the shorter name ERRP_GUARD, but I like that idea.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

[linux-linus test] 151705: regressions - FAIL

[osstest service owner]

flight 151705 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151705/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-xsm 16 guest-start/debian.repeat fail REGR. vs. 151214
 build-i386-pvops  6 kernel-build fail REGR. vs. 151214

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-arndale   7 xen-boot fail in 151690 pass in 151705
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeat  fail pass in 151690

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 17 guest-start.2  fail in 151690 REGR. vs. 151214

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-coresched-i386-xl  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-examine   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemut-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-i386-qemut-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 151214
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 151214
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 151214
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 151214
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 151214
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 151214
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietr

Re: [PATCH v2 2/3] xen/privcmd: Mark pages as dirty

[John Hubbard]

On 2020-07-07 04:43, Jürgen Groß wrote:

On 07.07.20 13:30, Souptick Joarder wrote:

On Tue, Jul 7, 2020 at 3:08 PM Jürgen Groß  wrote:

...

diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 33677ea..f6c1543 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -612,8 +612,11 @@ static void unlock_pages(struct page *pages[], unsigned 
int nr_pages)
   {
   unsigned int i;

- for (i = 0; i < nr_pages; i++)
+ for (i = 0; i < nr_pages; i++) {
+ if (!PageDirty(pages[i]))
+ set_page_dirty_lock(pages[i]);


With put_page() directly following I think you should be able to use
set_page_dirty() instead, as there is obviously a reference to the page
existing.


Patch [3/3] will convert above codes to use unpin_user_pages_dirty_lock()
which internally do the same check. So I thought to keep linux-stable and
linux-next code in sync. John had a similar concern [1] and later agreed to keep
this check.

Shall I keep this check ?  No ?


It doesn't matter *too* much, because patch 3/3 fixes up everything by
changing it all to unpin_user_pages_dirty_lock(). However, there is something
to be said for having correct interim patches, too. :)  Details:



[1] 
https://lore.kernel.org/xen-devel/a750e5e5-fd5d-663b-c5fd-261d7c939...@nvidia.com/


I wasn't referring to checking PageDirty(), but to the use of
set_page_dirty_lock().

Looking at the comment just before the implementation of
set_page_dirty_lock() suggests that it is fine to use set_page_dirty()
instead (so not calling lock_page()).



no no, that's a misreading of the comment. Unless this xen/privcmd code has
somehow taken a reference on page->mapping->host (which I do *not* think is
the case), then it is still racy to call set_page_dirty() here. Instead,
set_page_dirty_lock() should be used.




Only the transition from get_user_pages_fast() to pin_user_pages_fast()
requires to use the locked version IMO.



That's a different misunderstanding. :) pin_user_pages*() APIs are meant to be
functionally drop-in replacements for get_user_pages*(). Internally,
pin_user_pages*() functions do some additional tracking, but from a caller's
perspective, it should look the same. In other words, there is nothing
about pin_user_pages_fast() that requires set_page_dirty_lock() upon release.
The reason set_page_dirty_lock() was chosen is that there are very few
(none at all?) call sites that need to release and dirty a page, that also meet
the requirements to safely call set_page_dirty().

That's why there is a unpin_user_pages_dirty_lock(), but there is not a
corresponding unpin_user_pages_dirty() call: the latter has not been required
so far, even though the call site conversions are nearly done.


thanks,
--
John Hubbard
NVIDIA

Re: [PATCH v12 2/8] scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

[Eric Blake]

On 7/7/20 11:50 AM, Markus Armbruster wrote:

From: Vladimir Sementsov-Ogievskiy 

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
does corresponding changes in code (look for details in
include/qapi/error.h)

Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
  --max-width 80 FILES...

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Markus Armbruster 
Signed-off-by: Markus Armbruster 
---
  scripts/coccinelle/auto-propagated-errp.cocci | 337 ++
  include/qapi/error.h  |   3 +
  MAINTAINERS   |   1 +
  3 files changed, 341 insertions(+)
  create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci


Needs a tweak if we go with ERRP_GUARD.  But that's easy.


+
+// Convert special case with goto separately.
+// I tried merging this into the following rule the obvious way, but
+// it made Coccinelle hang on block.c
+//
+// Note interesting thing: if we don't do it here, and try to fixup
+// "out: }" things later after all transformations (the rule will be
+// the same, just without error_propagate() call), coccinelle fails to
+// match this "out: }".


"out: }" is not valid C; would referring to "out: ; }" fare any better?


+@ disable optional_qualifier@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(..., Error ** , ...)
+ {
+ <...
+-goto out;
++return;
+ ...>
+- out:
+-error_propagate(errp, local_err);
+ }
+
+// Convert most of local_err related stuff.
+//
+// Note, that we inherit rule1.fn and rule1.local_err names, not
+// objects themselves. We may match something not related to the
+// pattern matched by rule1. For example, local_err may be defined with
+// the same name in different blocks inside one function, and in one
+// block follow the propagation pattern and in other block doesn't.
+//
+// Note also that errp-cleaning functions
+//   error_free_errp
+//   error_report_errp
+//   error_reportf_errp
+//   warn_report_errp
+//   warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* -
+// freeing function and then set *errp to NULL, to avoid further
+// propagation to original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+//void error_free_errp(Error **errp)
+//{
+//error_free(*errp);
+//*errp = NULL;
+//}


I guess we can still decide later if we want these additional functions, 
or if they will even help after the number of places we have already 
improved after applying this script as-is and with Markus' cleanups in 
place.


While I won't call myself a Coccinelle expert, it at least looks sane 
enough that I'm comfortable if you add:


Reviewed-by: Eric Blake 

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

[PATCH v6 02/11] x86/vmx: add Intel PT MSR definitions

[Michał Leszczyński]

From: Michal Leszczynski 

Define constants related to Intel Processor Trace features.

Signed-off-by: Michal Leszczynski 
Acked-by: Andrew Cooper 
---
 xen/include/asm-x86/msr-index.h | 24 
 1 file changed, 24 insertions(+)

diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index 0fe98af923..4fd54fb5c9 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -72,7 +72,31 @@
 #define MSR_RTIT_OUTPUT_BASE0x0560
 #define MSR_RTIT_OUTPUT_MASK0x0561
 #define MSR_RTIT_CTL0x0570
+#define  RTIT_CTL_TRACE_EN  (_AC(1, ULL) <<  0)
+#define  RTIT_CTL_CYC_EN(_AC(1, ULL) <<  1)
+#define  RTIT_CTL_OS(_AC(1, ULL) <<  2)
+#define  RTIT_CTL_USR   (_AC(1, ULL) <<  3)
+#define  RTIT_CTL_PWR_EVT_EN(_AC(1, ULL) <<  4)
+#define  RTIT_CTL_FUP_ON_PTW(_AC(1, ULL) <<  5)
+#define  RTIT_CTL_FABRIC_EN (_AC(1, ULL) <<  6)
+#define  RTIT_CTL_CR3_FILTER(_AC(1, ULL) <<  7)
+#define  RTIT_CTL_TOPA  (_AC(1, ULL) <<  8)
+#define  RTIT_CTL_MTC_EN(_AC(1, ULL) <<  9)
+#define  RTIT_CTL_TSC_EN(_AC(1, ULL) << 10)
+#define  RTIT_CTL_DIS_RETC  (_AC(1, ULL) << 11)
+#define  RTIT_CTL_PTW_EN(_AC(1, ULL) << 12)
+#define  RTIT_CTL_BRANCH_EN (_AC(1, ULL) << 13)
+#define  RTIT_CTL_MTC_FREQ  (_AC(0xf, ULL) << 14)
+#define  RTIT_CTL_CYC_THRESH(_AC(0xf, ULL) << 19)
+#define  RTIT_CTL_PSB_FREQ  (_AC(0xf, ULL) << 24)
+#define  RTIT_CTL_ADDR(n)   (_AC(0xf, ULL) << (32 + 4 * (n)))
 #define MSR_RTIT_STATUS 0x0571
+#define  RTIT_STATUS_FILTER_EN  (_AC(1, ULL) <<  0)
+#define  RTIT_STATUS_CONTEXT_EN (_AC(1, ULL) <<  1)
+#define  RTIT_STATUS_TRIGGER_EN (_AC(1, ULL) <<  2)
+#define  RTIT_STATUS_ERROR  (_AC(1, ULL) <<  4)
+#define  RTIT_STATUS_STOPPED(_AC(1, ULL) <<  5)
+#define  RTIT_STATUS_BYTECNT(_AC(0x1, ULL) << 32)
 #define MSR_RTIT_CR3_MATCH  0x0572
 #define MSR_RTIT_ADDR_A(n) (0x0580 + (n) * 2)
 #define MSR_RTIT_ADDR_B(n) (0x0581 + (n) * 2)
-- 
2.17.1

[PATCH v6 01/11] memory: batch processing in acquire_resource()

[Michał Leszczyński]

From: Michal Leszczynski 

Allow to acquire large resources by allowing acquire_resource()
to process items in batches, using hypercall continuation.

Be aware that this modifies the behavior of acquire_resource
call with frame_list=NULL. While previously it would return
the size of internal array (32), with this patch it returns
the maximal quantity of frames that could be requested at once,
i.e. UINT_MAX >> MEMOP_EXTENT_SHIFT.

Signed-off-by: Michal Leszczynski 
---
 xen/common/memory.c | 49 -
 1 file changed, 44 insertions(+), 5 deletions(-)

diff --git a/xen/common/memory.c b/xen/common/memory.c
index 714077c1e5..eb42f883df 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -1046,10 +1046,12 @@ static int acquire_grant_table(struct domain *d, 
unsigned int id,
 }
 
 static int acquire_resource(
-XEN_GUEST_HANDLE_PARAM(xen_mem_acquire_resource_t) arg)
+XEN_GUEST_HANDLE_PARAM(xen_mem_acquire_resource_t) arg,
+unsigned long *start_extent)
 {
 struct domain *d, *currd = current->domain;
 xen_mem_acquire_resource_t xmar;
+uint32_t total_frames;
 /*
  * The mfn_list and gfn_list (below) arrays are ok on stack for the
  * moment since they are small, but if they need to grow in future
@@ -1069,7 +1071,7 @@ static int acquire_resource(
 if ( xmar.nr_frames )
 return -EINVAL;
 
-xmar.nr_frames = ARRAY_SIZE(mfn_list);
+xmar.nr_frames = UINT_MAX >> MEMOP_EXTENT_SHIFT;
 
 if ( __copy_field_to_guest(arg, &xmar, nr_frames) )
 return -EFAULT;
@@ -1077,8 +1079,28 @@ static int acquire_resource(
 return 0;
 }
 
+total_frames = xmar.nr_frames;
+
+/* Is the size too large for us to encode a continuation? */
+if ( unlikely(xmar.nr_frames > (UINT_MAX >> MEMOP_EXTENT_SHIFT)) )
+return -EINVAL;
+
+if ( *start_extent )
+{
+/*
+ * Check whether start_extent is in bounds, as this
+ * value if visible to the calling domain.
+ */
+if ( *start_extent > xmar.nr_frames )
+return -EINVAL;
+
+xmar.frame += *start_extent;
+xmar.nr_frames -= *start_extent;
+guest_handle_add_offset(xmar.frame_list, *start_extent);
+}
+
 if ( xmar.nr_frames > ARRAY_SIZE(mfn_list) )
-return -E2BIG;
+xmar.nr_frames = ARRAY_SIZE(mfn_list);
 
 rc = rcu_lock_remote_domain_by_id(xmar.domid, &d);
 if ( rc )
@@ -1135,6 +1157,14 @@ static int acquire_resource(
 }
 }
 
+if ( !rc )
+{
+*start_extent += xmar.nr_frames;
+
+if ( *start_extent != total_frames )
+rc = -ERESTART;
+}
+
  out:
 rcu_unlock_domain(d);
 
@@ -1599,8 +1629,17 @@ long do_memory_op(unsigned long cmd, 
XEN_GUEST_HANDLE_PARAM(void) arg)
 #endif
 
 case XENMEM_acquire_resource:
-rc = acquire_resource(
-guest_handle_cast(arg, xen_mem_acquire_resource_t));
+do {
+rc = acquire_resource(
+guest_handle_cast(arg, xen_mem_acquire_resource_t),
+&start_extent);
+
+if ( hypercall_preempt_check() )
+return hypercall_create_continuation(
+__HYPERVISOR_memory_op, "lh",
+op | (start_extent << MEMOP_EXTENT_SHIFT), arg);
+} while ( rc == -ERESTART );
+
 break;
 
 default:
-- 
2.17.1

[PATCH v6 03/11] x86/vmx: add IPT cpu feature

[Michał Leszczyński]

From: Michal Leszczynski 

Check if Intel Processor Trace feature is supported by current
processor. Define vmtrace_supported global variable.

Signed-off-by: Michal Leszczynski 
---
 xen/arch/x86/hvm/vmx/vmcs.c | 15 ++-
 xen/common/domain.c |  2 ++
 xen/include/asm-x86/cpufeature.h|  1 +
 xen/include/asm-x86/hvm/vmx/vmcs.h  |  1 +
 xen/include/public/arch-x86/cpufeatureset.h |  1 +
 xen/include/xen/domain.h|  2 ++
 6 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index ca94c2bedc..3a53553f10 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -291,6 +291,20 @@ static int vmx_init_vmcs_config(void)
 _vmx_cpu_based_exec_control &=
 ~(CPU_BASED_CR8_LOAD_EXITING | CPU_BASED_CR8_STORE_EXITING);
 
+rdmsrl(MSR_IA32_VMX_MISC, _vmx_misc_cap);
+
+/* Check whether IPT is supported in VMX operation. */
+if ( !smp_processor_id() )
+vmtrace_supported = cpu_has_ipt &&
+(_vmx_misc_cap & VMX_MISC_PROC_TRACE);
+else if ( vmtrace_supported &&
+  !(_vmx_misc_cap & VMX_MISC_PROC_TRACE) )
+{
+printk("VMX: IPT capabilities fatally differ between CPU%u and CPU0\n",
+   smp_processor_id());
+return -EINVAL;
+}
+
 if ( _vmx_cpu_based_exec_control & CPU_BASED_ACTIVATE_SECONDARY_CONTROLS )
 {
 min = 0;
@@ -305,7 +319,6 @@ static int vmx_init_vmcs_config(void)
SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS |
SECONDARY_EXEC_XSAVES |
SECONDARY_EXEC_TSC_SCALING);
-rdmsrl(MSR_IA32_VMX_MISC, _vmx_misc_cap);
 if ( _vmx_misc_cap & VMX_MISC_VMWRITE_ALL )
 opt |= SECONDARY_EXEC_ENABLE_VMCS_SHADOWING;
 if ( opt_vpid_enabled )
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 7cc9526139..a45cf023f7 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -82,6 +82,8 @@ struct vcpu *idle_vcpu[NR_CPUS] __read_mostly;
 
 vcpu_info_t dummy_vcpu_info;
 
+bool vmtrace_supported __read_mostly;
+
 static void __domain_finalise_shutdown(struct domain *d)
 {
 struct vcpu *v;
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index f790d5c1f8..555f696a26 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -104,6 +104,7 @@
 #define cpu_has_clwbboot_cpu_has(X86_FEATURE_CLWB)
 #define cpu_has_avx512erboot_cpu_has(X86_FEATURE_AVX512ER)
 #define cpu_has_avx512cdboot_cpu_has(X86_FEATURE_AVX512CD)
+#define cpu_has_ipt boot_cpu_has(X86_FEATURE_PROC_TRACE)
 #define cpu_has_sha boot_cpu_has(X86_FEATURE_SHA)
 #define cpu_has_avx512bwboot_cpu_has(X86_FEATURE_AVX512BW)
 #define cpu_has_avx512vlboot_cpu_has(X86_FEATURE_AVX512VL)
diff --git a/xen/include/asm-x86/hvm/vmx/vmcs.h 
b/xen/include/asm-x86/hvm/vmx/vmcs.h
index 906810592f..6153ba6769 100644
--- a/xen/include/asm-x86/hvm/vmx/vmcs.h
+++ b/xen/include/asm-x86/hvm/vmx/vmcs.h
@@ -283,6 +283,7 @@ extern u32 vmx_secondary_exec_control;
 #define VMX_VPID_INVVPID_SINGLE_CONTEXT_RETAINING_GLOBAL 0x800ULL
 extern u64 vmx_ept_vpid_cap;
 
+#define VMX_MISC_PROC_TRACE 0x4000
 #define VMX_MISC_CR3_TARGET 0x01ff
 #define VMX_MISC_VMWRITE_ALL0x2000
 
diff --git a/xen/include/public/arch-x86/cpufeatureset.h 
b/xen/include/public/arch-x86/cpufeatureset.h
index fe7492a225..2c91862f2d 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -217,6 +217,7 @@ XEN_CPUFEATURE(SMAP,  5*32+20) /*S  Supervisor Mode 
Access Prevention */
 XEN_CPUFEATURE(AVX512_IFMA,   5*32+21) /*A  AVX-512 Integer Fused Multiply Add 
*/
 XEN_CPUFEATURE(CLFLUSHOPT,5*32+23) /*A  CLFLUSHOPT instruction */
 XEN_CPUFEATURE(CLWB,  5*32+24) /*A  CLWB instruction */
+XEN_CPUFEATURE(PROC_TRACE,5*32+25) /*   Processor Tracing feature */
 XEN_CPUFEATURE(AVX512PF,  5*32+26) /*A  AVX-512 Prefetch Instructions */
 XEN_CPUFEATURE(AVX512ER,  5*32+27) /*A  AVX-512 Exponent & Reciprocal 
Instrs */
 XEN_CPUFEATURE(AVX512CD,  5*32+28) /*A  AVX-512 Conflict Detection Instrs 
*/
diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
index 7e51d361de..61ebc6c24d 100644
--- a/xen/include/xen/domain.h
+++ b/xen/include/xen/domain.h
@@ -130,4 +130,6 @@ struct vnuma_info {
 
 void vnuma_destroy(struct vnuma_info *vnuma);
 
+extern bool vmtrace_supported;
+
 #endif /* __XEN_DOMAIN_H__ */
-- 
2.17.1

[PATCH v6 04/11] common: add vmtrace_pt_size domain parameter

[Michał Leszczyński]

From: Michal Leszczynski 

Add vmtrace_pt_size domain parameter in live domain and
vmtrace_pt_order parameter in xen_domctl_createdomain.

Signed-off-by: Michal Leszczynski 
---
 xen/arch/x86/domain.c   | 6 ++
 xen/common/domain.c | 9 +
 xen/include/public/domctl.h | 1 +
 xen/include/xen/sched.h | 3 +++
 4 files changed, 19 insertions(+)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index fee6c3931a..b75017b28b 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -499,6 +499,12 @@ int arch_sanitise_domain_config(struct 
xen_domctl_createdomain *config)
  */
 config->flags |= XEN_DOMCTL_CDF_oos_off;
 
+if ( !hvm && config->processor_trace_buf_kb )
+{
+dprintk(XENLOG_INFO, "Processor trace is not supported on non-HVM\n");
+return -EINVAL;
+}
+
 return 0;
 }
 
diff --git a/xen/common/domain.c b/xen/common/domain.c
index a45cf023f7..e6e8f88da1 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -338,6 +338,12 @@ static int sanitise_domain_config(struct 
xen_domctl_createdomain *config)
 return -EINVAL;
 }
 
+if ( config->processor_trace_buf_kb && !vmtrace_supported )
+{
+dprintk(XENLOG_INFO, "Processor tracing is not supported\n");
+return -EINVAL;
+}
+
 return arch_sanitise_domain_config(config);
 }
 
@@ -443,6 +449,9 @@ struct domain *domain_create(domid_t domid,
 d->nr_pirqs = min(d->nr_pirqs, nr_irqs);
 
 radix_tree_init(&d->pirq_tree);
+
+if ( config->processor_trace_buf_kb )
+d->processor_trace_buf_kb = config->processor_trace_buf_kb;
 }
 
 if ( (err = arch_domain_create(d, config)) != 0 )
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 59bdc28c89..7681675a94 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -92,6 +92,7 @@ struct xen_domctl_createdomain {
 uint32_t max_evtchn_port;
 int32_t max_grant_frames;
 int32_t max_maptrack_frames;
+uint32_t processor_trace_buf_kb;
 
 struct xen_arch_domainconfig arch;
 };
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index ac53519d7f..c046e59886 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -457,6 +457,9 @@ struct domain
 unsignedpbuf_idx;
 spinlock_t  pbuf_lock;
 
+/* Used by vmtrace features */
+uint32_tprocessor_trace_buf_kb;
+
 /* OProfile support. */
 struct xenoprof *xenoprof;
 
-- 
2.17.1

[PATCH v6 00/11] Implement support for external IPT monitoring

[Michał Leszczyński]

Intel Processor Trace is an architectural extension available in modern Intel 
family CPUs. It allows recording the detailed trace of activity while the 
processor executes the code. One might use the recorded trace to reconstruct 
the code flow. It means, to find out the executed code paths, determine 
branches taken, and so forth.

The abovementioned feature is described in Intel(R) 64 and IA-32 Architectures 
Software Developer's Manual Volume 3C: System Programming Guide, Part 3, 
Chapter 36: "Intel Processor Trace."

This patch series implements an interface that Dom0 could use in order to 
enable IPT for particular vCPUs in DomU, allowing for external monitoring. Such 
a feature has numerous applications like malware monitoring, fuzzing, or 
performance testing.

Also thanks to Tamas K Lengyel for a few preliminary hints before
first version of this patch was submitted to xen-devel.

Changed since v1:
  * MSR_RTIT_CTL is managed using MSR load lists
  * other PT-related MSRs are modified only when vCPU goes out of context
  * trace buffer is now acquired as a resource
  * added vmtrace_pt_size parameter in xl.cfg, the size of trace buffer
must be specified in the moment of domain creation
  * trace buffers are allocated on domain creation, destructed on
domain destruction
  * HVMOP_vmtrace_ipt_enable/disable is limited to enabling/disabling PT
these calls don't manage buffer memory anymore
  * lifted 32 MFN/GFN array limit when acquiring resources
  * minor code style changes according to review

Changed since v2:
  * trace buffer is now allocated on domain creation (in v2 it was
allocated when hvm param was set)
  * restored 32-item limit in mfn/gfn arrays in acquire_resource
and instead implemented hypercall continuations
  * code changes according to Jan's and Roger's review

Changed since v3:
  * vmtrace HVMOPs are not implemented as DOMCTLs
  * patches splitted up according to Andrew's comments
  * code changes according to v3 review on the mailing list

Changed since v4:
  * rebased to commit be63d9d4
  * fixed dependencies between patches
(earlier patches don't reference further patches)
  * introduced preemption check in acquire_resource
  * moved buffer allocation to common code
  * splitted some patches according to code review
  * minor fixes according to code review

Changed since v5:
  * trace buffer size is now dynamically determined by the proctrace
tool
  * trace buffer size variable is uniformly defined as uint32_t
processor_trace_buf_kb in hypervisor, toolstack and ABI
  * buffer pages are not freed explicitly but reference count is
now used instead
  * minor fixes according to code review

This patch series is available on GitHub:
https://github.com/icedevml/xen/tree/ipt-patch-v6


Michal Leszczynski (11):
  memory: batch processing in acquire_resource()
  x86/vmx: add Intel PT MSR definitions
  x86/vmx: add IPT cpu feature
  common: add vmtrace_pt_size domain parameter
  tools/libxl: add vmtrace_pt_size parameter
  x86/hvm: processor trace interface in HVM
  x86/vmx: implement IPT in VMX
  x86/mm: add vmtrace_buf resource type
  x86/domctl: add XEN_DOMCTL_vmtrace_op
  tools/libxc: add xc_vmtrace_* functions
  tools/proctrace: add proctrace tool

 docs/man/xl.cfg.5.pod.in|  13 ++
 tools/golang/xenlight/helpers.gen.go|   2 +
 tools/golang/xenlight/types.gen.go  |   1 +
 tools/libxc/Makefile|   1 +
 tools/libxc/include/xenctrl.h   |  40 +
 tools/libxc/xc_vmtrace.c|  87 ++
 tools/libxl/libxl.h |   8 +
 tools/libxl/libxl_create.c  |   1 +
 tools/libxl/libxl_types.idl |   4 +
 tools/proctrace/Makefile|  45 +
 tools/proctrace/proctrace.c | 179 
 tools/xl/xl_parse.c |  22 +++
 xen/arch/x86/domain.c   |  27 +++
 xen/arch/x86/domctl.c   |  50 ++
 xen/arch/x86/hvm/vmx/vmcs.c |  15 +-
 xen/arch/x86/hvm/vmx/vmx.c  | 110 
 xen/common/domain.c |  46 +
 xen/common/memory.c |  80 -
 xen/include/asm-x86/cpufeature.h|   1 +
 xen/include/asm-x86/hvm/hvm.h   |  20 +++
 xen/include/asm-x86/hvm/vmx/vmcs.h  |   4 +
 xen/include/asm-x86/hvm/vmx/vmx.h   |  14 ++
 xen/include/asm-x86/msr-index.h |  24 +++
 xen/include/public/arch-x86/cpufeatureset.h |   1 +
 xen/include/public/domctl.h |  29 
 xen/include/public/memory.h |   1 +
 xen/include/xen/domain.h|   2 +
 xen/include/xen/sched.h |   7 +
 28 files changed, 828 insertions(+), 6 deletions(-)
 create mode 100644 tools/libxc/xc_vmtrace.c
 create mode 100644 tools/proctrace/Makefile
 create mode 100644 tools/pr

[PATCH v6 09/11] x86/domctl: add XEN_DOMCTL_vmtrace_op

[Michał Leszczyński]

From: Michal Leszczynski 

Implement domctl to manage the runtime state of
processor trace feature.

Signed-off-by: Michal Leszczynski 
---
 xen/arch/x86/domctl.c   | 50 +
 xen/include/public/domctl.h | 28 +
 2 files changed, 78 insertions(+)

diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 6f2c69788d..6132499db4 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -322,6 +322,50 @@ void arch_get_domain_info(const struct domain *d,
 info->arch_config.emulation_flags = d->arch.emulation_flags;
 }
 
+static int do_vmtrace_op(struct domain *d, struct xen_domctl_vmtrace_op *op,
+ XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+int rc;
+struct vcpu *v;
+
+if ( op->pad1 || op->pad2 )
+return -EINVAL;
+
+if ( !vmtrace_supported )
+return -EOPNOTSUPP;
+
+if ( !is_hvm_domain(d) )
+return -EOPNOTSUPP;
+
+if ( op->vcpu >= d->max_vcpus )
+return -EINVAL;
+
+v = domain_vcpu(d, op->vcpu);
+rc = 0;
+
+switch ( op->cmd )
+{
+case XEN_DOMCTL_vmtrace_pt_enable:
+case XEN_DOMCTL_vmtrace_pt_disable:
+vcpu_pause(v);
+rc = vmtrace_control_pt(v, op->cmd == XEN_DOMCTL_vmtrace_pt_enable);
+vcpu_unpause(v);
+break;
+
+case XEN_DOMCTL_vmtrace_pt_get_offset:
+rc = vmtrace_get_pt_offset(v, &op->offset, &op->size);
+
+if ( !rc && d->is_dying )
+rc = ENODATA;
+break;
+
+default:
+rc = -EOPNOTSUPP;
+}
+
+return rc;
+}
+
 #define MAX_IOPORTS 0x1
 
 long arch_do_domctl(
@@ -337,6 +381,12 @@ long arch_do_domctl(
 switch ( domctl->cmd )
 {
 
+case XEN_DOMCTL_vmtrace_op:
+ret = do_vmtrace_op(d, &domctl->u.vmtrace_op, u_domctl);
+if ( !ret )
+copyback = true;
+break;
+
 case XEN_DOMCTL_shadow_op:
 ret = paging_domctl(d, &domctl->u.shadow_op, u_domctl, 0);
 if ( ret == -ERESTART )
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 7681675a94..73c7ccbd16 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1136,6 +1136,30 @@ struct xen_domctl_vuart_op {
  */
 };
 
+/* XEN_DOMCTL_vmtrace_op: Perform VM tracing related operation */
+#if defined(__XEN__) || defined(__XEN_TOOLS__)
+
+struct xen_domctl_vmtrace_op {
+/* IN variable */
+uint32_t cmd;
+/* Enable/disable external vmtrace for given domain */
+#define XEN_DOMCTL_vmtrace_pt_enable  1
+#define XEN_DOMCTL_vmtrace_pt_disable 2
+#define XEN_DOMCTL_vmtrace_pt_get_offset  3
+domid_t domain;
+uint16_t pad1;
+uint32_t vcpu;
+uint16_t pad2;
+
+/* OUT variable */
+uint64_aligned_t size;
+uint64_aligned_t offset;
+};
+typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t;
+DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t);
+
+#endif /* defined(__XEN__) || defined(__XEN_TOOLS__) */
+
 struct xen_domctl {
 uint32_t cmd;
 #define XEN_DOMCTL_createdomain   1
@@ -1217,6 +1241,7 @@ struct xen_domctl {
 #define XEN_DOMCTL_vuart_op  81
 #define XEN_DOMCTL_get_cpu_policy82
 #define XEN_DOMCTL_set_cpu_policy83
+#define XEN_DOMCTL_vmtrace_op84
 #define XEN_DOMCTL_gdbsx_guestmemio1000
 #define XEN_DOMCTL_gdbsx_pausevcpu 1001
 #define XEN_DOMCTL_gdbsx_unpausevcpu   1002
@@ -1277,6 +1302,9 @@ struct xen_domctl {
 struct xen_domctl_monitor_opmonitor_op;
 struct xen_domctl_psr_alloc psr_alloc;
 struct xen_domctl_vuart_op  vuart_op;
+#if defined(__XEN__) || defined(__XEN_TOOLS__)
+struct xen_domctl_vmtrace_opvmtrace_op;
+#endif
 uint8_t pad[128];
 } u;
 };
-- 
2.17.1

[PATCH v6 07/11] x86/vmx: implement IPT in VMX

[Michał Leszczyński]

From: Michal Leszczynski 

Use Intel Processor Trace feature to provide vmtrace_pt_*
interface for HVM/VMX.

Signed-off-by: Michal Leszczynski 
---
 xen/arch/x86/hvm/vmx/vmx.c | 110 +
 xen/include/asm-x86/hvm/vmx/vmcs.h |   3 +
 xen/include/asm-x86/hvm/vmx/vmx.h  |  14 
 3 files changed, 127 insertions(+)

diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index cc6d4ece22..63a5a76e16 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -428,6 +428,56 @@ static void vmx_domain_relinquish_resources(struct domain 
*d)
 vmx_free_vlapic_mapping(d);
 }
 
+static int vmx_init_pt(struct vcpu *v)
+{
+int rc;
+uint64_t size = v->domain->processor_trace_buf_kb * KB(1);
+
+if ( !v->vmtrace.pt_buf || !size )
+return -EINVAL;
+
+/*
+ * We don't accept trace buffer size smaller than single page
+ * and the upper bound is defined as 4GB in the specification.
+ * The buffer size must be also a power of 2.
+ */
+if ( size < PAGE_SIZE || size > GB(4) || (size & (size - 1)) )
+return -EINVAL;
+
+v->arch.hvm.vmx.ipt_state = xzalloc(struct ipt_state);
+
+if ( !v->arch.hvm.vmx.ipt_state )
+return -ENOMEM;
+
+v->arch.hvm.vmx.ipt_state->output_base =
+page_to_maddr(v->vmtrace.pt_buf);
+v->arch.hvm.vmx.ipt_state->output_mask.raw = size - 1;
+
+rc = vmx_add_host_load_msr(v, MSR_RTIT_CTL, 0);
+
+if ( rc )
+return rc;
+
+rc = vmx_add_guest_msr(v, MSR_RTIT_CTL,
+  RTIT_CTL_TRACE_EN | RTIT_CTL_OS |
+  RTIT_CTL_USR | RTIT_CTL_BRANCH_EN);
+
+if ( rc )
+return rc;
+
+return 0;
+}
+
+static int vmx_destroy_pt(struct vcpu* v)
+{
+if ( v->arch.hvm.vmx.ipt_state )
+xfree(v->arch.hvm.vmx.ipt_state);
+
+v->arch.hvm.vmx.ipt_state = NULL;
+return 0;
+}
+
+
 static int vmx_vcpu_initialise(struct vcpu *v)
 {
 int rc;
@@ -471,6 +521,14 @@ static int vmx_vcpu_initialise(struct vcpu *v)
 
 vmx_install_vlapic_mapping(v);
 
+if ( v->domain->processor_trace_buf_kb )
+{
+rc = vmx_init_pt(v);
+
+if ( rc )
+return rc;
+}
+
 return 0;
 }
 
@@ -483,6 +541,7 @@ static void vmx_vcpu_destroy(struct vcpu *v)
  * prior to vmx_domain_destroy so we need to disable PML for each vcpu
  * separately here.
  */
+vmx_destroy_pt(v);
 vmx_vcpu_disable_pml(v);
 vmx_destroy_vmcs(v);
 passive_domain_destroy(v);
@@ -513,6 +572,18 @@ static void vmx_save_guest_msrs(struct vcpu *v)
  * be updated at any time via SWAPGS, which we cannot trap.
  */
 v->arch.hvm.vmx.shadow_gs = rdgsshadow();
+
+if ( unlikely(v->arch.hvm.vmx.ipt_state &&
+  v->arch.hvm.vmx.ipt_state->active) )
+{
+uint64_t rtit_ctl;
+rdmsrl(MSR_RTIT_CTL, rtit_ctl);
+BUG_ON(rtit_ctl & RTIT_CTL_TRACE_EN);
+
+rdmsrl(MSR_RTIT_STATUS, v->arch.hvm.vmx.ipt_state->status);
+rdmsrl(MSR_RTIT_OUTPUT_MASK,
+   v->arch.hvm.vmx.ipt_state->output_mask.raw);
+}
 }
 
 static void vmx_restore_guest_msrs(struct vcpu *v)
@@ -524,6 +595,17 @@ static void vmx_restore_guest_msrs(struct vcpu *v)
 
 if ( cpu_has_msr_tsc_aux )
 wrmsr_tsc_aux(v->arch.msrs->tsc_aux);
+
+if ( unlikely(v->arch.hvm.vmx.ipt_state &&
+  v->arch.hvm.vmx.ipt_state->active) )
+{
+wrmsrl(MSR_RTIT_OUTPUT_BASE,
+   v->arch.hvm.vmx.ipt_state->output_base);
+wrmsrl(MSR_RTIT_OUTPUT_MASK,
+   v->arch.hvm.vmx.ipt_state->output_mask.raw);
+wrmsrl(MSR_RTIT_STATUS,
+   v->arch.hvm.vmx.ipt_state->status);
+}
 }
 
 void vmx_update_cpu_exec_control(struct vcpu *v)
@@ -2240,6 +2322,25 @@ static bool vmx_get_pending_event(struct vcpu *v, struct 
x86_event *info)
 return true;
 }
 
+static int vmx_control_pt(struct vcpu *v, bool enable)
+{
+if ( !v->arch.hvm.vmx.ipt_state )
+return -EINVAL;
+
+v->arch.hvm.vmx.ipt_state->active = enable;
+return 0;
+}
+
+static int vmx_get_pt_offset(struct vcpu *v, uint64_t *offset, uint64_t *size)
+{
+if ( !v->arch.hvm.vmx.ipt_state )
+return -EINVAL;
+
+*offset = v->arch.hvm.vmx.ipt_state->output_mask.offset;
+*size = v->arch.hvm.vmx.ipt_state->output_mask.size + 1;
+return 0;
+}
+
 static struct hvm_function_table __initdata vmx_function_table = {
 .name = "VMX",
 .cpu_up_prepare   = vmx_cpu_up_prepare,
@@ -2295,6 +2396,8 @@ static struct hvm_function_table __initdata 
vmx_function_table = {
 .altp2m_vcpu_update_vmfunc_ve = vmx_vcpu_update_vmfunc_ve,
 .altp2m_vcpu_emulate_ve = vmx_vcpu_emulate_ve,
 .altp2m_vcpu_emulate_vmfunc = vmx_vcpu_emulate_vmfunc,
+.vmtrace_control_pt = vmx_control_pt,
+.vmtrace_get_pt_offset = vmx_get_pt_offset,
 .tsc_scaling = {
 .max_ratio = VMX_TSC_MULTIPLIER_MAX,
 },
@@ 

[PATCH v6 05/11] tools/libxl: add vmtrace_pt_size parameter

[Michał Leszczyński]

From: Michal Leszczynski 

Allow to specify the size of per-vCPU trace buffer upon
domain creation. This is zero by default (meaning: not enabled).

Signed-off-by: Michal Leszczynski 
---
 docs/man/xl.cfg.5.pod.in | 13 +
 tools/golang/xenlight/helpers.gen.go |  2 ++
 tools/golang/xenlight/types.gen.go   |  1 +
 tools/libxl/libxl.h  |  8 
 tools/libxl/libxl_create.c   |  1 +
 tools/libxl/libxl_types.idl  |  4 
 tools/xl/xl_parse.c  | 22 ++
 7 files changed, 51 insertions(+)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 0532739c1f..ddef9b6014 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -683,6 +683,19 @@ If this option is not specified then it will default to 
B.
 
 =back
 
+=item B
+
+Specifies the size of processor trace buffer that would be allocated
+for each vCPU belonging to this domain. Disabled (i.e.
+B by default. This must be set to
+non-zero value in order to be able to use processor tracing features
+with this domain.
+
+B: In order to use Intel Processor Trace feature, this value
+must be between 8 kB and 4 GB and it must be a power of 2.
+
+=back
+
 =head2 Devices
 
 The following options define the paravirtual, emulated and physical
diff --git a/tools/golang/xenlight/helpers.gen.go 
b/tools/golang/xenlight/helpers.gen.go
index 152c7e8e6b..3ce6f2374b 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1117,6 +1117,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version)
 x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart)
 x.Altp2M = Altp2MMode(xc.altp2m)
+x.ProcessorTraceBufKb = int(xc.processor_trace_buf_kb)
 
  return nil}
 
@@ -1592,6 +1593,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)}
 xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion)
 xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart)
 xc.altp2m = C.libxl_altp2m_mode(x.Altp2M)
+xc.processor_trace_buf_kb = C.int(x.ProcessorTraceBufKb)
 
  return nil
  }
diff --git a/tools/golang/xenlight/types.gen.go 
b/tools/golang/xenlight/types.gen.go
index 663c1e86b4..f4bc16c0fd 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -516,6 +516,7 @@ GicVersion GicVersion
 Vuart VuartType
 }
 Altp2M Altp2MMode
+ProcessorTraceBufKb int
 }
 
 type domainBuildInfoTypeUnion interface {
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 1cd6c38e83..fbf222967a 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -438,6 +438,14 @@
  */
 #define LIBXL_HAVE_CREATEINFO_PASSTHROUGH 1
 
+/*
+ * LIBXL_HAVE_PROCESSOR_TRACE_BUF_KB indicates that
+ * libxl_domain_create_info has a processor_trace_buf_kb parameter, which
+ * allows to enable pre-allocation of processor tracing buffers of given
+ * size.
+ */
+#define LIBXL_HAVE_PROCESSOR_TRACE_BUF_KB 1
+
 /*
  * libxl ABI compatibility
  *
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 2814818e34..4d6318124a 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -608,6 +608,7 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config 
*d_config,
 .max_evtchn_port = b_info->event_channels,
 .max_grant_frames = b_info->max_grant_frames,
 .max_maptrack_frames = b_info->max_maptrack_frames,
+.processor_trace_buf_kb = b_info->processor_trace_buf_kb,
 };
 
 if (info->type != LIBXL_DOMAIN_TYPE_PV) {
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 9d3f05f399..748fde65ab 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -645,6 +645,10 @@ libxl_domain_build_info = Struct("domain_build_info",[
 # supported by x86 HVM and ARM support is planned.
 ("altp2m", libxl_altp2m_mode),
 
+# Size of preallocated processor trace buffers (in KBYTES).
+# Use zero value to disable this feature.
+("processor_trace_buf_kb", integer),
+
 ], dir=DIR_IN,
copy_deprecated_fn="libxl__domain_build_info_copy_deprecated",
 )
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 61b4ef7b7e..87e373b413 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1861,6 +1861,28 @@ void parse_config_data(const char *config_source,
 }
 }
 
+if (!xlu_cfg_get_long(config, "processor_trace_buf_kb", &l, 1) && l) {
+if (l & (l - 1)) {
+fprintf(stderr, "ERROR: processor_trace_buf_kb"
+" - must be a power of 2\n");
+exit(1);
+}
+
+if (l < 8) {
+fprintf(stderr, "ERROR: processor_trace_buf_kb"
+" - value is too small\n");
+exit(1);
+}
+
+if (l > 1024*1024*4) {
+fprintf(stderr, "ERROR: processor_trace_buf_kb"
+" - value is

[PATCH v6 08/11] x86/mm: add vmtrace_buf resource type

[Michał Leszczyński]

From: Michal Leszczynski 

Allow to map processor trace buffer using
acquire_resource().

Signed-off-by: Michal Leszczynski 
---
 xen/common/memory.c | 31 +++
 xen/include/public/memory.h |  1 +
 2 files changed, 32 insertions(+)

diff --git a/xen/common/memory.c b/xen/common/memory.c
index eb42f883df..c0a22eb60f 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -1007,6 +1007,32 @@ static long xatp_permission_check(struct domain *d, 
unsigned int space)
 return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
 }
 
+static int acquire_vmtrace_buf(struct domain *d, unsigned int id,
+   uint64_t frame,
+   uint64_t nr_frames,
+   xen_pfn_t mfn_list[])
+{
+mfn_t mfn;
+unsigned int i;
+uint64_t size;
+struct vcpu *v = domain_vcpu(d, id);
+
+if ( !v || !v->vmtrace.pt_buf )
+return -EINVAL;
+
+mfn = page_to_mfn(v->vmtrace.pt_buf);
+size = v->domain->processor_trace_buf_kb * KB(1);
+
+if ( (frame > (size >> PAGE_SHIFT)) ||
+ (nr_frames > ((size >> PAGE_SHIFT) - frame)) )
+return -EINVAL;
+
+for ( i = 0; i < nr_frames; i++ )
+mfn_list[i] = mfn_x(mfn_add(mfn, frame + i));
+
+return 0;
+}
+
 static int acquire_grant_table(struct domain *d, unsigned int id,
unsigned long frame,
unsigned int nr_frames,
@@ -1117,6 +1143,11 @@ static int acquire_resource(
  mfn_list);
 break;
 
+case XENMEM_resource_vmtrace_buf:
+rc = acquire_vmtrace_buf(d, xmar.id, xmar.frame, xmar.nr_frames,
+ mfn_list);
+break;
+
 default:
 rc = arch_acquire_resource(d, xmar.type, xmar.id, xmar.frame,
xmar.nr_frames, mfn_list);
diff --git a/xen/include/public/memory.h b/xen/include/public/memory.h
index 21057ed78e..f4c905a10e 100644
--- a/xen/include/public/memory.h
+++ b/xen/include/public/memory.h
@@ -625,6 +625,7 @@ struct xen_mem_acquire_resource {
 
 #define XENMEM_resource_ioreq_server 0
 #define XENMEM_resource_grant_table 1
+#define XENMEM_resource_vmtrace_buf 2
 
 /*
  * IN - a type-specific resource identifier, which must be zero
-- 
2.17.1

[PATCH v6 11/11] tools/proctrace: add proctrace tool

[Michał Leszczyński]

From: Michal Leszczynski 

Add an demonstration tool that uses xc_vmtrace_* calls in order
to manage external IPT monitoring for DomU.

Signed-off-by: Michal Leszczynski 
---
 tools/proctrace/Makefile|  45 +
 tools/proctrace/proctrace.c | 179 
 2 files changed, 224 insertions(+)
 create mode 100644 tools/proctrace/Makefile
 create mode 100644 tools/proctrace/proctrace.c

diff --git a/tools/proctrace/Makefile b/tools/proctrace/Makefile
new file mode 100644
index 00..9c135229b9
--- /dev/null
+++ b/tools/proctrace/Makefile
@@ -0,0 +1,45 @@
+# Copyright (C) CERT Polska - NASK PIB
+# Author: Michał Leszczyński 
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; under version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+XEN_ROOT=$(CURDIR)/../..
+include $(XEN_ROOT)/tools/Rules.mk
+
+CFLAGS  += -Werror
+CFLAGS  += $(CFLAGS_libxenevtchn)
+CFLAGS  += $(CFLAGS_libxenctrl)
+LDLIBS  += $(LDLIBS_libxenctrl)
+LDLIBS  += $(LDLIBS_libxenevtchn)
+LDLIBS  += $(LDLIBS_libxenforeignmemory)
+
+.PHONY: all
+all: build
+
+.PHONY: build
+build: proctrace
+
+.PHONY: install
+install: build
+   $(INSTALL_DIR) $(DESTDIR)$(sbindir)
+   $(INSTALL_PROG) proctrace $(DESTDIR)$(sbindir)/proctrace
+
+.PHONY: uninstall
+uninstall:
+   rm -f $(DESTDIR)$(sbindir)/proctrace
+
+.PHONY: clean
+clean:
+   $(RM) -f proctrace $(DEPS_RM)
+
+.PHONY: distclean
+distclean: clean
+
+-include $(DEPS_INCLUDE)
diff --git a/tools/proctrace/proctrace.c b/tools/proctrace/proctrace.c
new file mode 100644
index 00..3c1ee8
--- /dev/null
+++ b/tools/proctrace/proctrace.c
@@ -0,0 +1,179 @@
+/**
+ * tools/proctrace.c
+ *
+ * Demonstrative tool for collecting Intel Processor Trace data from Xen.
+ *  Could be used to externally monitor a given vCPU in given DomU.
+ *
+ * Copyright (C) 2020 by CERT Polska - NASK PIB
+ *
+ * Authors: Michał Leszczyński, michal.leszczyn...@cert.pl
+ * Date:June, 2020
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; under version 2 of the License.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; If not, see .
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+
+volatile int interrupted = 0;
+volatile int domain_down = 0;
+
+void term_handler(int signum) {
+interrupted = 1;
+}
+
+int main(int argc, char* argv[]) {
+xc_interface *xc;
+uint32_t domid;
+uint32_t vcpu_id;
+uint64_t size;
+
+int rc = -1;
+uint8_t *buf = NULL;
+uint64_t last_offset = 0;
+
+xenforeignmemory_handle *fmem;
+xenforeignmemory_resource_handle *fres;
+
+if (signal(SIGINT, term_handler) == SIG_ERR)
+{
+fprintf(stderr, "Failed to register signal handler\n");
+return 1;
+}
+
+if (argc != 3) {
+fprintf(stderr, "Usage: %s  \n", argv[0]);
+fprintf(stderr, "It's recommended to redirect this"
+"program's output to file\n");
+fprintf(stderr, "or to pipe it's output to xxd or other program.\n");
+return 1;
+}
+
+domid = atoi(argv[1]);
+vcpu_id = atoi(argv[2]);
+
+xc = xc_interface_open(0, 0, 0);
+
+fmem = xenforeignmemory_open(0, 0);
+
+if (!xc) {
+fprintf(stderr, "Failed to open xc interface\n");
+return 1;
+}
+
+rc = xc_vmtrace_pt_enable(xc, domid, vcpu_id);
+
+if (rc) {
+fprintf(stderr, "Failed to call xc_vmtrace_pt_enable\n");
+return 1;
+}
+
+rc = xc_vmtrace_pt_get_offset(xc, domid, vcpu_id, NULL, &size);
+
+if (rc) {
+fprintf(stderr, "Failed to get trace buffer size\n");
+return 1;
+}
+
+fres = xenforeignmemory_map_resource(
+fmem, domid, XENMEM_resource_vmtrace_buf,
+/* vcpu: */ vcpu_id,
+/* frame: */ 0,
+/* num_frames: */ size >> XC_PAGE_SHIFT,
+(void **)&buf,
+PROT_READ, 0);
+
+if (!buf) {
+fprintf(stderr, "Failed to map trace buffer\n");
+return 1;
+}
+
+while (!interrupted) {
+uint64_t offset;
+rc = xc_vmtrace_pt_get_

[PATCH v6 10/11] tools/libxc: add xc_vmtrace_* functions

[Michał Leszczyński]

From: Michal Leszczynski 

Add functions in libxc that use the new XEN_DOMCTL_vmtrace interface.

Signed-off-by: Michal Leszczynski 
---
 tools/libxc/Makefile  |  1 +
 tools/libxc/include/xenctrl.h | 40 
 tools/libxc/xc_vmtrace.c  | 87 +++
 3 files changed, 128 insertions(+)
 create mode 100644 tools/libxc/xc_vmtrace.c

diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile
index fae5969a73..605e44501d 100644
--- a/tools/libxc/Makefile
+++ b/tools/libxc/Makefile
@@ -27,6 +27,7 @@ CTRL_SRCS-y   += xc_csched2.c
 CTRL_SRCS-y   += xc_arinc653.c
 CTRL_SRCS-y   += xc_rt.c
 CTRL_SRCS-y   += xc_tbuf.c
+CTRL_SRCS-y   += xc_vmtrace.c
 CTRL_SRCS-y   += xc_pm.c
 CTRL_SRCS-y   += xc_cpu_hotplug.c
 CTRL_SRCS-y   += xc_resume.c
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 4c89b7294c..491b2c3236 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -1585,6 +1585,46 @@ int xc_tbuf_set_cpu_mask(xc_interface *xch, xc_cpumap_t 
mask);
 
 int xc_tbuf_set_evt_mask(xc_interface *xch, uint32_t mask);
 
+/**
+ * Enable processor trace for given vCPU in given DomU.
+ * Allocate the trace ringbuffer with given size.
+ *
+ * @parm xch a handle to an open hypervisor interface
+ * @parm domid domain identifier
+ * @parm vcpu vcpu identifier
+ * @return 0 on success, -1 on failure
+ */
+int xc_vmtrace_pt_enable(xc_interface *xch, uint32_t domid,
+ uint32_t vcpu);
+
+/**
+ * Disable processor trace for given vCPU in given DomU.
+ * Deallocate the trace ringbuffer.
+ *
+ * @parm xch a handle to an open hypervisor interface
+ * @parm domid domain identifier
+ * @parm vcpu vcpu identifier
+ * @return 0 on success, -1 on failure
+ */
+int xc_vmtrace_pt_disable(xc_interface *xch, uint32_t domid,
+  uint32_t vcpu);
+
+/**
+ * Get current offset inside the trace ringbuffer.
+ * This allows to determine how much data was written into the buffer.
+ * Once buffer overflows, the offset will reset to 0 and the previous
+ * data will be overriden.
+ *
+ * @parm xch a handle to an open hypervisor interface
+ * @parm domid domain identifier
+ * @parm vcpu vcpu identifier
+ * @parm offset current offset inside trace buffer will be written there
+ * @parm size the total size of the trace buffer (in bytes)
+ * @return 0 on success, -1 on failure
+ */
+int xc_vmtrace_pt_get_offset(xc_interface *xch, uint32_t domid,
+ uint32_t vcpu, uint64_t *offset, uint64_t *size);
+
 int xc_domctl(xc_interface *xch, struct xen_domctl *domctl);
 int xc_sysctl(xc_interface *xch, struct xen_sysctl *sysctl);
 
diff --git a/tools/libxc/xc_vmtrace.c b/tools/libxc/xc_vmtrace.c
new file mode 100644
index 00..ee034da8d3
--- /dev/null
+++ b/tools/libxc/xc_vmtrace.c
@@ -0,0 +1,87 @@
+/**
+ * xc_vmtrace.c
+ *
+ * API for manipulating hardware tracing features
+ *
+ * Copyright (c) 2020, Michal Leszczynski
+ *
+ * Copyright 2020 CERT Polska. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; If not, see .
+ */
+
+#include "xc_private.h"
+#include 
+
+int xc_vmtrace_pt_enable(
+xc_interface *xch, uint32_t domid, uint32_t vcpu)
+{
+DECLARE_DOMCTL;
+int rc;
+
+domctl.cmd = XEN_DOMCTL_vmtrace_op;
+domctl.domain = domid;
+domctl.u.vmtrace_op.cmd = XEN_DOMCTL_vmtrace_pt_enable;
+domctl.u.vmtrace_op.vcpu = vcpu;
+domctl.u.vmtrace_op.pad1 = 0;
+domctl.u.vmtrace_op.pad2 = 0;
+
+rc = do_domctl(xch, &domctl);
+return rc;
+}
+
+int xc_vmtrace_pt_get_offset(
+xc_interface *xch, uint32_t domid, uint32_t vcpu,
+uint64_t *offset, uint64_t *size)
+{
+DECLARE_DOMCTL;
+int rc;
+
+domctl.cmd = XEN_DOMCTL_vmtrace_op;
+domctl.domain = domid;
+domctl.u.vmtrace_op.cmd = XEN_DOMCTL_vmtrace_pt_get_offset;
+domctl.u.vmtrace_op.vcpu = vcpu;
+domctl.u.vmtrace_op.pad1 = 0;
+domctl.u.vmtrace_op.pad2 = 0;
+
+rc = do_domctl(xch, &domctl);
+if ( !rc )
+{
+if (offset)
+*offset = domctl.u.vmtrace_op.offset;
+
+if (size)
+*size = domctl.u.vmtrace_op.size;
+}
+
+return rc;
+}
+
+int xc_vmtrace_pt_disable(xc_interface *xch, uint32_t domid

[PATCH v6 06/11] x86/hvm: processor trace interface in HVM

[Michał Leszczyński]

From: Michal Leszczynski 

Implement necessary changes in common code/HVM to support
processor trace features. Define vmtrace_pt_* API and
implement trace buffer allocation/deallocation in common
code.

Signed-off-by: Michal Leszczynski 
---
 xen/arch/x86/domain.c | 21 +
 xen/common/domain.c   | 35 +++
 xen/include/asm-x86/hvm/hvm.h | 20 
 xen/include/xen/sched.h   |  4 
 4 files changed, 80 insertions(+)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index b75017b28b..8ce2ab6b8f 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -2205,6 +2205,27 @@ int domain_relinquish_resources(struct domain *d)
 altp2m_vcpu_disable_ve(v);
 }
 
+for_each_vcpu ( d, v )
+{
+unsigned int i;
+uint64_t nr_pages = v->domain->processor_trace_buf_kb * KB(1);
+nr_pages >>= PAGE_SHIFT;
+
+if ( !v->vmtrace.pt_buf )
+continue;
+
+for ( i = 0; i < nr_pages; i++ )
+{
+struct page_info *pg = mfn_to_page(
+mfn_add(page_to_mfn(v->vmtrace.pt_buf), i));
+
+put_page_alloc_ref(pg);
+put_page_and_type(pg);
+}
+
+v->vmtrace.pt_buf = NULL;
+}
+
 if ( is_pv_domain(d) )
 {
 for_each_vcpu ( d, v )
diff --git a/xen/common/domain.c b/xen/common/domain.c
index e6e8f88da1..193099a2ab 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -137,6 +137,38 @@ static void vcpu_destroy(struct vcpu *v)
 free_vcpu_struct(v);
 }
 
+static int vmtrace_alloc_buffers(struct vcpu *v)
+{
+unsigned int i;
+struct page_info *pg;
+uint64_t size = v->domain->processor_trace_buf_kb * KB(1);
+
+pg = alloc_domheap_pages(v->domain, get_order_from_bytes(size),
+ MEMF_no_refcount);
+
+if ( !pg )
+return -ENOMEM;
+
+for ( i = 0; i < (size >> PAGE_SHIFT); i++ )
+{
+struct page_info *pg_iter = mfn_to_page(
+mfn_add(page_to_mfn(pg), i));
+
+if ( !get_page_and_type(pg_iter, v->domain, PGT_writable_page) )
+{
+/*
+ * The domain can't possibly know about this page yet, so failure
+ * here is a clear indication of something fishy going on.
+ */
+domain_crash(v->domain);
+return -ENODATA;
+}
+}
+
+v->vmtrace.pt_buf = pg;
+return 0;
+}
+
 struct vcpu *vcpu_create(struct domain *d, unsigned int vcpu_id)
 {
 struct vcpu *v;
@@ -162,6 +194,9 @@ struct vcpu *vcpu_create(struct domain *d, unsigned int 
vcpu_id)
 v->vcpu_id = vcpu_id;
 v->dirty_cpu = VCPU_CPU_CLEAN;
 
+if ( d->processor_trace_buf_kb && vmtrace_alloc_buffers(v) != 0 )
+return NULL;
+
 spin_lock_init(&v->virq_lock);
 
 tasklet_init(&v->continue_hypercall_tasklet, NULL, NULL);
diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h
index 1eb377dd82..476a216205 100644
--- a/xen/include/asm-x86/hvm/hvm.h
+++ b/xen/include/asm-x86/hvm/hvm.h
@@ -214,6 +214,10 @@ struct hvm_function_table {
 bool_t (*altp2m_vcpu_emulate_ve)(struct vcpu *v);
 int (*altp2m_vcpu_emulate_vmfunc)(const struct cpu_user_regs *regs);
 
+/* vmtrace */
+int (*vmtrace_control_pt)(struct vcpu *v, bool enable);
+int (*vmtrace_get_pt_offset)(struct vcpu *v, uint64_t *offset, uint64_t 
*size);
+
 /*
  * Parameters and callbacks for hardware-assisted TSC scaling,
  * which are valid only when the hardware feature is available.
@@ -655,6 +659,22 @@ static inline bool altp2m_vcpu_emulate_ve(struct vcpu *v)
 return false;
 }
 
+static inline int vmtrace_control_pt(struct vcpu *v, bool enable)
+{
+if ( hvm_funcs.vmtrace_control_pt )
+return hvm_funcs.vmtrace_control_pt(v, enable);
+
+return -EOPNOTSUPP;
+}
+
+static inline int vmtrace_get_pt_offset(struct vcpu *v, uint64_t *offset, 
uint64_t *size)
+{
+if ( hvm_funcs.vmtrace_get_pt_offset )
+return hvm_funcs.vmtrace_get_pt_offset(v, offset, size);
+
+return -EOPNOTSUPP;
+}
+
 /*
  * This must be defined as a macro instead of an inline function,
  * because it uses 'struct vcpu' and 'struct domain' which have
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index c046e59886..b6f39233aa 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -253,6 +253,10 @@ struct vcpu
 /* vPCI per-vCPU area, used to store data for long running operations. */
 struct vpci_vcpu vpci;
 
+struct {
+struct page_info *pt_buf;
+} vmtrace;
+
 struct arch_vcpu arch;
 };
 
-- 
2.17.1

Re: [PATCH v12 7/8] nbd: Use ERRP_AUTO_PROPAGATE()

[Eric Blake]

On 7/7/20 11:50 AM, Markus Armbruster wrote:

From: Vladimir Sementsov-Ogievskiy 

If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call).  Fix such cases.

If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
&error_fatal, this means that we don't break error_abort
(we'll abort on error_set, not on error_propagate)

This commit is generated by command

 sed -n '/^Network Block Device (NBD)$/,/^$/{s/^F: //p}' \
 MAINTAINERS | \
 xargs git ls-files | grep '\.[hc]$' | \
 xargs spatch \
 --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
 --macro-file scripts/cocci-macro-file.h \
 --in-place --no-show-diff --max-width 80

Reported-by: Kevin Wolf 
Reported-by: Greg Kurz 
Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Markus Armbruster 
[Commit message tweaked]
Signed-off-by: Markus Armbruster 
---


Reviewed-by: Eric Blake 

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH v12 1/8] error: New macro ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

Eric Blake  writes:

> On 7/7/20 11:50 AM, Markus Armbruster wrote:
>> From: Vladimir Sementsov-Ogievskiy 
>>
>> Introduce a new ERRP_AUTO_PROPAGATE macro, to be used at start of
>> functions with an errp OUT parameter.
>>
>> It has three goals:
>>
>> 1. Fix issue with error_fatal and error_prepend/error_append_hint: user
>
> the user

Yes.

>> can't see this additional information, because exit() happens in
>> error_setg earlier than information is added. [Reported by Greg Kurz]
>>
>> 2. Fix issue with error_abort and error_propagate: when we wrap
>> error_abort by local_err+error_propagate, the resulting coredump will
>> refer to error_propagate and not to the place where error happened.
>> (the macro itself doesn't fix the issue, but it allows us to [3.] drop
>> the local_err+error_propagate pattern, which will definitely fix the
>> issue) [Reported by Kevin Wolf]
>>
>> 3. Drop local_err+error_propagate pattern, which is used to workaround
>> void functions with errp parameter, when caller wants to know resulting
>> status. (Note: actually these functions could be merely updated to
>> return int error code).
>>
>> To achieve these goals, later patches will add invocations
>> of this macro at the start of functions with either use
>> error_prepend/error_append_hint (solving 1) or which use
>> local_err+error_propagate to check errors, switching those
>> functions to use *errp instead (solving 2 and 3).
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy 
>> Reviewed-by: Paul Durrant 
>> Reviewed-by: Greg Kurz 
>> Reviewed-by: Eric Blake 
>> [Comments merged properly with recent commit "error: Document Error
>> API usage rules", and edited for clarity.  Put ERRP_AUTO_PROPAGATE()
>> before its helpers, and touch up style.  Commit message tweaked.]
>> Signed-off-by: Markus Armbruster 
>> ---
>>   include/qapi/error.h | 160 ++-
>>   1 file changed, 141 insertions(+), 19 deletions(-)
>>
>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>> index 3fed49747d..c865a7d2f1 100644
>> --- a/include/qapi/error.h
>> +++ b/include/qapi/error.h
>
>> @@ -128,18 +122,26 @@
>>* handle the error...
>>* }
>>* when it doesn't, say a void function:
>> + * ERRP_AUTO_PROPAGATE();
>> + * foo(arg, errp);
>> + * if (*errp) {
>> + * handle the error...
>> + * }
>> + * More on ERRP_AUTO_PROPAGATE() below.
>> + *
>> + * Code predating ERRP_AUTO_PROPAGATE() still exits, and looks like this:
>
> exists

Fixing...

>>* Error *err = NULL;
>>* foo(arg, &err);
>>* if (err) {
>>* handle the error...
>> - * error_propagate(errp, err);
>> + * error_propagate(errp, err); // deprecated
>>* }
>> - * Do *not* "optimize" this to
>> + * Avoid in new code.  Do *not* "optimize" it to
>>* foo(arg, errp);
>>* if (*errp) { // WRONG!
>>* handle the error...
>>* }
>> - * because errp may be NULL!
>> + * because errp may be NULL!  Guard with ERRP_AUTO_PROPAGATE().
>
> maybe:
>
> because errp may be NULL without the ERRP_AUTO_PROPAGATE() guard.

Sold.

>>*
>>* But when all you do with the error is pass it on, please use
>>* foo(arg, errp);
>> @@ -158,6 +160,19 @@
>>* handle the error...
>>* }
>>*
>> + * Pass an existing error to the caller:
>
>> + * = Converting to ERRP_AUTO_PROPAGATE() =
>> + *
>> + * To convert a function to use ERRP_AUTO_PROPAGATE():
>> + *
>> + * 0. If the Error ** parameter is not named @errp, rename it to
>> + *@errp.
>> + *
>> + * 1. Add an ERRP_AUTO_PROPAGATE() invocation, by convention right at
>> + *the beginning of the function.  This makes @errp safe to use.
>> + *
>> + * 2. Replace &err by errp, and err by *errp.  Delete local variable
>> + *@err.
>> + *
>> + * 3. Delete error_propagate(errp, *errp), replace
>> + *error_propagate_prepend(errp, *errp, ...) by error_prepend(errp, ...),
>> + *
>
> Why a comma here?

Editing accident.

>> + * 4. Ensure @errp is valid at return: when you destroy *errp, set
>> + *errp = NULL.
>> + *
>> + * Example:
>> + *
>> + * bool fn(..., Error **errp)
>> + * {
>> + * Error *err = NULL;
>> + *
>> + * foo(arg, &err);
>> + * if (err) {
>> + * handle the error...
>> + * error_propagate(errp, err);
>> + * return false;
>> + * }
>> + * ...
>> + * }
>> + *
>> + * becomes
>> + *
>> + * bool fn(..., Error **errp)
>> + * {
>> + * ERRP_AUTO_PROPAGATE();
>> + *
>> + * foo(arg, errp);
>> + * if (*errp) {
>> + * handle the error...
>> + * return false;
>> + * }
>> + * ...
>> + * }
>
> Do we want the example to show the use of error_free and *errp = NULL?

Yes, but we're running out of time, so let's do it in the series that
introduces the usage to the code.

> Otherwise, this is looking good to me.  It w

Re: [PATCH v12 1/8] error: New macro ERRP_AUTO_PROPAGATE()

[Vladimir Sementsov-Ogievskiy]

07.07.2020 19:50, Markus Armbruster wrote:

From: Vladimir Sementsov-Ogievskiy

Introduce a new ERRP_AUTO_PROPAGATE macro, to be used at start of
functions with an errp OUT parameter.

It has three goals:

1. Fix issue with error_fatal and error_prepend/error_append_hint: user
can't see this additional information, because exit() happens in
error_setg earlier than information is added. [Reported by Greg Kurz]

2. Fix issue with error_abort and error_propagate: when we wrap
error_abort by local_err+error_propagate, the resulting coredump will
refer to error_propagate and not to the place where error happened.
(the macro itself doesn't fix the issue, but it allows us to [3.] drop
the local_err+error_propagate pattern, which will definitely fix the
issue) [Reported by Kevin Wolf]

3. Drop local_err+error_propagate pattern, which is used to workaround
void functions with errp parameter, when caller wants to know resulting
status. (Note: actually these functions could be merely updated to
return int error code).

To achieve these goals, later patches will add invocations
of this macro at the start of functions with either use
error_prepend/error_append_hint (solving 1) or which use
local_err+error_propagate to check errors, switching those
functions to use *errp instead (solving 2 and 3).

Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Paul Durrant
Reviewed-by: Greg Kurz
Reviewed-by: Eric Blake
[Comments merged properly with recent commit "error: Document Error
API usage rules", and edited for clarity.  Put ERRP_AUTO_PROPAGATE()
before its helpers, and touch up style.  Commit message tweaked.]
Signed-off-by: Markus Armbruster


Ok, I see you have mostly rewritten the big comment and not only in this patch, 
so, I go and read the whole comment on top of these series.

=

   * Pass an existing error to the caller with the message modified:
   * error_propagate_prepend(errp, err,
   * "Could not frobnicate '%s': ", name);
   * This is more concise than
   * error_propagate(errp, err); // don't do this
   * error_prepend(errp, "Could not frobnicate '%s': ", name);
   * and works even when @errp is &error_fatal.

- the latter doesn't consider ERRP_AUTO_PROPAGATE: as we know, that 
ERRP_AUTO_PROPAGATE should be used when we use error_prepend, the latter should 
look like


ERRP_AUTO_PROPAGATE();
...
error_propagate(errp, err); // don't do this
error_prepend(errp, "Could not frobnicate '%s': ", name);

- and it works even when @errp is &error_fatal, so the error_propagate_prepend 
now is just a shortcut, not the only correct way.


Still, the text is formally correct as is, and may be improved later.

=

   * 2. Replace &err by errp, and err by *errp.  Delete local variable
   *@err.

- hmm a bit not obvious,, It can be local_err. It can be (in some rare cases) 
still needed to handle the error locally, not passing to the caller..

may be just something like "Assume local Error *err variable is used to get errors 
from called functions and than propagated to caller's errp" before paragraph [2.] 
will help.


   *
   * 3. Delete error_propagate(errp, *errp), replace
   *error_propagate_prepend(errp, *errp, ...) by error_prepend(errp, ...),
   *
   * 4. Ensure @errp is valid at return: when you destroy *errp, set
   *errp = NULL.

=


May be good to add note about ERRP_AUTO_PROPAGATE() into comment above 
error_append_hint (and error_(v)prepend)).



=

  /*
   * Make @errp parameter easier to use regardless of argument value

may be s/argument/its/

   *
   * This macro is for use right at the beginning of a function that
   * takes an Error **errp parameter to pass errors to its caller.  The
   * parameter must be named @errp.
   *
   * It must be used when the function dereferences @errp or passes
   * @errp to error_prepend(), error_vprepend(), or error_append_hint().
   * It is safe to use even when it's not needed, but please avoid
   * cluttering the source with useless code.
   *
   * If @errp is NULL or &error_fatal, rewrite it to point to a local
   * Error variable, which will be automatically propagated to the
   * original @errp on function exit.
   *
   * Note: &error_abort is not rewritten, because that would move the
   * abort from the place where the error is created to the place where
   * it's propagated.
   */

=


All these are minor, the documentation is good as is, thank you!

--
Best regards,
Vladimir

Re: [PATCH v12 2/8] scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

Eric Blake  writes:

> On 7/7/20 11:50 AM, Markus Armbruster wrote:
>> From: Vladimir Sementsov-Ogievskiy 
>>
>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>> does corresponding changes in code (look for details in
>> include/qapi/error.h)
>>
>> Usage example:
>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>   --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>   --max-width 80 FILES...
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy 
>> Reviewed-by: Markus Armbruster 
>> Signed-off-by: Markus Armbruster 
>> ---
>>   scripts/coccinelle/auto-propagated-errp.cocci | 337 ++
>>   include/qapi/error.h  |   3 +
>>   MAINTAINERS   |   1 +
>>   3 files changed, 341 insertions(+)
>>   create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>
> Needs a tweak if we go with ERRP_GUARD.  But that's easy.
>
>> +
>> +// Convert special case with goto separately.
>> +// I tried merging this into the following rule the obvious way, but
>> +// it made Coccinelle hang on block.c
>> +//
>> +// Note interesting thing: if we don't do it here, and try to fixup
>> +// "out: }" things later after all transformations (the rule will be
>> +// the same, just without error_propagate() call), coccinelle fails to
>> +// match this "out: }".
>
> "out: }" is not valid C; would referring to "out: ; }" fare any better?

We can try for the next batch.

>> +@ disable optional_qualifier@
>> +identifier rule1.fn, rule1.local_err, out;
>> +symbol errp;
>> +@@
>> +
>> + fn(..., Error ** , ...)
>> + {
>> + <...
>> +-goto out;
>> ++return;
>> + ...>
>> +- out:
>> +-error_propagate(errp, local_err);
>> + }
>> +
>> +// Convert most of local_err related stuff.
>> +//
>> +// Note, that we inherit rule1.fn and rule1.local_err names, not
>> +// objects themselves. We may match something not related to the
>> +// pattern matched by rule1. For example, local_err may be defined with
>> +// the same name in different blocks inside one function, and in one
>> +// block follow the propagation pattern and in other block doesn't.
>> +//
>> +// Note also that errp-cleaning functions
>> +//   error_free_errp
>> +//   error_report_errp
>> +//   error_reportf_errp
>> +//   warn_report_errp
>> +//   warn_reportf_errp
>> +// are not yet implemented. They must call corresponding Error* -
>> +// freeing function and then set *errp to NULL, to avoid further
>> +// propagation to original errp (consider ERRP_AUTO_PROPAGATE in use).
>> +// For example, error_free_errp may look like this:
>> +//
>> +//void error_free_errp(Error **errp)
>> +//{
>> +//error_free(*errp);
>> +//*errp = NULL;
>> +//}
>
> I guess we can still decide later if we want these additional
> functions, or if they will even help after the number of places we
> have already improved after applying this script as-is and with
> Markus' cleanups in place.

Yes.

> While I won't call myself a Coccinelle expert, it at least looks sane
> enough that I'm comfortable if you add:
>
> Reviewed-by: Eric Blake 

Thanks!

Re: [PATCH v12 1/8] error: New macro ERRP_AUTO_PROPAGATE()

[Markus Armbruster]

Vladimir Sementsov-Ogievskiy  writes:

> 07.07.2020 19:50, Markus Armbruster wrote:
>> From: Vladimir Sementsov-Ogievskiy
>>
>> Introduce a new ERRP_AUTO_PROPAGATE macro, to be used at start of
>> functions with an errp OUT parameter.
>>
>> It has three goals:
>>
>> 1. Fix issue with error_fatal and error_prepend/error_append_hint: user
>> can't see this additional information, because exit() happens in
>> error_setg earlier than information is added. [Reported by Greg Kurz]
>>
>> 2. Fix issue with error_abort and error_propagate: when we wrap
>> error_abort by local_err+error_propagate, the resulting coredump will
>> refer to error_propagate and not to the place where error happened.
>> (the macro itself doesn't fix the issue, but it allows us to [3.] drop
>> the local_err+error_propagate pattern, which will definitely fix the
>> issue) [Reported by Kevin Wolf]
>>
>> 3. Drop local_err+error_propagate pattern, which is used to workaround
>> void functions with errp parameter, when caller wants to know resulting
>> status. (Note: actually these functions could be merely updated to
>> return int error code).
>>
>> To achieve these goals, later patches will add invocations
>> of this macro at the start of functions with either use
>> error_prepend/error_append_hint (solving 1) or which use
>> local_err+error_propagate to check errors, switching those
>> functions to use *errp instead (solving 2 and 3).
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>> Reviewed-by: Paul Durrant
>> Reviewed-by: Greg Kurz
>> Reviewed-by: Eric Blake
>> [Comments merged properly with recent commit "error: Document Error
>> API usage rules", and edited for clarity.  Put ERRP_AUTO_PROPAGATE()
>> before its helpers, and touch up style.  Commit message tweaked.]
>> Signed-off-by: Markus Armbruster
>
> Ok, I see you have mostly rewritten the big comment

Guilty as charged...  I was happy with the contents you provided (and
grateful for it), but our parallel work caused some redundancy.  I went
beyond a minimal merge to get a something that reads as one coherent
text.

> and not only in this patch, so, I go and read the whole comment on top of 
> these series.
>
> =
>
>* Pass an existing error to the caller with the message modified:
>* error_propagate_prepend(errp, err,
>* "Could not frobnicate '%s': ", name);
>* This is more concise than
>* error_propagate(errp, err); // don't do this
>* error_prepend(errp, "Could not frobnicate '%s': ", name);
>* and works even when @errp is &error_fatal.
>
> - the latter doesn't consider ERRP_AUTO_PROPAGATE: as we know, that 
> ERRP_AUTO_PROPAGATE should be used when we use error_prepend, the latter 
> should look like
>
>
> ERRP_AUTO_PROPAGATE();
> ...
> error_propagate(errp, err); // don't do this
> error_prepend(errp, "Could not frobnicate '%s': ", name);
>
> - and it works even when @errp is &error_fatal, so the 
> error_propagate_prepend now is just a shortcut, not the only correct way.

I can duplicate the advice from the paragraph preceding it, like this:

 * This is rarely needed.  When @err is a local variable, use of
 * ERRP_GUARD() commonly results in more readable code.
 * Where it is needed, it is more concise than
 * error_propagate(errp, err); // don't do this
 * error_prepend(errp, "Could not frobnicate '%s': ", name);
 * and works even when @errp is &error_fatal.

> Still, the text is formally correct as is, and may be improved later.
>
> =
>
>* 2. Replace &err by errp, and err by *errp.  Delete local variable
>*@err.
>
> - hmm a bit not obvious,, It can be local_err.

Yes, but I trust the reader can make that mental jump.

> It can be (in some rare cases) still needed to handle the error locally, not 
> passing to the caller..

I didn't think of this.

What about

 * To convert a function to use ERRP_GUARD(), assuming the local
 * variable it propagates to @errp is called @err:
 [...]
 * 2. Replace &err by errp, and err by *errp.  Delete local variable
 *@err if it s now unused.

Nope, still no good, if we replace like that, @err *will* be unused, and
the locally handled error will leak to the caller.

No time left for wordsmithing; let's improve on top.

> may be just something like "Assume local Error *err variable is used to get 
> errors from called functions and than propagated to caller's errp" before 
> paragraph [2.] will help.
>
>
>*
>* 3. Delete error_propagate(errp, *errp), replace
>*error_propagate_prepend(errp, *errp, ...) by error_prepend(errp, ...),
>*
>* 4. Ensure @errp is valid at return: when you destroy *errp, set
>*errp = NULL.
>
> =
>
>
> May be good to add note about ERRP_AUTO_PROPAGATE() into comment above 
> error_append_hint (and error_(v)prepend)).

Good point.

> ==

[xen-4.13-testing test] 151712: tolerable FAIL - PUSHED

[osstest service owner]

flight 151712 xen-4.13-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151712/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass

version targeted for testing:
 xen  378321bb1fd5272653ae64f0306827614a3bd196
baseline version:
 xen  9f7e8bac4ca279b3bfccb5f3730fb2e5398c95ab

Last test of basis   151337  2020-06-24 15:14:15 Z   13 days
Testing same since   151712  2020-07-07 13:13:16 Z0 days1 attempts


People who touched revisions under test:
  Jan Beulich 
  Julien Grall 
  Roger Pau Monné 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf 

[xen-4.10-testing test] 151713: regressions - trouble: fail/pass/starved

[osstest service owner]

flight 151713 xen-4.10-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151713/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 14 guest-saverestore.2 fail 
REGR. vs. 151255
 test-armhf-armhf-xl-vhd 15 guest-start/debian.repeat fail REGR. vs. 151255

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail like 151255
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2 fail like 151255
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-arm64-arm64-xl-thunderx  2 hosts-allocate   starved  n/a

version targeted for testing:
 xen  93be943e7d759015bd5db41a48f6dce58e580d5a
baseline version:
 xen  fd6e49ecae03840610fdc6a416a638590c0b6535

Last test of basis   151255  2020-06-20 12:32:09 Z   17 days
Testing same since   151713  2020-07-07 13:35:49 Z0 days1 attempts


People who touched revisions under test:
  Jan Beulich 
  Juli

Re: [PATCH v2 1/3] xen/privcmd: Corrected error handling path

[Souptick Joarder]

On Tue, Jul 7, 2020 at 5:15 PM Jürgen Groß  wrote:
>
> On 07.07.20 13:40, Souptick Joarder wrote:
> > On Tue, Jul 7, 2020 at 3:05 PM Jürgen Groß  wrote:
> >>
> >> On 06.07.20 20:16, Souptick Joarder wrote:
> >>> Previously, if lock_pages() end up partially mapping pages, it used
> >>> to return -ERRNO due to which unlock_pages() have to go through
> >>> each pages[i] till *nr_pages* to validate them. This can be avoided
> >>> by passing correct number of partially mapped pages & -ERRNO separately,
> >>> while returning from lock_pages() due to error.
> >>>
> >>> With this fix unlock_pages() doesn't need to validate pages[i] till
> >>> *nr_pages* for error scenario and few condition checks can be ignored.
> >>>
> >>> Signed-off-by: Souptick Joarder 
> >>> Cc: John Hubbard 
> >>> Cc: Boris Ostrovsky 
> >>> Cc: Paul Durrant 
> >>> ---
> >>>drivers/xen/privcmd.c | 31 +++
> >>>1 file changed, 15 insertions(+), 16 deletions(-)
> >>>
> >>> diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
> >>> index a250d11..33677ea 100644
> >>> --- a/drivers/xen/privcmd.c
> >>> +++ b/drivers/xen/privcmd.c
> >>> @@ -580,13 +580,13 @@ static long privcmd_ioctl_mmap_batch(
> >>>
> >>>static int lock_pages(
> >>>struct privcmd_dm_op_buf kbufs[], unsigned int num,
> >>> - struct page *pages[], unsigned int nr_pages)
> >>> + struct page *pages[], unsigned int nr_pages, unsigned int *pinned)
> >>>{
> >>>unsigned int i;
> >>> + int page_count = 0;
> >>
> >> Initial value shouldn't be needed, and ...
> >>
> >>>
> >>>for (i = 0; i < num; i++) {
> >>>unsigned int requested;
> >>> - int pinned;
> >>
> >> ... you could move the declaration here.
> >>
> >> With that done you can add my
> >>
> >> Reviewed-by: Juergen Gross 
> >
> > Ok. But does it going make any difference other than limiting scope ?
>
> Dropping the initializer surely does, and in the end page_count just
> replaces the former pinned variable, so why would we want to widen the
> scope with this patch?

Agree, no reason to move it up. Will change it in v3.

>
>
> Juergen

[xen-4.11-testing test] 151714: tolerable FAIL - PUSHED

[osstest service owner]

flight 151714 xen-4.11-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151714/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass

version targeted for testing:
 xen  ddaaccbbab6b19bf21ed2c097f3055a3c2544c8d
baseline version:
 xen  2b77729888fb851ab96e7f77bc854122626b4861

Last test of basis   151318  2020-06-23 18:34:16 Z   14 days
Testing same since   151714  2020-07-07 13:35:55 Z0 days1 attempts


People who touched revisions under test:
  Jan Beulich 
  Julien Grall 
  Roger Pau Monné 

jobs:
 build-amd64-xsm   

Re: [linux-linus test] 151705: regressions - FAIL

[Jürgen Groß]

On 07.07.20 21:23, osstest service owner wrote:

flight 151705 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/151705/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:


...


  build-i386-pvops  6 kernel-build fail REGR. vs. 151214


This one is a known problem with a fixing patch already queued for the
next rc.


Juergen

Re: [PATCH] trivial: Remove trailing whitespaces

[Markus Armbruster]

Daniel P. Berrangé  writes:

> On Mon, Jul 06, 2020 at 06:23:00PM +0200, Christophe de Dinechin wrote:
>> There are a number of unnecessary trailing whitespaces that have
>> accumulated over time in the source code. They cause stray changes
>> in patches if you use tools that automatically remove them.
>> 
>> Tested by doing a `git diff -w` after the change.
>> 
>> This could probably be turned into a pre-commit hook.

See .git/hooks/pre-commit.sample.

Expected test output is prone to flunk the whitespace test.  One
solution is to strip trailing whitespace from test output.

> scripts/checkpatch.pl ought to be made to check it.
>
>> 
>> Signed-off-by: Christophe de Dinechin 
>> ---
[...]
>>  78 files changed, 440 insertions(+), 440 deletions(-)
>
> The cleanup is a good idea, however, I think it is probably better to
> split the patch approx into subsystems. That will make it much easier
> to cherry-pick for people doing backports.

I doubt that's worth the trouble.

Acked-by: Markus Armbruster 

Re: [PATCH v2 2/3] xen/privcmd: Mark pages as dirty

[Jürgen Groß]

On 07.07.20 21:30, John Hubbard wrote:

On 2020-07-07 04:43, Jürgen Groß wrote:

On 07.07.20 13:30, Souptick Joarder wrote:

On Tue, Jul 7, 2020 at 3:08 PM Jürgen Groß  wrote:

...

diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 33677ea..f6c1543 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -612,8 +612,11 @@ static void unlock_pages(struct page *pages[], 
unsigned int nr_pages)

   {
   unsigned int i;

- for (i = 0; i < nr_pages; i++)
+ for (i = 0; i < nr_pages; i++) {
+ if (!PageDirty(pages[i]))
+ set_page_dirty_lock(pages[i]);


With put_page() directly following I think you should be able to use
set_page_dirty() instead, as there is obviously a reference to the page
existing.


Patch [3/3] will convert above codes to use 
unpin_user_pages_dirty_lock()
which internally do the same check. So I thought to keep linux-stable 
and
linux-next code in sync. John had a similar concern [1] and later 
agreed to keep

this check.

Shall I keep this check ?  No ?


It doesn't matter *too* much, because patch 3/3 fixes up everything by
changing it all to unpin_user_pages_dirty_lock(). However, there is 
something

to be said for having correct interim patches, too. :)  Details:



[1] 
https://lore.kernel.org/xen-devel/a750e5e5-fd5d-663b-c5fd-261d7c939...@nvidia.com/ 



I wasn't referring to checking PageDirty(), but to the use of
set_page_dirty_lock().

Looking at the comment just before the implementation of
set_page_dirty_lock() suggests that it is fine to use set_page_dirty()
instead (so not calling lock_page()).



no no, that's a misreading of the comment. Unless this xen/privcmd code has
somehow taken a reference on page->mapping->host (which I do *not* think is
the case), then it is still racy to call set_page_dirty() here. Instead,
set_page_dirty_lock() should be used.


Ah, okay. Thanks for the clarification.

So you can add my

Reviewed-by: Juergen Gross 


Juergen

Re: [PATCH v2] xen/displif: Protocol version 2

[Oleksandr Andrushchenko]

Hi, Paul!

On 7/2/20 11:42 AM, Jürgen Groß wrote:
> On 02.07.20 09:59, Oleksandr Andrushchenko wrote:
>>
>> On 7/2/20 10:20 AM, Jürgen Groß wrote:
>>> On 01.07.20 14:07, Oleksandr Andrushchenko wrote:
 On 7/1/20 1:46 PM, Jürgen Groß wrote:
> On 01.07.20 09:19, Oleksandr Andrushchenko wrote:
>> From: Oleksandr Andrushchenko 
>>
>> 1. Add protocol version as an integer
>>
>> Version string, which is in fact an integer, is hard to handle in the
>> code that supports different protocol versions. To simplify that
>> also add the version as an integer.
>>
>> 2. Pass buffer offset with XENDISPL_OP_DBUF_CREATE
>>
>> There are cases when display data buffer is created with non-zero
>> offset to the data start. Handle such cases and provide that offset
>> while creating a display buffer.
>>
>> 3. Add XENDISPL_OP_GET_EDID command
>>
>> Add an optional request for reading Extended Display Identification
>> Data (EDID) structure which allows better configuration of the
>> display connectors over the configuration set in XenStore.
>> With this change connectors may have multiple resolutions defined
>> with respect to detailed timing definitions and additional properties
>> normally provided by displays.
>>
>> If this request is not supported by the backend then visible area
>> is defined by the relevant XenStore's "resolution" property.
>>
>> If backend provides extended display identification data (EDID) with
>> XENDISPL_OP_GET_EDID request then EDID values must take precedence
>> over the resolutions defined in XenStore.
>>
>> 4. Bump protocol version to 2.
>>
>> Signed-off-by: Oleksandr Andrushchenko 
>
> Reviewed-by: Juergen Gross 

 Thank you, do you want me to prepare the same for the kernel so

 you have it at hand when the time comes?
>>>
>>> It should be added to the kernel only when really needed (i.e. a user of
>>> the new functionality is showing up).
>>
>> We have a patch for that which adds EDID to the existing PV DRM frontend,
>>
>> so while upstreaming those changes I will also include changes to the 
>> protocol
>>
>> in the kernel series: for that we need the header in Xen tree first, right?
>
> Yes.
>
Is it possible that we have this change in the release please?

This is not used by any piece of code in Xen, so it won't hurt anything.

But I need this change in so I can proceed with the Linux kernel part:

we would like to upstream the relevant EDID change to the display

frontend driver and without the header in Xen tree we are stuck

Thank you in advance,

Oleksandr

>
> Juergen