RE: [PATCH 4/9] xen/arm: static memory initialization

2021-07-05 Thread Penny Zheng
Hi Jan > -Original Message- > From: Penny Zheng > Sent: Monday, July 5, 2021 1:22 PM > To: Julien Grall ; Jan Beulich > Cc: Bertrand Marquis ; Wei Chen > ; xen-devel@lists.xenproject.org; > sstabell...@kernel.org > Subject: RE: [PATCH 4/9] xen/arm: static memory initialization > > Hi Ju

RE: [PATCH 4/9] xen/arm: static memory initialization

2021-07-05 Thread Penny Zheng
Hi Julien > -Original Message- > From: Julien Grall > Sent: Thursday, July 1, 2021 2:10 AM > To: Penny Zheng ; xen-devel@lists.xenproject.org; > sstabell...@kernel.org; jbeul...@suse.com > Cc: Bertrand Marquis ; Wei Chen > > Subject: Re: [PATCH 4/9] xen/arm: static memory initialization

Re: [PATCH v15 06/12] swiotlb: Use is_swiotlb_force_bounce for swiotlb data bouncing

2021-07-05 Thread Claire Chang
On Sat, Jul 3, 2021 at 1:55 PM Nathan Chancellor wrote: > > Hi Will and Robin, > > On Fri, Jul 02, 2021 at 04:13:50PM +0100, Robin Murphy wrote: > > On 2021-07-02 14:58, Will Deacon wrote: > > > Hi Nathan, > > > > > > On Thu, Jul 01, 2021 at 12:52:20AM -0700, Nathan Chancellor wrote: > > > > On 7/

Re: [PATCH] tools/libxenguest: Fix max_extd_leaf calculation for legacy restore

2021-07-05 Thread Olaf Hering
Am Fri, 2 Jul 2021 20:03:34 +0100 schrieb Andrew Cooper : > Fixes: 34990446ca91 ("libxl: don't ignore the return value from > xc_cpuid_apply_policy") I think it fixes 111c8c33a8a18588f3da3c5dbb7f5c63ddb98ce5 ("x86/cpuid: do not expand max leaves on restore"), 34990446ca91 just revealed the bug?

Re: [PATCH 4/9] xen/arm: static memory initialization

2021-07-05 Thread Jan Beulich
On 05.07.2021 07:22, Penny Zheng wrote: >> From: Julien Grall >> Sent: Thursday, July 1, 2021 1:46 AM >> >> On 10/06/2021 10:35, Jan Beulich wrote: >>> On 07.06.2021 04:43, Penny Zheng wrote: @@ -1512,6 +1530,38 @@ static void free_heap_pages( spin_unlock(&heap_lock); }

Re: [PATCH 4/9] xen/arm: static memory initialization

2021-07-05 Thread Jan Beulich
On 05.07.2021 09:14, Penny Zheng wrote: >> From: Penny Zheng >> Sent: Monday, July 5, 2021 1:22 PM >> >>> From: Julien Grall >>> Sent: Thursday, July 1, 2021 1:46 AM >>> >>> On 10/06/2021 10:35, Jan Beulich wrote: On 07.06.2021 04:43, Penny Zheng wrote: > @@ -1512,6 +1530,38 @@ static voi

[libvirt test] 163308: regressions - FAIL

2021-07-05 Thread osstest service owner
flight 163308 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/163308/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-armhf-libvirt 6 libvirt-buildfail REGR. vs. 151777 build-amd64-libvirt

Re: [PATCH] tools/libxenguest: Fix migration's debug option

2021-07-05 Thread Olaf Hering
Am Fri, 2 Jul 2021 20:03:42 +0100 schrieb Andrew Cooper : > The code has gone through many refactors, but the first refactor was the one > which broke it by inverting the check with respect to checkpointed streams. > > Fixes: 7449fb36c6c8 ("migration/save: pass checkpointed_stream from libxl to

Re: [PATCH] tools/libxenguest: Fix migration's debug option

2021-07-05 Thread Jan Beulich
On 02.07.2021 21:03, Andrew Cooper wrote: > The code has gone through many refactors, but the first refactor was the one > which broke it by inverting the check with respect to checkpointed streams. > > Fixes: 7449fb36c6c8 ("migration/save: pass checkpointed_stream from libxl to > libxc") > Repor

Re: [PATCH -next] xen: Use DEVICE_ATTR_*() macro

2021-07-05 Thread Juergen Gross
On 26.05.21 16:10, YueHaibing wrote: Use DEVICE_ATTR_*() helper instead of plain DEVICE_ATTR(), which makes the code a bit shorter and easier to read. Signed-off-by: YueHaibing Pushed to xen/tip.git for-linus-5.14 Juergen OpenPGP_0xB0DE9DD628BF132F.asc Description: OpenPGP public key Op

Re: [PATCH] tools/libxenguest: Fix max_extd_leaf calculation for legacy restore

2021-07-05 Thread Jan Beulich
On 02.07.2021 21:03, Andrew Cooper wrote: > 0x1c is lower than any value which will actually be observed in > p->extd.max_leaf, but higher than the logical 9 leaves worth of extended data > on Intel systems, causing x86_cpuid_copy_to_buffer() to fail with -ENOBUFS. > > Correct the calculation. >

Re: [PATCH] tools/libxenguest: Fix migration's debug option

2021-07-05 Thread Olaf Hering
Am Mon, 5 Jul 2021 09:57:21 +0200 schrieb Jan Beulich : > What is "the grant problem" referring to here? Neither anything above > nor the offending original commit has any reference to grants, or a > problem with them. When the guest is paused during final transit, the backends will continue to

Re: [PATCH v20210701 06/40] tools: fix Python3.4 TypeError in format string

2021-07-05 Thread Olaf Hering
Am Fri, 2 Jul 2021 18:19:39 +0200 schrieb Marek Marczykowski-Górecki : > Why bytes()? Encode does already return bytes type. You are right, this works as well: i = 123 b = ("str/%x" % (i, )).encode('utf-8') Any preference regarding the "encoding"? I picked UTF8, but 'ascii' might be more co

Re: [PATCH] xen/Makefile: drop -Werror

2021-07-05 Thread Jan Beulich
On 02.07.2021 20:52, Elliott Mitchell wrote: > On Fri, Jul 02, 2021 at 07:51:55PM +0200, Fabrice Fontaine wrote: >> >> I do agree that setting -Werror is generally perfectly valid for upstream. >> However, for downstream packager, it is generally seen as an issue as >> it will always raise unexepec

Re: [PATCH] xen/Makefile: drop -Werror

2021-07-05 Thread Jan Beulich
On 02.07.2021 19:51, Fabrice Fontaine wrote: > Le ven. 2 juil. 2021 à 19:34, Andrew Cooper > a écrit : >> >> On 02/07/2021 18:06, Fabrice Fontaine wrote: >>> Drop -Werror to avoid the following build failure with -DNDEBUG: >>> >>> In file included from :0:0: >>> /usr/lfs/hdd_v1/rc-buildroot-test/s

Re: [PATCH v20210701 06/40] tools: fix Python3.4 TypeError in format string

2021-07-05 Thread Olaf Hering
Am Fri, 2 Jul 2021 17:39:54 +0100 schrieb Andrew Cooper : > However, the % (phys, ) with the trailing comma is deliberate to work > around a common python error, so wants to remain if you're keeping the > %-formatting. What error is that? Olaf pgpOfD8Lydffy.pgp Description: Digitale Signatur v

Re: [PATCH v20210701 10/40] tools: add xc_is_known_page_type to libxenctrl

2021-07-05 Thread Olaf Hering
Am Fri, 2 Jul 2021 20:20:08 +0100 schrieb Andrew Cooper : > Subject needs correcting after v2. Apparently I missed some places while removing the old "xc_" prefix. > However, given that this is in the save/restore common header, does it > really need a prefix?  Simply is_known_page_type() seems

Re: [PATCH] tools/libxenguest: Fix migration's debug option

2021-07-05 Thread Jan Beulich
On 05.07.2021 10:02, Olaf Hering wrote: > Am Mon, 5 Jul 2021 09:57:21 +0200 > schrieb Jan Beulich : > >> What is "the grant problem" referring to here? Neither anything above >> nor the offending original commit has any reference to grants, or a >> problem with them. > > When the guest is paused

Re: [PATCH] tools/xenstored: Stash the correct request in lu_status->in

2021-07-05 Thread Luca Fancellu
> On 1 Jul 2021, at 15:03, Julien Grall wrote: > > From: Julien Grall > > When Live-Updating with some load, Xenstored may hit the assert > req->in == lu_status->in in do_lu_start(). > > This is happening because the request is stashed when Live-Update > begins. This happens in a different

Re: [PATCH v20210701 11/40] tools: use sr_is_known_page_type

2021-07-05 Thread Olaf Hering
Am Fri, 2 Jul 2021 20:27:21 +0100 schrieb Andrew Cooper : > Any reason this isn't folded into the previous patch, like your > subsequent two page type helper patches are? I think I wanted to separate this for simpler review, but I forgot to split the followup change as well. > > +ER

Re: [PATCH] tools/libxenguest: Fix migration's debug option

2021-07-05 Thread Olaf Hering
Am Mon, 5 Jul 2021 10:23:00 +0200 schrieb Jan Beulich : > I see. A similar problem then exists with at least the FIFO event > channel per-vCPU control blocks? I have not done any debugging how the pages differ and what they are actually used for. My guess was that it might be activity from the

Re: [PATCH v5 1/4] xen: XENMEM_exchange should only be used/compiled for arch supporting PV guest

2021-07-05 Thread Jan Beulich
On 03.07.2021 19:11, Julien Grall wrote: > Changes in v5: > - Removed the #ifdef CONFIG_X86 as they are not necessary anymore > - Used paging_mode_translate() rather than is_pv_domain() Is there a particular reason you use this in favor of steal_page()'s paging_mode_external()? > @@ -815,

Re: [PATCH v5 2/4] xen: arm: Stop returning a bogus GFN for the shared info

2021-07-05 Thread Jan Beulich
On 03.07.2021 19:11, Julien Grall wrote: > From: Julien Grall > > While Arm never had a M2P, the implementation of mfn_to_gfn() is pretty > bogus as we directly return the MFN passed in parameter. > > The last use of mfn_to_gfn() on Arm is in getdomaininfo(). It looks > like this is mostly used

Re: [PATCH v20210701 12/40] tools: unify type checking for data pfns in migration stream

2021-07-05 Thread Olaf Hering
Am Fri, 2 Jul 2021 20:43:13 +0100 schrieb Andrew Cooper : > Anyone adding a new page type is going to have to audit/edit each of > these helpers.  I think it would be better to write all the true cases > explicitly. You mean the check if a page has data or needs to be populated should look like

Re: [PATCH] xen/events: reset active flag for lateeoi events later

2021-07-05 Thread Ross Lagerwall
On 2021-06-23 14:09, Juergen Gross wrote: > In order to avoid a race condition for user events when changing > cpu affinity reset the active flag only when EOI-ing the event. > > This is working fine as all user events are lateeoi events. Note that > lateeoi_ack_mask_dynirq() is not modified as th

Re: [PATCH v5 4/4] xen/mm: Provide dummy M2P-related helpers when the M2P is not supported

2021-07-05 Thread Jan Beulich
On 03.07.2021 19:11, Julien Grall wrote: > From: Julien Grall > > At the moment, Arm is providing a dummy implementation for the M2P > helpers used in common code. However, they are quite isolated and could > be used by other architecture in the future. So move the helpers > necessary for compila

Re: [PATCH] xen/events: reset active flag for lateeoi events later

2021-07-05 Thread Juergen Gross
On 05.07.21 11:00, Ross Lagerwall wrote: On 2021-06-23 14:09, Juergen Gross wrote: In order to avoid a race condition for user events when changing cpu affinity reset the active flag only when EOI-ing the event. This is working fine as all user events are lateeoi events. Note that lateeoi_ack_m

RE: [PATCH 4/9] xen/arm: static memory initialization

2021-07-05 Thread Penny Zheng
Hi Jan > -Original Message- > From: Jan Beulich > Sent: Monday, July 5, 2021 3:51 PM > To: Penny Zheng > Cc: Bertrand Marquis ; Wei Chen > ; xen-devel@lists.xenproject.org; > sstabell...@kernel.org; Julien Grall > Subject: Re: [PATCH 4/9] xen/arm: static memory initialization > > On 05

Re: [PATCH] tools/libxenguest: Fix migration's debug option

2021-07-05 Thread Jan Beulich
On 05.07.2021 10:32, Olaf Hering wrote: > Am Mon, 5 Jul 2021 10:23:00 +0200 > schrieb Jan Beulich : > >> I see. A similar problem then exists with at least the FIFO event >> channel per-vCPU control blocks? > > I have not done any debugging how the pages differ and what they are actually > used

Re: [PATCH] tools/libxenguest: Fix migration's debug option

2021-07-05 Thread Olaf Hering
Am Mon, 5 Jul 2021 11:19:59 +0200 schrieb Jan Beulich : > "The interface" being which one? The tool stack can map the guest's > grant table, so it is in the position to find out about all grants > without further hypervisor help. The interface means the code behind verify_frames. If there are in

Re: [PATCH] xen/Makefile: drop -Werror

2021-07-05 Thread Fabrice Fontaine
Dear all, Le lun. 5 juil. 2021 à 10:16, Jan Beulich a écrit : > > On 02.07.2021 19:51, Fabrice Fontaine wrote: > > Le ven. 2 juil. 2021 à 19:34, Andrew Cooper > > a écrit : > >> > >> On 02/07/2021 18:06, Fabrice Fontaine wrote: > >>> Drop -Werror to avoid the following build failure with -DNDEBU

Re: [PATCH] tools/libxenguest: Fix migration's debug option

2021-07-05 Thread Jan Beulich
On 05.07.2021 11:25, Olaf Hering wrote: > Am Mon, 5 Jul 2021 11:19:59 +0200 > schrieb Jan Beulich : > >> "The interface" being which one? The tool stack can map the guest's >> grant table, so it is in the position to find out about all grants >> without further hypervisor help. > > The interface

Re: [PATCH v6 7/9] docs: Change Makefile and sphinx configuration for doxygen

2021-07-05 Thread Luca Fancellu
> On 2 Jul 2021, at 23:23, Stefano Stabellini wrote: > > On Fri, 2 Jul 2021, Luca Fancellu wrote: >>> On 1 Jul 2021, at 18:43, Stefano Stabellini wrote: >>> >>> On Thu, 1 Jul 2021, Luca Fancellu wrote: > On 24 Jun 2021, at 00:33, Stefano Stabellini > wrote: > > On Mon, 10

Re: [PATCH v20210701 06/40] tools: fix Python3.4 TypeError in format string

2021-07-05 Thread Andrew Cooper
On 05/07/2021 09:18, Olaf Hering wrote: > Am Fri, 2 Jul 2021 17:39:54 +0100 > schrieb Andrew Cooper : > >> However, the % (phys, ) with the trailing comma is deliberate to work >> around a common python error, so wants to remain if you're keeping the >> %-formatting. > What error is that? >>> def

Re: [PATCH v20210701 10/40] tools: add xc_is_known_page_type to libxenctrl

2021-07-05 Thread Andrew Cooper
On 05/07/2021 09:22, Olaf Hering wrote: > Am Fri, 2 Jul 2021 20:20:08 +0100 > schrieb Andrew Cooper : >>> +/* Sanitiy check for types returned by Xen */ >>> +static inline bool sr_is_known_page_type(xen_pfn_t type) >> uint32_t > Why is this better than returning 'bool'? For the parameter sorry,

Re: [PATCH v20210701 11/40] tools: use sr_is_known_page_type

2021-07-05 Thread Andrew Cooper
On 05/07/2021 09:25, Olaf Hering wrote: > Am Fri, 2 Jul 2021 20:27:21 +0100 > schrieb Andrew Cooper : > >> Any reason this isn't folded into the previous patch, like your >> subsequent two page type helper patches are? > I think I wanted to separate this for simpler review, but I forgot to split >

Re: [PATCH v20210701 12/40] tools: unify type checking for data pfns in migration stream

2021-07-05 Thread Andrew Cooper
On 05/07/2021 09:59, Olaf Hering wrote: > Am Fri, 2 Jul 2021 20:43:13 +0100 > schrieb Andrew Cooper : > >> Anyone adding a new page type is going to have to audit/edit each of >> these helpers.  I think it would be better to write all the true cases >> explicitly. > You mean the check if a page has

RE: [Kvmtool] Some thoughts on using kvmtool Virtio for Xen

2021-07-05 Thread Wei Chen
Hi Stefano, Thanks for your comments. > -Original Message- > From: Stefano Stabellini > Sent: 2021年6月30日 8:43 > To: w...@kernel.org; julien.thierry.k...@gmail.com; Wei Chen > > Cc: k...@vger.kernel.org; xen-de...@lists.xen.org; jean-phili...@linaro.org; > Julien Grall ; Andre Przywara ;

Re: [PATCH] tools/libxenguest: Fix migration's debug option

2021-07-05 Thread Andrew Cooper
On 05/07/2021 10:31, Jan Beulich wrote: > On 05.07.2021 11:25, Olaf Hering wrote: >> Am Mon, 5 Jul 2021 11:19:59 +0200 >> schrieb Jan Beulich : >> >>> "The interface" being which one? The tool stack can map the guest's >>> grant table, so it is in the position to find out about all grants >>> witho

Re: [PATCH v20210701 06/40] tools: fix Python3.4 TypeError in format string

2021-07-05 Thread Andrew Cooper
On 05/07/2021 09:07, Olaf Hering wrote: > Am Fri, 2 Jul 2021 18:19:39 +0200 > schrieb Marek Marczykowski-Górecki : > >> Why bytes()? Encode does already return bytes type. > You are right, this works as well: > i = 123 > b = ("str/%x" % (i, )).encode('utf-8') > > Any preference regarding the "e

Re: [PATCH] tools/libxenguest: Fix max_extd_leaf calculation for legacy restore

2021-07-05 Thread Andrew Cooper
On 05/07/2021 08:35, Olaf Hering wrote: > Am Fri, 2 Jul 2021 20:03:34 +0100 > schrieb Andrew Cooper : > >> Fixes: 34990446ca91 ("libxl: don't ignore the return value from >> xc_cpuid_apply_policy") > I think it fixes 111c8c33a8a18588f3da3c5dbb7f5c63ddb98ce5 ("x86/cpuid: do not > expand max leaves

Re: [PATCH] tools/libxenguest: Fix migration's debug option

2021-07-05 Thread Jan Beulich
On 05.07.2021 12:06, Andrew Cooper wrote: > On 05/07/2021 10:31, Jan Beulich wrote: >> On 05.07.2021 11:25, Olaf Hering wrote: >>> Am Mon, 5 Jul 2021 11:19:59 +0200 >>> schrieb Jan Beulich : >>> "The interface" being which one? The tool stack can map the guest's grant table, so it is in t

Re: [PATCH v20210701 15/40] tools: prepare to allocate saverestore arrays once

2021-07-05 Thread Andrew Cooper
On 01/07/2021 10:56, Olaf Hering wrote: > The hotpath 'send_dirty_pages' is supposed to do just one thing: sending. > The other end 'handle_page_data' is supposed to do just receiving. > > But instead both do other costly work like memory allocations and data moving. > Do the allocations once, the

Re: [PATCH v4] arm64: Change type of hsr, cpsr, spsr_el1 to uint64_t

2021-07-05 Thread Bertrand Marquis
Hi Michal, On 5 Jul 2021, at 07:39, Michal Orzel mailto:michal.or...@arm.com>> wrote: AArch64 registers are 64bit whereas AArch32 registers are 32bit or 64bit. MSR/MRS are expecting 64bit values thus we should get rid of helpers READ/WRITE_SYSREG32 in favour of using READ/WRITE_SYSREG. We should

[PATCH v7 0/9] Use Doxygen and sphinx for html documentation

2021-07-05 Thread Luca Fancellu
This serie introduce doxygen in the sphinx html docs generation. One benefit is to keep most of the documentation in the source files of xen so that it's more maintainable, on the other hand there are some limitation of doxygen that should be addressed modifying the current codebase (for example do

[PATCH v7 2/9] docs: add Xen png logo for the doxygen documentation

2021-07-05 Thread Luca Fancellu
Add the xen-doxygen folder for the doxygen template and add the Xen png logo in it. Signed-off-by: Luca Fancellu Acked-by: Stefano Stabellini --- docs/xen-doxygen/xen_project_logo_165x67.png | Bin 0 -> 18223 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 docs/xen-dox

[PATCH v7 3/9] docs: add doxygen templates

2021-07-05 Thread Luca Fancellu
Add doxygen templates for the doxygen documentation. Signed-off-by: Luca Fancellu Acked-by: Stefano Stabellini --- docs/xen-doxygen/customdoxygen.css | 36 +++ docs/xen-doxygen/footer.html | 21 +++ docs/xen-doxygen/header.html | 56 ++

[PATCH v7 1/9] docs: add doxygen configuration file

2021-07-05 Thread Luca Fancellu
Add xen.doxyfile.in as template for the doxygen configuration file, it will be used to generate the doxygen documentation. Signed-off-by: Luca Fancellu Acked-by: Stefano Stabellini --- docs/xen.doxyfile.in | 2316 ++ 1 file changed, 2316 insertions(+) cr

[PATCH v7 4/9] m4/python: add function to docs_tool.m4 and new m4 module

2021-07-05 Thread Luca Fancellu
Add ax_python_module.m4 to have a way to check if a python module is installed in the system. Add a function to docs_tool.m4 to throw an error if the required docs tool is missing. Signed-off-by: Luca Fancellu Acked-by: Stefano Stabellini --- m4/ax_python_module.m4 | 56 +++

[PATCH v7 8/9] docs: hypercalls sphinx skeleton for generated html

2021-07-05 Thread Luca Fancellu
Create a skeleton for the documentation about hypercalls. At this stage the documentation can be created only for one architecture at a time. Signed-off-by: Luca Fancellu --- v7 changes: - modify the commit message v6 changes: - Now every platform has the same sections in .rst files --- .gitigno

[PATCH v7 5/9] docs: add checks to configure for sphinx and doxygen

2021-07-05 Thread Luca Fancellu
Add checks in the configure files to see if the system is capable of generating the sphinx html docs using doxygen and sphinx-breathe tools. Signed-off-by: Luca Fancellu Acked-by: Stefano Stabellini --- config/Docs.mk.in | 2 + docs/configure| 258 +

[PATCH v7 9/9] docs/doxygen: doxygen documentation for grant_table.h

2021-07-05 Thread Luca Fancellu
Modification to include/public/grant_table.h: 1) Add doxygen tags to: - Create Grant tables section - include variables in the generated documentation - Used @keepindent/@endkeepindent to enclose comment section that are indented using spaces, to keep the indentation. 2) Add .rst file for

[PATCH v7 6/9] docs: add doxygen preprocessor and related files

2021-07-05 Thread Luca Fancellu
Add preprocessor called by doxygen before parsing headers, it will include in every header a doxygen_include.h file that provides missing defines and includes that are usually passed by the compiler, it will also handle the problem of anonymous union/struct not recognized by doxygen giving them nam

[PATCH v7 7/9] docs: Change Makefile and sphinx configuration for doxygen

2021-07-05 Thread Luca Fancellu
Modify docs/Makefile to call doxygen and generate sphinx html documentation given the doxygen XML output. Modify docs/conf.py sphinx configuration file to setup the breathe extension that works as bridge between sphinx and doxygen. Add some files to the .gitignore to ignore some generated files f

Re: [PATCH v20210701 15/40] tools: prepare to allocate saverestore arrays once

2021-07-05 Thread Olaf Hering
Am Mon, 5 Jul 2021 11:44:30 +0100 schrieb Andrew Cooper : > On 01/07/2021 10:56, Olaf Hering wrote: > I agree that the repeated alloc/free of same-sized memory regions on > each iteration is a waste.  However, if we are going to fix this by > using one-off allocations, then we want to compensate w

[xen-unstable test] 163305: tolerable FAIL

2021-07-05 Thread osstest service owner
flight 163305 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/163305/ Failures :-/ but no regressions. Tests which are failing intermittently (not blocking): test-amd64-amd64-libvirt-vhd 16 guest-saverestore fail in 163293 pass in 163305 test-amd64-amd64-xl-qemut-debia

Backport of commit 3de218ff39b9e3f0d4

2021-07-05 Thread Juergen Gross
Hi Greg, the attached patch is a backport of upstream commit 3de218ff39b9e3f0d4 for Linux 5.10 and older (I've checked it to apply down to 4.4). Juergen From 9535989b80d89932f34f7f1596b91b400d7adbb2 Mon Sep 17 00:00:00 2001 From: Juergen Gross Date: Mon, 5 Jul 2021 13:12:27 +0200 Subject: [PAT

[ovmf test] 163307: regressions - FAIL

2021-07-05 Thread osstest service owner
flight 163307 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/163307/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 162359 test-amd64-amd64-xl-qemuu

Re: [PATCH] tools/xenstored: Stash the correct request in lu_status->in

2021-07-05 Thread Juergen Gross
On 01.07.21 16:03, Julien Grall wrote: From: Julien Grall When Live-Updating with some load, Xenstored may hit the assert req->in == lu_status->in in do_lu_start(). This is happening because the request is stashed when Live-Update begins. This happens in a different request (see call lu_begin(

Re: [PATCH v20210701 15/40] tools: prepare to allocate saverestore arrays once

2021-07-05 Thread Andrew Cooper
On 05/07/2021 12:27, Olaf Hering wrote: > Am Mon, 5 Jul 2021 11:44:30 +0100 > schrieb Andrew Cooper : > >>> This patch is just prepartion, subsequent changes will populate the arrays. >>> >>> Once all changes are applied, migration of a busy HVM domU changes like >>> that: >>> >>> Without this ser

Re: [PATCH v7 9/9] docs/doxygen: doxygen documentation for grant_table.h

2021-07-05 Thread Jan Beulich
On 05.07.2021 12:51, Luca Fancellu wrote: > Modification to include/public/grant_table.h: > > 1) Add doxygen tags to: > - Create Grant tables section > - include variables in the generated documentation > - Used @keepindent/@endkeepindent to enclose comment >section that are indented using

Re: [PATCH v20210701 12/40] tools: unify type checking for data pfns in migration stream

2021-07-05 Thread Andrew Cooper
On 01/07/2021 10:56, Olaf Hering wrote: > Introduce a helper which decides if a given pfn in the migration > stream is backed by memory. > > This specifically deals with type XEN_DOMCTL_PFINFO_XALLOC, which was > a synthetic toolstack-only type used in Xen 4.2 to 4.5. It indicated a > dirty page on

Re: [PATCH v7 9/9] docs/doxygen: doxygen documentation for grant_table.h

2021-07-05 Thread Luca Fancellu
Hi Jan, > On 5 Jul 2021, at 14:03, Jan Beulich wrote: > > On 05.07.2021 12:51, Luca Fancellu wrote: >> Modification to include/public/grant_table.h: >> >> 1) Add doxygen tags to: >> - Create Grant tables section >> - include variables in the generated documentation >> - Used @keepindent/@endkee

Re: [PATCH v7 9/9] docs/doxygen: doxygen documentation for grant_table.h

2021-07-05 Thread Jan Beulich
On 05.07.2021 15:23, Luca Fancellu wrote: > Hi Jan, > >> On 5 Jul 2021, at 14:03, Jan Beulich wrote: >> >> On 05.07.2021 12:51, Luca Fancellu wrote: >>> Modification to include/public/grant_table.h: >>> >>> 1) Add doxygen tags to: >>> - Create Grant tables section >>> - include variables in the g

Re: [PATCH v20210701 12/40] tools: unify type checking for data pfns in migration stream

2021-07-05 Thread Olaf Hering
On Mon, Jul 05, Andrew Cooper wrote: > What do you mean "This specifically deals with" ? This was a result from Jürgen pointing out that XEN_DOMCTL_PFINFO_XALLOC is not handled. If all the type checking changes go into a single commit, the commig message has to be reworded. Olaf

Re: [PATCH v20210701 15/40] tools: prepare to allocate saverestore arrays once

2021-07-05 Thread Olaf Hering
Am Mon, 5 Jul 2021 14:01:07 +0100 schrieb Andrew Cooper : > > The last one is always way faster because apparently map/unmap is less > > costly with a stopped guest. > That's suspicious.  If true, we've got some very wonky behaviour in the > hypervisor... At least the transfer rate this last i

Re: [PATCH v7 9/9] docs/doxygen: doxygen documentation for grant_table.h

2021-07-05 Thread Julien Grall
Hi Luca, On 05/07/2021 11:51, Luca Fancellu wrote: Modification to include/public/grant_table.h: 1) Add doxygen tags to: - Create Grant tables section - include variables in the generated documentation - Used @keepindent/@endkeepindent to enclose comment section that are indented usin

[ANNOUNCE] Call for agenda items for July Community Call @ 1500 UTC

2021-07-05 Thread George Dunlap
Hi all, The proposed agenda is in https://cryptpad.fr/pad/#/2/pad/edit/iCXRLaMLoMY8t6n94MYywvLt/ and you can edit to add items. Alternatively, you can reply to this mail directly. Agenda items appreciated a few days before the call: please put your name besides items if you edit the document.

Re: [ANNOUNCE] Call for agenda items for July Community Call @ 1500 UTC

2021-07-05 Thread George Dunlap
Sorry, this should say 8 July; this coming Thursday. -George > On Jul 5, 2021, at 3:20 PM, George Dunlap wrote: > > Hi all, > > The proposed agenda is in > https://cryptpad.fr/pad/#/2/pad/edit/iCXRLaMLoMY8t6n94MYywvLt/ and you can > edit to add items. Alternatively, you can reply to this m

Re: [PATCH v20210701 10/40] tools: add xc_is_known_page_type to libxenctrl

2021-07-05 Thread Olaf Hering
Am Mon, 5 Jul 2021 10:51:50 +0100 schrieb Andrew Cooper : > All type fields are uniformly uint32_t elsewhere. To me it looks like xc_get_pfn_type_batch writes to an array of xen_pfn_t. Olaf pgpHzJtSAfDlr.pgp Description: Digitale Signatur von OpenPGP

[xen-unstable-smoke test] 163310: tolerable all pass - PUSHED

2021-07-05 Thread osstest service owner
flight 163310 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/163310/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 15 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 1

Re: [XEN PATCH v6 01/31] build: fix %.s: %.S rule

2021-07-05 Thread Jan Beulich
On 01.07.2021 16:09, Anthony PERARD wrote: > Fixes: e321576f4047 ("xen/build: start using if_changed") > Signed-off-by: Anthony PERARD Reviewed-by: Jan Beulich

[PATCH v2 00/13] x86: more or less log-dirty related improvements

2021-07-05 Thread Jan Beulich
... or so I hope. This series continues the attempt to deal with the ovmf change putting the shared info page at a very high address (which is now planned to get reverted there, but the general problem doesn't go away by them doing so). There are further issues with truncated value, which are being

[PATCH v2 01/13] libxl/x86: check return value of SHADOW_OP_SET_ALLOCATION domctl

2021-07-05 Thread Jan Beulich
The hypervisor may not have enough memory to satisfy the request. While there, make the unit of the value clear by renaming the local variable. Requested-by: Andrew Cooper Signed-off-by: Jan Beulich Reviewed-by: Andrew Cooper --- v2: Rename local variables. Also log requested value. Don't log e

[PATCH v2 02/13] libxc: split xc_logdirty_control() from xc_shadow_control()

2021-07-05 Thread Jan Beulich
For log-dirty operations a 64-bit field is being truncated to become an "int" return value. Seeing the large number of arguments the present function takes, reduce its set of parameters to that needed for all operations not involving the log-dirty bitmap, while introducing a new wrapper for the log

[PATCH v2 03/13] libxenguest: deal with log-dirty op stats overflow

2021-07-05 Thread Jan Beulich
In send_memory_live() the precise value the dirty_count struct field gets initialized to doesn't matter much (apart from the triggering of the log message in send_dirty_pages(), see below), but it is important that it not be zero on the first iteration (or else send_dirty_pages() won't get called a

[PATCH v2 04/13] libxenguest: short-circuit "all-dirty" handling

2021-07-05 Thread Jan Beulich
For one it is unnecessary to fill a perhaps large chunk of memory with all ones. Add a new parameter to send_dirty_pages() for callers to indicate so. Then it is further unnecessary to allocate the dirty bitmap altogether when all that's ever going to happen is a single all-dirty run. Signed-off-

[PATCH v2 05/13] libxenguest: avoid allocating unused deferred-pages bitmap

2021-07-05 Thread Jan Beulich
Like for the dirty bitmap, it is unnecessary to allocate the deferred- pages bitmap when all that's ever going to happen is a single all-dirty run. Signed-off-by: Jan Beulich --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -130,7 +130,7 @@ static int write_batch(struct

[PATCH v2 06/13] libxenguest: complete loops in xc_map_domain_meminfo()

2021-07-05 Thread Jan Beulich
minfo->p2m_size may have more than 31 significant bits. Change the induction variable to unsigned long, and (largely for signed-ness consistency) a helper variable to unsigned int. And while there also avoid open-coding min(). Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- v2: Use min().

[PATCH v2 07/13] libxenguest: guard against overflow from too large p2m when checkpointing

2021-07-05 Thread Jan Beulich
struct xc_sr_record's length field has just 32 bits. Fill it early and check that the calculated value hasn't overflowed. Additionally check for counter overflow early - there's no point even trying to allocate any memory in such an event. While there also limit an induction variable's type to uns

[PATCH v2 08/13] libxenguest: fix off-by-1 in colo-secondary-bitmap merging

2021-07-05 Thread Jan Beulich
Valid GFNs (having a representation in the dirty bitmap) need to be strictly below p2m_size. Signed-off-by: Jan Beulich Reviewed-by: Andrew Cooper --- a/tools/libs/guest/xg_sr_save.c +++ b/tools/libs/guest/xg_sr_save.c @@ -614,7 +614,7 @@ static int colo_merge_secondary_dirty_bi for ( i =

[PATCH v2 09/13] libxenguest: restrict PV guest size

2021-07-05 Thread Jan Beulich
The P2M, the use of PFNs, and hence the maximum valid PFN are purely software constructs in PV. In principle a guest is free to use arbitrary PFNs. However, at least page table normalization requires that PFN space be, like MFN space, limited to the architectural 40 bits (52 address bits). And of c

[PATCH v2 10/13] libxc: simplify HYPERCALL_BUFFER()

2021-07-05 Thread Jan Beulich
_hcbuf_buf1 has been there only for a pointer comparison to validate type compatibility. The same can be achieved by not using typeof() on the definition of what so far was _hcbuf_buf2, as the initializer has to also be type-compatible. Drop _hcbuf_buf1 and the comaprison; rename _hcbuf_buf2. Sinc

[PATCH v2 11/13] x86/paging: supply more useful log-dirty page count

2021-07-05 Thread Jan Beulich
In paging_log_dirty_op(), always update the count of pages field: - if more pages were specified than the guest has ever accessed (HVM) or marked part of the p2m (PV), there's no point for the caller to inspect bits beyond the one for that last page, - if the guest's p2m size has grown in the m

[PATCH v2 12/13] x86/mm: update log-dirty bitmap when manipulating P2M

2021-07-05 Thread Jan Beulich
Just like for PV guests MMU_MACHPHYS_UPDATE implies marking of the respective page as dirty, additions to a HVM guest's P2M should do so. For HVM the opposite is also true: Pages being removed from the P2M are no longer dirty at their prior GFN; there's no point in telling the tool stack to try an

[PATCH v2 13/13] SUPPORT.md: write down restriction of 32-bit tool stacks

2021-07-05 Thread Jan Beulich
Let's try to avoid giving the impression that 32-bit tool stacks are as capable as 64-bit ones. Signed-off-by: Jan Beulich --- v2: Wording adjustments as per review discussion. --- a/SUPPORT.md +++ b/SUPPORT.md @@ -131,6 +131,12 @@ ARM only has one guest type at the momen ## Toolstack +Whil

[linux-linus test] 163306: regressions - FAIL

2021-07-05 Thread osstest service owner
flight 163306 linux-linus real [real] http://logs.test-lab.xenproject.org/osstest/logs/163306/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-xl-xsm7 xen-install fail REGR. vs. 152332 test-amd64-i386-qem

Re: [PATCH v2 03/13] libxenguest: deal with log-dirty op stats overflow

2021-07-05 Thread Andrew Cooper
On 05/07/2021 16:13, Jan Beulich wrote: > In send_memory_live() the precise value the dirty_count struct field > gets initialized to doesn't matter much (apart from the triggering of > the log message in send_dirty_pages(), see below), but it is important > that it not be zero on the first iteratio

Re: [PATCH v2 10/13] libxc: simplify HYPERCALL_BUFFER()

2021-07-05 Thread Andrew Cooper
On 05/07/2021 16:16, Jan Beulich wrote: > _hcbuf_buf1 has been there only for a pointer comparison to validate > type compatibility. The same can be achieved by not using typeof() on > the definition of what so far was _hcbuf_buf2, as the initializer has > to also be type-compatible. Drop _hcbuf_bu

Re: [PATCH v2 03/13] libxenguest: deal with log-dirty op stats overflow

2021-07-05 Thread Jan Beulich
On 05.07.2021 17:41, Andrew Cooper wrote: > On 05/07/2021 16:13, Jan Beulich wrote: >> In send_memory_live() the precise value the dirty_count struct field >> gets initialized to doesn't matter much (apart from the triggering of >> the log message in send_dirty_pages(), see below), but it is import

[PATCH 00/16] x86/mm: large parts of P2M code and struct p2m_domain are HVM-only

2021-07-05 Thread Jan Beulich
The primary goal of this series is to leave p2m.c with, as its leading comment suggests, just code for "physical-to-machine mappings for automatically-translated domains". This requires splitting a few functions, with their non-HVM parts moved elsewhere. 01: x86/P2M: rename p2m_remove_page() 02: x

Re: [PATCH 00/16] x86/mm: large parts of P2M code and struct p2m_domain are HVM-only

2021-07-05 Thread Jan Beulich
On 05.07.2021 18:03, Jan Beulich wrote: > The primary goal of this series is to leave p2m.c with, as its leading > comment suggests, just code for "physical-to-machine mappings for > automatically-translated domains". This requires splitting a few > functions, with their non-HVM parts moved elsewhe

[PATCH 01/16] x86/P2M: rename p2m_remove_page()

2021-07-05 Thread Jan Beulich
This is in preparation to re-using the original name. Signed-off-by: Jan Beulich --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -788,8 +788,8 @@ void p2m_final_teardown(struct domain *d #ifdef CONFIG_HVM static int __must_check -p2m_remove_page(struct p2m_domain *p2m, gfn_t gfn,

[PATCH 02/16] x86/P2M: introduce p2m_{add,remove}_page()

2021-07-05 Thread Jan Beulich
p2m_add_page() is simply a rename from guest_physmap_add_entry(). p2m_remove_page() then is its counterpart, despite rendering guest_physmap_remove_page(). This way callers can use suitable pairs of functions (previously violated by hvm/grant_table.c). In HVM-specific code further avoid going thro

[PATCH 03/16] x86/P2M: drop a few CONFIG_HVM

2021-07-05 Thread Jan Beulich
This is to make it easier to see which parts of p2m.c still aren't HVM- specific: In one case the conditionals sat in an already guarded region, while in the other case P2M_AUDIT implies HVM. Signed-off-by: Jan Beulich --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -1584,11 +1584,10

[PATCH 04/16] x86/P2M: move map_domain_gfn() (again)

2021-07-05 Thread Jan Beulich
The main user is the guest walking code, so move it back there; commit 9a6787cc3809 ("x86/mm: build map_domain_gfn() just once") would perhaps better have kept it there in the first place. This way it'll only get built when it's actually needed (and still only once). This also eliminates one more

[PATCH 05/16] x86/mm: move guest_physmap_{add,remove}_page()

2021-07-05 Thread Jan Beulich
... to a new file, separating the functions from their HVM-specific backing ones, themselves only dealing with the non-translated case. To avoid having a new CONFIG_HVM conditional in there, do away with the inline placeholder. Signed-off-by: Jan Beulich --- a/xen/arch/x86/mm/Makefile +++ b/xen

[PATCH 06/16] x86/mm: split set_identity_p2m_entry() into PV and HVM parts

2021-07-05 Thread Jan Beulich
..., moving the former into the new physmap.c. Signed-off-by: Jan Beulich --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -43,6 +43,7 @@ #include #include "mm-locks.h" +#include "p2m.h" /* Override macro from asm/page.h to make work with mfn_t */ #undef virt_to_mfn @@ -1366,1

[PATCH 07/16] x86/P2M: p2m_{alloc,free}_ptp() and p2m_alloc_table() are HVM-only

2021-07-05 Thread Jan Beulich
This also includes the two p2m related fields. Signed-off-by: Jan Beulich --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -94,7 +94,9 @@ static int p2m_initialise(struct domain int ret = 0; mm_rwlock_init(&p2m->lock); +#ifdef CONFIG_HVM INIT_PAGE_LIST_HEAD(&p2m->pages

[PATCH 08/16] x86/P2M: PoD, altp2m, and nested-p2m are HVM-only

2021-07-05 Thread Jan Beulich
There's no need to initialize respective data for PV domains. Note that p2m_teardown_{alt,nested}p2m() will handle the lack-of-initialization case fine. Signed-off-by: Jan Beulich --- a/xen/arch/x86/mm/p2m.c +++ b/xen/arch/x86/mm/p2m.c @@ -102,6 +102,9 @@ static int p2m_initialise(struct domain

  1   2   >