Re: [xen-unstable test] 162845: regressions - FAIL

2021-06-16 Thread Jan Beulich
On 16.06.2021 08:54, osstest service owner wrote: > flight 162845 xen-unstable real [real] > flight 162853 xen-unstable real-retest [real] > http://logs.test-lab.xenproject.org/osstest/logs/162845/ > http://logs.test-lab.xenproject.org/osstest/logs/162853/ > > Regressions :-( > > Tests which did

[PATCH 0/2] xen: fix max_pfn handling for pv guests

2021-06-16 Thread Juergen Gross
Fix some bad naming and setting of max_pfn related variables. Juergen Gross (2): xen: fix setting of max_pfn in shared_info xen: rename wrong named pfn related variables arch/x86/include/asm/xen/page.h | 4 ++-- arch/x86/xen/p2m.c | 31 --- arch/x86/

[PATCH 2/2] xen: rename wrong named pfn related variables

2021-06-16 Thread Juergen Gross
There are some variables in Xen specific coding which names imply them holding a PFN, while in reality they are containing numbers of PFNs. Rename them accordingly. Signed-off-by: Juergen Gross --- arch/x86/include/asm/xen/page.h | 4 ++-- arch/x86/xen/p2m.c | 31 -

[PATCH 1/2] xen: fix setting of max_pfn in shared_info

2021-06-16 Thread Juergen Gross
Xen PV guests are specifying the highest used PFN via the max_pfn field in shared_info. This value is used by the Xen tools when saving or migrating the guest. Unfortunately this field is misnamed, as in reality it is specifying the number of pages (including any memory holes) of the guest, so it

Re: [PATCH v12 06/12] swiotlb: Use is_swiotlb_force_bounce for swiotlb data bouncing

2021-06-16 Thread Christoph Hellwig
On Wed, Jun 16, 2021 at 02:21:51PM +0800, Claire Chang wrote: > Propagate the swiotlb_force into io_tlb_default_mem->force_bounce and > use it to determine whether to bounce the data or not. This will be > useful later to allow for different pools. > > Signed-off-by: Claire Chang Looks good, Re

Re: [PATCH v12 09/12] swiotlb: Add restricted DMA alloc/free support

2021-06-16 Thread Christoph Hellwig
On Wed, Jun 16, 2021 at 02:21:54PM +0800, Claire Chang wrote: > Add the functions, swiotlb_{alloc,free} and is_swiotlb_for_alloc to > support the memory allocation from restricted DMA pool. > > The restricted DMA pool is preferred if available. > > Note that since coherent allocation needs remapp

[libvirt test] 162852: regressions - FAIL

2021-06-16 Thread osstest service owner
flight 162852 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/162852/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-armhf-libvirt 6 libvirt-buildfail REGR. vs. 151777 build-amd64-libvirt

[ovmf test] 162851: regressions - FAIL

2021-06-16 Thread osstest service owner
flight 162851 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/162851/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 162359 test-amd64-amd64-xl-qemuu

Re: Regressed XSA-286, was [xen-unstable test] 161917: regressions - FAIL

2021-06-16 Thread Jan Beulich
On 13.05.2021 22:15, Andrew Cooper wrote: > On 13/05/2021 04:56, osstest service owner wrote: >> flight 161917 xen-unstable real [real] >> http://logs.test-lab.xenproject.org/osstest/logs/161917/ >> >> Regressions :-( >> >> Tests which did not succeed and are blocking, >> including tests which coul

[linux-linus test] 162847: regressions - FAIL

2021-06-16 Thread osstest service owner
flight 162847 linux-linus real [real] http://logs.test-lab.xenproject.org/osstest/logs/162847/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-xl-xsm7 xen-install fail REGR. vs. 152332 test-amd64-i386-xl-

Re: [PATCH 1/2] xen: fix setting of max_pfn in shared_info

2021-06-16 Thread Jan Beulich
On 16.06.2021 09:30, Juergen Gross wrote: > Xen PV guests are specifying the highest used PFN via the max_pfn > field in shared_info. This value is used by the Xen tools when saving > or migrating the guest. > > Unfortunately this field is misnamed, as in reality it is specifying > the number of p

Re: [PATCH 2/2] xen: rename wrong named pfn related variables

2021-06-16 Thread Jan Beulich
On 16.06.2021 09:30, Juergen Gross wrote: > --- a/arch/x86/xen/p2m.c > +++ b/arch/x86/xen/p2m.c > @@ -95,8 +95,8 @@ unsigned long *xen_p2m_addr __read_mostly; > EXPORT_SYMBOL_GPL(xen_p2m_addr); > unsigned long xen_p2m_size __read_mostly; > EXPORT_SYMBOL_GPL(xen_p2m_size); > -unsigned long xen_ma

[xen-unstable-coverity test] 162857: all pass - PUSHED

2021-06-16 Thread osstest service owner
flight 162857 xen-unstable-coverity real [real] http://logs.test-lab.xenproject.org/osstest/logs/162857/ Perfect :-) All tests in this flight passed as required version targeted for testing: xen 4bcf6433eed3d9cbc00865ec62380a33ca832dac baseline version: xen 9303

Re: [PATCH 1/2] xen: fix setting of max_pfn in shared_info

2021-06-16 Thread Juergen Gross
On 16.06.21 11:52, Jan Beulich wrote: On 16.06.2021 09:30, Juergen Gross wrote: Xen PV guests are specifying the highest used PFN via the max_pfn field in shared_info. This value is used by the Xen tools when saving or migrating the guest. Unfortunately this field is misnamed, as in reality it

Re: [PATCH 2/2] xen: rename wrong named pfn related variables

2021-06-16 Thread Juergen Gross
On 16.06.21 11:56, Jan Beulich wrote: On 16.06.2021 09:30, Juergen Gross wrote: --- a/arch/x86/xen/p2m.c +++ b/arch/x86/xen/p2m.c @@ -95,8 +95,8 @@ unsigned long *xen_p2m_addr __read_mostly; EXPORT_SYMBOL_GPL(xen_p2m_addr); unsigned long xen_p2m_size __read_mostly; EXPORT_SYMBOL_GPL(xen_p2

Re: [PATCH 1/2] xen: fix setting of max_pfn in shared_info

2021-06-16 Thread Jan Beulich
On 16.06.2021 12:37, Juergen Gross wrote: > On 16.06.21 11:52, Jan Beulich wrote: >> On 16.06.2021 09:30, Juergen Gross wrote: >>> Xen PV guests are specifying the highest used PFN via the max_pfn >>> field in shared_info. This value is used by the Xen tools when saving >>> or migrating the guest.

Re: [PATCH 2/2] xen: rename wrong named pfn related variables

2021-06-16 Thread Jan Beulich
On 16.06.2021 12:43, Juergen Gross wrote: > On 16.06.21 11:56, Jan Beulich wrote: >> On 16.06.2021 09:30, Juergen Gross wrote: >>> --- a/arch/x86/xen/p2m.c >>> +++ b/arch/x86/xen/p2m.c >>> @@ -95,8 +95,8 @@ unsigned long *xen_p2m_addr __read_mostly; >>> EXPORT_SYMBOL_GPL(xen_p2m_addr); >>> unsi

[qemu-mainline bisection] complete test-amd64-amd64-xl-qemuu-debianhvm-amd64

2021-06-16 Thread osstest service owner
branch xen-unstable xenbranch xen-unstable job test-amd64-amd64-xl-qemuu-debianhvm-amd64 testid guest-saverestore Tree: linux git://xenbits.xen.org/linux-pvops.git Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git Tree: qemu git:

Re: [PATCH 1/2] xen: fix setting of max_pfn in shared_info

2021-06-16 Thread Juergen Gross
On 16.06.21 12:56, Jan Beulich wrote: On 16.06.2021 12:37, Juergen Gross wrote: On 16.06.21 11:52, Jan Beulich wrote: On 16.06.2021 09:30, Juergen Gross wrote: Xen PV guests are specifying the highest used PFN via the max_pfn field in shared_info. This value is used by the Xen tools when savin

[PATCH 1/2] Revert "swiotlb-xen: remove xen_swiotlb_dma_mmap and xen_swiotlb_dma_get_sgtable"

2021-06-16 Thread Roman Skakun
This reverts commit 922659ea771b3fd728149262c5ea15608fab9719. Signed-off-by: Roman Skakun --- drivers/xen/swiotlb-xen.c | 29 +++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c index 2b385c1b4a99..90b

[PATCH 2/2] swiotlb-xen: override common mmap and get_sgtable dma ops

2021-06-16 Thread Roman Skakun
This commit is dedicated to fix incorrect conversion from cpu_addr to page address in cases when we get virtual address which allocated through xen_swiotlb_alloc_coherent() and can be mapped in the vmalloc range. As the result, virt_to_page() cannot convert this address properly and return incorrec

Re: [PATCH] swiotlb-xen: override common mmap and get_sgtable dma ops

2021-06-16 Thread Roman Skakun
> We make sure that we allocate contiguous memory in > xen_swiotlb_alloc_coherent(). I understood. Thanks! -- Best Regards, Roman.

Re: [PATCH v12 00/12] Restricted DMA

2021-06-16 Thread Will Deacon
Hi Claire, On Wed, Jun 16, 2021 at 02:21:45PM +0800, Claire Chang wrote: > This series implements mitigations for lack of DMA access control on > systems without an IOMMU, which could result in the DMA accessing the > system memory at unexpected times and/or unexpected addresses, possibly > leadin

Re: [PATCH v1] tools: ipxe: update for fixing build with GCC11

2021-06-16 Thread Andrew Cooper
On 15/06/2021 22:26, Olaf Hering wrote: > Use a snapshot which includes commit > f3f568e382a5f19824b3bfc6081cde39eee661e8 ("[crypto] Add > memory output constraints for big-integer inline assembly"), > which fixes build with gcc11. > > Signed-off-by: Olaf Hering > --- > tools/firmware/etherboot/M

Re: [PATCH 1/5] tools/tests: Drop obsolete mce-test infrastructure

2021-06-16 Thread Andrew Cooper
On 16/06/2021 07:46, Jan Beulich wrote: > On 15.06.2021 18:19, Andrew Cooper wrote: >> mce-test has a test suite, but it depends on xend, needs to run in-tree, and >> requires manual setup of at least one guest, and manual parameters to pass >> into cases. Drop the test infrasturcture. >> >> Move

[PATCH v20210616 00/36] leftover from 2020

2021-06-16 Thread Olaf Hering
Various unreviewed changes, rebase to 4bcf6433ee. Olaf Hering (36): hotplug/Linux: fix starting of xenstored with restarting systemd tools: add API to work with sevaral bits at once xl: fix description of migrate --debug tools: create libxensaverestore MAINTAINERS: add myself as saverest

[PATCH v20210616 02/36] tools: add API to work with sevaral bits at once

2021-06-16 Thread Olaf Hering
Introduce new API to test if a fixed number of bits is clear or set, and clear or set them all at once. The caller has to make sure the input bitnumber is a multiple of BITS_PER_LONG. This API avoids the loop over each bit in a known range just to see if all of them are either clear or set. Sign

[PATCH v20210616 01/36] hotplug/Linux: fix starting of xenstored with restarting systemd

2021-06-16 Thread Olaf Hering
A hard to trigger race with another unrelated systemd service and xenstored.service unveiled a bug in the way how xenstored is launched with systemd. launch-xenstore may start either a daemon or a domain. In case a domain is used, systemd-notify was called. If another service triggered a restart o

[PATCH v20210616 06/36] tools: add readv_exact to libxenctrl

2021-06-16 Thread Olaf Hering
Read a batch of iovec's. Short reads are the common case, finish the trailing iov with read_exact. Signed-off-by: Olaf Hering v2: - add comment to short-read handling --- tools/libs/ctrl/xc_private.c | 57 +++- tools/libs/ctrl/xc_private.h | 1 + 2 files change

[PATCH v20210616 03/36] xl: fix description of migrate --debug

2021-06-16 Thread Olaf Hering
xl migrate --debug used to track every pfn in every batch of pages. But these times are gone. The code in xc_domain_save is the consumer of this knob, but it considers it only for the remus and colo case. Adjust the help text to tell what --debug does today: Nothing. Signed-off-by: Olaf Hering R

[PATCH v20210616 13/36] tools: save: move types array

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move types array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 7 ++- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/libs/save

[PATCH v20210616 08/36] tools: use sr_is_known_page_type

2021-06-16 Thread Olaf Hering
Verify pfn type on sending side, also verify incoming batch of pfns. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross v02: - use sr_is_known_page_type instead of xc_is_known_page_type --- tools/libs/saverestore/restore.c | 3 +-- tools/libs/saverestore/save.c| 6 ++ 2 files change

[PATCH v20210616 07/36] tools: add sr_is_known_page_type to libsaverestore

2021-06-16 Thread Olaf Hering
Users of xc_get_pfn_type_batch may want to sanity check the data returned by Xen. Add a simple helper for this purpose. Signed-off-by: Olaf Hering v02: - rename xc_is_known_page_type to sr_is_known_page_type - move from ctrl/xc_private.h to saverestore/common.h (jgross) --- tools/libs/saveresto

[PATCH v20210616 17/36] tools: save: move guest_data array

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move guest_data array into preallocated space. Because this was allocated with calloc: Adjust the loop to clear unused entries as needed. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/sa

[PATCH v20210616 19/36] tools: restore: move types array

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move types array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 1 + tools/libs/saverestore/restore.c | 12 +--- 2 files changed, 2 insertions(+), 11 deletions(-) diff --git a/tools/

[PATCH v20210616 09/36] tools: unify type checking for data pfns in migration stream

2021-06-16 Thread Olaf Hering
Introduce a helper which decides if a given pfn type has data for the migration stream. No change in behavior intended. Signed-off-by: Olaf Hering --- tools/libs/saverestore/common.h | 17 tools/libs/saverestore/restore.c | 34 +--- tools/libs/saver

[PATCH v20210616 20/36] tools: restore: move mfns array

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move mfns array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/restore.c | 5 ++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/tools/libs/saver

[PATCH v20210616 10/36] tools: show migration transfer rate in send_dirty_pages

2021-06-16 Thread Olaf Hering
Show how fast domU pages are transferred in each iteration. The relevant data is how fast the pfns travel, not so much how much protocol overhead exists. So the reported MiB/sec is just for pfns. Signed-off-by: Olaf Hering v02: - rearrange MiB_sec calculation (jgross) --- tools/libs/saverestor

[PATCH v20210616 21/36] tools: restore: move map_errs array

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move map_errs array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 1 + tools/libs/saverestore/restore.c | 12 +--- 2 files changed, 2 insertions(+), 11 deletions(-) diff --git a/too

[PATCH v20210616 12/36] tools: save: move mfns array

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move mfns array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 7 ++- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/libs/saver

[PATCH v20210616 22/36] tools: restore: move mfns array in populate_pfns

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move populate_pfns mfns array into preallocated space. Use some prefix to avoid conflict with an array used in handle_page_data. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/restore.c |

[PATCH v20210616 11/36] tools: prepare to allocate saverestore arrays once

2021-06-16 Thread Olaf Hering
The hotpath 'send_dirty_pages' is supposed to do just one thing: sending. The other end 'handle_page_data' is supposed to do just receiving. But instead both do other costly work like memory allocations and data moving. Do the allocations once, the array sizes are a compiletime constant. Avoid unn

[PATCH v20210616 23/36] tools: restore: move pfns array in populate_pfns

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move populate_pfns' pfns array into preallocated space. Use some prefix to avoid conflict with an array used in handle_page_data. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 1 + tools/libs/saverestore/restore.c

Re: [PATCH v1] tools: ipxe: update for fixing build with GCC11

2021-06-16 Thread Olaf Hering
Please revert bf4ccd4265ac614fbfa38bf168b6eeaf4c17d51e in ipxe.git, CentOS 7 apparently fails to handle '-D'. It worked in my testing with SLE12SP5 and SLE15SP3 as a base system. See below. I guess for xen.git, updating to just bf4ccd4265ac614fbfa38bf168b6eeaf4c17d51e^ will be good enough. O

[PATCH v20210616 29/36] tools: change struct precopy_stats to precopy_stats_t

2021-06-16 Thread Olaf Hering
This will help libxl_save_msgs_gen.pl to copy the struct as a region of memory. No change in behavior intented. Signed-off-by: Olaf Hering --- tools/include/xensaverestore.h | 7 +++ tools/libs/saverestore/common.h | 2 +- tools/libs/saverestore/save.c | 6 +++--- 3 files changed, 7 inse

[PATCH v20210616 04/36] tools: create libxensaverestore

2021-06-16 Thread Olaf Hering
Move all save/restore related code from libxenguest.so into a separate library libxensaverestore.so. The only consumer is libxl-save-helper. There is no need to have the moved code mapped all the time in binaries where libxenguest.so is used. According to size(1) the change is: textdata

[PATCH v20210616 14/36] tools: save: move errors array

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move errors array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 7 ++- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/libs/sav

[PATCH v20210616 33/36] tools: add --abort_if_busy to libxl_domain_suspend

2021-06-16 Thread Olaf Hering
Provide a knob to the host admin to abort the live migration of a running domU if the downtime during final transit will be too long for the workload within domU. Adjust error reporting. Add ERROR_MIGRATION_ABORTED to allow callers of libxl_domain_suspend to distinguish between errors and the requ

[PATCH v20210616 26/36] tools: restore: write data directly into guest

2021-06-16 Thread Olaf Hering
Read incoming migration stream directly into the guest memory. This avoids the memory allocation and copying, and the resulting performance penalty. Signed-off-by: Olaf Hering --- tools/libs/saverestore/common.h | 1 + tools/libs/saverestore/restore.c | 132 ++- 2

[PATCH v20210616 32/36] tools: add --min_remaining to libxl_domain_suspend

2021-06-16 Thread Olaf Hering
The decision to stop+move a domU to the new host must be based on two factors: - the available network bandwidth for the migration stream - the maximum time a workload within a domU can be savely suspended Both values define how many dirty pages a workload may produce prior the final stop+move. T

[PATCH v20210616 18/36] tools: save: move local_pages array

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move local_pages array into preallocated space. Adjust the code to use the src page as is in case of HVM. In case of PV the page may need to be normalised, use a private memory area for this purpose. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tool

[PATCH v20210616 36/36] tools: use superpages during restore of HVM guest

2021-06-16 Thread Olaf Hering
During creating of a HVM domU meminit_hvm() tries to map superpages. After save/restore or migration this mapping is lost, everything is allocated in single pages. This causes a performance degradation after migration. Add neccessary code to preallocate a superpage for an incoming chunk of pfns. I

[PATCH v20210616 31/36] tools: add --max_iters to libxl_domain_suspend

2021-06-16 Thread Olaf Hering
Migrating a large, and potentially busy, domU will take more time than neccessary due to excessive number of copying iterations. Allow to host admin to control the number of iterations which copy cumulated domU dirty pages to the target host. The default remains 5, which means one initial iterati

[PATCH v20210616 30/36] tools: add callback to libxl for precopy_policy and precopy_stats_t

2021-06-16 Thread Olaf Hering
This duplicates simple_precopy_policy. To recap its purpose: - do up to 5 iterations of copying dirty domU memory to target, including the initial copying of all domU memory, excluding the final copying while the domU is suspended - do fewer iterations in case the domU dirtied less than 50 page

[PATCH v20210616 35/36] tools: use sr_bitmap for populated_pfns

2021-06-16 Thread Olaf Hering
Signed-off-by: Olaf Hering v02: - remove xg_ prefix from called functions --- tools/libs/saverestore/common.h | 21 +++- tools/libs/saverestore/restore.c | 69 tools/libs/saverestore/restore_x86_hvm.c | 9 tools/libs/saverestore/restore_x86_pv.

[PATCH v20210616 34/36] tools: add API for expandable bitmaps

2021-06-16 Thread Olaf Hering
Since the incoming migration stream lacks info about what the highest pfn will be, data structures can not be allocated upfront. Add an API for expandable bitmaps, loosely based on pfn_set_populated. Signed-off-by: Olaf Hering v02: - remove xg_ prefix from functions --- tools/libs/saverestore/

Re: [PATCH 2/5] tools/tests: Drop run runes

2021-06-16 Thread Andrew Cooper
On 16/06/2021 07:44, Jan Beulich wrote: > On 15.06.2021 18:19, Andrew Cooper wrote: >> --- a/tools/tests/x86_emulator/Makefile >> +++ b/tools/tests/x86_emulator/Makefile >> @@ -7,10 +7,6 @@ TARGET := test_x86_emulator >> .PHONY: all >> all: >> >> -.PHONY: run >> -run: $(TARGET) >> -./$(TARG

[PATCH v20210616 16/36] tools: save: move rec_pfns array

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move rec_pfns array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 11 +-- 2 files changed, 3 insertions(+), 10 deletions(-) diff --git a/tools

[PATCH v20210616 24/36] tools: restore: split record processing

2021-06-16 Thread Olaf Hering
handle_page_data must be able to read directly into mapped guest memory. This will avoid unneccesary memcpy calls for data which can be consumed verbatim. Rearrange the code to allow decisions based on the incoming record. This change is preparation for future changes in handle_page_data, no cha

[PATCH v20210616 05/36] MAINTAINERS: add myself as saverestore maintainer

2021-06-16 Thread Olaf Hering
I touched it last. Signed-off-by: Olaf Hering --- MAINTAINERS | 6 ++ 1 file changed, 6 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 39750bb75d..dbb8f56ab3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -381,6 +381,12 @@ R: Juergen Gross S: Supported F: tools/libs/

[PATCH v20210616 15/36] tools: save: move iov array

2021-06-16 Thread Olaf Hering
Remove allocation from hotpath, move iov array into preallocated space. Signed-off-by: Olaf Hering Reviewed-by: Juergen Gross --- tools/libs/saverestore/common.h | 2 ++ tools/libs/saverestore/save.c | 7 ++- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/libs/savere

[PATCH v20210616 28/36] tools: adjust libxl_domain_suspend to receive a struct props

2021-06-16 Thread Olaf Hering
Upcoming changes will pass more knobs down to xc_domain_save. Adjust the libxl_domain_suspend API to allow easy adding of additional knobs. No change in behavior intented. Signed-off-by: Olaf Hering Acked-by: Christian Lindig --- tools/include/libxl.h| 26 ++

[PATCH v20210616 27/36] tools: recognize LIBXL_API_VERSION for 4.16

2021-06-16 Thread Olaf Hering
This is required by upcoming API changes. Signed-off-by: Olaf Hering --- tools/include/libxl.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/include/libxl.h b/tools/include/libxl.h index ae7fe27c1f..29931626a2 100644 --- a/tools/include/libxl.h +++ b/tools/include/l

[PATCH v20210616 25/36] tools: restore: split handle_page_data

2021-06-16 Thread Olaf Hering
handle_page_data must be able to read directly into mapped guest memory. This will avoid unneccesary memcpy calls for data that can be consumed verbatim. Split the various steps of record processing: - move processing to handle_buffered_page_data - adjust xenforeignmemory_map to set errno in case

[PATCH v2] tools: ipxe: update for fixing build with GCC11

2021-06-16 Thread Olaf Hering
Use a snapshot which includes commit f3f568e382a5f19824b3bfc6081cde39eee661e8 ("[crypto] Add memory output constraints for big-integer inline assembly"), which fixes build with gcc11. Signed-off-by: Olaf Hering --- tools/firmware/etherboot/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 del

Re: [PATCH 0/5] tools/tests: More cleanup for automation improvements

2021-06-16 Thread Andrew Cooper
On 16/06/2021 07:38, Jan Beulich wrote: > On 15.06.2021 18:19, Andrew Cooper wrote: >> Jan/Roger: x86_emulator and vpci use $(HOSTCC) not $(CC). While they are >> unit >> tests, we still potentially want to run them in dom0 rather than the build >> environment - particularly for x86_emulator whic

Re: [PATCH v1] tools: ipxe: update for fixing build with GCC11

2021-06-16 Thread Bernhard M. Wiedemann
So this means, CentOS7 binutils has 9cb80f72d8b from 2011-12-21 but not git blame binutils/objcopy.c|grep enable-determini 955d0b3bd75 (Roland McGrath 2013-01-07 17:40:59 + 549) -D --enable-deterministic-archives\n\ 2e30cb575a1 (Cary Coutant 2012-04-25 17:50:14 + 555) -D

[qemu-mainline test] 162850: regressions - FAIL

2021-06-16 Thread osstest service owner
flight 162850 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/162850/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-qemuu-freebsd11-amd64 16 guest-saverestore fail REGR. vs. 152631 test-amd64-am

Re: [PATCH 2/5] tools/tests: Drop run runes

2021-06-16 Thread Jan Beulich
On 16.06.2021 15:08, Andrew Cooper wrote: > On 16/06/2021 07:44, Jan Beulich wrote: >> On 15.06.2021 18:19, Andrew Cooper wrote: >>> --- a/tools/tests/x86_emulator/Makefile >>> +++ b/tools/tests/x86_emulator/Makefile >>> @@ -7,10 +7,6 @@ TARGET := test_x86_emulator >>> .PHONY: all >>> all: >>>

Re: [PATCH 2/2] swiotlb-xen: override common mmap and get_sgtable dma ops

2021-06-16 Thread Boris Ostrovsky
On 6/16/21 7:42 AM, Roman Skakun wrote: > This commit is dedicated to fix incorrect conversion from > cpu_addr to page address in cases when we get virtual > address which allocated through xen_swiotlb_alloc_coherent() > and can be mapped in the vmalloc range. > As the result, virt_to_page() cann

Re: [PATCH 0/5] tools/tests: More cleanup for automation improvements

2021-06-16 Thread Jan Beulich
On 16.06.2021 15:22, Andrew Cooper wrote: > On 16/06/2021 07:38, Jan Beulich wrote: >> On 15.06.2021 18:19, Andrew Cooper wrote: >>> Jan/Roger: x86_emulator and vpci use $(HOSTCC) not $(CC). While they are >>> unit >>> tests, we still potentially want to run them in dom0 rather than the build >>>

Re: [xen-unstable test] 162845: regressions - FAIL

2021-06-16 Thread Anthony PERARD
On Wed, Jun 16, 2021 at 09:12:52AM +0200, Jan Beulich wrote: > On 16.06.2021 08:54, osstest service owner wrote: > > flight 162845 xen-unstable real [real] > > flight 162853 xen-unstable real-retest [real] > > http://logs.test-lab.xenproject.org/osstest/logs/162845/ > > http://logs.test-lab.xenproj

Re: [PATCH 2/2] swiotlb-xen: override common mmap and get_sgtable dma ops

2021-06-16 Thread Christoph Hellwig
On Wed, Jun 16, 2021 at 10:12:55AM -0400, Boris Ostrovsky wrote: > I wonder now whether we could avoid code duplication between here and > dma_common_mmap()/dma_common_get_sgtable() and use your helper there. > > > Christoph, would that work?  I.e. something like You should not duplicate the co

[PATCH 02/10] tools/xenstored: Introduce lu_get_connection() and use it

2021-06-16 Thread Julien Grall
From: Julien Grall At the moment, dump_state_buffered_data() is taking two connections in parameters (one is the connection to dump, the other is the connection used to request LU). The naming doesn't help to distinguish (c vs conn) them and this already lead to several mistake while modifying th

[PATCH 00/10] tools/xenstored: Bug fixes + Improve Live-Update

2021-06-16 Thread Julien Grall
From: Julien Grall Hi all, At the moment, Live-Update will, by default, not proceed if there are in-flight transactions. It is possible force it by passing -F but this will break any connection with in-flight transactions. There are PV drivers out that may never terminate some transaction. On h

[PATCH 04/10] tools/xenstored: Limit the number of requests a connection can delay

2021-06-16 Thread Julien Grall
From: Julien Grall Currently, only liveupdate request can be delayed. The request can only be performed by a privileged connection (e.g. dom0). So it is fine to have no limits. In a follow-up patch we will want to delay request for unprivileged connection as well. So it is best to apply a limit.

[PATCH 01/10] MAINTAINERS: Add myself as reviewers for tools/xenstore

2021-06-16 Thread Julien Grall
I would like to help reviewing Xenstored patches. It is more convenient to find them if I am CCed. Signed-off-by: Julien Grall --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index 39750bb75db5..dd8c011456cd 100644 --- a/MAINTAINERS +++ b/MAINTAINERS

[PATCH 06/10] tools/xenstored: Introduce a wrapper for conn->funcs->can_{read, write}

2021-06-16 Thread Julien Grall
From: Julien Grall Currently, the callbacks can_read and can_write are called directly. This doesn't allow us to add generic check and therefore requires duplication. At the moment, one check that could benefit to be common is whether the connection should ignored. The position is slightly diffe

[PATCH 05/10] tools/xenstored: xenstored_core.h should include fcntl.h

2021-06-16 Thread Julien Grall
From: Julien Grall xenstored_core.h will consider live-udpate is not supported if O_CLOEXEC doesn't exist. However, the header doesn't include the one defining O_CLOEXEC (i.e. fcntl.h). This means that depending on the header included, some source file will think Live-Update is not supported. I

[PATCH 03/10] tools/xenstore: Don't assume conn->in points to the LU request

2021-06-16 Thread Julien Grall
From: Julien Grall call_delayed() is currently assuming that conn->in is NULL when handling delayed request. However, the connection is not paused. Therefore new request can be processed and conn->in may be non-NULL if we have only received a partial request. Furthermore, as we overwrite conn->i

[PATCH 08/10] tools/xenstored: Extend restore code to handle multiple input buffer

2021-06-16 Thread Julien Grall
From: Julien Grall Currently, the restore code is considering the stream will contain at most one in-flight request per connection. In a follow-up changes, we will want to transfer multiple in-flight requests. The function read_state_buffered() is now extended to restore multiple in-flight reque

[PATCH 07/10] tools/xenstored: delay_request: don't assume conn->in == in

2021-06-16 Thread Julien Grall
From: Julien Grall delay_request() is currently assuming that the request delayed is always conn->in. This is currently correct, but it is a call for a latent bug as the function allows the caller to specify any request. To prevent any future surprise, check if the request delayed is the current

[PATCH 09/10] tools/xenstored: Dump delayed requests

2021-06-16 Thread Julien Grall
From: Julien Grall Currently, only Live-Update request can be delayed. In a follow-up, we will want to delay more requests (e.g. transaction start). Therefore we want to preserve delayed requests across Live-Update. Delayed requests are just complete "in" buffer. So the code is refactored to all

Re: [xen-unstable test] 162845: regressions - FAIL

2021-06-16 Thread Jan Beulich
On 16.06.2021 16:21, Anthony PERARD wrote: > On Wed, Jun 16, 2021 at 09:12:52AM +0200, Jan Beulich wrote: >> On 16.06.2021 08:54, osstest service owner wrote: >>> flight 162845 xen-unstable real [real] >>> flight 162853 xen-unstable real-retest [real] >>> http://logs.test-lab.xenproject.org/osstest

Re: [PATCH v20210616 00/36] leftover from 2020

2021-06-16 Thread Andrew Cooper
On 16/06/2021 13:50, Olaf Hering wrote: > Various unreviewed changes, rebase to 4bcf6433ee. General CI run: https://gitlab.com/xen-project/patchew/xen/-/pipelines/322032419 Some specific failures. https://gitlab.com/xen-project/patchew/xen/-/jobs/1351977567 (32bit toolstack build): common.c: In

[PATCH 10/10] tools/xenstored: Delay new transaction while Live-Update is pending

2021-06-16 Thread Julien Grall
From: Julien Grall At the moment, Live-Update will, by default, not proceed if there are in-flight transactions. It is possible force it by passing -F but this will break any connection with in-flight transactions. There are PV drivers out that may never terminate some transaction. On host runni

Re: [xen-unstable test] 162845: regressions - FAIL

2021-06-16 Thread Jan Beulich
On 16.06.2021 16:49, Jan Beulich wrote: > On 16.06.2021 16:21, Anthony PERARD wrote: >> On Wed, Jun 16, 2021 at 09:12:52AM +0200, Jan Beulich wrote: >>> On 16.06.2021 08:54, osstest service owner wrote: flight 162845 xen-unstable real [real] flight 162853 xen-unstable real-retest [real] >

Re: [PATCH v20210616 00/36] leftover from 2020

2021-06-16 Thread Olaf Hering
Am Wed, 16 Jun 2021 15:50:24 +0100 schrieb Andrew Cooper : > new_max |= new_max >> 32; Lazy compiler? I hoped this is a compile-time constant, which evaluates to zero in 32bit builds. if ( sizeof(unsigned long) > 4 ) I guess a #ifdef as

Re: [xen-unstable test] 162845: regressions - FAIL

2021-06-16 Thread Anthony PERARD
On Wed, Jun 16, 2021 at 04:49:33PM +0200, Jan Beulich wrote: > I don't think it should. But I now notice I should have looked at the > logs of these tests: > > xc: info: Saving domain 2, type x86 HVM > xc: error: Unable to obtain the guest p2m size (1 = Operation not permitted): > Internal error

Re: [PATCH 01/10] MAINTAINERS: Add myself as reviewers for tools/xenstore

2021-06-16 Thread Juergen Gross
On 16.06.21 16:43, Julien Grall wrote: I would like to help reviewing Xenstored patches. It is more convenient to find them if I am CCed. Signed-off-by: Julien Grall Acked-by: Juergen Gross Juergen OpenPGP_0xB0DE9DD628BF132F.asc Description: OpenPGP public key OpenPGP_signature Descrip

Re: [PATCH v20210616 04/36] tools: create libxensaverestore

2021-06-16 Thread Olaf Hering
Am Wed, 16 Jun 2021 14:50:57 +0200 schrieb Olaf Hering : > Move all save/restore related code from libxenguest.so into a separate > library libxensaverestore.so. This additional change is required to cover non-x86. --- a/tools/libs/saverestore/nomigrate.c +++ b/tools/libs/saverestore/nomigrate.c

[PATCH] code-of-conduct.rst: Add Stefano Stabellini as a Conduct Team member

2021-06-16 Thread George Dunlap
With my upcoming leave, Ian will be the only person actively on the Conduct Team. Stefano has volunteered to join the team, so that there are at least two members. Signed-off-by: George Dunlap --- Membership of the Conduct Team is a global XenProject decision, and so needs a vote of the leadersh

Re: [PATCH] code-of-conduct.rst: Add Stefano Stabellini as a Conduct Team member

2021-06-16 Thread Julien Grall
Hi George, On 16/06/2021 17:18, George Dunlap wrote: With my upcoming leave, Ian will be the only person actively on the Conduct Team. Stefano has volunteered to join the team, so that there are at least two members. Signed-off-by: George Dunlap --- Membership of the Conduct Team is a global

Re: [PATCH 2/2] swiotlb-xen: override common mmap and get_sgtable dma ops

2021-06-16 Thread Boris Ostrovsky
On 6/16/21 10:21 AM, Christoph Hellwig wrote: > On Wed, Jun 16, 2021 at 10:12:55AM -0400, Boris Ostrovsky wrote: >> I wonder now whether we could avoid code duplication between here and >> dma_common_mmap()/dma_common_get_sgtable() and use your helper there. >> >> >> Christoph, would that work? 

Re: [xen-unstable test] 162845: regressions - FAIL

2021-06-16 Thread Jan Beulich
On 16.06.2021 17:12, Anthony PERARD wrote: > On Wed, Jun 16, 2021 at 04:49:33PM +0200, Jan Beulich wrote: >> I don't think it should. But I now notice I should have looked at the >> logs of these tests: >> >> xc: info: Saving domain 2, type x86 HVM >> xc: error: Unable to obtain the guest p2m size

Re: [PATCH 2/2] swiotlb-xen: override common mmap and get_sgtable dma ops

2021-06-16 Thread Christoph Hellwig
On Wed, Jun 16, 2021 at 11:33:50AM -0400, Boris Ostrovsky wrote: > Isn't the expectation of virt_to_page() that it only works on non-vmalloc'd > addresses? (This is not a rhetorical question, I actually don't know). Yes. Thus is why I'd suggest to just do the vmalloc_to_page or virt_to_page danc

Re: [PATCH v20210616 00/36] leftover from 2020

2021-06-16 Thread Olaf Hering
Am Wed, 16 Jun 2021 15:50:24 +0100 schrieb Andrew Cooper : > 32bit toolstack build as in i386? How is this used in practice? I guess such build should be marked as CONFIG_MIGRATE=n in config/x86_32.mk? Olaf pgpSd_TByeC2C.pgp Description: Digitale Signatur von OpenPGP

Re: [PATCH 2/2] swiotlb-xen: override common mmap and get_sgtable dma ops

2021-06-16 Thread Boris Ostrovsky
On 6/16/21 11:35 AM, Christoph Hellwig wrote: > On Wed, Jun 16, 2021 at 11:33:50AM -0400, Boris Ostrovsky wrote: >> Isn't the expectation of virt_to_page() that it only works on non-vmalloc'd >> addresses? (This is not a rhetorical question, I actually don't know). > Yes. Thus is why I'd sugges

Re: Regressed XSA-286, was [xen-unstable test] 161917: regressions - FAIL

2021-06-16 Thread Andrew Cooper
On 16/06/2021 09:48, Jan Beulich wrote: > On 13.05.2021 22:15, Andrew Cooper wrote: >> On 13/05/2021 04:56, osstest service owner wrote: >>> flight 161917 xen-unstable real [real] >>> http://logs.test-lab.xenproject.org/osstest/logs/161917/ >>> >>> Regressions :-( >>> >>> Tests which did not succee

Re: [PATCH 2/2] swiotlb-xen: override common mmap and get_sgtable dma ops

2021-06-16 Thread Christoph Hellwig
On Wed, Jun 16, 2021 at 11:39:07AM -0400, Boris Ostrovsky wrote: > > On 6/16/21 11:35 AM, Christoph Hellwig wrote: > > On Wed, Jun 16, 2021 at 11:33:50AM -0400, Boris Ostrovsky wrote: > >> Isn't the expectation of virt_to_page() that it only works on > >> non-vmalloc'd addresses? (This is not a r

  1   2   >