Re: [RFC] UEFI Secure Boot on Xen Hosts

2020-04-30 Thread Jan Beulich
On 30.04.2020 00:51, Bobby Eshleman wrote: > Hey all, > > We're looking to develop UEFI Secure Boot support for grub-loaded Xen and > ultimately for XCP-ng (I'm on the XCP-ng team at Vates). > > In addition to carrying the chain-of-trust through the entire boot sequence > into dom0, we would also

Re: [PATCH v1 2/3] mm/memory_hotplug: Introduce MHP_DRIVER_MANAGED

2020-04-30 Thread David Hildenbrand
On 29.04.20 18:08, David Hildenbrand wrote: > Some paravirtualized devices that add memory via add_memory() and > friends (esp. virtio-mem) don't want to create entries in > /sys/firmware/memmap/ - primarily to hinder kexec from adding this > memory to the boot memmap of the kexec kernel. > > In f

Re: [PATCH] x86/hap: be more selective with assisted TLB flush

2020-04-30 Thread Jan Beulich
On 29.04.2020 19:36, Roger Pau Monne wrote: > When doing an assisted flush on HAP the purpose of the > on_selected_cpus is just to trigger a vmexit on remote CPUs that are > in guest context, and hence just using is_vcpu_dirty_cpu is too lax, > also check that the vCPU is running. Am I right to un

Re: [PATCH 2/2] xen: credit2: limit the max number of CPUs in a runqueue

2020-04-30 Thread Jürgen Groß
On 29.04.20 19:36, Dario Faggioli wrote: In Credit2 CPUs (can) share runqueues, depending on the topology. For instance, with per-socket runqueues (the default) all the CPUs that are part of the same socket share a runqueue. On platform with a huge number of CPUs per socket, that could be a prob

Re: [PATCH v1 2/3] mm/memory_hotplug: Introduce MHP_DRIVER_MANAGED

2020-04-30 Thread Dan Williams
On Thu, Apr 30, 2020 at 12:20 AM David Hildenbrand wrote: > > On 29.04.20 18:08, David Hildenbrand wrote: > > Some paravirtualized devices that add memory via add_memory() and > > friends (esp. virtio-mem) don't want to create entries in > > /sys/firmware/memmap/ - primarily to hinder kexec from a

Re: Xen Compilation Error on Ubuntu 20.04

2020-04-30 Thread Ayush Dosaj
Hi Andrew, Xen Development team. I compiled and installed Xen by appending -fcf-protection=none to CFLAGS on Ubuntu 20.04 but it still crashes on startup. On Wed, Apr 29, 2020 at 10:58 PM Ayush Dosaj wrote: > Awesome, thanks! > > On Wed, Apr 29, 2020 at 10:55 PM Andrew Cooper > wrote: > >> On

Re: [PATCH v1 2/3] mm/memory_hotplug: Introduce MHP_DRIVER_MANAGED

2020-04-30 Thread David Hildenbrand
On 30.04.20 10:11, Dan Williams wrote: > On Thu, Apr 30, 2020 at 12:20 AM David Hildenbrand wrote: >> >> On 29.04.20 18:08, David Hildenbrand wrote: >>> Some paravirtualized devices that add memory via add_memory() and >>> friends (esp. virtio-mem) don't want to create entries in >>> /sys/firmware

Re: [PATCH] x86/hap: be more selective with assisted TLB flush

2020-04-30 Thread Roger Pau Monné
On Thu, Apr 30, 2020 at 09:20:58AM +0200, Jan Beulich wrote: > On 29.04.2020 19:36, Roger Pau Monne wrote: > > When doing an assisted flush on HAP the purpose of the > > on_selected_cpus is just to trigger a vmexit on remote CPUs that are > > in guest context, and hence just using is_vcpu_dirty_cpu

Re: [PATCH v1 2/3] mm/memory_hotplug: Introduce MHP_DRIVER_MANAGED

2020-04-30 Thread Dan Williams
On Thu, Apr 30, 2020 at 1:21 AM David Hildenbrand wrote: > >> Just because we decided to use some DAX memory in the current kernel as > >> system ram, doesn't mean we should make that decision for the kexec > >> kernel (e.g., using it as initial memory, placing kexec binaries onto > >> it, etc.).

Re: [PATCH] x86/hap: be more selective with assisted TLB flush

2020-04-30 Thread Jan Beulich
On 30.04.2020 10:28, Roger Pau Monné wrote: > On Thu, Apr 30, 2020 at 09:20:58AM +0200, Jan Beulich wrote: >> On 29.04.2020 19:36, Roger Pau Monne wrote: >>> When doing an assisted flush on HAP the purpose of the >>> on_selected_cpus is just to trigger a vmexit on remote CPUs that are >>> in guest

Is it an April Fools' Day Joke?

2020-04-30 Thread Jason Long
Hello,According to "https://xenproject.org/2017/04/01/xen-hypervisor-to-be-rewritten/"; article, is just a joke or real? Thanks.

[PATCH] x86/hvm: allow for more fine grained assisted flush

2020-04-30 Thread Roger Pau Monne
Improve the assisted flush by expanding the interface and allowing for more fine grained TLB flushes to be issued using the HVMOP_flush_tlbs hypercall. Support for such advanced flushes is signaled in CPUID using the XEN_HVM_CPUID_ADVANCED_FLUSH flag. The new features make use of the NULL paramete

[xen-unstable test] 149881: tolerable FAIL - PUSHED

2020-04-30 Thread osstest service owner
flight 149881 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/149881/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-rtds 17 guest-saverestore.2 fail blocked in 149871 test-amd64-amd64-xl-qemuu-win7-amd64

[PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Andrew Cooper
Fam19h is very similar to Fam17h in these regards. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/acpi/cpu_idle.c | 1 + xen/arch/x86/cpu/microcode/amd.c | 1 + xen/arch/x86/cpu/vpmu_amd.c | 1 + xen/arch/x86/nmi.c | 2

Re: [PATCH] x86/hyperv: stash and use the configured max VP index

2020-04-30 Thread Roger Pau Monné
On Wed, Apr 29, 2020 at 11:47:18AM +, Wei Liu wrote: > On Wed, Apr 29, 2020 at 11:41:44AM +0100, Wei Liu wrote: > > The value returned from CPUID is the maximum number for virtual > > processors supported by Hyper-V. It could be larger than the maximum > > number of virtual processors configure

Re: [PATCH] x86/hyperv: stash and use the configured max VP index

2020-04-30 Thread Roger Pau Monné
On Thu, Apr 30, 2020 at 12:15:58PM +0200, Roger Pau Monné wrote: > On Wed, Apr 29, 2020 at 11:47:18AM +, Wei Liu wrote: > > On Wed, Apr 29, 2020 at 11:41:44AM +0100, Wei Liu wrote: > > > The value returned from CPUID is the maximum number for virtual > > > processors supported by Hyper-V. It co

[PATCH] x86/HyperV: correct hv_hcall_page for xen.efi build

2020-04-30 Thread Jan Beulich
Along the lines of what the not reverted part of 3c4b2eef4941 ("x86: refine link time stub area related assertion") did, we need to transform the absolute HV_HCALL_PAGE into the image base relative hv_hcall_page (or else there'd be no need for two distinct symbols). Otherwise mkreloc, as used for g

[PATCH] x86/EFI: correct section offsets in mkreloc diagnostics

2020-04-30 Thread Jan Beulich
These are more helpful if they point at the address where the relocated value starts, rather than at the specific byte of the difference. Signed-off-by: Jan Beulich --- a/xen/arch/x86/efi/mkreloc.c +++ b/xen/arch/x86/efi/mkreloc.c @@ -238,7 +238,7 @@ static void diff_sections(const unsigned

[PATCH v2 1/3] mm/memory_hotplug: Prepare passing flags to add_memory() and friends

2020-04-30 Thread David Hildenbrand
We soon want to pass flags - prepare for that. This patch is based on a similar patch by Oscar Salvador: https://lkml.kernel.org/r/20190625075227.15193-3-osalva...@suse.de Acked-by: Wei Liu Cc: Michael Ellerman Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: "Rafael J. Wysocki" Cc: Len Br

[PATCH v2 3/3] device-dax: Add system ram (add_memory()) with MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread David Hildenbrand
Currently, when adding memory, we create entries in /sys/firmware/memmap/ as "System RAM". This does not reflect the reality and will lead to kexec-tools to add that memory to the fixed-up initial memmap for a kexec kernel (loaded via kexec_load()). The memory will be considered initial System RAM

[PATCH v2 0/3] mm/memory_hotplug: Allow to not create firmware memmap entries

2020-04-30 Thread David Hildenbrand
This is the follow up of [1]: [PATCH v1 0/3] mm/memory_hotplug: Make virtio-mem play nicely with kexec-tools I realized that this is not only helpful for virtio-mem, but also for dax/kmem - it's a fix for that use case (see patch #3) of persistent memory. Also, while testing, I di

[PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread David Hildenbrand
Some devices/drivers that add memory via add_memory() and friends (e.g., dax/kmem, but also virtio-mem in the future) don't want to create entries in /sys/firmware/memmap/ - primarily to hinder kexec from adding this memory to the boot memmap of the kexec kernel. In fact, such memory is never expo

Re: [PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Roger Pau Monné
On Thu, Apr 30, 2020 at 10:59:47AM +0100, Andrew Cooper wrote: > diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c > index c3f92ed231..014524486f 100644 > --- a/xen/arch/x86/nmi.c > +++ b/xen/arch/x86/nmi.c > @@ -398,7 +398,7 @@ void setup_apic_nmi_watchdog(void) > case X86_VENDOR_AMD: >

Re: [PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Andrew Cooper
On 30/04/2020 11:35, Roger Pau Monné wrote: > On Thu, Apr 30, 2020 at 10:59:47AM +0100, Andrew Cooper wrote: >> diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c >> index c3f92ed231..014524486f 100644 >> --- a/xen/arch/x86/nmi.c >> +++ b/xen/arch/x86/nmi.c >> @@ -398,7 +398,7 @@ void setup_apic_

Re: [PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Roger Pau Monné
On Thu, Apr 30, 2020 at 11:38:14AM +0100, Andrew Cooper wrote: > On 30/04/2020 11:35, Roger Pau Monné wrote: > > On Thu, Apr 30, 2020 at 10:59:47AM +0100, Andrew Cooper wrote: > >> diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c > >> index c3f92ed231..014524486f 100644 > >> --- a/xen/arch/x86/

[PATCH] tools/xl: vcpu-pin: Skip global affinity when the hard affinity is not changed

2020-04-30 Thread Julien Grall
From: Julien Grall After XSA-273, it is not possible to modify the vCPU soft affinity using xl vcpu-pin without modifying the hard affinity. Instead the command will crash. 42sh> gdb /usr/local/sbin/xl (gdb) r vcpu-pin 0 0 - 10 [...] Program received signal SIGSEGV, Segmentation fault. [...] (gd

Re: [PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Jan Beulich
On 30.04.2020 11:59, Andrew Cooper wrote: > Fam19h is very similar to Fam17h in these regards. > > Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich Nevertheless a question: > --- a/xen/arch/x86/cpu/microcode/amd.c > +++ b/xen/arch/x86/cpu/microcode/amd.c > @@ -125,6 +125,7 @@ static bool

Re: [RFC] UEFI Secure Boot on Xen Hosts

2020-04-30 Thread Daniel Kiper
Adding Ard... On Thu, Apr 30, 2020 at 09:01:45AM +0200, Jan Beulich wrote: > On 30.04.2020 00:51, Bobby Eshleman wrote: > > Hey all, > > > > We're looking to develop UEFI Secure Boot support for grub-loaded Xen and > > ultimately for XCP-ng (I'm on the XCP-ng team at Vates). > > > > In addition to

[xen-unstable-smoke test] 149888: tolerable all pass - PUSHED

2020-04-30 Thread osstest service owner
flight 149888 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/149888/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 1

Re: [PATCH v2 3/3] device-dax: Add system ram (add_memory()) with MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Dave Hansen
On 4/30/20 3:29 AM, David Hildenbrand wrote: > Currently, when adding memory, we create entries in /sys/firmware/memmap/ > as "System RAM". This does not reflect the reality and will lead to > kexec-tools to add that memory to the fixed-up initial memmap for a > kexec kernel (loaded via kexec_load(

Re: [PATCH v2 4/5] common/domain: add a domain context record for shared_info...

2020-04-30 Thread Jan Beulich
On 28.04.2020 17:37, Paul Durrant wrote: >> -Original Message- >> From: Julien Grall >> Sent: 20 April 2020 18:35 >> To: Paul Durrant ; xen-devel@lists.xenproject.org >> Cc: Paul Durrant ; Ian Jackson >> ; Wei Liu ; >> Andrew Cooper ; George Dunlap >> ; Jan Beulich >> ; Stefano Stabellin

Re: [PATCH v2 4/5] common/domain: add a domain context record for shared_info...

2020-04-30 Thread Jan Beulich
On 07.04.2020 19:38, Paul Durrant wrote: > --- a/tools/misc/xen-domctx.c > +++ b/tools/misc/xen-domctx.c > @@ -59,6 +59,16 @@ static void dump_header(struct domain_save_descriptor > *desc) > off += desc->length; > } > > +static void dump_shared_info(struct domain_save_descriptor *desc) > +

Re: [PATCH v2 5/5] tools/libxc: make use of domain context SHARED_INFO record...

2020-04-30 Thread Jan Beulich
On 07.04.2020 19:38, Paul Durrant wrote: > ... in the save/restore code. > > This patch replaces direct mapping of the shared_info_frame (retrieved > using XEN_DOMCTL_getdomaininfo) with save/load of the domain context > SHARED_INFO record. > > No modifications are made to the definition of the m

Re: [RFC] UEFI Secure Boot on Xen Hosts

2020-04-30 Thread Ard Biesheuvel
On Thu, 30 Apr 2020 at 13:15, Daniel Kiper wrote: > > Adding Ard... > > On Thu, Apr 30, 2020 at 09:01:45AM +0200, Jan Beulich wrote: > > On 30.04.2020 00:51, Bobby Eshleman wrote: > > > Hey all, > > > > > > We're looking to develop UEFI Secure Boot support for grub-loaded Xen and > > > ultimately

Re: [PATCH 2/2] xen: credit2: limit the max number of CPUs in a runqueue

2020-04-30 Thread Dario Faggioli
On Thu, 2020-04-30 at 09:35 +0200, Jürgen Groß wrote: > On 29.04.20 19:36, Dario Faggioli wrote: > > > > Therefore, let's set a limit to the max number of CPUs that can > > share a > > Credit2 runqueue. The actual value is configurable (at boot time), > > the > > default being 16. If, for instance

Re: [PATCH v6 09/15] efi: use new page table APIs in copy_mapping

2020-04-30 Thread Jan Beulich
On 24.04.2020 16:09, Hongyan Xia wrote: > --- a/xen/common/efi/boot.c > +++ b/xen/common/efi/boot.c > @@ -1454,16 +1454,20 @@ static __init void copy_mapping(unsigned long mfn, > unsigned long end, > continue; > if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) ) > { > -

[PATCH 2/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
From: Julien Grall EXPERT mode is currently used to gate any options that are in technical preview or not security supported At the moment, the only way to select it is to use XEN_CONFIG_EXPERT=y on the make command line. However, if the user forget to add the option of one of the make command (

[PATCH 0/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
From: Julien Grall Hi all, This small series is meant to make easier to experiment when using Xen. See patch #2 for more details. Cheers, Julien Grall (2): xen/Kconfig: define EXPERT a bool rather than a string xen: Allow EXPERT mode to be selected from the menuconfig directly xen/Kconfi

[PATCH 1/2] xen/Kconfig: define EXPERT a bool rather than a string

2020-04-30 Thread Julien Grall
From: Julien Grall Since commit f80fe2b34f08 "xen: Update Kconfig to Linux v5.4" EXPERT can only have two values (enabled or disabled). So switch from a string to a bool. Take the opportunity to replace all "EXPERT = y" to "EXPERT". Signed-off-by: Julien Grall --- xen/Kconfig

Re: [PATCH 2/2] xen: credit2: limit the max number of CPUs in a runqueue

2020-04-30 Thread Jürgen Groß
On 30.04.20 14:28, Dario Faggioli wrote: On Thu, 2020-04-30 at 09:35 +0200, Jürgen Groß wrote: On 29.04.20 19:36, Dario Faggioli wrote: Therefore, let's set a limit to the max number of CPUs that can share a Credit2 runqueue. The actual value is configurable (at boot time), the default being 1

Re: [PATCH 0/12] direct-map DomUs

2020-04-30 Thread Julien Grall
Hi Stefano, On 29/04/2020 21:16, Stefano Stabellini wrote: On Thu, 16 Apr 2020, Julien Grall wrote: Stefano Stabellini (12): xen: introduce xen_dom_flags xen/arm: introduce arch_xen_dom_flags and direct_map xen/arm: introduce 1:1 mapping for domUs xen: split allo

Re: [PATCH v6 10/15] efi: switch to new APIs in EFI code

2020-04-30 Thread Jan Beulich
On 24.04.2020 16:09, Hongyan Xia wrote: > --- a/xen/arch/x86/efi/runtime.h > +++ b/xen/arch/x86/efi/runtime.h > @@ -1,12 +1,18 @@ > +#include > +#include > #include > #include > > #ifndef COMPAT > -l4_pgentry_t *__read_mostly efi_l4_pgtable; > +mfn_t __read_mostly efi_l4_mfn = INVALID_MFN_

Re: [PATCH 12/12] xen/arm: call iomem_permit_access for passthrough devices

2020-04-30 Thread Julien Grall
Hi Stefano, On 29/04/2020 21:47, Stefano Stabellini wrote: On Wed, 15 Apr 2020, Julien Grall wrote: But doesn't it make sense to give domU permission if it is going to get the memory mapped? But admittedly I can't think of something that would break because of the lack of the iomem_permit_acces

Re: [PATCH] x86/EFI: correct section offsets in mkreloc diagnostics

2020-04-30 Thread Andrew Cooper
On 30/04/2020 11:24, Jan Beulich wrote: > These are more helpful if they point at the address where the relocated > value starts, rather than at the specific byte of the difference. > > Signed-off-by: Jan Beulich Acked-by: Andrew Cooper

Re: [PATCH 11/12] xen/arm: if xen_force don't try to setup the IOMMU

2020-04-30 Thread Julien Grall
Hi Stefano, On 29/04/2020 22:55, Stefano Stabellini wrote: On Wed, 15 Apr 2020, Julien Grall wrote: Hi Stefano, On 15/04/2020 02:02, Stefano Stabellini wrote: If xen_force (which means xen,force-assign-without-iommu was requested) don't try to add the device to the IOMMU. Return early instead

Re: [PATCH 2/2] xen: credit2: limit the max number of CPUs in a runqueue

2020-04-30 Thread Dario Faggioli
On Thu, 2020-04-30 at 14:52 +0200, Jürgen Groß wrote: > On 30.04.20 14:28, Dario Faggioli wrote: > > On Thu, 2020-04-30 at 09:35 +0200, Jürgen Groß wrote: > > > > > With that, when the user removes 4 CPUs, we will have the 6 vs 10 > > situation. But we would make sure that, when she adds them back

[libvirt test] 149886: regressions - FAIL

2020-04-30 Thread osstest service owner
flight 149886 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/149886/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-amd64-libvirt 6 libvirt-buildfail REGR. vs. 146182 build-i386-libvirt

Re: [PATCH 0/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
Hmmm I have just realized I forgot to CC the REST. I will resend it. On 30/04/2020 13:43, Julien Grall wrote: From: Julien Grall Hi all, This small series is meant to make easier to experiment when using Xen. See patch #2 for more details. Cheers, Julien Grall (2): xen/Kconfig: define E

[PATCH RESEND 1/2] xen/Kconfig: define EXPERT a bool rather than a string

2020-04-30 Thread Julien Grall
From: Julien Grall Since commit f80fe2b34f08 "xen: Update Kconfig to Linux v5.4" EXPERT can only have two values (enabled or disabled). So switch from a string to a bool. Take the opportunity to replace all "EXPERT = y" to "EXPERT". Signed-off-by: Julien Grall --- xen/Kconfig

[PATCH RESEND 0/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
From: Julien Grall Hi all, This is a resend as I forgot to CC the maintainers. This small series is meant to make easier to experiment when using Xen. See patch #2 for more details. Cheers, Julien Grall (2): xen/Kconfig: define EXPERT a bool rather than a string xen: Allow EXPERT mode to

[PATCH RESEND 2/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
From: Julien Grall EXPERT mode is currently used to gate any options that are in technical preview or not security supported At the moment, the only way to select it is to use XEN_CONFIG_EXPERT=y on the make command line. However, if the user forget to add the option of one of the make command (

Re: [PATCH 1/2] xen/Kconfig: define EXPERT a bool rather than a string

2020-04-30 Thread Jan Beulich
On 30.04.2020 14:43, Julien Grall wrote: > From: Julien Grall > > Since commit f80fe2b34f08 "xen: Update Kconfig to Linux v5.4" EXPERT > can only have two values (enabled or disabled). So switch from a string > to a bool. > > Take the opportunity to replace all "EXPERT = y" to "EXPERT". > > Sig

[qemu-mainline test] 149885: tolerable FAIL - PUSHED

2020-04-30 Thread osstest service owner
flight 149885 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/149885/ Failures :-/ but no regressions. Regressions which are regarded as allowable (not blocking): test-amd64-amd64-xl-rtds 16 guest-localmigrate fail REGR. vs. 149877 test-armhf-armhf-xl-rtds

Re: [PATCH] x86/hvm: allow for more fine grained assisted flush

2020-04-30 Thread Roger Pau Monné
On Thu, Apr 30, 2020 at 11:17:25AM +0200, Roger Pau Monne wrote: > diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c > index 814b7020d8..1d41b6e2ea 100644 > --- a/xen/arch/x86/hvm/hvm.c > +++ b/xen/arch/x86/hvm/hvm.c > @@ -4011,17 +4011,42 @@ static void hvm_s3_resume(struct domain *d) >

Re: [PATCH RESEND 2/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Jan Beulich
On 30.04.2020 16:25, Julien Grall wrote: > EXPERT mode is currently used to gate any options that are in technical > preview or not security supported At the moment, the only way to select > it is to use XEN_CONFIG_EXPERT=y on the make command line. > > However, if the user forget to add the optio

Re: [PATCH 05/12] xen: introduce reserve_heap_pages

2020-04-30 Thread Julien Grall
Hi, On 29/04/2020 23:46, Stefano Stabellini wrote: On Fri, 17 Apr 2020, Jan Beulich wrote: On 15.04.2020 03:02, Stefano Stabellini wrote: Introduce a function named reserve_heap_pages (similar to alloc_heap_pages) that allocates a requested memory range. Call __alloc_heap_pages for the impleme

Re: [PATCH v6 11/15] x86/smpboot: clone_mapping should have one exit path

2020-04-30 Thread Jan Beulich
On 24.04.2020 16:09, Hongyan Xia wrote: > From: Wei Liu Like for patches 1 and 2 in this series, and as iirc mentioned already long ago for those, "should" or alike are misleading here: It's not a mistake that they don't, i.e. this is not a bug fix. You _want_ these functions to have a single (ma

Re: Cpu on/offlining crash with core scheduling

2020-04-30 Thread Jürgen Groß
On 29.04.20 11:15, Sergey Dyasli wrote: On 29/04/2020 09:09, Jürgen Groß wrote: On 27.04.20 15:49, Sergey Dyasli wrote: Hi Juergen, When I'm testing vcpu pinning with something like:   # xl vcpu-pin 0 0 2   # xen-hptool cpu-offline 3   (offline / online CPUs {2,3} if the above

[PATCH 1/3] xen/sched: allow rcu work to happen when syncing cpus in core scheduling

2020-04-30 Thread Juergen Gross
With RCU barriers moved from tasklets to normal RCU processing cpu offlining in core scheduling might deadlock due to cpu synchronization required by RCU processing and core scheduling concurrently. Fix that by bailing out from core scheduling synchronization in case of pending RCU work. Additiona

[PATCH 3/3] xen/cpupool: fix removing cpu from a cpupool

2020-04-30 Thread Juergen Gross
Commit cb563d7665f2 ("xen/sched: support core scheduling for moving cpus to/from cpupools") introduced a regression when trying to remove an offline cpu from a cpupool, as the system would crash in this situation. Fix that by testing the cpu to be online. Fixes: cb563d7665f2 ("xen/sched: support

Re: [PATCH v6 12/15] x86/smpboot: switch pl*e to use new APIs in clone_mapping

2020-04-30 Thread Jan Beulich
On 24.04.2020 16:09, Hongyan Xia wrote: > From: Wei Liu Nit: Why the emphasis on pl*e in the title? Is there anything left unconverted in the function? IOW how about "switch clone_mapping() to new page table APIs"? > --- a/xen/arch/x86/smpboot.c > +++ b/xen/arch/x86/smpboot.c > @@ -672,9 +672,9

[PATCH 0/3] xen: Fix some bugs in scheduling

2020-04-30 Thread Juergen Gross
Some bugs I found when trying to find a problem with cpu-on/offlining in core scheduling mode. Patches 1 and 3 are fixing observed problems, while patch 2 is more of a theoretical issue. Juergen Gross (3): xen/sched: allow rcu work to happen when syncing cpus in core scheduling xen/sched:

[PATCH 2/3] xen/sched: fix theoretical races accessing vcpu->dirty_cpu

2020-04-30 Thread Juergen Gross
The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases. There are some instances where accesses are not atomically done, and even wors

Re: [PATCH 2/3] xen/sched: fix theoretical races accessing vcpu->dirty_cpu

2020-04-30 Thread Jürgen Groß
On 30.04.20 17:15, Juergen Gross wrote: The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases. There are some instances where acces

[PATCH v1.1 2/3] xen/sched: fix theoretical races accessing vcpu->dirty_cpu

2020-04-30 Thread Juergen Gross
The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases. There are some instances where accesses are not atomically done, and even wors

Re: [PATCH v2 3/3] device-dax: Add system ram (add_memory()) with MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread David Hildenbrand
On 30.04.20 13:23, Dave Hansen wrote: > On 4/30/20 3:29 AM, David Hildenbrand wrote: >> Currently, when adding memory, we create entries in /sys/firmware/memmap/ >> as "System RAM". This does not reflect the reality and will lead to >> kexec-tools to add that memory to the fixed-up initial memmap f

Re: [PATCH RESEND 2/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
Hi Jan, On 30/04/2020 15:50, Jan Beulich wrote: On 30.04.2020 16:25, Julien Grall wrote: EXPERT mode is currently used to gate any options that are in technical preview or not security supported At the moment, the only way to select it is to use XEN_CONFIG_EXPERT=y on the make command line. Ho

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Eric W. Biederman
David Hildenbrand writes: > Some devices/drivers that add memory via add_memory() and friends (e.g., > dax/kmem, but also virtio-mem in the future) don't want to create entries > in /sys/firmware/memmap/ - primarily to hinder kexec from adding this > memory to the boot memmap of the kexec kernel.

Re: [PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Andrew Cooper
On 30/04/2020 12:09, Jan Beulich wrote: > On 30.04.2020 11:59, Andrew Cooper wrote: >> Fam19h is very similar to Fam17h in these regards. >> >> Signed-off-by: Andrew Cooper > Reviewed-by: Jan Beulich Thanks. > > Nevertheless a question: > >> --- a/xen/arch/x86/cpu/microcode/amd.c >> +++ b/xen/a

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread David Hildenbrand
On 30.04.20 17:38, Eric W. Biederman wrote: > David Hildenbrand writes: > >> Some devices/drivers that add memory via add_memory() and friends (e.g., >> dax/kmem, but also virtio-mem in the future) don't want to create entries >> in /sys/firmware/memmap/ - primarily to hinder kexec from adding th

[ovmf test] 149887: all pass - PUSHED

2020-04-30 Thread osstest service owner
flight 149887 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/149887/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf f07fb43b2d3f92a15d6992205b72ba5df0e74fe2 baseline version: ovmf b2034179e8feed9c7d3bc

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Dave Hansen
On 4/30/20 8:52 AM, David Hildenbrand wrote: >> Justifying behavior by documentation that does not consider memory >> hotplug is bad thinking. > Are you maybe confusing this patch series with the arm64 approach? This > is not about ordinary hotplugged DIMMs. > > I'd love to get Dan's, Dave's and M

Re: [PATCH] x86/hap: be more selective with assisted TLB flush

2020-04-30 Thread Andrew Cooper
On 30/04/2020 09:33, Jan Beulich wrote: > On 30.04.2020 10:28, Roger Pau Monné wrote: >> On Thu, Apr 30, 2020 at 09:20:58AM +0200, Jan Beulich wrote: >>> On 29.04.2020 19:36, Roger Pau Monne wrote: When doing an assisted flush on HAP the purpose of the on_selected_cpus is just to trigger

Re: [PATCH 05/12] xen: introduce reserve_heap_pages

2020-04-30 Thread Stefano Stabellini
On Thu, 30 Apr 2020, Jan Beulich wrote: > On 30.04.2020 00:46, Stefano Stabellini wrote: > > On Fri, 17 Apr 2020, Jan Beulich wrote: > >> On 15.04.2020 03:02, Stefano Stabellini wrote: > >>> Introduce a function named reserve_heap_pages (similar to > >>> alloc_heap_pages) that allocates a requested

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Eric W. Biederman
David Hildenbrand writes: > On 30.04.20 17:38, Eric W. Biederman wrote: >> David Hildenbrand writes: >> >>> Some devices/drivers that add memory via add_memory() and friends (e.g., >>> dax/kmem, but also virtio-mem in the future) don't want to create entries >>> in /sys/firmware/memmap/ - prima

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread David Hildenbrand
On 30.04.20 18:33, Eric W. Biederman wrote: > David Hildenbrand writes: > >> On 30.04.20 17:38, Eric W. Biederman wrote: >>> David Hildenbrand writes: >>> Some devices/drivers that add memory via add_memory() and friends (e.g., dax/kmem, but also virtio-mem in the future) don't want to

Re: [PATCH 05/12] xen: introduce reserve_heap_pages

2020-04-30 Thread Stefano Stabellini
On Thu, 30 Apr 2020, Julien Grall wrote: > > > > +pg = maddr_to_page(start); > > > > +node = phys_to_nid(start); > > > > +zone = page_to_zone(pg); > > > > +page_list_del(pg, &heap(node, zone, order)); > > > > + > > > > +__alloc_heap_pages(pg, order, memflags, d); > > > > > > I

Re: [PATCH] x86/hap: be more selective with assisted TLB flush

2020-04-30 Thread Roger Pau Monné
On Thu, Apr 30, 2020 at 05:19:19PM +0100, Andrew Cooper wrote: > On 30/04/2020 09:33, Jan Beulich wrote: > > On 30.04.2020 10:28, Roger Pau Monné wrote: > >> On Thu, Apr 30, 2020 at 09:20:58AM +0200, Jan Beulich wrote: > >>> On 29.04.2020 19:36, Roger Pau Monne wrote: > When doing an assisted

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Eric W. Biederman
David Hildenbrand writes: > On 30.04.20 18:33, Eric W. Biederman wrote: >> David Hildenbrand writes: >> >>> On 30.04.20 17:38, Eric W. Biederman wrote: David Hildenbrand writes: > Some devices/drivers that add memory via add_memory() and friends (e.g., > dax/kmem, but also vi

[PATCH 0/3] Automation: improve openSUSE containers + podman

2020-04-30 Thread Dario Faggioli
Hello, This short series contains some improvements for building Xen in openSUSE containers. In fact, the build dependencies inside the Tumbleweed container are updated and more handy helpers are added, in containerize, for referring to both Leap and Tumbleweed containers. In addition to that, in

[PATCH 1/3] automation: update openSUSE Tumbleweed building dependencies

2020-04-30 Thread Dario Faggioli
We need python3 (and the respective -devel package), these days. Signed-off-by: Dario Faggioli --- Cc: Doug Goldstein Cc: Andrew Cooper --- This patch was submitted already, but not as part of this series. Anyway, changes from v1: * add python3 instead of replacing python2 with it. --- I think

[PATCH 3/3] automation: implement (rootless) podman support in containerize

2020-04-30 Thread Dario Faggioli
Right now only docker is supported, when using the containerize script for building inside containers. Enable podman as well. Note that podman can be use in rootless mode too, but for that to work the files /etc/subuid and /etc/subgid must be properly configured. For instance: dario@localhost> c

[linux-linus test] 149882: tolerable FAIL - PUSHED

2020-04-30 Thread osstest service owner
flight 149882 linux-linus real [real] http://logs.test-lab.xenproject.org/osstest/logs/149882/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10 fail blocked in 149868 test-amd64-amd64-xl-qemut-win7-amd64

[PATCH 2/3] automation: openSUSE distro names helpers for containerize.

2020-04-30 Thread Dario Faggioli
Signed-off-by: Dario Faggioli --- Cc: Doug Goldstein --- automation/scripts/containerize |2 ++ 1 file changed, 2 insertions(+) diff --git a/automation/scripts/containerize b/automation/scripts/containerize index fbc4bc22d6..eb805bf96c 100755 --- a/automation/scripts/containerize +++ b/auto

Re: [PATCH 05/12] xen: introduce reserve_heap_pages

2020-04-30 Thread Julien Grall
Hi, On 30/04/2020 18:00, Stefano Stabellini wrote: On Thu, 30 Apr 2020, Julien Grall wrote: +pg = maddr_to_page(start); +node = phys_to_nid(start); +zone = page_to_zone(pg); +page_list_del(pg, &heap(node, zone, order)); + +__alloc_heap_pages(pg, order, memflags, d); I agre

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread David Hildenbrand
>>> If the class of memory is different then please by all means let's mark >>> it differently in struct resource so everyone knows it is different. >>> But that difference needs to be more than hotplug. >>> >>> That difference needs to be the hypervisor loaned us memory and might >>> take it back

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Dan Williams
On Thu, Apr 30, 2020 at 11:44 AM David Hildenbrand wrote: > > >>> If the class of memory is different then please by all means let's mark > >>> it differently in struct resource so everyone knows it is different. > >>> But that difference needs to be more than hotplug. > >>> > >>> That difference

[PATCH 09/16] x86: lift mapcache variable to the arch level

2020-04-30 Thread Hongyan Xia
From: Wei Liu It is going to be needed by HVM and idle domain as well, because without the direct map, both need a mapcache to map pages. This only lifts the mapcache variable up. Whether we populate the mapcache for a domain is unchanged in this patch. Signed-off-by: Wei Liu Signed-off-by: We

[PATCH 05/16] x86: map/unmap pages in restore_all_guests.

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia Before, it assumed the pv cr3 could be accessed via a direct map. This is no longer true. Note that we do not map and unmap root_pgt for now since it is still a xenheap page. Signed-off-by: Hongyan Xia --- xen/arch/x86/x86_64/entry.S | 27 ++- 1 file

[PATCH 00/16] Remove the direct map

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia This series depends on Xen page table domheap conversion: https://lists.xenproject.org/archives/html/xen-devel/2020-04/msg01374.html. After breaking the reliance on the direct map to manipulate Xen page tables, we can now finally remove the direct map altogether. This series:

[PATCH 07/16] x86/pv: rewrite how building PV dom0 handles domheap mappings

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia Building a PV dom0 is allocating from the domheap but uses it like the xenheap. This is clearly wrong. Fix. Signed-off-by: Hongyan Xia --- xen/arch/x86/pv/dom0_build.c | 58 ++-- 1 file changed, 43 insertions(+), 15 deletions(-) diff --git a/x

[PATCH 06/16] x86/pv: domheap pages should be mapped while relocating initrd

2020-04-30 Thread Hongyan Xia
From: Wei Liu Xen shouldn't use domheap page as if they were xenheap pages. Map and unmap pages accordingly. Signed-off-by: Wei Liu Signed-off-by: Wei Wang --- xen/arch/x86/pv/dom0_build.c | 17 +++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/pv/d

[PATCH 02/16] acpi: vmap pages in acpi_os_alloc_memory

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia Also, introduce a wrapper around vmap that maps a contiguous range for boot allocations. Signed-off-by: Hongyan Xia --- xen/drivers/acpi/osl.c | 9 - xen/include/xen/vmap.h | 5 + 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/xen/drivers/acpi/osl.

[PATCH 04/16] x86/srat: vmap the pages for acpi_slit

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia This avoids the assumption that boot pages are in the direct map. Signed-off-by: Hongyan Xia --- xen/arch/x86/srat.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index 506a56d66b..9a84c6c8a8 100644 --- a/xen/a

[PATCH 01/16] x86/setup: move vm_init() before acpi calls

2020-04-30 Thread Hongyan Xia
From: Wei Liu After the direct map removal, pages from the boot allocator are not mapped at all in the direct map. Although we have map_domain_page, they are ephemeral and are less helpful for mappings that are more than a page, so we want a mechanism to globally map a range of pages, which is wh

[PATCH 08/16] x86: add Persistent Map (PMAP) infrastructure

2020-04-30 Thread Hongyan Xia
From: Wei Liu The basic idea is like Persistent Kernel Map (PKMAP) in linux. We pre-populate all the relevant page tables before system is fully set up. It is needed to bootstrap map domain page infrastructure -- we need some way to map pages to set up the mapcache without a direct map. This in

[PATCH 03/16] x86/numa: vmap the pages for memnodemap

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia This avoids the assumption that there is a direct map and boot pages fall inside the direct map. Clean up the variables so that mfn actually stores a type-safe mfn. Signed-off-by: Hongyan Xia --- xen/arch/x86/numa.c | 8 1 file changed, 4 insertions(+), 4 deletions(

[PATCH 12/16] x86/domain_page: remove the fast paths when mfn is not in the directmap

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia When mfn is not in direct map, never use mfn_to_virt for any mappings. We replace mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) with arch_mfn_in_direct_map(mfn) because these two are equivalent. The extra comparison in arch_mfn_in_direct_map() looks different but becaus

[PATCH 16/16] x86/setup: do not create valid mappings when directmap=no

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia Create empty mappings in the second e820 pass. Also, destroy existing direct map mappings created in the first pass. To make xenheap pages visible in guests, it is necessary to create empty L3 tables in the direct map even when directmap=no, since guest cr3s copy idle domain's

[PATCH 11/16] x86: add a boot option to enable and disable the direct map

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia Also add a helper function to retrieve it. Change arch_mfn_in_direct_map to check this option before returning. This is added as a boot command line option, not a Kconfig. We do not produce different builds for EC2 so this is not introduced as a compile-time configuration. Sig

  1   2   >