Question...

2020-04-30 Thread Samuel P. Felton - GPT LLC
So, I'm trying to get a Xen Dom0 and DomU, both running Ubuntu 20.04 LTS up on our brand-new Gigabyte ThunderX2 ARM64 box. I can get Ubuntu up and running, but after installing the Xen bits, it dies after the UEFI layer launches GRUB. I haven't been able to get any logfiles because it doesn't ge

Troubles running Xen on Raspberry Pi 4 with 5.6.1 DomU

2020-04-30 Thread Roman Shaposhnik
Hi! I'm trying to run Xen on Raspberry Pi 4 with 5.6.1 stock, upstream kernel. The kernel itself works perfectly well on the board. When I try booting it as Dom0 under Xen, it goes into a stacktrace (attached). Looking at what nice folks over at Dornerworks have previously done to make RPi kernel

Re: [PATCH 09/12] xen/arm: if is_domain_direct_mapped use native addresses for GICv3

2020-04-30 Thread Stefano Stabellini
On Wed, 15 Apr 2020, Julien Grall wrote: > > diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c > > index 4e60ba15cc..4cf430f865 100644 > > --- a/xen/arch/arm/vgic-v3.c > > +++ b/xen/arch/arm/vgic-v3.c > > @@ -1677,13 +1677,25 @@ static int vgic_v3_domain_init(struct domain *d) > > > I

Re: [PATCH 03/12] xen/arm: introduce 1:1 mapping for domUs

2020-04-30 Thread Stefano Stabellini
On Wed, 15 Apr 2020, Julien Grall wrote: > On 15/04/2020 02:02, Stefano Stabellini wrote: > > In some cases it is desirable to map domU memory 1:1 (guest physical == > > physical.) For instance, because we want to assign a device to the domU > > but the IOMMU is not present or cannot be used. In th

Re: [PATCH 10/12] xen/arm: if is_domain_direct_mapped use native UART address for vPL011

2020-04-30 Thread Stefano Stabellini
On Wed, 15 Apr 2020, Julien Grall wrote: > Hi Stefano, > > On 15/04/2020 02:02, Stefano Stabellini wrote: > > We always use a fix address to map the vPL011 to domains. The address > > could be a problem for domains that are directly mapped. > > > > Instead, for domains that are directly mapped, r

Re: [PATCH 11/12] xen/arm: if xen_force don't try to setup the IOMMU

2020-04-30 Thread Stefano Stabellini
On Thu, 30 Apr 2020, Julien Grall wrote: > Hi Stefano, > > On 29/04/2020 22:55, Stefano Stabellini wrote: > > On Wed, 15 Apr 2020, Julien Grall wrote: > > > Hi Stefano, > > > > > > On 15/04/2020 02:02, Stefano Stabellini wrote: > > > > If xen_force (which means xen,force-assign-without-iommu was

Re: [PATCH 08/12] xen/arm: if is_domain_direct_mapped use native addresses for GICv2

2020-04-30 Thread Stefano Stabellini
On Wed, 15 Apr 2020, Julien Grall wrote: > On 15/04/2020 02:02, Stefano Stabellini wrote: > > Today we use native addresses to map the GICv2 for Dom0 and fixed > > addresses for DomUs. > > > > This patch changes the behavior so that native addresses are used for > > any domain that is_domain_direc

[qemu-mainline test] 149890: tolerable FAIL - PUSHED

2020-04-30 Thread osstest service owner
flight 149890 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/149890/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-rtds 16 guest-localmigrate fail like 149885 test-amd64-amd64-xl-qemuu-win7-amd6

Re: Xen Compilation Error on Ubuntu 20.04

2020-04-30 Thread Pry Mar
Hello Ayush Dosaj, https://bugs.launchpad.net/ubuntu/+source/xen/+bug/1851091 see the bug report above ^. As soon as Ubuntu 19.10 went to LZ4 kernel compression this bug appeared. As Stefan Bader reports in his testing, the hypervisor not only crashes, but resets the machine. The same trouble c

[xen-unstable test] 149889: tolerable FAIL - PUSHED

2020-04-30 Thread osstest service owner
flight 149889 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/149889/ Failures :-/ but no regressions. Regressions which are regarded as allowable (not blocking): test-amd64-amd64-xl-rtds 16 guest-localmigrate fail REGR. vs. 149881 Tests which did not succeed

Re: [RFC] UEFI Secure Boot on Xen Hosts

2020-04-30 Thread Christopher Clark
On Thu, Apr 30, 2020 at 3:28 PM Marek Marczykowski-Górecki wrote: > > On Wed, Apr 29, 2020 at 05:51:08PM -0500, Bobby Eshleman wrote: > > # Option #3: Lean on Grub2's LoadFile2() Verification > > > > Grub2 will provide a LoadFile2() method to subsequent programs that supports > > signature verific

[ovmf test] 149891: all pass - PUSHED

2020-04-30 Thread osstest service owner
flight 149891 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/149891/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf e54310451f1ac2ce4ccb90a110f45bb9b4f3ccd6 baseline version: ovmf f07fb43b2d3f92a15d699

Re: [RFC] UEFI Secure Boot on Xen Hosts

2020-04-30 Thread Marek Marczykowski-Górecki
On Wed, Apr 29, 2020 at 05:51:08PM -0500, Bobby Eshleman wrote: > # Option #3: Lean on Grub2's LoadFile2() Verification > > Grub2 will provide a LoadFile2() method to subsequent programs that supports > signature verification of arbitrary files. Linux is moving in the > direction of using LoadFil

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Andrew Morton
On Thu, 30 Apr 2020 20:43:39 +0200 David Hildenbrand wrote: > > > > Why does the firmware map support hotplug entries? > > I assume: > > The firmware memmap was added primarily for x86-64 kexec (and still, is > mostly used on x86-64 only IIRC). There, we had ACPI hotplug. When DIMMs > get hotp

[PATCH 3/3] golang/xenlight: add necessary module/package documentation

2020-04-30 Thread Nick Rosbrook
Add a README and package comment giving a brief overview of the package. These also help pkg.go.dev generate better documentation. Also, add a copy of Xen's license to tools/golang/xenlight. This is required for the package to be shown on pkg.go.dev and added to the default module proxy, proxy.gol

[PATCH 2/3] golang/xenlight: init xenlight go module

2020-04-30 Thread Nick Rosbrook
Initialize the xenlight Go module using the xenbits git-http URL, xenbits.xen.org/git-http/xen.git/tools/golang/xenlight, and update the XEN_GOCODE_URL variable in tools/Rules.mk accordingly. Signed-off-by: Nick Rosbrook --- tools/Rules.mk | 2 +- tools/golang/xenlight/go.mod | 1 +

[PATCH 0/3] initialize xenlight go module

2020-04-30 Thread Nick Rosbrook
These patches setup an initial Go module for the xenlight package. The go code is tracked again, since the module is defined in xen.git as xenbits.xen.org/git-http/xen.git/tools/golang/xenlight. The final patch adds a README and LICENSE to tools/golang/xenlight so that the package will show up prop

Re: Xen Compilation Error on Ubuntu 20.04

2020-04-30 Thread Jason Andryuk
On Thu, Apr 30, 2020 at 4:18 AM Ayush Dosaj wrote: > > Hi Andrew, Xen Development team. > > I compiled and installed Xen by appending -fcf-protection=none to CFLAGS on > Ubuntu 20.04 but it still crashes on startup. Ayush, try the patch below. EMBEDDED_EXTRA_CFLAGS ensures it is set down in xen

[PATCH 14/16] x86/setup: leave early boot slightly earlier

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia When we do not have a direct map, memory for metadata of heap nodes in init_node_heap() is allocated from xenheap, which needs to be mapped and unmapped on demand. However, we cannot just take memory from the boot allocator to create the PTEs while we are passing memory to the h

[PATCH 15/16] x86/setup: vmap heap nodes when they are outside the direct map

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia When we do not have a direct map, arch_mfn_in_direct_map() will always return false, thus init_node_heap() will allocate xenheap pages from an existing node for the metadata of a new node. This means that the metadata of a new node is in a different node, slowing down heap alloc

[PATCH 10/16] x86/mapcache: initialise the mapcache for the idle domain

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia In order to use the mapcache in the idle domain, we also have to populate its page tables in the PERDOMAIN region, and we need to move mapcache_domain_init() earlier in arch_domain_create(). Note, commit 'x86: lift mapcache variable to the arch level' has initialised the mapcac

[PATCH 13/16] xen/page_alloc: add a path for xenheap when there is no direct map

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia When there is not an always-mapped direct map, xenheap allocations need to be mapped and unmapped on-demand. Signed-off-by: Hongyan Xia --- xen/common/page_alloc.c | 45 ++--- 1 file changed, 42 insertions(+), 3 deletions(-) diff --git a/x

[PATCH 16/16] x86/setup: do not create valid mappings when directmap=no

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia Create empty mappings in the second e820 pass. Also, destroy existing direct map mappings created in the first pass. To make xenheap pages visible in guests, it is necessary to create empty L3 tables in the direct map even when directmap=no, since guest cr3s copy idle domain's

[PATCH 11/16] x86: add a boot option to enable and disable the direct map

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia Also add a helper function to retrieve it. Change arch_mfn_in_direct_map to check this option before returning. This is added as a boot command line option, not a Kconfig. We do not produce different builds for EC2 so this is not introduced as a compile-time configuration. Sig

[PATCH 12/16] x86/domain_page: remove the fast paths when mfn is not in the directmap

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia When mfn is not in direct map, never use mfn_to_virt for any mappings. We replace mfn_x(mfn) <= PFN_DOWN(__pa(HYPERVISOR_VIRT_END - 1)) with arch_mfn_in_direct_map(mfn) because these two are equivalent. The extra comparison in arch_mfn_in_direct_map() looks different but becaus

[PATCH 04/16] x86/srat: vmap the pages for acpi_slit

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia This avoids the assumption that boot pages are in the direct map. Signed-off-by: Hongyan Xia --- xen/arch/x86/srat.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index 506a56d66b..9a84c6c8a8 100644 --- a/xen/a

[PATCH 08/16] x86: add Persistent Map (PMAP) infrastructure

2020-04-30 Thread Hongyan Xia
From: Wei Liu The basic idea is like Persistent Kernel Map (PKMAP) in linux. We pre-populate all the relevant page tables before system is fully set up. It is needed to bootstrap map domain page infrastructure -- we need some way to map pages to set up the mapcache without a direct map. This in

[PATCH 06/16] x86/pv: domheap pages should be mapped while relocating initrd

2020-04-30 Thread Hongyan Xia
From: Wei Liu Xen shouldn't use domheap page as if they were xenheap pages. Map and unmap pages accordingly. Signed-off-by: Wei Liu Signed-off-by: Wei Wang --- xen/arch/x86/pv/dom0_build.c | 17 +++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/pv/d

[PATCH 02/16] acpi: vmap pages in acpi_os_alloc_memory

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia Also, introduce a wrapper around vmap that maps a contiguous range for boot allocations. Signed-off-by: Hongyan Xia --- xen/drivers/acpi/osl.c | 9 - xen/include/xen/vmap.h | 5 + 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/xen/drivers/acpi/osl.

[PATCH 01/16] x86/setup: move vm_init() before acpi calls

2020-04-30 Thread Hongyan Xia
From: Wei Liu After the direct map removal, pages from the boot allocator are not mapped at all in the direct map. Although we have map_domain_page, they are ephemeral and are less helpful for mappings that are more than a page, so we want a mechanism to globally map a range of pages, which is wh

[PATCH 03/16] x86/numa: vmap the pages for memnodemap

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia This avoids the assumption that there is a direct map and boot pages fall inside the direct map. Clean up the variables so that mfn actually stores a type-safe mfn. Signed-off-by: Hongyan Xia --- xen/arch/x86/numa.c | 8 1 file changed, 4 insertions(+), 4 deletions(

[PATCH 09/16] x86: lift mapcache variable to the arch level

2020-04-30 Thread Hongyan Xia
From: Wei Liu It is going to be needed by HVM and idle domain as well, because without the direct map, both need a mapcache to map pages. This only lifts the mapcache variable up. Whether we populate the mapcache for a domain is unchanged in this patch. Signed-off-by: Wei Liu Signed-off-by: We

[PATCH 05/16] x86: map/unmap pages in restore_all_guests.

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia Before, it assumed the pv cr3 could be accessed via a direct map. This is no longer true. Note that we do not map and unmap root_pgt for now since it is still a xenheap page. Signed-off-by: Hongyan Xia --- xen/arch/x86/x86_64/entry.S | 27 ++- 1 file

[PATCH 00/16] Remove the direct map

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia This series depends on Xen page table domheap conversion: https://lists.xenproject.org/archives/html/xen-devel/2020-04/msg01374.html. After breaking the reliance on the direct map to manipulate Xen page tables, we can now finally remove the direct map altogether. This series:

[PATCH 07/16] x86/pv: rewrite how building PV dom0 handles domheap mappings

2020-04-30 Thread Hongyan Xia
From: Hongyan Xia Building a PV dom0 is allocating from the domheap but uses it like the xenheap. This is clearly wrong. Fix. Signed-off-by: Hongyan Xia --- xen/arch/x86/pv/dom0_build.c | 58 ++-- 1 file changed, 43 insertions(+), 15 deletions(-) diff --git a/x

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Dan Williams
On Thu, Apr 30, 2020 at 11:44 AM David Hildenbrand wrote: > > >>> If the class of memory is different then please by all means let's mark > >>> it differently in struct resource so everyone knows it is different. > >>> But that difference needs to be more than hotplug. > >>> > >>> That difference

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread David Hildenbrand
>>> If the class of memory is different then please by all means let's mark >>> it differently in struct resource so everyone knows it is different. >>> But that difference needs to be more than hotplug. >>> >>> That difference needs to be the hypervisor loaned us memory and might >>> take it back

[PATCH 2/3] automation: openSUSE distro names helpers for containerize.

2020-04-30 Thread Dario Faggioli
Signed-off-by: Dario Faggioli --- Cc: Doug Goldstein --- automation/scripts/containerize |2 ++ 1 file changed, 2 insertions(+) diff --git a/automation/scripts/containerize b/automation/scripts/containerize index fbc4bc22d6..eb805bf96c 100755 --- a/automation/scripts/containerize +++ b/auto

Re: [PATCH 05/12] xen: introduce reserve_heap_pages

2020-04-30 Thread Julien Grall
Hi, On 30/04/2020 18:00, Stefano Stabellini wrote: On Thu, 30 Apr 2020, Julien Grall wrote: +pg = maddr_to_page(start); +node = phys_to_nid(start); +zone = page_to_zone(pg); +page_list_del(pg, &heap(node, zone, order)); + +__alloc_heap_pages(pg, order, memflags, d); I agre

[linux-linus test] 149882: tolerable FAIL - PUSHED

2020-04-30 Thread osstest service owner
flight 149882 linux-linus real [real] http://logs.test-lab.xenproject.org/osstest/logs/149882/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10 fail blocked in 149868 test-amd64-amd64-xl-qemut-win7-amd64

[PATCH 1/3] automation: update openSUSE Tumbleweed building dependencies

2020-04-30 Thread Dario Faggioli
We need python3 (and the respective -devel package), these days. Signed-off-by: Dario Faggioli --- Cc: Doug Goldstein Cc: Andrew Cooper --- This patch was submitted already, but not as part of this series. Anyway, changes from v1: * add python3 instead of replacing python2 with it. --- I think

[PATCH 3/3] automation: implement (rootless) podman support in containerize

2020-04-30 Thread Dario Faggioli
Right now only docker is supported, when using the containerize script for building inside containers. Enable podman as well. Note that podman can be use in rootless mode too, but for that to work the files /etc/subuid and /etc/subgid must be properly configured. For instance: dario@localhost> c

[PATCH 0/3] Automation: improve openSUSE containers + podman

2020-04-30 Thread Dario Faggioli
Hello, This short series contains some improvements for building Xen in openSUSE containers. In fact, the build dependencies inside the Tumbleweed container are updated and more handy helpers are added, in containerize, for referring to both Leap and Tumbleweed containers. In addition to that, in

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Eric W. Biederman
David Hildenbrand writes: > On 30.04.20 18:33, Eric W. Biederman wrote: >> David Hildenbrand writes: >> >>> On 30.04.20 17:38, Eric W. Biederman wrote: David Hildenbrand writes: > Some devices/drivers that add memory via add_memory() and friends (e.g., > dax/kmem, but also vi

Re: [PATCH] x86/hap: be more selective with assisted TLB flush

2020-04-30 Thread Roger Pau Monné
On Thu, Apr 30, 2020 at 05:19:19PM +0100, Andrew Cooper wrote: > On 30/04/2020 09:33, Jan Beulich wrote: > > On 30.04.2020 10:28, Roger Pau Monné wrote: > >> On Thu, Apr 30, 2020 at 09:20:58AM +0200, Jan Beulich wrote: > >>> On 29.04.2020 19:36, Roger Pau Monne wrote: > When doing an assisted

Re: [PATCH 05/12] xen: introduce reserve_heap_pages

2020-04-30 Thread Stefano Stabellini
On Thu, 30 Apr 2020, Julien Grall wrote: > > > > +pg = maddr_to_page(start); > > > > +node = phys_to_nid(start); > > > > +zone = page_to_zone(pg); > > > > +page_list_del(pg, &heap(node, zone, order)); > > > > + > > > > +__alloc_heap_pages(pg, order, memflags, d); > > > > > > I

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread David Hildenbrand
On 30.04.20 18:33, Eric W. Biederman wrote: > David Hildenbrand writes: > >> On 30.04.20 17:38, Eric W. Biederman wrote: >>> David Hildenbrand writes: >>> Some devices/drivers that add memory via add_memory() and friends (e.g., dax/kmem, but also virtio-mem in the future) don't want to

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Eric W. Biederman
David Hildenbrand writes: > On 30.04.20 17:38, Eric W. Biederman wrote: >> David Hildenbrand writes: >> >>> Some devices/drivers that add memory via add_memory() and friends (e.g., >>> dax/kmem, but also virtio-mem in the future) don't want to create entries >>> in /sys/firmware/memmap/ - prima

Re: [PATCH 05/12] xen: introduce reserve_heap_pages

2020-04-30 Thread Stefano Stabellini
On Thu, 30 Apr 2020, Jan Beulich wrote: > On 30.04.2020 00:46, Stefano Stabellini wrote: > > On Fri, 17 Apr 2020, Jan Beulich wrote: > >> On 15.04.2020 03:02, Stefano Stabellini wrote: > >>> Introduce a function named reserve_heap_pages (similar to > >>> alloc_heap_pages) that allocates a requested

Re: [PATCH] x86/hap: be more selective with assisted TLB flush

2020-04-30 Thread Andrew Cooper
On 30/04/2020 09:33, Jan Beulich wrote: > On 30.04.2020 10:28, Roger Pau Monné wrote: >> On Thu, Apr 30, 2020 at 09:20:58AM +0200, Jan Beulich wrote: >>> On 29.04.2020 19:36, Roger Pau Monne wrote: When doing an assisted flush on HAP the purpose of the on_selected_cpus is just to trigger

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Dave Hansen
On 4/30/20 8:52 AM, David Hildenbrand wrote: >> Justifying behavior by documentation that does not consider memory >> hotplug is bad thinking. > Are you maybe confusing this patch series with the arm64 approach? This > is not about ordinary hotplugged DIMMs. > > I'd love to get Dan's, Dave's and M

[ovmf test] 149887: all pass - PUSHED

2020-04-30 Thread osstest service owner
flight 149887 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/149887/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf f07fb43b2d3f92a15d6992205b72ba5df0e74fe2 baseline version: ovmf b2034179e8feed9c7d3bc

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread David Hildenbrand
On 30.04.20 17:38, Eric W. Biederman wrote: > David Hildenbrand writes: > >> Some devices/drivers that add memory via add_memory() and friends (e.g., >> dax/kmem, but also virtio-mem in the future) don't want to create entries >> in /sys/firmware/memmap/ - primarily to hinder kexec from adding th

Re: [PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Andrew Cooper
On 30/04/2020 12:09, Jan Beulich wrote: > On 30.04.2020 11:59, Andrew Cooper wrote: >> Fam19h is very similar to Fam17h in these regards. >> >> Signed-off-by: Andrew Cooper > Reviewed-by: Jan Beulich Thanks. > > Nevertheless a question: > >> --- a/xen/arch/x86/cpu/microcode/amd.c >> +++ b/xen/a

Re: [PATCH v2 2/3] mm/memory_hotplug: Introduce MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Eric W. Biederman
David Hildenbrand writes: > Some devices/drivers that add memory via add_memory() and friends (e.g., > dax/kmem, but also virtio-mem in the future) don't want to create entries > in /sys/firmware/memmap/ - primarily to hinder kexec from adding this > memory to the boot memmap of the kexec kernel.

Re: [PATCH RESEND 2/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
Hi Jan, On 30/04/2020 15:50, Jan Beulich wrote: On 30.04.2020 16:25, Julien Grall wrote: EXPERT mode is currently used to gate any options that are in technical preview or not security supported At the moment, the only way to select it is to use XEN_CONFIG_EXPERT=y on the make command line. Ho

Re: [PATCH v2 3/3] device-dax: Add system ram (add_memory()) with MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread David Hildenbrand
On 30.04.20 13:23, Dave Hansen wrote: > On 4/30/20 3:29 AM, David Hildenbrand wrote: >> Currently, when adding memory, we create entries in /sys/firmware/memmap/ >> as "System RAM". This does not reflect the reality and will lead to >> kexec-tools to add that memory to the fixed-up initial memmap f

[PATCH v1.1 2/3] xen/sched: fix theoretical races accessing vcpu->dirty_cpu

2020-04-30 Thread Juergen Gross
The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases. There are some instances where accesses are not atomically done, and even wors

Re: [PATCH 2/3] xen/sched: fix theoretical races accessing vcpu->dirty_cpu

2020-04-30 Thread Jürgen Groß
On 30.04.20 17:15, Juergen Gross wrote: The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases. There are some instances where acces

[PATCH 3/3] xen/cpupool: fix removing cpu from a cpupool

2020-04-30 Thread Juergen Gross
Commit cb563d7665f2 ("xen/sched: support core scheduling for moving cpus to/from cpupools") introduced a regression when trying to remove an offline cpu from a cpupool, as the system would crash in this situation. Fix that by testing the cpu to be online. Fixes: cb563d7665f2 ("xen/sched: support

Re: [PATCH v6 12/15] x86/smpboot: switch pl*e to use new APIs in clone_mapping

2020-04-30 Thread Jan Beulich
On 24.04.2020 16:09, Hongyan Xia wrote: > From: Wei Liu Nit: Why the emphasis on pl*e in the title? Is there anything left unconverted in the function? IOW how about "switch clone_mapping() to new page table APIs"? > --- a/xen/arch/x86/smpboot.c > +++ b/xen/arch/x86/smpboot.c > @@ -672,9 +672,9

[PATCH 0/3] xen: Fix some bugs in scheduling

2020-04-30 Thread Juergen Gross
Some bugs I found when trying to find a problem with cpu-on/offlining in core scheduling mode. Patches 1 and 3 are fixing observed problems, while patch 2 is more of a theoretical issue. Juergen Gross (3): xen/sched: allow rcu work to happen when syncing cpus in core scheduling xen/sched:

[PATCH 2/3] xen/sched: fix theoretical races accessing vcpu->dirty_cpu

2020-04-30 Thread Juergen Gross
The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases. There are some instances where accesses are not atomically done, and even wors

[PATCH 1/3] xen/sched: allow rcu work to happen when syncing cpus in core scheduling

2020-04-30 Thread Juergen Gross
With RCU barriers moved from tasklets to normal RCU processing cpu offlining in core scheduling might deadlock due to cpu synchronization required by RCU processing and core scheduling concurrently. Fix that by bailing out from core scheduling synchronization in case of pending RCU work. Additiona

Re: Cpu on/offlining crash with core scheduling

2020-04-30 Thread Jürgen Groß
On 29.04.20 11:15, Sergey Dyasli wrote: On 29/04/2020 09:09, Jürgen Groß wrote: On 27.04.20 15:49, Sergey Dyasli wrote: Hi Juergen, When I'm testing vcpu pinning with something like:   # xl vcpu-pin 0 0 2   # xen-hptool cpu-offline 3   (offline / online CPUs {2,3} if the above

Re: [PATCH v6 11/15] x86/smpboot: clone_mapping should have one exit path

2020-04-30 Thread Jan Beulich
On 24.04.2020 16:09, Hongyan Xia wrote: > From: Wei Liu Like for patches 1 and 2 in this series, and as iirc mentioned already long ago for those, "should" or alike are misleading here: It's not a mistake that they don't, i.e. this is not a bug fix. You _want_ these functions to have a single (ma

Re: [PATCH 05/12] xen: introduce reserve_heap_pages

2020-04-30 Thread Julien Grall
Hi, On 29/04/2020 23:46, Stefano Stabellini wrote: On Fri, 17 Apr 2020, Jan Beulich wrote: On 15.04.2020 03:02, Stefano Stabellini wrote: Introduce a function named reserve_heap_pages (similar to alloc_heap_pages) that allocates a requested memory range. Call __alloc_heap_pages for the impleme

Re: [PATCH RESEND 2/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Jan Beulich
On 30.04.2020 16:25, Julien Grall wrote: > EXPERT mode is currently used to gate any options that are in technical > preview or not security supported At the moment, the only way to select > it is to use XEN_CONFIG_EXPERT=y on the make command line. > > However, if the user forget to add the optio

Re: [PATCH] x86/hvm: allow for more fine grained assisted flush

2020-04-30 Thread Roger Pau Monné
On Thu, Apr 30, 2020 at 11:17:25AM +0200, Roger Pau Monne wrote: > diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c > index 814b7020d8..1d41b6e2ea 100644 > --- a/xen/arch/x86/hvm/hvm.c > +++ b/xen/arch/x86/hvm/hvm.c > @@ -4011,17 +4011,42 @@ static void hvm_s3_resume(struct domain *d) >

Re: [PATCH 1/2] xen/Kconfig: define EXPERT a bool rather than a string

2020-04-30 Thread Jan Beulich
On 30.04.2020 14:43, Julien Grall wrote: > From: Julien Grall > > Since commit f80fe2b34f08 "xen: Update Kconfig to Linux v5.4" EXPERT > can only have two values (enabled or disabled). So switch from a string > to a bool. > > Take the opportunity to replace all "EXPERT = y" to "EXPERT". > > Sig

[qemu-mainline test] 149885: tolerable FAIL - PUSHED

2020-04-30 Thread osstest service owner
flight 149885 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/149885/ Failures :-/ but no regressions. Regressions which are regarded as allowable (not blocking): test-amd64-amd64-xl-rtds 16 guest-localmigrate fail REGR. vs. 149877 test-armhf-armhf-xl-rtds

[PATCH RESEND 0/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
From: Julien Grall Hi all, This is a resend as I forgot to CC the maintainers. This small series is meant to make easier to experiment when using Xen. See patch #2 for more details. Cheers, Julien Grall (2): xen/Kconfig: define EXPERT a bool rather than a string xen: Allow EXPERT mode to

[PATCH RESEND 2/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
From: Julien Grall EXPERT mode is currently used to gate any options that are in technical preview or not security supported At the moment, the only way to select it is to use XEN_CONFIG_EXPERT=y on the make command line. However, if the user forget to add the option of one of the make command (

[PATCH RESEND 1/2] xen/Kconfig: define EXPERT a bool rather than a string

2020-04-30 Thread Julien Grall
From: Julien Grall Since commit f80fe2b34f08 "xen: Update Kconfig to Linux v5.4" EXPERT can only have two values (enabled or disabled). So switch from a string to a bool. Take the opportunity to replace all "EXPERT = y" to "EXPERT". Signed-off-by: Julien Grall --- xen/Kconfig

Re: [PATCH 0/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
Hmmm I have just realized I forgot to CC the REST. I will resend it. On 30/04/2020 13:43, Julien Grall wrote: From: Julien Grall Hi all, This small series is meant to make easier to experiment when using Xen. See patch #2 for more details. Cheers, Julien Grall (2): xen/Kconfig: define E

[libvirt test] 149886: regressions - FAIL

2020-04-30 Thread osstest service owner
flight 149886 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/149886/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-amd64-libvirt 6 libvirt-buildfail REGR. vs. 146182 build-i386-libvirt

Re: [PATCH 2/2] xen: credit2: limit the max number of CPUs in a runqueue

2020-04-30 Thread Dario Faggioli
On Thu, 2020-04-30 at 14:52 +0200, Jürgen Groß wrote: > On 30.04.20 14:28, Dario Faggioli wrote: > > On Thu, 2020-04-30 at 09:35 +0200, Jürgen Groß wrote: > > > > > With that, when the user removes 4 CPUs, we will have the 6 vs 10 > > situation. But we would make sure that, when she adds them back

Re: [PATCH 11/12] xen/arm: if xen_force don't try to setup the IOMMU

2020-04-30 Thread Julien Grall
Hi Stefano, On 29/04/2020 22:55, Stefano Stabellini wrote: On Wed, 15 Apr 2020, Julien Grall wrote: Hi Stefano, On 15/04/2020 02:02, Stefano Stabellini wrote: If xen_force (which means xen,force-assign-without-iommu was requested) don't try to add the device to the IOMMU. Return early instead

Re: [PATCH] x86/EFI: correct section offsets in mkreloc diagnostics

2020-04-30 Thread Andrew Cooper
On 30/04/2020 11:24, Jan Beulich wrote: > These are more helpful if they point at the address where the relocated > value starts, rather than at the specific byte of the difference. > > Signed-off-by: Jan Beulich Acked-by: Andrew Cooper

Re: [PATCH 12/12] xen/arm: call iomem_permit_access for passthrough devices

2020-04-30 Thread Julien Grall
Hi Stefano, On 29/04/2020 21:47, Stefano Stabellini wrote: On Wed, 15 Apr 2020, Julien Grall wrote: But doesn't it make sense to give domU permission if it is going to get the memory mapped? But admittedly I can't think of something that would break because of the lack of the iomem_permit_acces

Re: [PATCH v6 10/15] efi: switch to new APIs in EFI code

2020-04-30 Thread Jan Beulich
On 24.04.2020 16:09, Hongyan Xia wrote: > --- a/xen/arch/x86/efi/runtime.h > +++ b/xen/arch/x86/efi/runtime.h > @@ -1,12 +1,18 @@ > +#include > +#include > #include > #include > > #ifndef COMPAT > -l4_pgentry_t *__read_mostly efi_l4_pgtable; > +mfn_t __read_mostly efi_l4_mfn = INVALID_MFN_

Re: [PATCH 0/12] direct-map DomUs

2020-04-30 Thread Julien Grall
Hi Stefano, On 29/04/2020 21:16, Stefano Stabellini wrote: On Thu, 16 Apr 2020, Julien Grall wrote: Stefano Stabellini (12): xen: introduce xen_dom_flags xen/arm: introduce arch_xen_dom_flags and direct_map xen/arm: introduce 1:1 mapping for domUs xen: split allo

Re: [PATCH 2/2] xen: credit2: limit the max number of CPUs in a runqueue

2020-04-30 Thread Jürgen Groß
On 30.04.20 14:28, Dario Faggioli wrote: On Thu, 2020-04-30 at 09:35 +0200, Jürgen Groß wrote: On 29.04.20 19:36, Dario Faggioli wrote: Therefore, let's set a limit to the max number of CPUs that can share a Credit2 runqueue. The actual value is configurable (at boot time), the default being 1

[PATCH 0/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
From: Julien Grall Hi all, This small series is meant to make easier to experiment when using Xen. See patch #2 for more details. Cheers, Julien Grall (2): xen/Kconfig: define EXPERT a bool rather than a string xen: Allow EXPERT mode to be selected from the menuconfig directly xen/Kconfi

[PATCH 1/2] xen/Kconfig: define EXPERT a bool rather than a string

2020-04-30 Thread Julien Grall
From: Julien Grall Since commit f80fe2b34f08 "xen: Update Kconfig to Linux v5.4" EXPERT can only have two values (enabled or disabled). So switch from a string to a bool. Take the opportunity to replace all "EXPERT = y" to "EXPERT". Signed-off-by: Julien Grall --- xen/Kconfig

[PATCH 2/2] xen: Allow EXPERT mode to be selected from the menuconfig directly

2020-04-30 Thread Julien Grall
From: Julien Grall EXPERT mode is currently used to gate any options that are in technical preview or not security supported At the moment, the only way to select it is to use XEN_CONFIG_EXPERT=y on the make command line. However, if the user forget to add the option of one of the make command (

Re: [PATCH v6 09/15] efi: use new page table APIs in copy_mapping

2020-04-30 Thread Jan Beulich
On 24.04.2020 16:09, Hongyan Xia wrote: > --- a/xen/common/efi/boot.c > +++ b/xen/common/efi/boot.c > @@ -1454,16 +1454,20 @@ static __init void copy_mapping(unsigned long mfn, > unsigned long end, > continue; > if ( !(l4e_get_flags(l4e) & _PAGE_PRESENT) ) > { > -

Re: [PATCH 2/2] xen: credit2: limit the max number of CPUs in a runqueue

2020-04-30 Thread Dario Faggioli
On Thu, 2020-04-30 at 09:35 +0200, Jürgen Groß wrote: > On 29.04.20 19:36, Dario Faggioli wrote: > > > > Therefore, let's set a limit to the max number of CPUs that can > > share a > > Credit2 runqueue. The actual value is configurable (at boot time), > > the > > default being 16. If, for instance

Re: [RFC] UEFI Secure Boot on Xen Hosts

2020-04-30 Thread Ard Biesheuvel
On Thu, 30 Apr 2020 at 13:15, Daniel Kiper wrote: > > Adding Ard... > > On Thu, Apr 30, 2020 at 09:01:45AM +0200, Jan Beulich wrote: > > On 30.04.2020 00:51, Bobby Eshleman wrote: > > > Hey all, > > > > > > We're looking to develop UEFI Secure Boot support for grub-loaded Xen and > > > ultimately

Re: [PATCH v2 5/5] tools/libxc: make use of domain context SHARED_INFO record...

2020-04-30 Thread Jan Beulich
On 07.04.2020 19:38, Paul Durrant wrote: > ... in the save/restore code. > > This patch replaces direct mapping of the shared_info_frame (retrieved > using XEN_DOMCTL_getdomaininfo) with save/load of the domain context > SHARED_INFO record. > > No modifications are made to the definition of the m

Re: [PATCH v2 4/5] common/domain: add a domain context record for shared_info...

2020-04-30 Thread Jan Beulich
On 07.04.2020 19:38, Paul Durrant wrote: > --- a/tools/misc/xen-domctx.c > +++ b/tools/misc/xen-domctx.c > @@ -59,6 +59,16 @@ static void dump_header(struct domain_save_descriptor > *desc) > off += desc->length; > } > > +static void dump_shared_info(struct domain_save_descriptor *desc) > +

Re: [PATCH v2 4/5] common/domain: add a domain context record for shared_info...

2020-04-30 Thread Jan Beulich
On 28.04.2020 17:37, Paul Durrant wrote: >> -Original Message- >> From: Julien Grall >> Sent: 20 April 2020 18:35 >> To: Paul Durrant ; xen-devel@lists.xenproject.org >> Cc: Paul Durrant ; Ian Jackson >> ; Wei Liu ; >> Andrew Cooper ; George Dunlap >> ; Jan Beulich >> ; Stefano Stabellin

Re: [PATCH v2 3/3] device-dax: Add system ram (add_memory()) with MHP_NO_FIRMWARE_MEMMAP

2020-04-30 Thread Dave Hansen
On 4/30/20 3:29 AM, David Hildenbrand wrote: > Currently, when adding memory, we create entries in /sys/firmware/memmap/ > as "System RAM". This does not reflect the reality and will lead to > kexec-tools to add that memory to the fixed-up initial memmap for a > kexec kernel (loaded via kexec_load(

[xen-unstable-smoke test] 149888: tolerable all pass - PUSHED

2020-04-30 Thread osstest service owner
flight 149888 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/149888/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 1

Re: [RFC] UEFI Secure Boot on Xen Hosts

2020-04-30 Thread Daniel Kiper
Adding Ard... On Thu, Apr 30, 2020 at 09:01:45AM +0200, Jan Beulich wrote: > On 30.04.2020 00:51, Bobby Eshleman wrote: > > Hey all, > > > > We're looking to develop UEFI Secure Boot support for grub-loaded Xen and > > ultimately for XCP-ng (I'm on the XCP-ng team at Vates). > > > > In addition to

Re: [PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Jan Beulich
On 30.04.2020 11:59, Andrew Cooper wrote: > Fam19h is very similar to Fam17h in these regards. > > Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich Nevertheless a question: > --- a/xen/arch/x86/cpu/microcode/amd.c > +++ b/xen/arch/x86/cpu/microcode/amd.c > @@ -125,6 +125,7 @@ static bool

[PATCH] tools/xl: vcpu-pin: Skip global affinity when the hard affinity is not changed

2020-04-30 Thread Julien Grall
From: Julien Grall After XSA-273, it is not possible to modify the vCPU soft affinity using xl vcpu-pin without modifying the hard affinity. Instead the command will crash. 42sh> gdb /usr/local/sbin/xl (gdb) r vcpu-pin 0 0 - 10 [...] Program received signal SIGSEGV, Segmentation fault. [...] (gd

Re: [PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Roger Pau Monné
On Thu, Apr 30, 2020 at 11:38:14AM +0100, Andrew Cooper wrote: > On 30/04/2020 11:35, Roger Pau Monné wrote: > > On Thu, Apr 30, 2020 at 10:59:47AM +0100, Andrew Cooper wrote: > >> diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c > >> index c3f92ed231..014524486f 100644 > >> --- a/xen/arch/x86/

Re: [PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Andrew Cooper
On 30/04/2020 11:35, Roger Pau Monné wrote: > On Thu, Apr 30, 2020 at 10:59:47AM +0100, Andrew Cooper wrote: >> diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c >> index c3f92ed231..014524486f 100644 >> --- a/xen/arch/x86/nmi.c >> +++ b/xen/arch/x86/nmi.c >> @@ -398,7 +398,7 @@ void setup_apic_

Re: [PATCH] x86/amd: Initial support for Fam19h processors

2020-04-30 Thread Roger Pau Monné
On Thu, Apr 30, 2020 at 10:59:47AM +0100, Andrew Cooper wrote: > diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c > index c3f92ed231..014524486f 100644 > --- a/xen/arch/x86/nmi.c > +++ b/xen/arch/x86/nmi.c > @@ -398,7 +398,7 @@ void setup_apic_nmi_watchdog(void) > case X86_VENDOR_AMD: >

  1   2   >