Re: [PATCH v2 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu

2020-05-14 Thread Jan Beulich
On 11.05.2020 13:28, Juergen Gross wrote: > @@ -1956,13 +1958,17 @@ void sync_local_execstate(void) > > void sync_vcpu_execstate(struct vcpu *v) > { > -if ( v->dirty_cpu == smp_processor_id() ) > +unsigned int dirty_cpu = read_atomic(&v->dirty_cpu); > + > +if ( dirty_cpu == smp_proc

Re: [PATCH v8 04/12] xen: add basic hypervisor filesystem support

2020-05-14 Thread Jan Beulich
On 08.05.2020 17:34, Juergen Gross wrote: > --- a/xen/common/Makefile > +++ b/xen/common/Makefile > @@ -10,6 +10,7 @@ obj-y += domain.o > obj-y += event_2l.o > obj-y += event_channel.o > obj-y += event_fifo.o > +obj-$(CONFIG_HYPFS) += hypfs.o > obj-$(CONFIG_CRASH_DEBUG) += gdbstub.o > obj-$(CO

[xen-unstable-smoke test] 150171: tolerable all pass - PUSHED

2020-05-14 Thread osstest service owner
flight 150171 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/150171/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 1

Re: [PATCH v2 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu

2020-05-14 Thread Jürgen Groß
On 14.05.20 09:10, Jan Beulich wrote: On 11.05.2020 13:28, Juergen Gross wrote: @@ -1956,13 +1958,17 @@ void sync_local_execstate(void) void sync_vcpu_execstate(struct vcpu *v) { -if ( v->dirty_cpu == smp_processor_id() ) +unsigned int dirty_cpu = read_atomic(&v->dirty_cpu); + +

Re: [PATCH v8 12/12] x86/HVM: don't needlessly intercept APERF/MPERF/TSC MSR reads

2020-05-14 Thread Jan Beulich
On 13.05.2020 15:35, Jan Beulich wrote: > On 08.05.2020 23:04, Andrew Cooper wrote: >> On 05/05/2020 09:20, Jan Beulich wrote: >>> If the hardware can handle accesses, we should allow it to do so. This >>> way we can expose EFRO to HVM guests, >> >> I'm reminded now of the conversation I'm sure we'

[PATCH v9 0/9] x86emul: further work

2020-05-14 Thread Jan Beulich
1: address x86_insn_is_mem_{access,write}() omissions 2: disable FPU/MMX/SIMD insn emulation when !HVM 3: support MOVDIR{I,64B} insns 4: support ENQCMD insn 5: support SERIALIZE 6: support X{SUS,RES}LDTRK 7: support FNSTENV and FNSAVE 8: support FLDENV and FRSTOR 9: support FXSAVE/FXRSTOR Main ch

[PATCH v9 3/9] x86emul: support MOVDIR{I,64B} insns

2020-05-14 Thread Jan Beulich
Introduce a new blk() hook, paralleling the rmw() one in a certain way, but being intended for larger data sizes, and hence its HVM intermediate handling function doesn't fall back to splitting the operation if the requested virtual address can't be mapped. Note that SDM revision 071 doesn't speci

[PATCH v9 1/9] x86emul: address x86_insn_is_mem_{access,write}() omissions

2020-05-14 Thread Jan Beulich
First of all explain in comments what the functions' purposes are. Then make them actually match their comments. Note that fc6fa977be54 ("x86emul: extend x86_insn_is_mem_write() coverage") didn't actually fix the function's behavior for {,V}STMXCSR: Both are covered by generic code higher up in th

[PATCH v9 2/9] x86emul: disable FPU/MMX/SIMD insn emulation when !HVM

2020-05-14 Thread Jan Beulich
In a pure PV environment (the PV shim in particular) we don't really need emulation of all these. To limit #ifdef-ary utilize some of the CASE_*() macros we have, by providing variants expanding to (effectively) nothing (really a label, which in turn requires passing -Wno-unused-label to the compil

[PATCH v9 4/9] x86emul: support ENQCMD insns

2020-05-14 Thread Jan Beulich
Note that the ISA extensions document revision 038 doesn't specify exception behavior for ModRM.mod == 0b11; assuming #UD here. No tests are being added to the harness - this would be quite hard, we can't just issue the insns against RAM. Their similarity with MOVDIR64B should have the test case t

[PATCH v9 6/9] x86emul: support X{SUS,RES}LDTRK

2020-05-14 Thread Jan Beulich
There's nothing to be done by the emulator, as we unconditionally abort any XBEGIN. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- v9: A -> a in public header. Add comments. v6: New. --- a/tools/libxl/libxl_cpuid.c +++ b/tools/libxl/libxl_cpuid.c @@ -208,6 +208,7 @@ int libxl_cpuid_pars

[PATCH v9 5/9] x86emul: support SERIALIZE

2020-05-14 Thread Jan Beulich
... enabling its use by all guest kinds at the same time. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- v9: A -> a in public header. v7: Re-base. v6: New. --- a/tools/libxl/libxl_cpuid.c +++ b/tools/libxl/libxl_cpuid.c @@ -214,6 +214,7 @@ int libxl_cpuid_parse_config(libxl_cpuid

[PATCH v9 7/9] x86emul: support FNSTENV and FNSAVE

2020-05-14 Thread Jan Beulich
To avoid introducing another boolean into emulator state, the rex_prefix field gets (ab)used to convey the real/VM86 vs protected mode info (affecting structure layout, albeit not size) to x86_emul_blk(). Signed-off-by: Jan Beulich --- TBD: The full 16-bit padding fields in the 32-bit structures

[PATCH v9 9/9] x86emul: support FXSAVE/FXRSTOR

2020-05-14 Thread Jan Beulich
Note that FPU selector handling as well as MXCSR mask saving for now does not honor differences between host and guest visible featuresets. While for Intel operation of the insns with CR4.OSFXSR=0 is implementation dependent, use the easiest solution there: Simply don't look at the bit in the firs

[PATCH v9 8/9] x86emul: support FLDENV and FRSTOR

2020-05-14 Thread Jan Beulich
While the Intel SDM claims that FRSTOR itself may raise #MF upon completion, this was confirmed by Intel to be a doc error which will be corrected in due course; behavior is like FLDENV, and like old hard copy manuals describe it. Re-arrange a switch() statement's case label order to allow for fal

Re: [PATCH v9 0/9] x86emul: further work

2020-05-14 Thread Jan Beulich
On 14.05.2020 11:03, Jan Beulich wrote: > 1: address x86_insn_is_mem_{access,write}() omissions > 2: disable FPU/MMX/SIMD insn emulation when !HVM > 3: support MOVDIR{I,64B} insns > 4: support ENQCMD insn > 5: support SERIALIZE > 6: support X{SUS,RES}LDTRK > 7: support FNSTENV and FNSAVE > 8: supp

Re: [PATCH v2 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu

2020-05-14 Thread Jan Beulich
On 14.05.2020 10:50, Jürgen Groß wrote: > On 14.05.20 09:10, Jan Beulich wrote: >> On 11.05.2020 13:28, Juergen Gross wrote: >>> @@ -1956,13 +1958,17 @@ void sync_local_execstate(void) >>>     void sync_vcpu_execstate(struct vcpu *v) >>>   { >>> -    if ( v->dirty_cpu == smp_processor_id() ) >>> + 

Re: [PATCH v2 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu

2020-05-14 Thread Jürgen Groß
On 14.05.20 11:24, Jan Beulich wrote: On 14.05.2020 10:50, Jürgen Groß wrote: On 14.05.20 09:10, Jan Beulich wrote: On 11.05.2020 13:28, Juergen Gross wrote: @@ -1956,13 +1958,17 @@ void sync_local_execstate(void)     void sync_vcpu_execstate(struct vcpu *v)   { -    if ( v->dirty_cpu == smp

Re: [PATCH v8 08/12] xen: add /buildinfo/config entry to hypervisor filesystem

2020-05-14 Thread Jan Beulich
On 08.05.2020 17:34, Juergen Gross wrote: > Add the /buildinfo/config entry to the hypervisor filesystem. This > entry contains the .config file used to build the hypervisor. > > Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich with a remark and one further adjustment: > @@ -73,3 +74,14 @

Re: [PATCH v8 04/12] xen: add basic hypervisor filesystem support

2020-05-14 Thread Jürgen Groß
On 14.05.20 09:59, Jan Beulich wrote: On 08.05.2020 17:34, Juergen Gross wrote: --- a/xen/common/Makefile +++ b/xen/common/Makefile @@ -10,6 +10,7 @@ obj-y += domain.o obj-y += event_2l.o obj-y += event_channel.o obj-y += event_fifo.o +obj-$(CONFIG_HYPFS) += hypfs.o obj-$(CONFIG_CRASH_DE

Re: [PATCH v8 08/12] xen: add /buildinfo/config entry to hypervisor filesystem

2020-05-14 Thread Jürgen Groß
On 14.05.20 11:32, Jan Beulich wrote: On 08.05.2020 17:34, Juergen Gross wrote: Add the /buildinfo/config entry to the hypervisor filesystem. This entry contains the .config file used to build the hypervisor. Signed-off-by: Juergen Gross Reviewed-by: Jan Beulich with a remark and one furthe

Re: [PATCH v8 04/12] xen: add basic hypervisor filesystem support

2020-05-14 Thread Jan Beulich
On 08.05.2020 17:34, Juergen Gross wrote: > --- /dev/null > +++ b/xen/include/xen/hypfs.h > @@ -0,0 +1,122 @@ > +#ifndef __XEN_HYPFS_H__ > +#define __XEN_HYPFS_H__ > + > +#ifdef CONFIG_HYPFS > +#include > +#include > +#include > + > +struct hypfs_entry_leaf; > + > +struct hypfs_entry { > +un

Re: [PATCH v8 04/12] xen: add basic hypervisor filesystem support

2020-05-14 Thread Jürgen Groß
On 14.05.20 11:55, Jan Beulich wrote: On 08.05.2020 17:34, Juergen Gross wrote: --- /dev/null +++ b/xen/include/xen/hypfs.h @@ -0,0 +1,122 @@ +#ifndef __XEN_HYPFS_H__ +#define __XEN_HYPFS_H__ + +#ifdef CONFIG_HYPFS +#include +#include +#include + +struct hypfs_entry_leaf; + +struct hypfs_entr

Re: [PATCH 4/4] x86/APIC: restrict certain messages to BSP

2020-05-14 Thread Roger Pau Monné
On Fri, Mar 13, 2020 at 10:26:47AM +0100, Jan Beulich wrote: > All CPUs get an equal setting of EOI broadcast suppression; no need to > log one message per CPU, even if it's only in verbose APIC mode. > > Only the BSP is eligible to possibly get ExtINT enabled; no need to log > that it gets disabl

Re: [PATCH] domain_page: handle NULL within unmap_domain_page() itself

2020-05-14 Thread Wei Liu
On Wed, May 13, 2020 at 04:43:33PM +0100, Hongyan Xia wrote: > From: Hongyan Xia > > The macro version UNMAP_DOMAIN_PAGE() does both NULL checking and > variable clearing. Move NULL checking into the function itself so that > the semantics is consistent with other similar constructs like XFREE().

Re: [PATCH v8 09/12] xen: add runtime parameter access support to hypfs

2020-05-14 Thread Jan Beulich
On 08.05.2020 17:34, Juergen Gross wrote: > --- a/xen/arch/arm/xen.lds.S > +++ b/xen/arch/arm/xen.lds.S > @@ -89,6 +89,13 @@ SECTIONS > __start_schedulers_array = .; > *(.data.schedulers) > __end_schedulers_array = .; > + > +#ifdef CONFIG_HYPFS > + . = ALIGN(8); > +

[PATCH v3 4/5] common/domain: add a domain context record for shared_info...

2020-05-14 Thread Paul Durrant
From: Paul Durrant ... and update xen-domctx to dump some information describing the record. NOTE: The domain may or may not be using the embedded vcpu_info array so ultimately separate context records will be added for vcpu_info when this becomes necessary. Signed-off-by: Paul Durr

[PATCH v3 2/5] xen/common/domctl: introduce XEN_DOMCTL_get/setdomaincontext

2020-05-14 Thread Paul Durrant
These domctls provide a mechanism to get and set domain context from the toolstack. Signed-off-by: Paul Durrant --- Cc: Daniel De Graaf Cc: Ian Jackson Cc: Wei Liu Cc: Andrew Cooper Cc: George Dunlap Cc: Jan Beulich Cc: Julien Grall Cc: Stefano Stabellini v3: - Addressed comments from J

[PATCH v3 1/5] xen/common: introduce a new framework for save/restore of 'domain' context

2020-05-14 Thread Paul Durrant
To allow enlightened HVM guests (i.e. those that have PV drivers) to be migrated without their co-operation it will be necessary to transfer 'PV' state such as event channel state, grant entry state, etc. Currently there is a framework (entered via the hvm_save/load() functions) that allows a doma

[PATCH v3 5/5] tools/libxc: make use of domain context SHARED_INFO record...

2020-05-14 Thread Paul Durrant
... in the save/restore code. This patch replaces direct mapping of the shared_info_frame (retrieved using XEN_DOMCTL_getdomaininfo) with save/load of the domain context SHARED_INFO record. No modifications are made to the definition of the migration stream at this point. Subsequent patches will

[PATCH v3 0/5] domain context infrastructure

2020-05-14 Thread Paul Durrant
From: Paul Durrant Paul Durrant (5): xen/common: introduce a new framework for save/restore of 'domain' context xen/common/domctl: introduce XEN_DOMCTL_get/setdomaincontext tools/misc: add xen-domctx to present domain context common/domain: add a domain context record for shared_info.

[PATCH v3 3/5] tools/misc: add xen-domctx to present domain context

2020-05-14 Thread Paul Durrant
This tool is analogous to 'xen-hvmctx' which presents HVM context. Subsequent patches will add 'dump' functions when new records are introduced. Signed-off-by: Paul Durrant --- Cc: Ian Jackson Cc: Wei Liu v3: - Re-worked to avoid copying onto stack - Added optional typecode and instance argu

Re: use of "stat -"

2020-05-14 Thread Ian Jackson
I've read this thread. Jan, I'm sorry that this causes you inconvenience. I'm hoping it won't come down to a choice between supporting people who want to ship a dom0 without perl, and people who want a dom0 using more-than-a-decade-old coreutils. Jan, can you tell me what the output is of this o

Re: [PATCH v8 09/12] xen: add runtime parameter access support to hypfs

2020-05-14 Thread Jürgen Groß
On 14.05.20 12:20, Jan Beulich wrote: On 08.05.2020 17:34, Juergen Gross wrote: --- a/xen/arch/arm/xen.lds.S +++ b/xen/arch/arm/xen.lds.S @@ -89,6 +89,13 @@ SECTIONS __start_schedulers_array = .; *(.data.schedulers) __end_schedulers_array = .; + +#ifdef CONFIG_HYPFS +

[xen-unstable-smoke test] 150173: tolerable all pass - PUSHED

2020-05-14 Thread osstest service owner
flight 150173 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/150173/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 1

Re: [PATCH v8 04/12] xen: add basic hypervisor filesystem support

2020-05-14 Thread Jan Beulich
On 14.05.2020 11:50, Jürgen Groß wrote: > On 14.05.20 09:59, Jan Beulich wrote: >> On 08.05.2020 17:34, Juergen Gross wrote: >>> +#define HYPFS_FIXEDSIZE_INIT(var, typ, nam, contvar, wr) \ >>> +    struct hypfs_entry_leaf __read_mostly var = {    \ >>> +    .e.type = typ,   

Re: [PATCH v8 09/12] xen: add runtime parameter access support to hypfs

2020-05-14 Thread Jan Beulich
On 14.05.2020 13:48, Jürgen Groß wrote: > On 14.05.20 12:20, Jan Beulich wrote: >> On 08.05.2020 17:34, Juergen Gross wrote: >>> --- a/xen/common/grant_table.c >>> +++ b/xen/common/grant_table.c >>> @@ -85,8 +85,43 @@ struct grant_table { >>>   struct grant_table_arch arch; >>>   }; >>>   -stat

Re: [PATCH v8 04/12] xen: add basic hypervisor filesystem support

2020-05-14 Thread Jürgen Groß
On 14.05.20 13:58, Jan Beulich wrote: On 14.05.2020 11:50, Jürgen Groß wrote: On 14.05.20 09:59, Jan Beulich wrote: On 08.05.2020 17:34, Juergen Gross wrote: +#define HYPFS_FIXEDSIZE_INIT(var, typ, nam, contvar, wr) \ +    struct hypfs_entry_leaf __read_mostly var = {    \ +    .e.type

Re: [PATCH 4/4] x86/APIC: restrict certain messages to BSP

2020-05-14 Thread Jan Beulich
On 14.05.2020 12:01, Roger Pau Monné wrote: > On Fri, Mar 13, 2020 at 10:26:47AM +0100, Jan Beulich wrote: >> All CPUs get an equal setting of EOI broadcast suppression; no need to >> log one message per CPU, even if it's only in verbose APIC mode. >> >> Only the BSP is eligible to possibly get Ext

Re: use of "stat -"

2020-05-14 Thread Jan Beulich
On 14.05.2020 13:02, Ian Jackson wrote: > I've read this thread. Jan, I'm sorry that this causes you > inconvenience. I'm hoping it won't come down to a choice between > supporting people who want to ship a dom0 without perl, and people who > want a dom0 using more-than-a-decade-old coreutils. W

Re: [PATCH v8 12/12] xen: remove XEN_SYSCTL_set_parameter support

2020-05-14 Thread Jan Beulich
On 08.05.2020 17:34, Juergen Gross wrote: Besides the changes made, wouldn't it be worthwhile to change HYPFS Kconfig help wording from "result in some features not being available" to something mentioning runtime parameter setting in particular, and perhaps also from "might" to "will"? > --- a/x

Re: [PATCH] x86: retrieve and log CPU frequency information

2020-05-14 Thread Roger Pau Monné
On Wed, Apr 15, 2020 at 01:55:24PM +0200, Jan Beulich wrote: > While from just a single Skylake system it is already clear that we > can't base any of our logic on CPUID leaf 15 [1] (leaf 16 is > documented to be used for display purposes only anyway), logging this > information may still give us s

Re: [PATCH] domain_page: handle NULL within unmap_domain_page() itself

2020-05-14 Thread Jan Beulich
On 14.05.2020 12:01, Wei Liu wrote: > On Wed, May 13, 2020 at 04:43:33PM +0100, Hongyan Xia wrote: >> From: Hongyan Xia >> >> The macro version UNMAP_DOMAIN_PAGE() does both NULL checking and >> variable clearing. Move NULL checking into the function itself so that >> the semantics is consistent w

Re: [PATCH 4/4] x86/APIC: restrict certain messages to BSP

2020-05-14 Thread Roger Pau Monné
On Thu, May 14, 2020 at 02:31:33PM +0200, Jan Beulich wrote: > On 14.05.2020 12:01, Roger Pau Monné wrote: > > On Fri, Mar 13, 2020 at 10:26:47AM +0100, Jan Beulich wrote: > >> --- a/xen/arch/x86/apic.c > >> +++ b/xen/arch/x86/apic.c > >> @@ -499,7 +499,7 @@ static void resume_x2apic(void) > >>

[xen-unstable test] 150169: regressions - FAIL

2020-05-14 Thread osstest service owner
flight 150169 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/150169/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-libvirt-vhd 10 debian-di-installfail REGR. vs. 150164 Tests which did no

Re: [PATCH] x86: retrieve and log CPU frequency information

2020-05-14 Thread Jan Beulich
On 14.05.2020 15:10, Roger Pau Monné wrote: > On Wed, Apr 15, 2020 at 01:55:24PM +0200, Jan Beulich wrote: >> While from just a single Skylake system it is already clear that we >> can't base any of our logic on CPUID leaf 15 [1] (leaf 16 is >> documented to be used for display purposes only anyway

Re: [PATCH v2 2/3] xen/sched: don't call sync_vcpu_execstate() in sched_unit_migrate_finish()

2020-05-14 Thread Jan Beulich
On 11.05.2020 13:28, Juergen Gross wrote: > With support of core scheduling sched_unit_migrate_finish() gained a > call of sync_vcpu_execstate() as it was believed to be called as a > result of vcpu migration in any case. > > In case of migrating a vcpu away from a physical cpu for a short period

Re: [PATCH v2 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu

2020-05-14 Thread Jan Beulich
On 14.05.2020 11:29, Jürgen Groß wrote: > On 14.05.20 11:24, Jan Beulich wrote: >> On 14.05.2020 10:50, Jürgen Groß wrote: >>> On 14.05.20 09:10, Jan Beulich wrote: On 11.05.2020 13:28, Juergen Gross wrote: > @@ -1956,13 +1958,17 @@ void sync_local_execstate(void) >      void sync_vcpu

Re: [PATCH v2] x86: use POPCNT for hweight() when available

2020-05-14 Thread Roger Pau Monné
On Mon, Jul 15, 2019 at 02:39:04PM +, Jan Beulich wrote: > This is faster than using the software implementation, and the insn is > available on all half-way recent hardware. Therefore convert > generic_hweight() to out-of-line functions (without affecting Arm) > and use alternatives patching t

[OSSTEST PATCH] cri-args-hostlists: Do not include transcript in reports

2020-05-14 Thread Ian Jackson
These transcripts are huge and not useful. Still include them if sg-secute-flight failed. Signed-off-by: Ian Jackson --- cri-args-hostlists | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/cri-args-hostlists b/cri-args-hostlists index 3578fe1c..28d576db 100644 --- a/cr

[OSSTEST PATCH 2/4] ts-repeat-test: Honour repeat_mult

2020-05-14 Thread Ian Jackson
Nothing automatic uses repeat_mult but this script should definitely honour it. Signed-off-by: Ian Jackson --- ts-repeat-test | 4 1 file changed, 4 insertions(+) diff --git a/ts-repeat-test b/ts-repeat-test index 4625add8..e6b52465 100755 --- a/ts-repeat-test +++ b/ts-repeat-test @@ -51,6

[OSSTEST PATCH 3/4] ts-repeat-test: Show total planned repeat count info

2020-05-14 Thread Ian Jackson
Message change only Signed-off-by: Ian Jackson --- ts-repeat-test | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ts-repeat-test b/ts-repeat-test index e6b52465..5e17c335 100755 --- a/ts-repeat-test +++ b/ts-repeat-test @@ -56,7 +56,7 @@ my $orgreps = $reps; $reps *= $times;

[OSSTEST PATCH 4/4] TestSupport: honour OSSTEST_PDU_MANUAL override env var

2020-05-14 Thread Ian Jackson
This takes effect on everything that uses selecthost(). The result is that PDU operations are made manual. This can be useful for testing etc. Signed-off-by: Ian Jackson --- Osstest/TestSupport.pm | 6 ++ 1 file changed, 6 insertions(+) diff --git a/Osstest/TestSupport.pm b/Osstest/TestSu

[OSSTEST PATCH 1/4] Executive: Do not print "shared ... marked ready" when not shared

2020-05-14 Thread Ian Jackson
Signed-off-by: Ian Jackson --- Osstest/Executive.pm | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm index e741f529..c3dc1261 100644 --- a/Osstest/Executive.pm +++ b/Osstest/Executive.pm @@ -1017,11 +1017,12 @@ sub executive_

[OSSTEST PATCH 0/4] Four misc small fixes

2020-05-14 Thread Ian Jackson
Thse four small fixes/improvements have been generated as the result of my buster work. They are not really related to buster. Many more changes will follow...

Re: [PATCH v2 2/3] xen/sched: don't call sync_vcpu_execstate() in sched_unit_migrate_finish()

2020-05-14 Thread Jürgen Groß
On 14.05.20 15:57, Jan Beulich wrote: On 11.05.2020 13:28, Juergen Gross wrote: With support of core scheduling sched_unit_migrate_finish() gained a call of sync_vcpu_execstate() as it was believed to be called as a result of vcpu migration in any case. In case of migrating a vcpu away from a p

Error during update_runstate_area with KPTI activated

2020-05-14 Thread Bertrand Marquis
Hi, When executing linux on arm64 with KPTI activated (in Dom0 or in a DomU), I have a lot of walk page table errors like this: (XEN) p2m.c:1890: d1v0: Failed to walk page-table va 0xff837ebe0cd0 After implementing a call trace, I found that the problem was coming from the update_runstate_a

[xen-unstable-smoke test] 150175: tolerable all pass - PUSHED

2020-05-14 Thread osstest service owner
flight 150175 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/150175/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 1

Re: [PATCH v18 2/2] tools/libxl: VM forking toolstack side

2020-05-14 Thread Ian Jackson
Tamas K Lengyel writes ("[PATCH v18 2/2] tools/libxl: VM forking toolstack side"): > Add necessary bits to implement "xl fork-vm" commands. The command allows the > user to specify how to launch the device model allowing for a late-launch > model > in which the user can execute the fork without t

[linux-5.4 test] 150172: tolerable FAIL - PUSHED

2020-05-14 Thread osstest service owner
flight 150172 linux-5.4 real [real] http://logs.test-lab.xenproject.org/osstest/logs/150172/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-rtds 16 guest-localmigrate fail like 150122 test-armhf-armhf-xl-rtds 16 guest-s

Re: [PATCH v8 09/12] xen: add runtime parameter access support to hypfs

2020-05-14 Thread Jürgen Groß
On 14.05.20 14:10, Jan Beulich wrote: On 14.05.2020 13:48, Jürgen Groß wrote: On 14.05.20 12:20, Jan Beulich wrote: On 08.05.2020 17:34, Juergen Gross wrote: --- a/xen/common/grant_table.c +++ b/xen/common/grant_table.c @@ -85,8 +85,43 @@ struct grant_table {   struct grant_table_arch arc

Re: [PATCH v8 12/12] xen: remove XEN_SYSCTL_set_parameter support

2020-05-14 Thread Jürgen Groß
On 14.05.20 15:03, Jan Beulich wrote: On 08.05.2020 17:34, Juergen Gross wrote: Besides the changes made, wouldn't it be worthwhile to change HYPFS Kconfig help wording from "result in some features not being available" to something mentioning runtime parameter setting in particular, and perhaps

Re: [PATCH v2 2/3] xen/sched: don't call sync_vcpu_execstate() in sched_unit_migrate_finish()

2020-05-14 Thread Jan Beulich
On 14.05.2020 16:21, Jürgen Groß wrote: > On 14.05.20 15:57, Jan Beulich wrote: >> On 11.05.2020 13:28, Juergen Gross wrote: >>> With support of core scheduling sched_unit_migrate_finish() gained a >>> call of sync_vcpu_execstate() as it was believed to be called as a >>> result of vcpu migration i

Re: [PATCH v8 09/12] xen: add runtime parameter access support to hypfs

2020-05-14 Thread Jan Beulich
On 14.05.2020 16:56, Jürgen Groß wrote: > On 14.05.20 14:10, Jan Beulich wrote: >> On 14.05.2020 13:48, Jürgen Groß wrote: >>> On 14.05.20 12:20, Jan Beulich wrote: On 08.05.2020 17:34, Juergen Gross wrote: > --- a/xen/common/grant_table.c > +++ b/xen/common/grant_table.c > @@ -85,

Re: [PATCH v18 2/2] tools/libxl: VM forking toolstack side

2020-05-14 Thread Tamas K Lengyel
On Thu, May 14, 2020 at 8:52 AM Ian Jackson wrote: > > Tamas K Lengyel writes ("[PATCH v18 2/2] tools/libxl: VM forking toolstack > side"): > > Add necessary bits to implement "xl fork-vm" commands. The command allows > > the > > user to specify how to launch the device model allowing for a late

Re: [PATCH] x86: retrieve and log CPU frequency information

2020-05-14 Thread Roger Pau Monné
On Thu, May 14, 2020 at 03:38:18PM +0200, Jan Beulich wrote: > On 14.05.2020 15:10, Roger Pau Monné wrote: > > On Wed, Apr 15, 2020 at 01:55:24PM +0200, Jan Beulich wrote: > >> While from just a single Skylake system it is already clear that we > >> can't base any of our logic on CPUID leaf 15 [1]

[PATCH v3 1/3] xen/sched: allow rcu work to happen when syncing cpus in core scheduling

2020-05-14 Thread Juergen Gross
With RCU barriers moved from tasklets to normal RCU processing cpu offlining in core scheduling might deadlock due to cpu synchronization required by RCU processing and core scheduling concurrently. Fix that by bailing out from core scheduling synchronization in case of pending RCU work. Additiona

[PATCH v3 3/3] xen/sched: fix latent races accessing vcpu->dirty_cpu

2020-05-14 Thread Juergen Gross
The dirty_cpu field of struct vcpu denotes which cpu still holds data of a vcpu. All accesses to this field should be atomic in case the vcpu could just be running, as it is accessed without any lock held in most cases. Especially sync_local_execstate() and context_switch() for the same vcpu runnin

[PATCH v3 0/3] xen: Fix some bugs in scheduling

2020-05-14 Thread Juergen Gross
Some problems I found when trying to find a problem with cpu-on/offlining in core scheduling mode. Juergen Gross (3): xen/sched: allow rcu work to happen when syncing cpus in core scheduling xen/sched: don't call sync_vcpu_execstate() in sched_unit_migrate_finish() xen/sched: fix lat

[PATCH v3 2/3] xen/sched: don't call sync_vcpu_execstate() in sched_unit_migrate_finish()

2020-05-14 Thread Juergen Gross
With support of core scheduling sched_unit_migrate_finish() gained a call of sync_vcpu_execstate() as it was believed to be called as a result of vcpu migration in any case. In case of migrating a vcpu away from a physical cpu for a short period of time ionly without ever being scheduled on the se

Re: [PATCH v8 09/12] xen: add runtime parameter access support to hypfs

2020-05-14 Thread Jürgen Groß
On 14.05.20 17:02, Jan Beulich wrote: On 14.05.2020 16:56, Jürgen Groß wrote: On 14.05.20 14:10, Jan Beulich wrote: On 14.05.2020 13:48, Jürgen Groß wrote: On 14.05.20 12:20, Jan Beulich wrote: On 08.05.2020 17:34, Juergen Gross wrote: --- a/xen/common/grant_table.c +++ b/xen/common/grant_ta

Re: [PATCH v2 1/6] x86/mem-paging: fold p2m_mem_paging_prep()'s main if()-s

2020-05-14 Thread Roger Pau Monné
On Thu, Apr 23, 2020 at 10:37:06AM +0200, Jan Beulich wrote: > The condition of the second can be true only if the condition of the > first was met; the second half of the condition of the second then also > is redundant with an earlier check. Combine them, drop a pointless > local variable, and re

Re: [PATCH] x86: retrieve and log CPU frequency information

2020-05-14 Thread Jan Beulich
On 14.05.2020 17:32, Roger Pau Monné wrote: > On Thu, May 14, 2020 at 03:38:18PM +0200, Jan Beulich wrote: >> On 14.05.2020 15:10, Roger Pau Monné wrote: >>> On Wed, Apr 15, 2020 at 01:55:24PM +0200, Jan Beulich wrote: While from just a single Skylake system it is already clear that we ca

Re: [PATCH v2 1/6] x86/mem-paging: fold p2m_mem_paging_prep()'s main if()-s

2020-05-14 Thread Jan Beulich
On 14.05.2020 17:45, Roger Pau Monné wrote: > On Thu, Apr 23, 2020 at 10:37:06AM +0200, Jan Beulich wrote: >> The condition of the second can be true only if the condition of the >> first was met; the second half of the condition of the second then also >> is redundant with an earlier check. Combin

Re: Error during update_runstate_area with KPTI activated

2020-05-14 Thread Julien Grall
On 14/05/2020 15:28, Bertrand Marquis wrote: Hi, Hi, When executing linux on arm64 with KPTI activated (in Dom0 or in a DomU), I have a lot of walk page table errors like this: (XEN) p2m.c:1890: d1v0: Failed to walk page-table va 0xff837ebe0cd0 After implementing a call trace, I fou

Re: [PATCH v2 2/6] x86/mem-paging: correct p2m_mem_paging_prep()'s error handling

2020-05-14 Thread Roger Pau Monné
On Thu, Apr 23, 2020 at 10:37:46AM +0200, Jan Beulich wrote: > Communicating errors from p2m_set_entry() to the caller is not enough: > Neither the M2P nor the stats updates should occur in such a case. > Instead the allocated page needs to be freed again; for cleanliness > reasons also properly ta

Re: [PATCH v5 00/21] Add support for qemu-xen runnning in a Linux-based stubdomain

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 00/21] Add support for qemu-xen runnning in a Linux-based stubdomain"): > In coordination with Marek, I'm making a submission of his patches for Linux > stubdomain device-model support. I made a few of my own additions, but Marek > did the heavy lifting. Thank yo

Re: [PATCH v5 01/21] Document ioemu MiniOS stubdomain protocol

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 01/21] Document ioemu MiniOS stubdomain protocol"): > From: Marek Marczykowski-Górecki > > Add documentation based on reverse-engineered toolstack-ioemu stubdomain > protocol. > > Signed-off-by: Marek Marczykowski-Górecki > Signed-off-by: Jason Andryuk Acked-

Re: [PATCH v5 02/21] Document ioemu Linux stubdomain protocol

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 02/21] Document ioemu Linux stubdomain protocol"): > From: Marek Marczykowski-Górecki > > Add documentation for upcoming Linux stubdomain for qemu-upstream. Acked-by: Ian Jackson This probably shouldn't be committed except with the implmeentation :-). Ian.

Re: [PATCH v5 04/21] libxl: Allow running qemu-xen in stubdomain

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 04/21] libxl: Allow running qemu-xen in stubdomain"): > From: Marek Marczykowski-Górecki > > Do not prohibit anymore using stubdomain with qemu-xen. > To help distingushing MiniOS and Linux stubdomain, add helper inline > functions libxl__stubdomain_is_linux() an

Re: [PATCH v2 3/6] x86/mem-paging: use guest handle for XENMEM_paging_op_prep

2020-05-14 Thread Roger Pau Monné
On Thu, Apr 23, 2020 at 10:38:18AM +0200, Jan Beulich wrote: > While it should have been this way from the beginning, not doing so will > become an actual problem with PVH Dom0. The interface change is binary > compatible, but requires tools side producers to be re-built. > > Drop the bogus/unnece

Re: Error during update_runstate_area with KPTI activated

2020-05-14 Thread Bertrand Marquis
> On 14 May 2020, at 16:57, Julien Grall wrote: > > > > On 14/05/2020 15:28, Bertrand Marquis wrote: >> Hi, > > Hi, > >> When executing linux on arm64 with KPTI activated (in Dom0 or in a DomU), I >> have a lot of walk page table errors like this: >> (XEN) p2m.c:1890: d1v0: Failed to walk

Re: [PATCH v5 05/21] libxl: Handle Linux stubdomain specific QEMU options.

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 05/21] libxl: Handle Linux stubdomain specific QEMU options."): > From: Eric Shelton > > This patch creates an appropriate command line for the QEMU instance > running in a Linux-based stubdomain. > > NOTE: a number of items are not currently implemented for Lin

Re: [PATCH v2 4/6] x86/mem-paging: add minimal lock order enforcement to p2m_mem_paging_prep()

2020-05-14 Thread Roger Pau Monné
On Thu, Apr 23, 2020 at 10:38:44AM +0200, Jan Beulich wrote: > While full checking is impossible (as the lock is being acquired/ > released down the call tree), perform at least a lock level check. I'm slightly confused, doesn't alloc_domheap_page already have it's own lock order checking? Thanks

Re: [PATCH v5 06/21] libxl: write qemu arguments into separate xenstore keys

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 06/21] libxl: write qemu arguments into separate xenstore keys"): > +static int libxl__write_stub_linux_dmargs(libxl__gc *gc, > +int dm_domid, int guest_domid, > +char **args) > +{ ... > +v

Re: [PATCH v5 07/21] xl: add stubdomain related options to xl config parser

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 07/21] xl: add stubdomain related options to xl config parser"): > From: Marek Marczykowski-Górecki > > Signed-off-by: Marek Marczykowski-Górecki > Reviewed-by: Jason Andryuk > Signed-off-by: Jason Andryuk Acked-by: Ian Jackson

Re: [PATCH v5 08/21] tools/libvchan: notify server when client is connected

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 08/21] tools/libvchan: notify server when client is connected"): > From: Marek Marczykowski-Górecki > > Let the server know when the client is connected. Otherwise server will > notice only when client send some data. > This change does not break existing clients

Re: [PATCH v2 5/6] x86/mem-paging: move code to its dedicated source file

2020-05-14 Thread Roger Pau Monné
On Thu, Apr 23, 2020 at 10:39:17AM +0200, Jan Beulich wrote: > Do a little bit of style adjustment along the way, and drop the > "p2m_mem_paging_" prefixes from the now static functions. > > Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné Thanks, Roger.

Re: [PATCH v5 09/21] libxl: add save/restore support for qemu-xen in stubdomain

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 09/21] libxl: add save/restore support for qemu-xen in stubdomain"): > From: Marek Marczykowski-Górecki > > Rely on a wrapper script in stubdomain to attach FD 3/4 of qemu to > relevant consoles. > > Signed-off-by: Marek Marczykowski-Górecki > Address TODO in d

Re: [PATCH v5 10/21] tools: add missing libxenvchan cflags

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 10/21] tools: add missing libxenvchan cflags"): > From: Marek Marczykowski-Górecki > > libxenvchan.h include xenevtchn.h and xengnttab.h, so applications built > with it needs applicable -I in CFLAGS too. Acked-by: Ian Jackson

Re: [PATCH v2 6/6] x86/mem-paging: consistently use gfn_t

2020-05-14 Thread Roger Pau Monné
On Thu, Apr 23, 2020 at 10:39:39AM +0200, Jan Beulich wrote: > Where gprintk()s get touched anyway to switch to PRI_gfn, also switch to > %pd for the domain logged. > > Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné Thanks, Roger.

Re: [PATCH v5 11/21] tools: add simple vchan-socket-proxy

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 11/21] tools: add simple vchan-socket-proxy"): > From: Marek Marczykowski-Górecki > > Add a simple proxy for tunneling socket connection over vchan. This is > based on existing vchan-node* applications, but extended with socket > support. vchan-socket-proxy serves

Re: [PATCH v5 12/21] libxl: use vchan for QMP access with Linux stubdomain

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 12/21] libxl: use vchan for QMP access with Linux stubdomain"): > From: Marek Marczykowski-Górecki > > Access to QMP of QEMU in Linux stubdomain is possible over vchan > connection. Handle the actual vchan connection in a separate process > (vchan-socket-proxy).

Re: [PATCH v5 13/21] Regenerate autotools files

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 13/21] Regenerate autotools files"): > From: Marek Marczykowski-Górecki > > Since we have those generated files committed to the repo (why?!), > update them after changing configure.ac. This should be folded into the patch that changes the input. Can you please

Re: [PATCH v5 14/21] libxl: require qemu in dom0 even if stubdomain is in use

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 14/21] libxl: require qemu in dom0 even if stubdomain is in use"): > From: Marek Marczykowski-Górecki > > Until xenconsoled learns how to handle multiple consoles, this is needed > for save/restore support (qemu state is transferred over secondary > consoles). >

Re: [PATCH v5 15/21] libxl: ignore emulated IDE disks beyond the first 4

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 15/21] libxl: ignore emulated IDE disks beyond the first 4"): > From: Marek Marczykowski-Górecki > > Qemu supports only 4 emulated IDE disks, when given more (or with higher > indexes), it will fail to start. Since the disks can still be accessible > using PV int

Re: [PATCH v5 16/21] libxl: consider also qemu in stubdomain in libxl__dm_active check

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 16/21] libxl: consider also qemu in stubdomain in libxl__dm_active check"): > From: Marek Marczykowski-Górecki > > Since qemu-xen can now run in stubdomain too, handle this case when > checking it's state too. Acked-by: Ian Jackson

Re: [PATCH v5 17/21] docs: Add device-model-domid to xenstore-paths

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 17/21] docs: Add device-model-domid to xenstore-paths"): > Document device-model-domid for when using a device model stubdomain. > Acked-by: Ian Jackson

Re: [PATCH v5 19/21] libxl: Refactor kill_device_model to libxl__kill_xs_path

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 19/21] libxl: Refactor kill_device_model to libxl__kill_xs_path"): > Move kill_device_model to libxl__kill_xs_path so we have a helper to > kill a process from a pid stored in xenstore. We'll be using it to kill > vchan-qmp-proxy. > > libxl__kill_xs_path takes a

Re: [PATCH v5 18/21] libxl: Check stubdomain kernel & ramdisk presence

2020-05-14 Thread Ian Jackson
Jason Andryuk writes ("[PATCH v5 18/21] libxl: Check stubdomain kernel & ramdisk presence"): > Just out of context is the following comment for libxl__domain_make: > /* fixme: this function can leak the stubdom if it fails */ > > When the stubdomain kernel or ramdisk is not present, the domid and

  1   2   >