[Xen-devel] [linux-next test] 128580: regressions - FAIL

2018-10-12 Thread osstest service owner
flight 128580 linux-next real [real]
http://logs.test-lab.xenproject.org/osstest/logs/128580/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-boot fail REGR. vs. 128493
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 
128493
 test-amd64-amd64-xl-credit2   7 xen-boot fail REGR. vs. 128493
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-boot fail REGR. vs. 128493
 test-amd64-i386-freebsd10-i386  7 xen-boot   fail REGR. vs. 128493
 test-amd64-i386-xl-qemuu-win10-i386  7 xen-boot  fail REGR. vs. 128493
 test-amd64-amd64-qemuu-nested-intel  7 xen-boot  fail REGR. vs. 128493
 test-amd64-amd64-xl-qemut-ws16-amd64  7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  7 xen-bootfail REGR. vs. 128493
 test-amd64-amd64-xl-qemuu-ws16-amd64  7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-xl-qemut-win7-amd64  7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-i386-pvgrub  7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 7 xen-boot fail REGR. vs. 
128493
 test-amd64-i386-libvirt   7 xen-boot fail REGR. vs. 128493
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 
128493
 test-amd64-amd64-xl-qemuu-ovmf-amd64  7 xen-boot fail REGR. vs. 128493
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-boot  fail REGR. vs. 128493
 test-amd64-amd64-xl-shadow7 xen-boot fail REGR. vs. 128493
 test-amd64-i386-freebsd10-amd64  7 xen-boot  fail REGR. vs. 128493
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-boot fail REGR. vs. 
128493
 test-amd64-i386-xl-raw7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-qemuu-nested-amd  7 xen-bootfail REGR. vs. 128493
 test-amd64-amd64-xl-qemut-win10-i386  7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-xl-pvhv2-amd  7 xen-bootfail REGR. vs. 128493
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 
128493
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-boot  fail REGR. vs. 128493
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-boot   fail REGR. vs. 128493
 test-amd64-amd64-xl-xsm   7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-xl   7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-boot fail REGR. 
vs. 128493
 test-amd64-amd64-libvirt  7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-xl-pvshim7 xen-boot fail REGR. vs. 128493
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  7 xen-boot fail REGR. vs. 128493
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-boot fail REGR. vs. 
128493
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-boot  fail REGR. vs. 128493
 test-amd64-i386-libvirt-xsm   7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-xl-qemut-debianhvm-amd64  7 xen-bootfail REGR. vs. 128493
 test-amd64-amd64-xl-qemuu-win10-i386  7 xen-boot fail REGR. vs. 128493
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-boot  fail REGR. vs. 128493
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-boot fail REGR. 
vs. 128493
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-boot   fail REGR. vs. 128493
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-boot  fail REGR. vs. 128493
 test-amd64-i386-xl-pvshim 7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 128493
 test-amd64-amd64-amd64-pvgrub  7 xen-bootfail REGR. vs. 128493
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm  7 xen-boot fail REGR. vs. 128493

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-rtds  7 xen-boot fail  like 128493
 test-amd64-amd64-xl-qcow2 7 xen-boot fail  like 128493
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-boot  fail like 128493
 test-amd64-i386-rumprun-i386  7 xen-boot fail  like 128493
 test-amd64-amd64-xl-multivcpu  7 xen-boot fail like 128493
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm  7 xen-boot fail like 128493
 test-amd64-i386-examine   8 reboot   fail  like 128493
 test-amd64-amd64-libvirt-xsm  7 xen-boot fail  like 128493
 test-amd64-i386-pair 10 xen-boot/src_hostfail  like 128493
 test-amd64-i386-pair 11 xen-boot/dst_hostfail  like 128493
 test-amd64-i386-xl7 xen-boot fail  like 128493
 test-amd64-i386-xl-qemut-win10-i386  7 xen-boot   fail like 128493
 test-amd64-amd64-xl-credit1   7 xen-boot

Re: [Xen-devel] [RFC PATCH v1 00/16] xen: sched: implement core-scheduling

2018-10-12 Thread Dario Faggioli
On Fri, 2018-10-12 at 07:15 +0200, Juergen Gross wrote:
> On 11/10/2018 19:37, Dario Faggioli wrote:
> > 
> > So, for example:
> > - domain A has vCore0 and vCore1
> > - each vCore has 2 threads ({vCore0.0, vCore0.1} and
> >   {vCore1.0, vCore1.1})
> > - we're on a 2-way SMT host
> > - vCore1 is running on physical core 3 on the host
> > - more specifically, vCore1.0 is currently executing on thread 0 of
> >   physical core 3 of the host, and vCore1.1 is currently executing
> > on
> >   thread 1 of core 3 of the host
> > - say that both vCore1.0 and vCore1.1 are in guest context
> > 
> > Now:
> > * vCore1.0 blocks. What happens?
> 
> It is going to vBlocked (the physical thread is sitting in the
> hypervisor waiting for either a (core-)scheduling event or for
> unblocking vCore1.0). vCore1.1 keeps running. Or, if vCore1.1
> is already vIdle/vBlocked, vCore1 is switching to blocked and the
> scheduler is looking for another vCore to schedule on the physical
> core.
> 
Ok. And then we'll have one thread in guest context, and one thread in
Xen (albeit, idle, in this case). In these other cases...

> > * vCore1.0 makes an hypercall. What happens?
> 
> Same as today. The hypercall is being executed.
> 
> > * vCore1.0 VMEXITs. What happens?
> 
> Same as today. The VMEXIT is handled.
> 
... we have one thread in guest context, and one thread in Xen, and the
one in Xen is not just staying idle, it's doing hypercalls and VMEXIT
handling.

> In case you referring to a potential rendezvous for e.g. L1TF
> mitigation: this would be handled scheduler agnostic.
> 
Yes, that was what I was thinking to. I.e., in order to be able to use
core-scheduling as a _fully_effective_ mitigation for stuff like L1TF,
we'd need something like that.

In fact, core-scheduling "per-se" mitigates leaks among guests, but if
we want to fully avoid for two threads to ever be in different security
contexts (like one in guest and one in Xen, to prevent Xen data leaking
to a guest), we do need some kind of synchronized Xen enters/exits,
AFAIUI.

What I'm trying to understand right now, is whether implementing things
in this way you're proposing, would help achieving that. And what I've
understood so far is that, no it doesn't.

The main difference between the two approaches would be that we
implement it once in schedule.c, for all schedulers. But this, I see it
as something having both up and down sides (yeah, like everything on
Earth, I know! :-P). More on this later.

> > All in all, I like the idea, because it is about introducing nice
> > abstractions, it is general, etc., but it looks like a major rework
> > of
> > the scheduler.
> 
> Correct. Finally something to do :-p
> 
Indeed! :-)

> > Note that, while this series which tries to implement core-
> > scheduling
> > for Credit1 is rather long and messy, doing the same (and with a
> > similar approach) for Credit2 is a lot easier and nicer. I have it
> > almost ready, and will send it soon.
> 
> Okay, but would it keep vThreads of the same vCore let always running
> together on the same physical core?
> 
It doesn't right now, as we don't have a way to expose such information
to the guest, yet. And since without such a mechanism, the guest can't
take advantage of something like this (neither from a performance nor
from a vuln. mitigation point of view), I kept that out.

But I certainly can see about making it do so (I was already planning
to).

> > Right. But again, in Credit2, I've been able to implement socket-
> > wise
> > coscheduling with this approach (I mean, an approach similar to the
> > one
> > in this series, but adapted to Credit2).
> 
> And then there still is sched_rt.c
> 
Ok, so I think this is the main benefit of this approach. We do the
thing once, and all schedulers get core-scheduling (or whatever
granularity of group scheduling we implement/allow).

But how easy it is to opt out, if one doesn't want it? E.g., in the
context of L1TF, what if I'm not affected, and hence am not interested
in core-scheduling? What if I want a cpupool with core-scheduling and
one without?

I may be wrong, but out of the top of my head, but it seems to me that
doing things in schedule.c makes this a lot harder, if possible at all.

Thanks and Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/


signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH v1 00/16] xen: sched: implement core-scheduling

2018-10-12 Thread Juergen Gross
On 12/10/2018 09:49, Dario Faggioli wrote:
> On Fri, 2018-10-12 at 07:15 +0200, Juergen Gross wrote:
>> On 11/10/2018 19:37, Dario Faggioli wrote:
>>>
>>> So, for example:
>>> - domain A has vCore0 and vCore1
>>> - each vCore has 2 threads ({vCore0.0, vCore0.1} and
>>>   {vCore1.0, vCore1.1})
>>> - we're on a 2-way SMT host
>>> - vCore1 is running on physical core 3 on the host
>>> - more specifically, vCore1.0 is currently executing on thread 0 of
>>>   physical core 3 of the host, and vCore1.1 is currently executing
>>> on
>>>   thread 1 of core 3 of the host
>>> - say that both vCore1.0 and vCore1.1 are in guest context
>>>
>>> Now:
>>> * vCore1.0 blocks. What happens?
>>
>> It is going to vBlocked (the physical thread is sitting in the
>> hypervisor waiting for either a (core-)scheduling event or for
>> unblocking vCore1.0). vCore1.1 keeps running. Or, if vCore1.1
>> is already vIdle/vBlocked, vCore1 is switching to blocked and the
>> scheduler is looking for another vCore to schedule on the physical
>> core.
>>
> Ok. And then we'll have one thread in guest context, and one thread in
> Xen (albeit, idle, in this case). In these other cases...
> 
>>> * vCore1.0 makes an hypercall. What happens?
>>
>> Same as today. The hypercall is being executed.
>>
>>> * vCore1.0 VMEXITs. What happens?
>>
>> Same as today. The VMEXIT is handled.
>>
> ... we have one thread in guest context, and one thread in Xen, and the
> one in Xen is not just staying idle, it's doing hypercalls and VMEXIT
> handling.
> 
>> In case you referring to a potential rendezvous for e.g. L1TF
>> mitigation: this would be handled scheduler agnostic.
>>
> Yes, that was what I was thinking to. I.e., in order to be able to use
> core-scheduling as a _fully_effective_ mitigation for stuff like L1TF,
> we'd need something like that.
> 
> In fact, core-scheduling "per-se" mitigates leaks among guests, but if
> we want to fully avoid for two threads to ever be in different security
> contexts (like one in guest and one in Xen, to prevent Xen data leaking
> to a guest), we do need some kind of synchronized Xen enters/exits,
> AFAIUI.
> 
> What I'm trying to understand right now, is whether implementing things
> in this way you're proposing, would help achieving that. And what I've
> understood so far is that, no it doesn't.

This aspect will need about the same effort in both solutions.
Maybe my proposal would make it easier to decide whether such a
rendezvous is needed, as there would be only one instance to ask
(schedule.c) instead of multiple instances (sched_*.c).

> 
> The main difference between the two approaches would be that we
> implement it once in schedule.c, for all schedulers. But this, I see it
> as something having both up and down sides (yeah, like everything on
> Earth, I know! :-P). More on this later.
> 
>>> All in all, I like the idea, because it is about introducing nice
>>> abstractions, it is general, etc., but it looks like a major rework
>>> of
>>> the scheduler.
>>
>> Correct. Finally something to do :-p
>>
> Indeed! :-)
> 
>>> Note that, while this series which tries to implement core-
>>> scheduling
>>> for Credit1 is rather long and messy, doing the same (and with a
>>> similar approach) for Credit2 is a lot easier and nicer. I have it
>>> almost ready, and will send it soon.
>>
>> Okay, but would it keep vThreads of the same vCore let always running
>> together on the same physical core?
>>
> It doesn't right now, as we don't have a way to expose such information
> to the guest, yet. And since without such a mechanism, the guest can't
> take advantage of something like this (neither from a performance nor
> from a vuln. mitigation point of view), I kept that out.
> 
> But I certainly can see about making it do so (I was already planning
> to).
> 
>>> Right. But again, in Credit2, I've been able to implement socket-
>>> wise
>>> coscheduling with this approach (I mean, an approach similar to the
>>> one
>>> in this series, but adapted to Credit2).
>>
>> And then there still is sched_rt.c
>>
> Ok, so I think this is the main benefit of this approach. We do the
> thing once, and all schedulers get core-scheduling (or whatever
> granularity of group scheduling we implement/allow).
> 
> But how easy it is to opt out, if one doesn't want it? E.g., in the
> context of L1TF, what if I'm not affected, and hence am not interested
> in core-scheduling? What if I want a cpupool with core-scheduling and
> one without?
> 
> I may be wrong, but out of the top of my head, but it seems to me that
> doing things in schedule.c makes this a lot harder, if possible at all.

Why? This would be a per-cpupool setting, so the scheduling granularity
would live in struct scheduler.


Juergen


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH v1 00/16] xen: sched: implement core-scheduling

2018-10-12 Thread Dario Faggioli
On Fri, 2018-10-12 at 10:35 +0200, Juergen Gross wrote:
> On 12/10/2018 09:49, Dario Faggioli wrote:
> > 
> > But how easy it is to opt out, if one doesn't want it? E.g., in the
> > context of L1TF, what if I'm not affected, and hence am not
> > interested
> > in core-scheduling? What if I want a cpupool with core-scheduling
> > and
> > one without?
> > 
> > I may be wrong, but out of the top of my head, but it seems to me
> > that
> > doing things in schedule.c makes this a lot harder, if possible at
> > all.
> 
> Why? This would be a per-cpupool setting, so the scheduling
> granularity
> would live in struct scheduler.
> 
Mmm... it's already starting to be a bit hard to reason, without
looking at at least a prototype of the code. :-/

But, anyway, if I'm in sched_dario.c, and schedule.c makes me reason in
terms of vCores, how do I deal with single CPUs for a particular
cpupool that does not want core-scheduling?

Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/


signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH v1 00/16] xen: sched: implement core-scheduling

2018-10-12 Thread Juergen Gross
On 12/10/2018 11:15, Dario Faggioli wrote:
> On Fri, 2018-10-12 at 10:35 +0200, Juergen Gross wrote:
>> On 12/10/2018 09:49, Dario Faggioli wrote:
>>>
>>> But how easy it is to opt out, if one doesn't want it? E.g., in the
>>> context of L1TF, what if I'm not affected, and hence am not
>>> interested
>>> in core-scheduling? What if I want a cpupool with core-scheduling
>>> and
>>> one without?
>>>
>>> I may be wrong, but out of the top of my head, but it seems to me
>>> that
>>> doing things in schedule.c makes this a lot harder, if possible at
>>> all.
>>
>> Why? This would be a per-cpupool setting, so the scheduling
>> granularity
>> would live in struct scheduler.
>>
> Mmm... it's already starting to be a bit hard to reason, without
> looking at at least a prototype of the code. :-/
> 
> But, anyway, if I'm in sched_dario.c, and schedule.c makes me reason in
> terms of vCores, how do I deal with single CPUs for a particular
> cpupool that does not want core-scheduling?

sched_dario.c would only know of scheduling entities. Mapping of vcpus
to scheduling entities is part of the infrastructure.

schedule.c would receive vcpu state changes. In case such a vcpu
state change leads to a scheduling entity state change the related
scheduler is informed about that to be able to react.

In case the scheduling entity is a thread (no core scheduling) each
vcpu state change will result in a scheduling entity state change,
so it will be as today.


Juergen


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH RFC] VMX: fix vmx_handle_eoi()

2018-10-12 Thread Jan Beulich
In commit 303066fdb1e ("VMX: fix interaction of APIC-V and Viridian
emulation") I screwed up quite heavily: Instead of clearing SVI, RVI was
cleared. Furthermore, unconditional clearing of SVI is wrong too - other
ISR bits should be taken into account.

Introduce a new helper set_svi(), split out of vmx_process_isr(), and
use it also from vmx_handle_eoi().

Following the problems in vmx_intr_assist() (see the still present big
block of debugging code there) also warn (once) if EOI'd vector and
original SVI don't match.

Signed-off-by: Jan Beulich 
---
RFC: Untested, as I didn't see any of the possibly resulting problems
 in any of my environments. But perhaps this is (part of) the
 reason of lost interrupts XenServer folks have been observing.

--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -448,7 +448,7 @@ void vlapic_EOI_set(struct vlapic *vlapi
 vlapic_clear_vector(vector, &vlapic->regs->data[APIC_ISR]);
 
 if ( hvm_funcs.handle_eoi )
-hvm_funcs.handle_eoi(vector);
+hvm_funcs.handle_eoi(vector, vlapic_find_highest_isr(vlapic));
 
 vlapic_handle_EOI(vlapic, vector);
 
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -1941,17 +1941,14 @@ static void vmx_update_eoi_exit_bitmap(s
 vmx_clear_eoi_exit_bitmap(v, vector);
 }
 
-static void vmx_process_isr(int isr, struct vcpu *v)
+static u8 set_svi(int isr)
 {
 unsigned long status;
 u8 old;
-unsigned int i;
-const struct vlapic *vlapic = vcpu_vlapic(v);
 
 if ( isr < 0 )
 isr = 0;
 
-vmx_vmcs_enter(v);
 __vmread(GUEST_INTR_STATUS, &status);
 old = status >> VMX_GUEST_INTR_STATUS_SVI_OFFSET;
 if ( isr != old )
@@ -1961,6 +1958,18 @@ static void vmx_process_isr(int isr, str
 __vmwrite(GUEST_INTR_STATUS, status);
 }
 
+return old;
+}
+
+static void vmx_process_isr(int isr, struct vcpu *v)
+{
+unsigned int i;
+const struct vlapic *vlapic = vcpu_vlapic(v);
+
+vmx_vmcs_enter(v);
+
+set_svi(isr);
+
 /*
  * Theoretically, only level triggered interrupts can have their
  * corresponding bits set in the eoi exit bitmap. That is, the bits
@@ -2111,14 +2120,13 @@ static bool vmx_test_pir(const struct vc
 return pi_test_pir(vec, &v->arch.hvm.vmx.pi_desc);
 }
 
-static void vmx_handle_eoi(u8 vector)
+static void vmx_handle_eoi(uint8_t vector, int isr)
 {
-unsigned long status;
+u8 old_svi = set_svi(isr);
+static bool warned;
 
-/* We need to clear the SVI field. */
-__vmread(GUEST_INTR_STATUS, &status);
-status &= VMX_GUEST_INTR_STATUS_SUBFIELD_BITMASK;
-__vmwrite(GUEST_INTR_STATUS, status);
+if ( vector != old_svi && !test_and_set_bool(warned) )
+printk(XENLOG_WARNING "EOI for %02x but SVI=%02x\n", vector, old_svi);
 }
 
 static void vmx_enable_msr_interception(struct domain *d, uint32_t msr)
--- a/xen/include/asm-x86/hvm/hvm.h
+++ b/xen/include/asm-x86/hvm/hvm.h
@@ -201,7 +201,7 @@ struct hvm_function_table {
 void (*deliver_posted_intr)(struct vcpu *v, u8 vector);
 void (*sync_pir_to_irr)(struct vcpu *v);
 bool (*test_pir)(const struct vcpu *v, uint8_t vector);
-void (*handle_eoi)(u8 vector);
+void (*handle_eoi)(uint8_t vector, int isr);
 
 /*Walk nested p2m  */
 int (*nhvm_hap_walk_L1_p2m)(struct vcpu *v, paddr_t L2_gpa,





___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [distros-debian-jessie test] 75400: trouble: blocked/broken

2018-10-12 Thread Platform Team regression test user
flight 75400 distros-debian-jessie real [real]
http://osstest.xensource.com/osstest/logs/75400/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf-pvopsbroken
 build-i386   broken
 build-amd64-pvopsbroken
 build-armhf  broken
 build-amd64  broken
 build-i386-pvops broken

Tests which did not succeed, but are not blocking:
 test-amd64-i386-i386-jessie-netboot-pvgrub  1 build-check(1)   blocked n/a
 test-amd64-i386-amd64-jessie-netboot-pygrub  1 build-check(1)  blocked n/a
 test-amd64-amd64-i386-jessie-netboot-pygrub  1 build-check(1)  blocked n/a
 test-armhf-armhf-armhf-jessie-netboot-pygrub  1 build-check(1) blocked n/a
 test-amd64-amd64-amd64-jessie-netboot-pvgrub  1 build-check(1) blocked n/a
 build-armhf-pvops 4 host-install(4)  broken like 75355
 build-armhf   4 host-install(4)  broken like 75355
 build-i3864 host-install(4)  broken like 75355
 build-amd64-pvops 4 host-install(4)  broken like 75355
 build-amd64   4 host-install(4)  broken like 75355
 build-i386-pvops  4 host-install(4)  broken like 75355

baseline version:
 flight   75355

jobs:
 build-amd64  broken  
 build-armhf  broken  
 build-i386   broken  
 build-amd64-pvopsbroken  
 build-armhf-pvopsbroken  
 build-i386-pvops broken  
 test-amd64-amd64-amd64-jessie-netboot-pvgrub blocked 
 test-amd64-i386-i386-jessie-netboot-pvgrub   blocked 
 test-amd64-i386-amd64-jessie-netboot-pygrub  blocked 
 test-armhf-armhf-armhf-jessie-netboot-pygrub blocked 
 test-amd64-amd64-i386-jessie-netboot-pygrub  blocked 



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xensource.com/osstest/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [linux-linus test] 128599: regressions - FAIL

2018-10-12 Thread osstest service owner
flight 128599 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/128599/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail REGR. 
vs. 125898
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 10 debian-hvm-install fail 
REGR. vs. 125898
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail 
REGR. vs. 125898
 test-amd64-i386-freebsd10-i386 11 guest-startfail REGR. vs. 125898
 test-amd64-i386-qemuu-rhel6hvm-intel 10 redhat-install   fail REGR. vs. 125898
 test-amd64-i386-qemuu-rhel6hvm-amd 10 redhat-install fail REGR. vs. 125898
 test-amd64-i386-xl-qemuu-win7-amd64 10 windows-install   fail REGR. vs. 125898
 test-amd64-i386-xl-qemuu-ws16-amd64 10 windows-install   fail REGR. vs. 125898
 test-amd64-i386-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail REGR. vs. 
125898
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 125898
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 7 xen-boot fail REGR. vs. 125898
 test-amd64-amd64-rumprun-amd64  7 xen-boot   fail REGR. vs. 125898
 test-amd64-amd64-xl-multivcpu  7 xen-bootfail REGR. vs. 125898
 test-amd64-amd64-pygrub   7 xen-boot fail REGR. vs. 125898
 test-amd64-amd64-xl-pvhv2-intel  7 xen-boot  fail REGR. vs. 125898
 test-amd64-amd64-xl-qemuu-win7-amd64  7 xen-boot fail REGR. vs. 125898
 test-amd64-i386-examine   8 reboot   fail REGR. vs. 125898
 test-amd64-amd64-libvirt-vhd  7 xen-boot fail REGR. vs. 125898
 test-amd64-i386-libvirt   7 xen-boot fail REGR. vs. 125898
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-boot fail REGR. vs. 125898
 test-amd64-i386-xl-xsm7 xen-boot fail REGR. vs. 125898
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-boot fail REGR. vs. 125898
 test-amd64-amd64-libvirt-xsm  7 xen-boot fail REGR. vs. 125898
 test-amd64-i386-xl-shadow 7 xen-boot fail REGR. vs. 125898
 test-amd64-i386-xl-qemut-win10-i386  7 xen-boot  fail REGR. vs. 125898
 test-amd64-i386-xl7 xen-boot fail REGR. vs. 125898
 test-amd64-i386-rumprun-i386  7 xen-boot fail REGR. vs. 125898
 test-amd64-amd64-xl-qcow2 7 xen-boot fail REGR. vs. 125898
 test-amd64-i386-freebsd10-amd64 11 guest-start   fail REGR. vs. 125898
 test-amd64-amd64-examine  8 reboot   fail REGR. vs. 125898
 test-armhf-armhf-xl-credit2   7 xen-boot fail REGR. vs. 125898

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds  7 xen-boot fail REGR. vs. 125898

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-credit1   7 xen-bootfail baseline untested
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 125898
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 125898
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 125898
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 125898
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 125898
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 125898
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-ch

[Xen-devel] [libvirt test] 128603: tolerable all pass - PUSHED

2018-10-12 Thread osstest service owner
flight 128603 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/128603/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 128521
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 128521
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-qcow2 12 migrate-support-checkfail never pass
 test-arm64-arm64-libvirt-qcow2 13 saverestore-support-checkfail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass

version targeted for testing:
 libvirt  1dbf6222dd5e1fedcbe335fc0852ef66e7a92901
baseline version:
 libvirt  0d981bcefcb5defa27c208a976165c7eb9cda56a

Last test of basis   128521  2018-10-09 04:19:04 Z3 days
Testing same since   128603  2018-10-10 17:19:03 Z1 days1 attempts


People who touched revisions under test:
  Bjoern Walk 
  Ján Tomko 
  Marc Hartmayer 
  Michal Privoznik 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-arm64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-arm64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-arm64-arm64-libvirt-xsm pass
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-arm64-arm64-libvirt pass
 test-armhf-armhf-libvirt pass
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-arm64-arm64-libvirt-qcow2   pass
 test-armhf-armhf-libvirt-raw pass
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/libvirt.git
   0d981bcefc..1dbf6222dd  1dbf6222dd5e1fedcbe335fc0852ef66e7a92901 -> 
xen-tested-master

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH RFC] VMX: fix vmx_handle_eoi()

2018-10-12 Thread Jan Beulich
>>> On 12.10.18 at 11:32,  wrote:
> In commit 303066fdb1e ("VMX: fix interaction of APIC-V and Viridian
> emulation") I screwed up quite heavily: Instead of clearing SVI, RVI was
> cleared.

I was wrong here:

> @@ -2111,14 +2120,13 @@ static bool vmx_test_pir(const struct vc
>  return pi_test_pir(vec, &v->arch.hvm.vmx.pi_desc);
>  }
>  
> -static void vmx_handle_eoi(u8 vector)
> +static void vmx_handle_eoi(uint8_t vector, int isr)
>  {
> -unsigned long status;
> +u8 old_svi = set_svi(isr);
> +static bool warned;
>  
> -/* We need to clear the SVI field. */
> -__vmread(GUEST_INTR_STATUS, &status);
> -status &= VMX_GUEST_INTR_STATUS_SUBFIELD_BITMASK;

This indeed kept RVI, but wrongly masked off bits 16 and up. But
that's benign as long as those bits don#t have a meaning anyway.

The second aspect - ignoring ISR bits - still holds, so the change
itself is still needed afaict. Just the first paragraph of the
description would need to change.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [freebsd-master bisection] complete build-amd64-xen-freebsd

2018-10-12 Thread osstest service owner
branch xen-unstable
xenbranch xen-unstable
job build-amd64-xen-freebsd
testid xen-build-freebsd

Tree: freebsd git://github.com/freebsd/freebsd.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  freebsd git://github.com/freebsd/freebsd.git
  Bug introduced:  fcf5119e8342a641960ca12bd4af98e1174d2817
  Bug not present: 95afcc1f4d12c3c3093d17a2fc4840a80b68996b
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/128666/


  commit fcf5119e8342a641960ca12bd4af98e1174d2817
  Merge: 46d1f3a998f 95afcc1f4d1
  Author: gjb 
  Date:   Fri Oct 5 17:53:47 2018 +
  
  MFH r338661 through r339200.
  
  Sponsored by:   The FreeBSD Foundation
  
  commit 46d1f3a998f71e6e35c685e92b760f5488d6c82a
  Author: jhb 
  Date:   Fri Oct 5 16:35:24 2018 +
  
  Update the existing heimdal implementation for OpenSSL 1.1.
  
  Existing work is underway to import a newer version of heimdal, but
  this patchset gets us to a fully working tree to enable more wide
  spread testing of OpenSSL 1.1 for now.
  
  I've also enabled WARNS=1 for kerberos (which is the reason for the
  change in libroken).  Having -Werror enabled was useful during the
  1.1 updates and we probably should have warnings enabled by default
  for kerberos anyway.
  
  This passes make tinderbox, and I have also done some very light
  runtime testing on amd64.
  
  Reviewed by:bjk, jkim, emaste
  Differential Revision:  https://reviews.freebsd.org/D17276
  
  commit e0d48e3a143aa236ff9e07ca350f1504a8e01bdb
  Author: emaste 
  Date:   Wed Oct 3 16:38:36 2018 +
  
  openssh: connect libressl-api-compat.c and regen config.h
  
  Differential Revision:  https://reviews.freebsd.org/D17390
  
  commit 7921dde60d5e4521e83aafc84157a65c73851a0f
  Author: emaste 
  Date:   Wed Oct 3 16:06:17 2018 +
  
  openssh: add openbsd-compat/libressl-api-compat.c
  
  Missed in migrating changeset from git to svn for r338811
  
  Reported by:jhb
  
  commit 3b1a96ee16f9b9adfe04f236b20f5e3687ec4974
  Author: jhb 
  Date:   Tue Oct 2 21:40:57 2018 +
  
  Update obsolete files list for OpenSSL 1.1.1.
  
  This will need a real date once this is merged to head.
  
  One weird thing to note: the 32-bit engines get dumped into /usr/lib32
  rather than /usr/lib32/engines, and I bet the 32-bit libcrypto.so i
  looking for the .so files in the wrong place.  We should probably fix
  both of those at some point.
  
  Reviewed by:emaste, jkim
  Differential Revision:  https://reviews.freebsd.org/D17384
  
  commit 23093832969ea9837d452d40b2121cf76debf487
  Author: jkim 
  Date:   Mon Oct 1 20:55:01 2018 +
  
  Make sendmail work with OpenSSL 1.1 API.  Taken from the ports tree.
  
  
https://svnweb.freebsd.org/ports/head/mail/sendmail/files/patch-tls.c?revision=466240
  
  Requested by:   gshapiro
  
  commit bad3dbcb47b5bbd5da073dcf0254395206fddb04
  Author: jkim 
  Date:   Mon Oct 1 20:51:26 2018 +
  
  Revert r338773.  A patch from the ports tree will be committed.
  
  Requested by:   gshapiro
  
  commit 683d164a600bd265966d3955e29f87c9200eb7bd
  Author: jkim 
  Date:   Mon Oct 1 18:16:36 2018 +
  
  Drop pre-AVX toolchain for amd64 and i386 to simplify the makefile.
  
  Especially, head does not support old toolchains because of ifunc support.
  
  commit a178e72a82102e8a7b617870131622ae95edd7a0
  Author: jkim 
  Date:   Tue Sep 25 22:21:36 2018 +
  
  Remove MD dirdeps from Makefile.depend.
  
  It can't be right. :-(
  
  commit e4b73ece31f1d11ab07c2179dfcc4c938119acb1
  Author: jkim 
  Date:   Tue Sep 25 22:15:47 2018 +
  
  Make it more meta mode friendly.
  
  commit 6ac49d7d55e60ae1d1081c3ac55bc5a0b5be4eeb
  Author: jkim 
  Date:   Tue Sep 25 22:14:52 2018 +
  
  Fix CLEANFILES.
  
  commit 2ef0b644bd6b4da7990af5da33ed55ca6040a499
  Author: jkim 
  Date:   Tue Sep 25 21:12:36 2018 +
  
  Regen Makefile.depend.
  
  commit 7becfdd73975ba028350cb8655536432e3718174
  Author: emaste 
  Date:   Tue Sep 25 17:41:48 2018 +
  
  libevent: eliminate in-tree usage of arc4random_addrandom
  
  Apply r338059 to newly-added libevent 2.1.18.
  
  Sponsored by:   The FreeBSD Foundation
  
  commit 89758c50f6dce9f9071938dacf253f99d052154b
  Author: emaste 
  Date:   Mon Sep 24 17:51:56 2018 +
  
  Switch ntp's embedded libevent to 2.1.18
  
  For OpenSSL 1.1.1 compatibility.
  
  Sponsored by:   The FreeBSD Foundation.
  
  commit 2bdba85773226bdea53f9c5ced1595d9e111c8cc
  Merge: ae332003d31 eac58b99dde
  Author: emaste 
  Date:   Mon Sep 24 16:48:54 2018 +
  
  Copy libevent sources to contrib
  
  To replace the libevent embedded in ntp, for Op

[Xen-devel] [freebsd-master test] 128662: regressions - trouble: blocked/fail

2018-10-12 Thread osstest service owner
flight 128662 freebsd-master real [real]
http://logs.test-lab.xenproject.org/osstest/logs/128662/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-freebsd   7 freebsd-buildfail REGR. vs. 128497

Tests which did not succeed, but are not blocking:
 build-amd64-xen-freebsd   1 build-check(1)   blocked  n/a
 build-amd64-freebsd-again 1 build-check(1)   blocked  n/a

version targeted for testing:
 freebsd  0a0171cbae68744e7831380eba9705b28bfba3a4
baseline version:
 freebsd  c0b412ce93b9d3ee750e5f262b50e64c52d300cc

Last test of basis   128497  2018-10-08 09:19:52 Z4 days
Failing since128582  2018-10-10 09:19:25 Z2 days2 attempts
Testing same since   128662  2018-10-12 09:18:52 Z0 days1 attempts


People who touched revisions under test:
  0mp <0...@freebsd.org>
  allanjude 
  brooks 
  des 
  egypcio 
  emaste 
  gjb 
  hselasky 
  jhb 
  jkim 
  jtl 
  kevans 
  kib 
  mav 
  mjg 
  mw 
  nwhitehorn 
  peter 
  rmacklem 
  shurd 
  yuripv 

jobs:
 build-amd64-freebsd-againblocked 
 build-amd64-freebsd  fail
 build-amd64-xen-freebsd  blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 1394 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf test] 128655: all pass - PUSHED

2018-10-12 Thread osstest service owner
flight 128655 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/128655/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 7177be0bd8d372ef7eaa86df538bd45ec841ed23
baseline version:
 ovmf 8122c6bc8b6f1fde31f2af6c1560ec552204980d

Last test of basis   128632  2018-10-11 09:56:33 Z1 days
Testing same since   128655  2018-10-12 04:10:52 Z0 days1 attempts


People who touched revisions under test:
  Jim Dailey 
  jim.dai...@dell.com 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   8122c6bc8b..7177be0bd8  7177be0bd8d372ef7eaa86df538bd45ec841ed23 -> 
xen-tested-master

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] x86/hvm/ioreq: allow ioreq servers to use HVM_PARAM_[BUF]IOREQ_PFN

2018-10-12 Thread Jan Beulich
>>> On 09.10.18 at 10:25,  wrote:
> Since commit 2c257bd6 "x86/hvm: remove default ioreq server (again)" the
> GFNs allocated by the toolstack and set in HVM_PARAM_IOREQ_PFN and
> HVM_PARAM_BUFIOREQ_PFN have been unused. This patch allows them to be used
> by (non-default) ioreq servers.
> 
> While in the area, also make sure HVM_PARAM_[BUF]IOREQ_PFN can only be set
> once. These parameters should have always been in the 'set once' category
> but this has, so far, not been enforced.
> 
> NOTE: This fixes a compatibility issue. A guest created on a version of
>   Xen that pre-dates the initial ioreq server implementation and then
>   migrated in will currently fail to resume because its migration
>   stream will lack values for HVM_PARAM_IOREQ_SERVER_PFN and
>   HVM_PARAM_NR_IOREQ_SERVER_PAGES *unless* the system has an
>   emulator domain that uses direct resource mapping (which depends
>   on the version of privcmd it happens to have) in which case it
>   will not require use of GFNs for the ioreq server shared
>   pages.
> 
> Signed-off-by: Paul Durrant 

Reviewed-by: Jan Beulich 



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] mm/page_alloc: make bootscrub happen in idle-loop

2018-10-12 Thread Jan Beulich
>>> On 09.10.18 at 17:21,  wrote:
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -161,8 +161,42 @@ string_param("badpage", opt_badpage);
>  /*
>   * no-bootscrub -> Free pages are not zeroed during boot.
>   */
> -static bool_t opt_bootscrub __initdata = 1;
> -boolean_param("bootscrub", opt_bootscrub);
> +enum bootscrub_mode {
> +BOOTSCRUB_OFF = 0,

The "= 0" is pointless.

> @@ -2039,8 +2077,24 @@ void __init heap_init_late(void)
>   */
>  setup_low_mem_virq();
>  
> -if ( opt_bootscrub )
> +switch ( opt_bootscrub )
> +{
> +default:
> +ASSERT_UNREACHABLE();
> +/* Fall through */
> +
> +case BOOTSCRUB_IDLE:
> +printk("Scrubbing Free RAM on %d nodes in background\n",
> +   num_online_nodes());

Still the question whether this printk(), and in particular the inclusion
of the node count, is meaningful in any way. Other than this
Reviewed-by: Jan Beulich 
and one or both changes would be easy enough to make while
committing, provided we can reach agreement.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 03/17] x86: turn is_pv_{, 32bit_}{domain, vcpu} into inline functions

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -877,8 +877,25 @@ void watchdog_domain_destroy(struct domain *d);
>  
>  #define VM_ASSIST(d, t) (test_bit(VMASST_TYPE_ ## t, &(d)->vm_assist))
>  
> -#define is_pv_domain(d) ((d)->guest_type == guest_type_pv)
> -#define is_pv_vcpu(v)   (is_pv_domain((v)->domain))
> +static inline bool is_pv_domain(const struct domain *d)
> +{
> +return IS_ENABLED(CONFIG_PV) ? d->guest_type == guest_type_pv : false;
> +}
> +
> +static inline bool is_pv_vcpu(const struct vcpu *v)
> +{
> +return is_pv_domain(v->domain);
> +}
> +
> +static inline bool is_pv_32bit_domain(const struct domain *d)
> +{
> +return is_pv_domain(d) && d->arch.is_32bit_pv;
> +}
> +
> +static inline bool is_pv_32bit_vcpu(const struct vcpu *v)
> +{
> +return is_pv_32bit_domain(v->domain);
> +}

Afaict this breaks the Arm build, i.e. I think you need e.g.
#ifdef CONFIG_COMPAT around the last two. (I can see why,
being inline functions now, they can't remain in the arch-specific
header.) With this
Reviewed-by: Jan Beulich 

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 04/17] x86: introduce is_pv_64bit_{vcpu, domain}

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> This is useful to rewrite the following pattern (v is PV vcpu)
> 
>if ( is_pv_32bit_vcpu(v) )
>do_foo;
>else
>do_bar;
> 
> to
> 
>if ( is_pv_32bit_vcpu(v) )
>do_foo;
>else if ( is_pv_64bit_vcpu(v) )
>do_bar;
>else
>ASSERT_UNREACHABLE;
> .
> 
> Previously it is not possible to rely on DCE to eliminate the do_bar
> part. It becomes possible with the new code structure.
> 
> Signed-off-by: Wei Liu 

Provided the additions go inside the #ifdef-s mentioned for patch 3
Acked-by: Jan Beulich 

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 05/17] x86: make x86_64/traps.c build with !CONFIG_PV

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> Provide declarations for hypercall_page_initialise_ring*_kernel, make
> sure DCE work as expected.
> 
> Signed-off-by: Wei Liu 

Acked-by: Jan Beulich 



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 0/4] x86/HVM: implement memory read caching

2018-10-12 Thread Andrew Cooper
On 11/09/18 14:10, Jan Beulich wrote:
> Emulation requiring device model assistance uses a form of instruction
> re-execution, assuming that the second (and any further) pass takes
> exactly the same path. This is a valid assumption as far as use of CPU
> registers goes (as those can't change without any other instruction
> executing in between), but is wrong for memory accesses. In particular
> it has been observed that Windows might page out buffers underneath
> an instruction currently under emulation (hitting between two passes).
> If the first pass translated a linear address successfully, any subsequent
> pass needs to do so too, yielding the exact same translation.
>
> Introduce a cache (used just by guest page table accesses for now, i.e.
> a form of "paging structure cache") to make sure above described
> assumption holds. This is a very simplistic implementation for now: Only
> exact matches are satisfied (no overlaps or partial reads or anything).
>
> There's also some seemingly unrelated cleanup here which was found
> desirable on the way.
>
> 1: x86/mm: add optional cache to GLA->GFN translation
> 2: x86/mm: use optional cache in guest_walk_tables()
> 3: x86/HVM: implement memory read caching
> 4: x86/HVM: prefill cache with PDPTEs when possible
>
> "VMX: correct PDPTE load checks" is omitted from v2, as I can't
> currently find enough time to carry out the requested further
> rework.

Following the x86 call, I've had some thoughts and suggestions about how
to make this work in a reasonable way, without resorting to the full
caching approach.

First and foremost, I'd like recommend against trying to combine the fix
for repeated PDPTR reading, and repeated PTE reading.  While they are
both repeated reading problems, one really is a knoblly corner case of
32bit PAE paging, and one is a general emulation problem.  Fixing these
problems independently makes the result rather more simple, and far
closer to how real CPUs work.

For naming, how about "access once" in place of cache?  This is the best
description of the purpose I can come up with.

Next, there should be a single hvmemul_read_once(gaddr, bytes, ...)
(name subject to improvement), which does a transparent read-through of
the "access once cache" in terms of a single flag guest physical address
space.  This allows individual callers to opt into using the access-once
semantics, and doesn't hoist them with the substantial boilerplate of
the sole correct way to use this interface.  Furthermore, this behaviour
has the same semantics as the correct longer term fix.

That alone should fix the windows issue, because there is no chance that
windows will ever page out the PDPTRs.

For the PDPTRs, this corner case is special, and should be handled by
the pagewalk code.  I'm still going to go with my previous suggestion of
having top_map point onto the caller stack.  For the VT-x case, the
values can be pulled straight out of the VMCS, while for AMD, the values
can be read through the "access once cache", which matches the behaviour
of being read from memory, but ensures they won't be repeatedly read.

Overall, I think this should be fairly architecturally clean, solve the
underlying bug, and moves things in the general direction of the longer
term goal, even if it doesn't get all the way there in one step.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 06/17] x86: provide stub for arch_do_multicall_call

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> --- a/xen/arch/x86/hypercall.c
> +++ b/xen/arch/x86/hypercall.c
> @@ -248,6 +248,15 @@ int hypercall_xlat_continuation(unsigned int *id, 
> unsigned int nr,
>  return rc;
>  }
>  
> +#ifndef CONFIG_PV
> +/* Stub for arch_do_multicall_call */
> +enum mc_disposition arch_do_multicall_call(struct mc_state *mc)
> +{
> +return mc_exit;
> +}
> +
> +#endif

Please drop the blank line above here. For the moment I'm fine, so
Acked-by: Jan Beulich 
but I really think HVM should gain multicall support, seeing that ARM
has this as well. At that point, the #ifdef above will start to look a
little funny.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 07/17] x86/traps: put PV code handlers under CONFIG_PV

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> The trap handlers in hypervisor need to forward events to PV guests if
> necessary. If there is no PV guest relevant code should be excluded.
> 
> Put code under CONFIG_PV and add ASSERT_UNREACHABLE.

-ETOOMANYIFDEFS

What's wrong with an inline stub for pv_inject_event()? This won't
eliminate all #ifdef-s, but quite a few of them afaics.

> @@ -1140,6 +1152,7 @@ static int handle_ldt_mapping_fault(unsigned int offset,
>  if ( !guest_mode(regs) )
>  return 0;
>  
> +#ifdef CONFIG_PV
>  /* Access would have become non-canonical? Pass #GP[sel] back. */
>  if ( unlikely(!is_canonical_address(curr->arch.pv.ldt_base + 
> offset)) )
>  {
> @@ -1151,6 +1164,9 @@ static int handle_ldt_mapping_fault(unsigned int offset,
>  /* else pass the #PF back, with adjusted %cr2. */
>  pv_inject_page_fault(regs->error_code,
>   curr->arch.pv.ldt_base + offset);
> +#else
> +ASSERT_UNREACHABLE();
> +#endif

I think here it should be the call site to gain #ifdef (around the
entire if()), and the function as a whole should then be put in
another #ifdef, or be moved to pv/traps.c.

> @@ -1494,7 +1514,9 @@ void __init do_early_page_fault(struct cpu_user_regs 
> *regs)
>  
>  void do_general_protection(struct cpu_user_regs *regs)
>  {
> +#ifdef CONFIG_PV
>  struct vcpu *v = current;
> +#endif

I think this would better be resolved by moving the declaration into
the block which has multiple references, and use plain "current" in
the subsequent "else if()". But it's a matter of taste, I agree.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 08/17] x86: make construct_dom0 build with !CONFIG_PV

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> --- a/xen/arch/x86/dom0_build.c
> +++ b/xen/arch/x86/dom0_build.c
> @@ -509,8 +509,16 @@ int __init construct_dom0(struct domain *d, const 
> module_t *image,
>  }
>  #endif
>  
> -rc = (is_hvm_domain(d) ? dom0_construct_pvh : dom0_construct_pv)
> - (d, image, image_headroom, initrd, cmdline);
> +if ( is_hvm_domain(d) )
> +rc = dom0_construct_pvh(d, image, image_headroom, initrd, cmdline);
> +else if ( is_pv_domain(d) )
> +rc = dom0_construct_pv(d, image, image_headroom, initrd, cmdline);
> +else
> +{
> +ASSERT_UNREACHABLE();
> +rc = -EINVAL;
> +}

Depending on what the plans are wrt simultaneous PV=n and HVM=n,
this may better need to be panic(). The assertion is certainly not valid
in that case - it is very much expected to get there in such a case. It
is only valid if the Kconfig change doesn't allow for that combination.
In any event I see that patch 13 doesn't change the code above
again.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 09/17] x86/amd: put setting pv_post_outb_hook under CONFIG_PV

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> It is used by PV code only.

And wrongly so - the same is needed for a PVH Dom0 afaict.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 0/4] x86/HVM: implement memory read caching

2018-10-12 Thread Jan Beulich
>>> On 12.10.18 at 15:55,  wrote:
> On 11/09/18 14:10, Jan Beulich wrote:
>> Emulation requiring device model assistance uses a form of instruction
>> re-execution, assuming that the second (and any further) pass takes
>> exactly the same path. This is a valid assumption as far as use of CPU
>> registers goes (as those can't change without any other instruction
>> executing in between), but is wrong for memory accesses. In particular
>> it has been observed that Windows might page out buffers underneath
>> an instruction currently under emulation (hitting between two passes).
>> If the first pass translated a linear address successfully, any subsequent
>> pass needs to do so too, yielding the exact same translation.
>>
>> Introduce a cache (used just by guest page table accesses for now, i.e.
>> a form of "paging structure cache") to make sure above described
>> assumption holds. This is a very simplistic implementation for now: Only
>> exact matches are satisfied (no overlaps or partial reads or anything).
>>
>> There's also some seemingly unrelated cleanup here which was found
>> desirable on the way.
>>
>> 1: x86/mm: add optional cache to GLA->GFN translation
>> 2: x86/mm: use optional cache in guest_walk_tables()
>> 3: x86/HVM: implement memory read caching
>> 4: x86/HVM: prefill cache with PDPTEs when possible
>>
>> "VMX: correct PDPTE load checks" is omitted from v2, as I can't
>> currently find enough time to carry out the requested further
>> rework.
> 
> Following the x86 call, I've had some thoughts and suggestions about how
> to make this work in a reasonable way, without resorting to the full
> caching approach.

Thanks, but one question before I start thinking about this in
more detail: Before writing this, did you read my mail from the
11th? I ask because what you suggest does not look to match
the behavior I've described there as what I think it ought to be.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 10/17] x86: provide stub for switch_compat

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> --- a/xen/include/xen/compat.h
> +++ b/xen/include/xen/compat.h
> @@ -228,7 +228,16 @@ struct vcpu_runstate_info;
>  void xlat_vcpu_runstate_info(struct vcpu_runstate_info *);
>  
>  struct domain;
> +
> +#ifdef CONFIG_PV
>  int switch_compat(struct domain *);
> +#else
> +# include 
> +static inline int switch_compat(struct domain *d)
> +{
> +return -EINVAL;
> +}
> +#endif

I see two options here: Either we go with th above, but with -EACCES
as the return value (to match the current one for HVM domains), or
the #ifdef goes around the case block in domctl.c. Personally I'd prefer
the latter.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 10/17] x86: provide stub for switch_compat

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 08:33:48AM -0600, Jan Beulich wrote:
> >>> On 04.10.18 at 17:43,  wrote:
> > --- a/xen/include/xen/compat.h
> > +++ b/xen/include/xen/compat.h
> > @@ -228,7 +228,16 @@ struct vcpu_runstate_info;
> >  void xlat_vcpu_runstate_info(struct vcpu_runstate_info *);
> >  
> >  struct domain;
> > +
> > +#ifdef CONFIG_PV
> >  int switch_compat(struct domain *);
> > +#else
> > +# include 
> > +static inline int switch_compat(struct domain *d)
> > +{
> > +return -EINVAL;
> > +}
> > +#endif
> 
> I see two options here: Either we go with th above, but with -EACCES
> as the return value (to match the current one for HVM domains), or
> the #ifdef goes around the case block in domctl.c. Personally I'd prefer
> the latter.

This makes me chuckle because originally I opted for what you want here
but decided to switch to using stub before posting.

I will do that in the next version of this series.

Wei.

> 
> Jan
> 
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86: restrict HVMOP_pagetable_dying to current

2018-10-12 Thread Wei Liu
On Fri, Oct 05, 2018 at 05:29:30AM -0600, Jan Beulich wrote:
> This is not used (and probably was never meant to be) by the tool stack.
> Limiting it to the current domain in particular allows to eliminate a
> bogus use of vCPU 0 in pagetable_dying().
> 
> Remove the now unnecessary domain/vCPU parameters from the wrapper/hook
> functions at the same time.
> 
> Signed-off-by: Jan Beulich 

Reviewed-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 11/17] x86: provide stubs for entry point

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> Instead of trying to split up entry.S and compat/entry.S, simply
> provide some stubs for them.

I'm not convinced; I'd much rather see the call sites recognizably
compiled out with !PV. I'm curious what Andrew thinks here.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 18/18] tools/debugger/kdd: Install as `xen-kdd', not just `kdd'

2018-10-12 Thread Ian Jackson
Juergen Gross writes ("Re: [Xen-devel] [PATCH 18/18] tools/debugger/kdd: 
Install as `xen-kdd', not just `kdd'"):
> On 08/10/2018 16:01, Ian Jackson wrote:
> > Tim Deegan writes ("Re: [PATCH 18/18] tools/debugger/kdd: Install as 
> > `xen-kdd', not just `kdd'"):
> >> At 18:29 +0100 on 05 Oct (1538764157), Ian Jackson wrote:
> >>> `kdd' is an unfortunate namespace landgrab.
> >>
> >> Bah, humbug, etc. :)  Can we have a note in the changelog for the next
> >> release to warn the few kdd users that we've done this?
> > 
> > Mmm, sorry.  Err, where are such release notes collected ?
> 
> I believe this is a manual process triggered by the Community Manager.
> 
> I'm fine with collecting RN snipplets for the upcoming release, of
> course.

Thanks.  Can you add kdd -> xen-kdd to that list then please ?

> Ian, another question: could we add a link to the release note of the
> related version on:
> 
> https://xenbits.xen.org/docs/unstable/support-matrix.html

Yes, I think so.  That page is generated by
  docs/support-matrix-generate
  docs/parse-support-md
which consumes
  SUPPORT.md
from each tree.

I think if we change SUPPORT.md to have a proper hyperlink to the
release notes, in each tree, in the Release Support indented stanza,
it will appear on the support matrix page.

That is better than having the url be magically inserted by the matrix
generation machinery because it will appear in places like this too
  https://xenbits.xen.org/docs/4.11-testing/SUPPORT.html

Would you prepare a patch for 4.11 to add the RN URL ?  You can
perhaps test the support matrix output by running a variant of the
rune at the top of
  docs/support-matrix-generate
and I will fix the matrix code it if it doesn't seem to work :-).

Regards,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 12/17] x86: connect guest creation with CONFIG_PV

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -322,17 +322,34 @@ struct domain *domain_create(domid_t domid,
>  }
>  
>  /* Sort out our idea of is_{pv,hvm}_domain(). */
> -if ( config && (config->flags & XEN_DOMCTL_CDF_hvm_guest) )
> +if ( config )
>  {
> +ASSERT(!is_system_domain(d));

This and the other ASSERT() are redundant with what's earlier in
the function. Do we really need them?

> +if ( config->flags & XEN_DOMCTL_CDF_hvm_guest)
> +{
>  #ifdef CONFIG_HVM
> -d->guest_type = guest_type_hvm;
> +d->guest_type = guest_type_hvm;
> +#else
> +err = -EINVAL;
> +goto fail;
> +#endif
> +}
> +else
> +{
> +#ifdef CONFIG_PV
> +d->guest_type = guest_type_pv;
>  #else
>  err = -EINVAL;
>  goto fail;
>  #endif
> +}
>  }
>  else
> +{
> +/* The type of system domain shouldn't matter. */
> +ASSERT(is_system_domain(d));
>  d->guest_type = guest_type_pv;
> +}

I'm afraid this comment may cause ambiguity. I think we had (and
perhaps still have) a number of places where we assume that in
particular the idle domain is a PV one. So I'd like to ask to eithr
extend the comment to explain reality, or to drop it.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 11/12] tools/pygrub: Add `xen' to fsimage python module name

2018-10-12 Thread Ian Jackson
This module should be called `libxenfsimage' for the same reasons that
the C library should.

Signed-off-by: Ian Jackson 
---
v2: New in this version of the series
---
 tools/pygrub/setup.py  | 4 ++--
 tools/pygrub/src/fsimage/fsimage.c | 8 
 tools/pygrub/src/pygrub| 6 +++---
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/tools/pygrub/setup.py b/tools/pygrub/setup.py
index b58cc1c4e6..b8f1dc4590 100644
--- a/tools/pygrub/setup.py
+++ b/tools/pygrub/setup.py
@@ -7,7 +7,7 @@ extra_compile_args  = [ "-fno-strict-aliasing", "-Werror" ]
 
 XEN_ROOT = "../.."
 
-fsimage = Extension("fsimage",
+xenfsimage = Extension("xenfsimage",
 extra_compile_args = extra_compile_args,
 include_dirs = [ XEN_ROOT + "/tools/libfsimage/common/" ],
 library_dirs = [ XEN_ROOT + "/tools/libfsimage/common/" ],
@@ -25,5 +25,5 @@ setup(name='pygrub',
   package_dir={'grub': 'src', 'fsimage': 'src'},
   scripts = ["src/pygrub"],
   packages=pkgs,
-  ext_modules = [ fsimage ]
+  ext_modules = [ xenfsimage ]
   )
diff --git a/tools/pygrub/src/fsimage/fsimage.c 
b/tools/pygrub/src/fsimage/fsimage.c
index 47940572a8..743a3fb7b8 100644
--- a/tools/pygrub/src/fsimage/fsimage.c
+++ b/tools/pygrub/src/fsimage/fsimage.c
@@ -132,7 +132,7 @@ static char fsimage_file_type__doc__[] = "Filesystem image 
file";
 PyTypeObject fsimage_file_type = {
PyObject_HEAD_INIT(&PyType_Type)
0,  /* ob_size */
-   "fsimage.file", /* tp_name */
+   "xenfsimage.file",  /* tp_name */
sizeof(fsimage_file_t), /* tp_size */
0,  /* tp_itemsize */
(destructor) fsimage_file_dealloc,  /* tp_dealloc */
@@ -234,7 +234,7 @@ PyDoc_STRVAR(fsimage_fs_type__doc__, "Filesystem image");
 PyTypeObject fsimage_fs_type = {
PyObject_HEAD_INIT(&PyType_Type)
0,  /* ob_size */
-   "fsimage.fs",   /* tp_name */
+   "xenfsimage.fs",/* tp_name */
sizeof(fsimage_fs_t),   /* tp_size */
0,  /* tp_itemsize */
(destructor) fsimage_fs_dealloc,/* tp_dealloc */
@@ -317,7 +317,7 @@ static struct PyMethodDef fsimage_module_methods[] = {
 };
 
 PyMODINIT_FUNC
-initfsimage(void)
+initxenfsimage(void)
 {
-   Py_InitModule("fsimage", fsimage_module_methods);
+   Py_InitModule("xenfsimage", fsimage_module_methods);
 }
diff --git a/tools/pygrub/src/pygrub b/tools/pygrub/src/pygrub
index dd0c8f77df..52a8965ad9 100755
--- a/tools/pygrub/src/pygrub
+++ b/tools/pygrub/src/pygrub
@@ -21,7 +21,7 @@ import xen.lowlevel.xc
 import curses, _curses, curses.wrapper, curses.textpad, curses.ascii
 import getopt
 
-import fsimage
+import xenfsimage
 import grub.GrubConf
 import grub.LiloConf
 import grub.ExtLinuxConf
@@ -897,7 +897,7 @@ if __name__ == "__main__":
 
 for offset in part_offs:
 try:
-fs = fsimage.open(file, offset, bootfsoptions)
+fs = xenfsimage.open(file, offset, bootfsoptions)
 
 chosencfg = sniff_solaris(fs, incfg)
 
@@ -945,7 +945,7 @@ if __name__ == "__main__":
 
 args = None
 if chosencfg["args"]:
-zfsinfo = fsimage.getbootstring(fs)
+zfsinfo = xenfsimage.getbootstring(fs)
 if zfsinfo is not None:
 e = re.compile("zfs-bootfs=[\w\-\.\:@/]+" )
 (chosencfg["args"],count) = e.subn(zfsinfo, chosencfg["args"])
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 10/12] tools/libfsimage: Add `xen' to .h names and principal .so name

2018-10-12 Thread Ian Jackson
`fsimage' is rather general.  And we do not expect this library to be
very useful out of tree because of its unstable ABI.

So add the word `xen'.  This will avoid naming conflicts with anyone
else's fsimage library.

Signed-off-by: Ian Jackson 
---
v2: New in this version of the series
---
 tools/libfsimage/Rules.mk   |  2 +-
 tools/libfsimage/common/Makefile| 34 ++---
 tools/libfsimage/common/fsimage_grub.c  |  2 +-
 tools/libfsimage/common/fsimage_plugin.c|  2 +-
 tools/libfsimage/common/fsimage_priv.h  |  4 ++--
 tools/libfsimage/common/xenfsimage_grub.h   |  4 ++--
 tools/libfsimage/common/xenfsimage_plugin.h |  2 +-
 tools/libfsimage/ext2fs-lib/ext2fs-lib.c|  2 +-
 tools/libfsimage/ext2fs/fsys_ext2fs.c   |  2 +-
 tools/libfsimage/fat/fsys_fat.c |  2 +-
 tools/libfsimage/iso9660/fsys_iso9660.c |  2 +-
 tools/libfsimage/reiserfs/fsys_reiserfs.c   |  2 +-
 tools/libfsimage/ufs/fsys_ufs.c |  2 +-
 tools/libfsimage/xfs/fsys_xfs.c |  2 +-
 tools/libfsimage/zfs/fsi_zfs.c  |  2 +-
 tools/libfsimage/zfs/fsi_zfs.h  |  2 +-
 tools/pygrub/setup.py   |  2 +-
 tools/pygrub/src/fsimage/fsimage.c  |  2 +-
 18 files changed, 36 insertions(+), 36 deletions(-)

diff --git a/tools/libfsimage/Rules.mk b/tools/libfsimage/Rules.mk
index eab4ecb35e..2a29d9ef2b 100644
--- a/tools/libfsimage/Rules.mk
+++ b/tools/libfsimage/Rules.mk
@@ -26,7 +26,7 @@ fs-uninstall:
fi
 
 $(FSLIB): $(PIC_OBJS)
-   $(CC) $(LDFLAGS) $(SHLIB_LDFLAGS) -o $@ $^ -lfsimage $(FS_LIBDEPS) 
$(APPEND_LDFLAGS)
+   $(CC) $(LDFLAGS) $(SHLIB_LDFLAGS) -o $@ $^ -lxenfsimage $(FS_LIBDEPS) 
$(APPEND_LDFLAGS)
 
 clean distclean::
rm -f $(PIC_OBJS) $(FSLIB) $(DEPS_RM)
diff --git a/tools/libfsimage/common/Makefile b/tools/libfsimage/common/Makefile
index beda8f5f3a..f20e1394a8 100644
--- a/tools/libfsimage/common/Makefile
+++ b/tools/libfsimage/common/Makefile
@@ -15,7 +15,7 @@ LIB_SRCS-y = fsimage.c fsimage_plugin.c fsimage_grub.c
 
 PIC_OBJS := $(patsubst %.c,%.opic,$(LIB_SRCS-y))
 
-LIB = libfsimage.so libfsimage.so.$(MAJOR) libfsimage.so.$(MAJOR).$(MINOR)
+LIB = libxenfsimage.so libxenfsimage.so.$(MAJOR) 
libxenfsimage.so.$(MAJOR).$(MINOR)
 
 .PHONY: all
 all: $(LIB)
@@ -24,32 +24,32 @@ all: $(LIB)
 install: all
$(INSTALL_DIR) $(DESTDIR)$(libdir)
$(INSTALL_DIR) $(DESTDIR)$(includedir)
-   $(INSTALL_PROG) libfsimage.so.$(MAJOR).$(MINOR) $(DESTDIR)$(libdir)
-   ln -sf libfsimage.so.$(MAJOR).$(MINOR) 
$(DESTDIR)$(libdir)/libfsimage.so.$(MAJOR)
-   ln -sf libfsimage.so.$(MAJOR) $(DESTDIR)$(libdir)/libfsimage.so
-   $(INSTALL_DATA) fsimage.h $(DESTDIR)$(includedir)
-   $(INSTALL_DATA) fsimage_plugin.h $(DESTDIR)$(includedir)
-   $(INSTALL_DATA) fsimage_grub.h $(DESTDIR)$(includedir)
+   $(INSTALL_PROG) libxenfsimage.so.$(MAJOR).$(MINOR) $(DESTDIR)$(libdir)
+   ln -sf libxenfsimage.so.$(MAJOR).$(MINOR) 
$(DESTDIR)$(libdir)/libxenfsimage.so.$(MAJOR)
+   ln -sf libxenfsimage.so.$(MAJOR) $(DESTDIR)$(libdir)/libxenfsimage.so
+   $(INSTALL_DATA) xenfsimage.h $(DESTDIR)$(includedir)
+   $(INSTALL_DATA) xenfsimage_plugin.h $(DESTDIR)$(includedir)
+   $(INSTALL_DATA) xenfsimage_grub.h $(DESTDIR)$(includedir)
 
 .PHONY: uninstall
 uninstall:
-   rm -f $(DESTDIR)$(includedir)/fsimage_grub.h
-   rm -f $(DESTDIR)$(includedir)/fsimage_plugin.h
-   rm -f $(DESTDIR)$(includedir)/fsimage.h
-   rm -f $(DESTDIR)$(libdir)/libfsimage.so
-   rm -f $(DESTDIR)$(libdir)/libfsimage.so.$(MAJOR)
-   rm -f $(DESTDIR)$(libdir)/libfsimage.so.$(MAJOR).$(MINOR)
+   rm -f $(DESTDIR)$(includedir)/xenfsimage_grub.h
+   rm -f $(DESTDIR)$(includedir)/xenfsimage_plugin.h
+   rm -f $(DESTDIR)$(includedir)/xenfsimage.h
+   rm -f $(DESTDIR)$(libdir)/libxenfsimage.so
+   rm -f $(DESTDIR)$(libdir)/libxenfsimage.so.$(MAJOR)
+   rm -f $(DESTDIR)$(libdir)/libxenfsimage.so.$(MAJOR).$(MINOR)
 
 clean distclean::
rm -f $(LIB)
 
-libfsimage.so: libfsimage.so.$(MAJOR)
+libxenfsimage.so: libxenfsimage.so.$(MAJOR)
ln -sf $< $@
-libfsimage.so.$(MAJOR): libfsimage.so.$(MAJOR).$(MINOR)
+libxenfsimage.so.$(MAJOR): libxenfsimage.so.$(MAJOR).$(MINOR)
ln -sf $< $@
 
-libfsimage.so.$(MAJOR).$(MINOR): $(PIC_OBJS)
-   $(CC) $(LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,libfsimage.so.$(MAJOR) 
$(SHLIB_LDFLAGS) -o $@ $^ $(PTHREAD_LIBS) $(APPEND_LDFLAGS)
+libxenfsimage.so.$(MAJOR).$(MINOR): $(PIC_OBJS)
+   $(CC) $(LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,libxenfsimage.so.$(MAJOR) 
$(SHLIB_LDFLAGS) -o $@ $^ $(PTHREAD_LIBS) $(APPEND_LDFLAGS)
 
 -include $(DEPS_INCLUDE)
 
diff --git a/tools/libfsimage/common/fsimage_grub.c 
b/tools/libfsimage/common/fsimage_grub.c
index ef71d6cceb..258d48bfbb 100644
--- a/tools/libfsimage/common/fsimage_grub.c
+++ b/tools/libfsimage/common/fsimage_grub.c
@@ -28,7 +28,7 @@
 #include 
 #include 

[Xen-devel] [PATCH 07/12] tools/xenstore: Re-introduce (fake) xs_restrict call to preserve ABI

2018-10-12 Thread Ian Jackson
From: Hans van Kranenburg 

libxenstore3.0 in Xen 4.8 had this function.  We don't really want to
bump the ABI version (soname) just for this, since we don't think
there are actual callers anywhere.  But tools complain about the
symbol going away.

So, provide a function xs_restrict which conforms to the original
semantics, although it always fails.

Gbp-Pq: Topic xenstore
Gbp-Pq: Name tools-fake-xs-restrict.patch
Signed-off-by: Ian Jackson 
---
v2: New in this version of the series
---
 tools/xenstore/include/xenstore.h | 5 +
 tools/xenstore/xs.c   | 6 ++
 2 files changed, 11 insertions(+)

diff --git a/tools/xenstore/include/xenstore.h 
b/tools/xenstore/include/xenstore.h
index f460b8c5e5..0d95bb0e5c 100644
--- a/tools/xenstore/include/xenstore.h
+++ b/tools/xenstore/include/xenstore.h
@@ -132,6 +132,11 @@ bool xs_mkdir(struct xs_handle *h, xs_transaction_t t,
 bool xs_rm(struct xs_handle *h, xs_transaction_t t,
   const char *path);
 
+/* Fake function which will always return false (required to let
+ * libxenstore remain at 3.0 version.
+ */
+bool xs_restrict(struct xs_handle *h, unsigned domid);
+
 /* Get permissions of node (first element is owner, first perms is "other").
  * Returns malloced array, or NULL: call free() after use.
  */
diff --git a/tools/xenstore/xs.c b/tools/xenstore/xs.c
index 77700bff2b..cbcebb2bce 100644
--- a/tools/xenstore/xs.c
+++ b/tools/xenstore/xs.c
@@ -796,6 +796,12 @@ unwind:
return false;
 }
 
+/* Always return false a functionality has been removed in Xen 4.9 */
+bool xs_restrict(struct xs_handle *h, unsigned domid)
+{
+   return false;
+}
+
 /* Watch a node for changes (poll on fd to detect, or call read_watch()).
  * When the node (or any child) changes, fd will become readable.
  * Token is returned when watch is read, to allow matching.
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 05/12] xenmon: Install as xenmon, not xenmon.py

2018-10-12 Thread Ian Jackson
From: Ian Jackson 

Adding the implementation language as a suffix to a program name is
poor practice.

Signed-off-by: Ian Jackson 
Acked-by: Wei Liu 
---
 tools/xenmon/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/xenmon/Makefile b/tools/xenmon/Makefile
index e45c5b8c14..e1712304d0 100644
--- a/tools/xenmon/Makefile
+++ b/tools/xenmon/Makefile
@@ -32,13 +32,13 @@ install: build
$(INSTALL_DIR) $(DESTDIR)$(sbindir)
$(INSTALL_PROG) xenbaked $(DESTDIR)$(sbindir)/xenbaked
$(INSTALL_PROG) xentrace_setmask  $(DESTDIR)$(sbindir)/xentrace_setmask
-   $(INSTALL_PROG) xenmon.py  $(DESTDIR)$(sbindir)/xenmon.py
+   $(INSTALL_PROG) xenmon.py  $(DESTDIR)$(sbindir)/xenmon
 
 .PHONY: uninstall
 uninstall:
rm -f $(DESTDIR)$(sbindir)/xenbaked
rm -f $(DESTDIR)$(sbindir)/xentrace_setmask
-   rm -f $(DESTDIR)$(sbindir)/xenmon.py
+   rm -f $(DESTDIR)$(sbindir)/xenmon
 
 .PHONY: clean
 clean:
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 00/12] Miscellaneous fixes from Debian

2018-10-12 Thread Ian Jackson
This is v2 of my series of stuff from Debian.  (I already pushed the
uncontroversial parts of v1, so those patches are now missing from
v2.)  The remainder are build fixes, and addressing the namespace
pollution of the pygrub `fsimage' libraries by renaming it to
`xenfsimage'.

Hans van Kranenburg (1):
  tools/xenstore: Re-introduce (fake) xs_restrict call to preserve ABI

Ian Jackson (11):
  tools/Rules.mk: Honour PREPEND_LDFLAGS_XEN_TOOLS
  INSTALL: Mention kconfig
  gdbsx: Honour LDFLAGS when linking
  pygrub fsimage.so: Honour LDFLAGS when building
  xenmon: Install as xenmon, not xenmon.py
  tools/debugger/kdd: Install as `xen-kdd', not just `kdd'
  xenstore.h: Put ( ) around XS_* define shifts
  tools/libfsimage: Bump soname to 4.12
  tools/libfsimage: Add `xen' to .h names and principal .so name
  tools/pygrub: Add `xen' to fsimage python module name
  tools/libfsimage: Rename /usr/lib/fs to /usr/lib/xenfsimage

 INSTALL| 20 
 tools/Rules.mk |  2 ++
 tools/debugger/gdbsx/Makefile  |  2 +-
 tools/debugger/kdd/Makefile|  4 +--
 tools/libfsimage/Rules.mk  |  4 +--
 tools/libfsimage/common/Makefile   | 38 +++---
 tools/libfsimage/common/fsimage.c  |  2 +-
 tools/libfsimage/common/fsimage_grub.c |  2 +-
 tools/libfsimage/common/fsimage_plugin.c   |  2 +-
 tools/libfsimage/common/fsimage_priv.h |  4 +--
 .../libfsimage/common/{fsimage.h => xenfsimage.h}  |  0
 .../common/{fsimage_grub.h => xenfsimage_grub.h}   |  4 +--
 .../{fsimage_plugin.h => xenfsimage_plugin.h}  |  2 +-
 tools/libfsimage/ext2fs-lib/ext2fs-lib.c   |  2 +-
 tools/libfsimage/ext2fs/fsys_ext2fs.c  |  2 +-
 tools/libfsimage/fat/fsys_fat.c|  2 +-
 tools/libfsimage/iso9660/fsys_iso9660.c|  2 +-
 tools/libfsimage/reiserfs/fsys_reiserfs.c  |  2 +-
 tools/libfsimage/ufs/fsys_ufs.c|  2 +-
 tools/libfsimage/xfs/fsys_xfs.c|  2 +-
 tools/libfsimage/zfs/fsi_zfs.c |  2 +-
 tools/libfsimage/zfs/fsi_zfs.h |  2 +-
 tools/pygrub/Makefile  |  2 +-
 tools/pygrub/setup.py  |  6 ++--
 tools/pygrub/src/fsimage/fsimage.c | 10 +++---
 tools/pygrub/src/pygrub|  6 ++--
 tools/xenmon/Makefile  |  4 +--
 tools/xenstore/include/xenstore.h  | 11 +--
 tools/xenstore/xs.c|  6 
 29 files changed, 91 insertions(+), 58 deletions(-)
 rename tools/libfsimage/common/{fsimage.h => xenfsimage.h} (100%)
 rename tools/libfsimage/common/{fsimage_grub.h => xenfsimage_grub.h} (98%)
 rename tools/libfsimage/common/{fsimage_plugin.h => xenfsimage_plugin.h} (98%)

-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 08/12] xenstore.h: Put ( ) around XS_* define shifts

2018-10-12 Thread Ian Jackson
These definitions were not properly protected from unwanted operator
precedence interactions.

Existing use sites in-tree all use & or |, so this does not change any
actual behaviour in-tree.

The same seems likely to be true in external callers.

Signed-off-by: Ian Jackson 
---
v2: New in this version of the series
---
 tools/xenstore/include/xenstore.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/xenstore/include/xenstore.h 
b/tools/xenstore/include/xenstore.h
index 0d95bb0e5c..996b75ab0d 100644
--- a/tools/xenstore/include/xenstore.h
+++ b/tools/xenstore/include/xenstore.h
@@ -23,8 +23,8 @@
 
 #define XBT_NULL 0
 
-#define XS_OPEN_READONLY   1UL<<0
-#define XS_OPEN_SOCKETONLY  1UL<<1
+#define XS_OPEN_READONLY   (1UL<<0)
+#define XS_OPEN_SOCKETONLY  (1UL<<1)
 
 /*
  * Setting XS_UNWATCH_FILTER arranges that after xs_unwatch, no
@@ -45,7 +45,7 @@
  *  xs_unwatch for the first watch
  *  has returned.
  */
-#define XS_UNWATCH_FILTER 1UL<<2
+#define XS_UNWATCH_FILTER (1UL<<2)
 
 struct xs_handle;
 typedef uint32_t xs_transaction_t;
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 01/12] tools/Rules.mk: Honour PREPEND_LDFLAGS_XEN_TOOLS

2018-10-12 Thread Ian Jackson
From: Ian Jackson 

This allows the caller to provide some LDFLAGS to the Xen build
system.

Signed-off-by: Ian Jackson 
---
 tools/Rules.mk | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/Rules.mk b/tools/Rules.mk
index 296b722372..68f2ed7ce1 100644
--- a/tools/Rules.mk
+++ b/tools/Rules.mk
@@ -9,6 +9,8 @@ include $(XEN_ROOT)/Config.mk
 export _INSTALL := $(INSTALL)
 INSTALL = $(XEN_ROOT)/tools/cross-install
 
+LDFLAGS += $(PREPEND_LDFLAGS_XEN_TOOLS)
+
 XEN_INCLUDE= $(XEN_ROOT)/tools/include
 XEN_LIBXENTOOLCORE  = $(XEN_ROOT)/tools/libs/toolcore
 XEN_LIBXENTOOLLOG  = $(XEN_ROOT)/tools/libs/toollog
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 02/12] INSTALL: Mention kconfig

2018-10-12 Thread Ian Jackson
From: Ian Jackson 

Firstly, add a reference to the documentation for the kconfig system.

Secondly, warn the user about the XEN_CONFIG_EXPERT problem.

CC: Wei Liu 
CC: Andrew Cooper 
CC: Jan Beulich 
Signed-off-by: Ian Jackson 
Reviewed-by: Doug Goldstein 

---
v2: Fix typos
---
 INSTALL | 20 
 1 file changed, 20 insertions(+)

diff --git a/INSTALL b/INSTALL
index 9aa9ebdddc..e28348509f 100644
--- a/INSTALL
+++ b/INSTALL
@@ -19,6 +19,26 @@ following compile process. Once configure is done, make(1) 
has to be
 called. Also make(1) recognizes certain arguments. The following sections
 will give an overview.
 
+Xen Hypervisor
+==
+
+Xen itself is configured via a `kconfig' system borrowed from Linux.
+See docs/misc/kconfig.txt.
+
+Note that unlike with Linux, and contrary to that document, you cannot
+look at Kconfig files, or the default or generated config files etc.,
+to find available configuration options.  This is because it is only
+supported (and security supported) by the Xen Project, to change a
+small subset of the options.  Attempts to change other options will be
+silently overridden.  The only way to find which configuration options
+are available is to run `make menuconfig' or the like.
+
+You can counter-override this behaviour by setting XEN_CONFIG_EXPERT=y
+in your environment.  However, doing this is not supported and the
+resulting configurations do not receive security support.  If you set
+this variable there is nothing stopping you setting dangerously
+experimental combinations of features - not even any warnings.
+
 Options recognized by configure
 ===
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 03/12] gdbsx: Honour LDFLAGS when linking

2018-10-12 Thread Ian Jackson
From: Ian Jackson 

This command does the link, so it needs LDFLAGS.

Signed-off-by: Ian Jackson 
---
 tools/debugger/gdbsx/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/debugger/gdbsx/Makefile b/tools/debugger/gdbsx/Makefile
index 723a2743cc..8d7cd94a31 100644
--- a/tools/debugger/gdbsx/Makefile
+++ b/tools/debugger/gdbsx/Makefile
@@ -26,7 +26,7 @@ uninstall:
rm -f $(DESTDIR)$(sbindir)/gdbsx
 
 gdbsx: gx/gx_all.a xg/xg_all.a 
-   $(CC) -o $@ $^
+   $(CC) $(LDFLAGS) -o $@ $^
 
 xg/xg_all.a:
$(MAKE) -C xg
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 06/12] tools/debugger/kdd: Install as `xen-kdd', not just `kdd'

2018-10-12 Thread Ian Jackson
From: Ian Jackson 

`kdd' is an unfortunate namespace landgrab.

Signed-off-by: Ian Jackson 
Acked-by: Tim Deegan 
---
 tools/debugger/kdd/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/debugger/kdd/Makefile b/tools/debugger/kdd/Makefile
index 5509eee68c..26116949d4 100644
--- a/tools/debugger/kdd/Makefile
+++ b/tools/debugger/kdd/Makefile
@@ -24,8 +24,8 @@ distclean: clean
 .PHONY: install
 install: all
[ -d $(DESTDIR)$(sbindir) ] || $(INSTALL_DIR) $(DESTDIR)$(sbindir)
-   $(INSTALL_PROG) kdd $(DESTDIR)$(sbindir)/kdd
+   $(INSTALL_PROG) kdd $(DESTDIR)$(sbindir)/xen-kdd
 
 .PHONY: uninstall
 uninstall:
-   rm -f $(DESTDIR)$(sbindir)/kdd
+   rm -f $(DESTDIR)$(sbindir)/xen-kdd
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 12/12] tools/libfsimage: Rename /usr/lib/fs to /usr/lib/xenfsimage

2018-10-12 Thread Ian Jackson
Again, avoid namespace pollution.  These paths are purely internal to
libfsimage and its fs-specific modules, so no visible change from the
outside.

Signed-off-by: Ian Jackson 
---
v2: New in this version of the series
---
 tools/libfsimage/Rules.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libfsimage/Rules.mk b/tools/libfsimage/Rules.mk
index 2a29d9ef2b..bb6d42abb4 100644
--- a/tools/libfsimage/Rules.mk
+++ b/tools/libfsimage/Rules.mk
@@ -6,7 +6,7 @@ LDFLAGS += -L../common/
 
 PIC_OBJS := $(patsubst %.c,%.opic,$(LIB_SRCS-y))
 
-FSDIR = $(libdir)/fs
+FSDIR = $(libdir)/xenfsimage
 
 FSLIB = fsimage.so
 
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 09/12] tools/libfsimage: Bump soname to 4.12

2018-10-12 Thread Ian Jackson
This library does not have a stable ABI promise.  As far as we know it
is used only by pygrub.  Bump its soname to the Xen version (and
intend to change it each time).

Signed-off-by: Ian Jackson 
---
v2: New in this version of the series
---
 tools/libfsimage/common/Makefile  | 4 ++--
 tools/libfsimage/common/fsimage.c | 2 +-
 tools/libfsimage/common/{fsimage.h => xenfsimage.h}   | 0
 tools/libfsimage/common/{fsimage_grub.h => xenfsimage_grub.h} | 0
 tools/libfsimage/common/{fsimage_plugin.h => xenfsimage_plugin.h} | 0
 5 files changed, 3 insertions(+), 3 deletions(-)
 rename tools/libfsimage/common/{fsimage.h => xenfsimage.h} (100%)
 rename tools/libfsimage/common/{fsimage_grub.h => xenfsimage_grub.h} (100%)
 rename tools/libfsimage/common/{fsimage_plugin.h => xenfsimage_plugin.h} (100%)

diff --git a/tools/libfsimage/common/Makefile b/tools/libfsimage/common/Makefile
index a4655c421c..beda8f5f3a 100644
--- a/tools/libfsimage/common/Makefile
+++ b/tools/libfsimage/common/Makefile
@@ -1,8 +1,8 @@
 XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/libfsimage/Rules.mk
 
-MAJOR = 1.0
-MINOR = 0
+MAJOR = 0
+MINOR = 4.12
 
 LDFLAGS-$(CONFIG_SunOS) = -Wl,-M -Wl,mapfile-SunOS
 LDFLAGS-$(CONFIG_Linux) = -Wl,mapfile-GNU
diff --git a/tools/libfsimage/common/fsimage.c 
b/tools/libfsimage/common/fsimage.c
index 21d6c38ac6..5cfa56a84d 100644
--- a/tools/libfsimage/common/fsimage.c
+++ b/tools/libfsimage/common/fsimage.c
@@ -31,7 +31,7 @@
 #include 
 #include 
 
-#include "fsimage_plugin.h"
+#include "xenfsimage_plugin.h"
 #include "fsimage_priv.h"
 
 static pthread_mutex_t fsi_lock = PTHREAD_MUTEX_INITIALIZER;
diff --git a/tools/libfsimage/common/fsimage.h 
b/tools/libfsimage/common/xenfsimage.h
similarity index 100%
rename from tools/libfsimage/common/fsimage.h
rename to tools/libfsimage/common/xenfsimage.h
diff --git a/tools/libfsimage/common/fsimage_grub.h 
b/tools/libfsimage/common/xenfsimage_grub.h
similarity index 100%
rename from tools/libfsimage/common/fsimage_grub.h
rename to tools/libfsimage/common/xenfsimage_grub.h
diff --git a/tools/libfsimage/common/fsimage_plugin.h 
b/tools/libfsimage/common/xenfsimage_plugin.h
similarity index 100%
rename from tools/libfsimage/common/fsimage_plugin.h
rename to tools/libfsimage/common/xenfsimage_plugin.h
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 04/12] pygrub fsimage.so: Honour LDFLAGS when building

2018-10-12 Thread Ian Jackson
From: Ian Jackson 

This seems to have been simply omitted.  Obviously this is needed when
building and not just when installing.  Passing only when installing
is ineffective.

Signed-off-by: Ian Jackson 
---
 tools/pygrub/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/pygrub/Makefile b/tools/pygrub/Makefile
index 536af07932..3063c4998f 100644
--- a/tools/pygrub/Makefile
+++ b/tools/pygrub/Makefile
@@ -10,7 +10,7 @@ INSTALL_LOG = build/installed_files.txt
 all: build
 .PHONY: build
 build:
-   CC="$(CC)" CFLAGS="$(PY_CFLAGS)" $(PYTHON) setup.py build
+   CC="$(CC)" CFLAGS="$(PY_CFLAGS)" LDFLAGS="$(PY_LDFLAGS)" $(PYTHON) 
setup.py build
 
 .PHONY: install
 install: all
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 13/17] x86: don't create Dom0 if neither PV nor HVM is available

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> This will give us a Xen setting in idle loop. This doesn't have
> practical use except for debugging purpose.

Hmm, that's an acceptable alternative to the panic() variant, but
I'd still prefer that one. Again - let's see if Andrew has an opinion
either way.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 14/17] x86: don't report PV support when !CONFIG_PV

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> Signed-off-by: Wei Liu 

Acked-by: Jan Beulich 



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 15/17] x86: expose CONFIG_PV

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -37,6 +37,14 @@ source "arch/Kconfig"
>  
>  config PV
>   def_bool y
> + prompt "PV support"
> + ---help---
> +   Interfaces to support PV guests which don't require hardware
> +   support like Intel's VT-x or AMD's SVM.
> +
> +   This option is needed if you want to run PV guests.

We've had a discussion with (iirc) George and perhaps a few others
about "guest" vs "domain", and I've learned that generally the
former is meant to exclude Dom0 while the latter would include it.
In that light either s/guest/domain/ for the help text, or please add
explicit mention of Dom0 there (as PV is still the prevailing kind of
Dom0, and you wouldn't want to trip people into disabling this just
because they only run HVM guests).

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 16/17] x86: introduce new defconfigs for PV and HVM

2018-10-12 Thread Jan Beulich
>>> On 04.10.18 at 17:43,  wrote:
> They will be used by build tests.

And is it difficult for the build tests to set these up themselves?
I don't really like such additions, in particular the neither-PV-nor-
HVM one (which is otherwise completely useless). At the very
least that one should gain a comment saying it exists for build
testing only.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1 1/6] x86/vvmx: introduce vvmcx_valid()

2018-10-12 Thread Sergey Dyasli
As a convenient helper function and refactor the code to use it.

No functional change.

Signed-off-by: Sergey Dyasli 
---
 xen/arch/x86/hvm/vmx/vvmx.c | 17 -
 xen/include/asm-x86/hvm/nestedhvm.h |  5 +
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 0e45db83e5..8eee6d0ea8 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -521,8 +521,7 @@ static void vmfail(struct cpu_user_regs *regs, enum 
vmx_insn_errno errno)
 if ( errno == VMX_INSN_SUCCEED )
 return;
 
-if ( vcpu_nestedhvm(current).nv_vvmcxaddr != INVALID_PADDR &&
- errno != VMX_INSN_FAIL_INVALID )
+if ( vvmcx_valid(current) && errno != VMX_INSN_FAIL_INVALID )
 vmfail_valid(regs, errno);
 else
 vmfail_invalid(regs);
@@ -805,7 +804,7 @@ static void nvmx_purge_vvmcs(struct vcpu *v)
 int i;
 
 __clear_current_vvmcs(v);
-if ( nvcpu->nv_vvmcxaddr != INVALID_PADDR )
+if ( vvmcx_valid(v) )
 hvm_unmap_guest_frame(nvcpu->nv_vvmcx, 1);
 nvcpu->nv_vvmcx = NULL;
 nvcpu->nv_vvmcxaddr = INVALID_PADDR;
@@ -1601,7 +1600,7 @@ static int nvmx_vmresume(struct vcpu *v, struct 
cpu_user_regs *regs)
 struct nestedvcpu *nvcpu = &vcpu_nestedhvm(v);
 
 /* check VMCS is valid and IO BITMAP is set */
-if ( (nvcpu->nv_vvmcxaddr != INVALID_PADDR) &&
+if ( vvmcx_valid(v) &&
 ((nvmx->iobitmap[0] && nvmx->iobitmap[1]) ||
 !(__n2_exec_control(v) & CPU_BASED_ACTIVATE_IO_BITMAP) ) )
 nvcpu->nv_vmentry_pending = 1;
@@ -1622,7 +1621,7 @@ int nvmx_handle_vmresume(struct cpu_user_regs *regs)
 if ( rc != X86EMUL_OKAY )
 return rc;
 
-if ( vcpu_nestedhvm(v).nv_vvmcxaddr == INVALID_PADDR )
+if ( !vvmcx_valid(v) )
 {
 vmfail_invalid(regs);
 return X86EMUL_OKAY;
@@ -1656,7 +1655,7 @@ int nvmx_handle_vmlaunch(struct cpu_user_regs *regs)
 if ( rc != X86EMUL_OKAY )
 return rc;
 
-if ( vcpu_nestedhvm(v).nv_vvmcxaddr == INVALID_PADDR )
+if ( !vvmcx_valid(v) )
 {
 vmfail_invalid(regs);
 return X86EMUL_OKAY;
@@ -1709,7 +1708,7 @@ int nvmx_handle_vmptrld(struct cpu_user_regs *regs)
 if ( nvcpu->nv_vvmcxaddr != gpa )
 nvmx_purge_vvmcs(v);
 
-if ( nvcpu->nv_vvmcxaddr == INVALID_PADDR )
+if ( !vvmcx_valid(v) )
 {
 bool_t writable;
 void *vvmcx = hvm_map_guest_frame_rw(paddr_to_pfn(gpa), 1, &writable);
@@ -1848,7 +1847,7 @@ int nvmx_handle_vmread(struct cpu_user_regs *regs)
 if ( rc != X86EMUL_OKAY )
 return rc;
 
-if ( vcpu_nestedhvm(v).nv_vvmcxaddr == INVALID_PADDR )
+if ( !vvmcx_valid(v) )
 {
 vmfail_invalid(regs);
 return X86EMUL_OKAY;
@@ -1891,7 +1890,7 @@ int nvmx_handle_vmwrite(struct cpu_user_regs *regs)
  != X86EMUL_OKAY )
 return X86EMUL_EXCEPTION;
 
-if ( vcpu_nestedhvm(v).nv_vvmcxaddr == INVALID_PADDR )
+if ( !vvmcx_valid(v) )
 {
 vmfail_invalid(regs);
 return X86EMUL_OKAY;
diff --git a/xen/include/asm-x86/hvm/nestedhvm.h 
b/xen/include/asm-x86/hvm/nestedhvm.h
index 9d1c2742b5..e09fa9d47d 100644
--- a/xen/include/asm-x86/hvm/nestedhvm.h
+++ b/xen/include/asm-x86/hvm/nestedhvm.h
@@ -92,4 +92,9 @@ static inline void nestedhvm_set_cr(struct vcpu *v, unsigned 
int cr,
 v->arch.hvm.nvcpu.guest_cr[cr] = value;
 }
 
+static inline bool vvmcx_valid(const struct vcpu *v)
+{
+return vcpu_nestedhvm(v).nv_vvmcxaddr != INVALID_PADDR;
+}
+
 #endif /* _HVM_NESTEDHVM_H */
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1 4/6] x86/vvmx: add VMX_INSN_VMCLEAR_WITH_VMXON_PTR errno

2018-10-12 Thread Sergey Dyasli
And make nvmx_handle_vmclear() return the new errno in case the provided
address is the same as vmxon region address.

While at it, correct the return value for not-4KB-aligned case and for
invalid physaddr.

Signed-off-by: Sergey Dyasli 
---
 xen/arch/x86/hvm/vmx/vvmx.c| 23 ++-
 xen/include/asm-x86/hvm/vmx/vmcs.h |  1 +
 2 files changed, 19 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 4caa5811a1..8b691bfc04 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -1804,9 +1804,20 @@ int nvmx_handle_vmclear(struct cpu_user_regs *regs)
 return rc;
 
 BUILD_BUG_ON(X86EMUL_OKAY != VMSUCCEED); /* rc = VMSUCCEED; */
+
+if ( gpa == vcpu_2_nvmx(v).vmxon_region_pa )
+{
+vmfail(regs, VMX_INSN_VMCLEAR_WITH_VMXON_PTR);
+goto out;
+}
+
 if ( gpa & 0xfff )
-rc = VMFAIL_INVALID;
-else if ( gpa == nvcpu->nv_vvmcxaddr )
+{
+vmfail(regs, VMX_INSN_VMCLEAR_INVALID_PHYADDR);
+goto out;
+}
+
+if ( gpa == nvcpu->nv_vvmcxaddr )
 {
 if ( cpu_has_vmx_vmcs_shadowing )
 nvmx_clear_vmcs_pointer(v, nvcpu->nv_vvmcx);
@@ -1820,7 +1831,10 @@ int nvmx_handle_vmclear(struct cpu_user_regs *regs)
 bool_t writable;
 
 vvmcs = hvm_map_guest_frame_rw(paddr_to_pfn(gpa), 0, &writable);
-if ( vvmcs ) 
+
+if ( !vvmcs )
+rc = VMFAIL_VALID;
+else
 {
 if ( writable )
 clear_vvmcs_launched(&nvmx->launched_list,
@@ -1835,9 +1849,8 @@ int nvmx_handle_vmclear(struct cpu_user_regs *regs)
 vmsucceed(regs);
 else if ( rc == VMFAIL_VALID )
 vmfail(regs, VMX_INSN_VMCLEAR_INVALID_PHYADDR);
-else
-vmfail_invalid(regs);
 
+out:
 return X86EMUL_OKAY;
 }
 
diff --git a/xen/include/asm-x86/hvm/vmx/vmcs.h 
b/xen/include/asm-x86/hvm/vmx/vmcs.h
index cae1861610..e84d2e482b 100644
--- a/xen/include/asm-x86/hvm/vmx/vmcs.h
+++ b/xen/include/asm-x86/hvm/vmx/vmcs.h
@@ -529,6 +529,7 @@ enum vmx_insn_errno
 {
 VMX_INSN_SUCCEED   = 0,
 VMX_INSN_VMCLEAR_INVALID_PHYADDR   = 2,
+VMX_INSN_VMCLEAR_WITH_VMXON_PTR= 3,
 VMX_INSN_VMLAUNCH_NONCLEAR_VMCS= 4,
 VMX_INSN_VMRESUME_NONLAUNCHED_VMCS = 5,
 VMX_INSN_INVALID_CONTROL_STATE = 7,
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 13/17] x86: don't create Dom0 if neither PV nor HVM is available

2018-10-12 Thread Andrew Cooper
On 12/10/18 16:12, Jan Beulich wrote:
 On 04.10.18 at 17:43,  wrote:
>> This will give us a Xen setting in idle loop. This doesn't have
>> practical use except for debugging purpose.
> Hmm, that's an acceptable alternative to the panic() variant, but
> I'd still prefer that one. Again - let's see if Andrew has an opinion
> either way.

I think that, for the purpose of keeping our interfaces clean, being
able to compile without PV and without HVM is a useful property
(especially as randconfig should hit it one in every 4 cases).

I've specifically needed to have no dom0 for certain debugging in the
past.  (The HPET series to fix the broken MSI handler, which I realise
still hasn't made its way upstream yet.)

I'm less certain however if implementing it like this is wise.  I
implemented it with a "nodom0" command line parameter, and left the rest
of the functionality intact.

In particular, one fuzzing idea I have is to have a tiny AFL stub on the
end of the serial port, at which point, having no dom0 but PV and HVM
fully compiled in would be the ideal position.

I'm sorry if this isn't necessarily a very helpful answer.  I'd be
tempted to drop this patch for now, and let the first person with a
concrete usecase to decide exactly how a no-dom0 system would look.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1 0/6] x86/vvmx: various fixes

2018-10-12 Thread Sergey Dyasli
These were found by running nested VMX tests from kvm-unit-tests.

Sergey Dyasli (6):
  x86/vvmx: introduce vvmcx_valid()
  x86/vvmx: correct vmfail() usage for vmptrld and vmclear
  x86/vvmx: add VMX_INSN_VMPTRLD_WITH_VMXON_PTR errno
  x86/vvmx: add VMX_INSN_VMCLEAR_WITH_VMXON_PTR errno
  x86/vvmx: correctly report vvmcs size
  x86/vvmx: fix I/O and MSR bitmaps mapping

 xen/arch/x86/hvm/vmx/vvmx.c | 164 +++-
 xen/include/asm-x86/hvm/nestedhvm.h |   5 +
 xen/include/asm-x86/hvm/vmx/vmcs.h  |   2 +
 3 files changed, 117 insertions(+), 54 deletions(-)

-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1 5/6] x86/vvmx: correctly report vvmcs size

2018-10-12 Thread Sergey Dyasli
The size of Xen's virtual vmcs region is 4096 bytes. Correctly report
it to the guest in case when VMCS shadowing is not available instead of
providing H/W value (which is usually smaller).

Signed-off-by: Sergey Dyasli 
---
 xen/arch/x86/hvm/vmx/vvmx.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 8b691bfc04..2c2ba36d94 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -2064,6 +2064,14 @@ int nvmx_msr_read_intercept(unsigned int msr, u64 
*msr_content)
 data = (host_data & (~0ul << 32)) |
(vmcs->vmcs_revision_id & 0x7fff);
 unmap_domain_page(vmcs);
+
+if ( !cpu_has_vmx_vmcs_shadowing )
+{
+/* Report vmcs_region_size as 4096 */
+data &= ~VMX_BASIC_VMCS_SIZE_MASK;
+data |= 1ULL << 44;
+}
+
 break;
 }
 case MSR_IA32_VMX_PINBASED_CTLS:
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1 3/6] x86/vvmx: add VMX_INSN_VMPTRLD_WITH_VMXON_PTR errno

2018-10-12 Thread Sergey Dyasli
And make nvmx_handle_vmptrld() return the new errno in case the provided
address is the same as vmxon region address.

While at it, correct the return value for not-4KB-aligned case.

Signed-off-by: Sergey Dyasli 
---
 xen/arch/x86/hvm/vmx/vvmx.c| 10 --
 xen/include/asm-x86/hvm/vmx/vmcs.h |  1 +
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 26b7d72660..4caa5811a1 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -1699,9 +1699,15 @@ int nvmx_handle_vmptrld(struct cpu_user_regs *regs)
 if ( rc != X86EMUL_OKAY )
 return rc;
 
-if ( gpa == vcpu_2_nvmx(v).vmxon_region_pa || gpa & 0xfff )
+if ( gpa & 0xfff )
 {
-vmfail_invalid(regs);
+vmfail(regs, VMX_INSN_VMPTRLD_INVALID_PHYADDR);
+goto out;
+}
+
+if ( gpa == vcpu_2_nvmx(v).vmxon_region_pa )
+{
+vmfail(regs, VMX_INSN_VMPTRLD_WITH_VMXON_PTR);
 goto out;
 }
 
diff --git a/xen/include/asm-x86/hvm/vmx/vmcs.h 
b/xen/include/asm-x86/hvm/vmx/vmcs.h
index 76dd04a72d..cae1861610 100644
--- a/xen/include/asm-x86/hvm/vmx/vmcs.h
+++ b/xen/include/asm-x86/hvm/vmx/vmcs.h
@@ -534,6 +534,7 @@ enum vmx_insn_errno
 VMX_INSN_INVALID_CONTROL_STATE = 7,
 VMX_INSN_INVALID_HOST_STATE= 8,
 VMX_INSN_VMPTRLD_INVALID_PHYADDR   = 9,
+VMX_INSN_VMPTRLD_WITH_VMXON_PTR= 10,
 VMX_INSN_VMPTRLD_INCORRECT_VMCS_ID = 11,
 VMX_INSN_UNSUPPORTED_VMCS_COMPONENT= 12,
 VMX_INSN_VMXON_IN_VMX_ROOT = 15,
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v1 6/6] x86/vvmx: fix I/O and MSR bitmaps mapping

2018-10-12 Thread Sergey Dyasli
Currently Xen tries to map bitmaps during emulation of vmptrld and
vmwrite. This is wrong: the guest can store arbitrary values in those
fields.

Make bitmaps mapping happen only during a nested vmentry and only if
the appropriate execution controls are turned on by L1 hypervisor.
Mapping happens only:

1. During the first nested vmentry
2. After L1 has changed an appropriate vmcs field
3. After nvmx_purge_vvmcs() was previously called

Signed-off-by: Sergey Dyasli 
---
 xen/arch/x86/hvm/vmx/vvmx.c | 104 +++-
 1 file changed, 67 insertions(+), 37 deletions(-)

diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 2c2ba36d94..e10ed79f66 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -750,6 +750,17 @@ static void __clear_current_vvmcs(struct vcpu *v)
 __vmpclear(nvcpu->nv_n2vmcx_pa);
 }
 
+static void unmap_msr_bitmap(struct vcpu *v)
+{
+struct nestedvmx *nvmx = &vcpu_2_nvmx(v);
+
+if ( nvmx->msrbitmap )
+{
+hvm_unmap_guest_frame(nvmx->msrbitmap, 1);
+nvmx->msrbitmap = NULL;
+}
+}
+
 /*
  * Refreshes the MSR bitmap mapping for the current nested vcpu.  Returns true
  * for a successful mapping, and returns false for MSR_BITMAP parameter errors
@@ -760,12 +771,7 @@ static bool __must_check _map_msr_bitmap(struct vcpu *v)
 struct nestedvmx *nvmx = &vcpu_2_nvmx(v);
 uint64_t gpa;
 
-if ( nvmx->msrbitmap )
-{
-hvm_unmap_guest_frame(nvmx->msrbitmap, 1);
-nvmx->msrbitmap = NULL;
-}
-
+unmap_msr_bitmap(v);
 gpa = get_vvmcs(v, MSR_BITMAP);
 
 if ( !IS_ALIGNED(gpa, PAGE_SIZE) )
@@ -776,6 +782,17 @@ static bool __must_check _map_msr_bitmap(struct vcpu *v)
 return nvmx->msrbitmap != NULL;
 }
 
+static void unmap_io_bitmap(struct vcpu *v, unsigned int idx)
+{
+struct nestedvmx *nvmx = &vcpu_2_nvmx(v);
+
+if ( nvmx->iobitmap[idx] )
+{
+hvm_unmap_guest_frame(nvmx->iobitmap[idx], 1);
+nvmx->iobitmap[idx] = NULL;
+}
+}
+
 static bool_t __must_check _map_io_bitmap(struct vcpu *v, u64 vmcs_reg)
 {
 struct nestedvmx *nvmx = &vcpu_2_nvmx(v);
@@ -783,8 +800,7 @@ static bool_t __must_check _map_io_bitmap(struct vcpu *v, 
u64 vmcs_reg)
 int index;
 
 index = vmcs_reg == IO_BITMAP_A ? 0 : 1;
-if (nvmx->iobitmap[index])
-hvm_unmap_guest_frame(nvmx->iobitmap[index], 1);
+unmap_io_bitmap(v, index);
 gpa = get_vvmcs(v, vmcs_reg);
 nvmx->iobitmap[index] = hvm_map_guest_frame_ro(gpa >> PAGE_SHIFT, 1);
 
@@ -799,7 +815,6 @@ static inline bool_t __must_check map_io_bitmap_all(struct 
vcpu *v)
 
 static void nvmx_purge_vvmcs(struct vcpu *v)
 {
-struct nestedvmx *nvmx = &vcpu_2_nvmx(v);
 struct nestedvcpu *nvcpu = &vcpu_nestedhvm(v);
 int i;
 
@@ -809,16 +824,11 @@ static void nvmx_purge_vvmcs(struct vcpu *v)
 nvcpu->nv_vvmcx = NULL;
 nvcpu->nv_vvmcxaddr = INVALID_PADDR;
 v->arch.hvm.vmx.vmcs_shadow_maddr = 0;
-for (i=0; i<2; i++) {
-if ( nvmx->iobitmap[i] ) {
-hvm_unmap_guest_frame(nvmx->iobitmap[i], 1);
-nvmx->iobitmap[i] = NULL;
-}
-}
-if ( nvmx->msrbitmap ) {
-hvm_unmap_guest_frame(nvmx->msrbitmap, 1);
-nvmx->msrbitmap = NULL;
-}
+
+for ( i = 0; i < 2; i++ )
+unmap_io_bitmap(v, i);
+
+unmap_msr_bitmap(v);
 }
 
 u64 nvmx_get_tsc_offset(struct vcpu *v)
@@ -1594,20 +1604,34 @@ static void clear_vvmcs_launched(struct list_head 
*launched_list,
 }
 }
 
-static int nvmx_vmresume(struct vcpu *v, struct cpu_user_regs *regs)
+static enum vmx_insn_errno nvmx_vmresume(struct vcpu *v)
 {
 struct nestedvmx *nvmx = &vcpu_2_nvmx(v);
 struct nestedvcpu *nvcpu = &vcpu_nestedhvm(v);
+unsigned int exec_ctrl;
 
-/* check VMCS is valid and IO BITMAP is set */
-if ( vvmcx_valid(v) &&
-((nvmx->iobitmap[0] && nvmx->iobitmap[1]) ||
-!(__n2_exec_control(v) & CPU_BASED_ACTIVATE_IO_BITMAP) ) )
-nvcpu->nv_vmentry_pending = 1;
-else
-vmfail_invalid(regs);
+ASSERT(vvmcx_valid(v));
+exec_ctrl = __n2_exec_control(v);
 
-return X86EMUL_OKAY;
+if ( exec_ctrl & CPU_BASED_ACTIVATE_IO_BITMAP )
+{
+if ( (nvmx->iobitmap[0] == NULL || nvmx->iobitmap[1] == NULL) &&
+ !map_io_bitmap_all(v) )
+goto invalid_control_state;
+}
+
+if ( exec_ctrl & CPU_BASED_ACTIVATE_MSR_BITMAP )
+{
+if ( nvmx->msrbitmap == NULL && !_map_msr_bitmap(v) )
+goto invalid_control_state;
+}
+
+nvcpu->nv_vmentry_pending = 1;
+
+return VMX_INSN_SUCCEED;
+
+invalid_control_state:
+return VMX_INSN_INVALID_CONTROL_STATE;
 }
 
 int nvmx_handle_vmresume(struct cpu_user_regs *regs)
@@ -1641,7 +1665,12 @@ int nvmx_handle_vmresume(struct cpu_user_regs *regs)
 vmfail_valid(regs, VMX_INSN_VMRESUME_NONLAUNCHED_VMCS);
 return X86EMUL_OKAY;
 }
-return nvmx_vmresume(v,regs

[Xen-devel] [PATCH v1 2/6] x86/vvmx: correct vmfail() usage for vmptrld and vmclear

2018-10-12 Thread Sergey Dyasli
Calling vmfail_valid() is correct only if vvmcx is valid. Modify
functions to use vmfail() instead which performs the necessary check.

Signed-off-by: Sergey Dyasli 
---
 xen/arch/x86/hvm/vmx/vvmx.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 8eee6d0ea8..26b7d72660 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -1744,7 +1744,7 @@ int nvmx_handle_vmptrld(struct cpu_user_regs *regs)
  !map_io_bitmap_all(v) ||
  !_map_msr_bitmap(v) )
 {
-vmfail_valid(regs, VMX_INSN_VMPTRLD_INVALID_PHYADDR);
+vmfail(regs, VMX_INSN_VMPTRLD_INVALID_PHYADDR);
 goto out;
 }
 }
@@ -1828,7 +1828,7 @@ int nvmx_handle_vmclear(struct cpu_user_regs *regs)
 if ( rc == VMSUCCEED )
 vmsucceed(regs);
 else if ( rc == VMFAIL_VALID )
-vmfail_valid(regs, VMX_INSN_VMCLEAR_INVALID_PHYADDR);
+vmfail(regs, VMX_INSN_VMCLEAR_INVALID_PHYADDR);
 else
 vmfail_invalid(regs);
 
-- 
2.17.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen optimization

2018-10-12 Thread Milan Boberic
Hi Stefano, glad to have you back :D,
this is my setup:
- dom0 is PetaLinux, has 1 vCPU and it's pinned for pCPU0
- there is only one domU and this is my bare-metal app that also
have one vCPU and it's pinned for pCPU1
so yeah, there is only dom0 and bare-metal app on the board.

Jitter is the same with and without Dario's patch.

I'm still not sure about timer's passthrough because there is no mention of
triple timer counter is device tree so I added:

&ttc0 {
   xen,passthrough = <0x1>;
};

at the end of the xen-overlay.dtsi file which I included in attachment.

About patch you sent, I can't find this funcion void vgic_inject_irq in
/xen/arch/arm/vgic.c file, this is link of git repository from where I
build my xen so you can take a look if that printk can be put somewhere
else.

https://github.com/Xilinx/xen/

I ran some more testing and realized that results are the same with or
without vwfi=native, which I think again points out that passthrough that I
need to provide in device tree isn't valid.

 And of course, higher the frequency of interrupts results in higher
jitter. I'm still battling with Xilinx SDK and triple timer counter that's
why I can't figure out what is the exact frequency set (I'm just rising it
and lowering it), I'll give my best to solve that ASAP because we need to
know exact value of frequency set.

Thanks in advance!

Milan



On Fri, Oct 12, 2018 at 12:29 AM Stefano Stabellini <
stefano.stabell...@xilinx.com> wrote:

> On Thu, 11 Oct 2018, Milan Boberic wrote:
> > On Wed, Oct 10, 2018 at 6:41 PM Meng Xu  wrote:
> > >
> > > The jitter may come from Xen or the OS in dom0.
> > > It will be useful to know what is the jitter if you run the test on
> PetaLinux.
> > > (It's understandable the jitter is gone without OS. It is also common
> > > that OS introduces various interferences.)
> >
> > Hi Meng,
> > well... I'm using bare-metal application and I need it exclusively to
> > be ran on one CPU as domU (guest) without OS (and I'm not sure how
> > would I make the same app to be ran on PetaLinux dom0 :D haha).
> > Is there a chance that PetaLinux as dom0 is creating this jitter and
> > how? Is there a way of decreasing it?
> >
> > Yes, there are no prints.
> >
> > I'm not sure about this timer interrupt passthrough because I didn't
> > find any example of it, in attachment I included xen-overlay.dtsi file
> > which I edited to add passthrough, in earlier replies there are
> > bare-metal configuration file. It would be helpful to know if those
> > setting are correct. If they are not correct it would explain the
> > jitter.
> >
> > Thanks in advance, Milan Boberic!
>
> Hi Milan,
>
> Sorry for taking so long to go back to this thread. But I am here now :)
>
> First, let me ask a couple of questions to understand the scenario
> better: is there any interference from other virtual machines while you
> measure the jitter? Or is the baremetal app the only thing actively
> running on the board?
>
> Second, it would be worth double-checking that Dario's patch to fix
> sched=null is not having unexpected side effects. I don't think so, it
> would be worth testing with it and without it to be sure.
>
> I gave a look at your VM configuration. The configuration looks correct.
> There is no dtdev settings, but given that none of the devices you are
> assigning to the guest does any DMA, it should be OK. You want to make
> sure that Dom0 is not trying to use those same devices -- make sure to
> add "xen,passthrough;" to each corresponding node on the host device
> tree.
>
> The error messages "No valid vCPU found" are due to the baremetal
> applications trying to configure as target cpu for the interrupt cpu1
> (the second cpu in the system), while actually only 1 vcpu is assigned
> to the VM. Hence, only cpu0 is allowed. I don't think it should cause
> any jitter issues, because the request is simply ignored. Just to be
> safe, you might want to double check that the physical interrupt is
> delivered to the right physical cpu, which would be cpu1 in your
> configuration, the one running the only vcpu of the baremetal app. You
> can do that by adding a printk to xen/arch/arm/vgic.c:vgic_inject_irq,
> for example:
>
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 5a4f082..208fde7 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -591,6 +591,7 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v,
> unsigned int virq,
>  out:
>  spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
>
> +if (v != current) printk("DEBUG irq slow path!\n");
>  /* we have a new higher priority irq, inject it into the guest */
>  vcpu_kick(v);
>
> You don't want "DEBUG irq slow path!" to get printed.
>
> Finally, I would try to set the timer to generate events less frequently
> than every 1us and see what happens, maybe every 5-10us. In my tests,
> the IRQ latency overhead caused by Xen is around 1us, so injecting 1
> interrupt every 1us, plus 1us of latency

Re: [Xen-devel] [PATCH 16/17] x86: introduce new defconfigs for PV and HVM

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 09:19:45AM -0600, Jan Beulich wrote:
> >>> On 04.10.18 at 17:43,  wrote:
> > They will be used by build tests.
> 
> And is it difficult for the build tests to set these up themselves?

I specifically said "build tests" but not "CI systems".

It wouldn't be very difficult for a CI system to generate these files,
but my idea is that providing these in tree can make things more
convenient for humans -- developers and maintainers.

Suppose you want a contributor to test their changes, instead of writing
"first please generate the following files, and then ...", you just give
them a rune / some runes to reference these files directly. There
doesn't need explaining and no ambiguity will arise. And if there is
disagreement on what work or what doesn't, it would be easy to
reproduce.

To me this is a clear win: a small investment that comes with long term
benefit.

> I don't really like such additions, in particular the neither-PV-nor-
> HVM one (which is otherwise completely useless). At the very
> least that one should gain a comment saying it exists for build
> testing only.
> 

Sure, I can do that.

Wei.

> Jan
> 
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/6] x86/pvh: fix TSC mode setup for PVH Dom0

2018-10-12 Thread Jan Beulich
>>> On 09.10.18 at 11:42,  wrote:
> A PVH Dom0 might use TSC scaling or other HVM specific TSC
> adjustments, so only short-circuit the TSC setup for a classic PV
> Dom0.

How that?

> --- a/xen/arch/x86/time.c
> +++ b/xen/arch/x86/time.c
> @@ -2125,7 +2125,7 @@ void tsc_set_info(struct domain *d,
>  {
>  ASSERT(!is_system_domain(d));
>  
> -if ( is_hardware_domain(d) )
> +if ( is_pv_domain(d) && is_hardware_domain(d) )

This code path is reachable only from arch_domain_create()
(setting defaults) and XEN_DOMCTL_settscinfo (which Dom0 can't
issue against itself). So either there's some important part missing
from the description, or I fail to see what use this change is. Hmm,
on a platform where host_tsc_is_safe() returns false the setting
of defaults would have an effect, but I think the description
should then say so, instead of giving the impression that the
functionality would become available in full.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 09/12] tools/libfsimage: Bump soname to 4.12

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:12:12PM +0100, Ian Jackson wrote:
> This library does not have a stable ABI promise.  As far as we know it
> is used only by pygrub.  Bump its soname to the Xen version (and
> intend to change it each time).
> 
> Signed-off-by: Ian Jackson 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 04/12] pygrub fsimage.so: Honour LDFLAGS when building

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:12:07PM +0100, Ian Jackson wrote:
> From: Ian Jackson 
> 
> This seems to have been simply omitted.  Obviously this is needed when
> building and not just when installing.  Passing only when installing
> is ineffective.
> 
> Signed-off-by: Ian Jackson 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 03/12] gdbsx: Honour LDFLAGS when linking

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:12:06PM +0100, Ian Jackson wrote:
> From: Ian Jackson 
> 
> This command does the link, so it needs LDFLAGS.
> 
> Signed-off-by: Ian Jackson 

Acked-by: Wei Liu 

I think this needs an ack from Elena as well.

> ---
>  tools/debugger/gdbsx/Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/debugger/gdbsx/Makefile b/tools/debugger/gdbsx/Makefile
> index 723a2743cc..8d7cd94a31 100644
> --- a/tools/debugger/gdbsx/Makefile
> +++ b/tools/debugger/gdbsx/Makefile
> @@ -26,7 +26,7 @@ uninstall:
>   rm -f $(DESTDIR)$(sbindir)/gdbsx
>  
>  gdbsx: gx/gx_all.a xg/xg_all.a 
> - $(CC) -o $@ $^
> + $(CC) $(LDFLAGS) -o $@ $^
>  
>  xg/xg_all.a:
>   $(MAKE) -C xg
> -- 
> 2.11.0
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 07/12] tools/xenstore: Re-introduce (fake) xs_restrict call to preserve ABI

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:12:10PM +0100, Ian Jackson wrote:
> From: Hans van Kranenburg 
> 
> libxenstore3.0 in Xen 4.8 had this function.  We don't really want to
> bump the ABI version (soname) just for this, since we don't think
> there are actual callers anywhere.  But tools complain about the
> symbol going away.
> 
> So, provide a function xs_restrict which conforms to the original
> semantics, although it always fails.
> 
> Gbp-Pq: Topic xenstore
> Gbp-Pq: Name tools-fake-xs-restrict.patch
> Signed-off-by: Ian Jackson 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 10/12] tools/libfsimage: Add `xen' to .h names and principal .so name

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:12:13PM +0100, Ian Jackson wrote:
> `fsimage' is rather general.  And we do not expect this library to be
> very useful out of tree because of its unstable ABI.
> 
> So add the word `xen'.  This will avoid naming conflicts with anyone
> else's fsimage library.
> 
> Signed-off-by: Ian Jackson 

Acked-by: Wei Liu 

I think we will need another upgrade note for 4.12.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 01/12] tools/Rules.mk: Honour PREPEND_LDFLAGS_XEN_TOOLS

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:12:04PM +0100, Ian Jackson wrote:
> From: Ian Jackson 
> 
> This allows the caller to provide some LDFLAGS to the Xen build
> system.
> 
> Signed-off-by: Ian Jackson 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 08/12] xenstore.h: Put ( ) around XS_* define shifts

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:12:11PM +0100, Ian Jackson wrote:
> These definitions were not properly protected from unwanted operator
> precedence interactions.
> 
> Existing use sites in-tree all use & or |, so this does not change any
> actual behaviour in-tree.
> 
> The same seems likely to be true in external callers.
> 
> Signed-off-by: Ian Jackson 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 12/12] tools/libfsimage: Rename /usr/lib/fs to /usr/lib/xenfsimage

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:12:15PM +0100, Ian Jackson wrote:
> Again, avoid namespace pollution.  These paths are purely internal to
> libfsimage and its fs-specific modules, so no visible change from the
> outside.
> 
> Signed-off-by: Ian Jackson 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 11/12] tools/pygrub: Add `xen' to fsimage python module name

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:12:14PM +0100, Ian Jackson wrote:
> This module should be called `libxenfsimage' for the same reasons that
> the C library should.
> 
> Signed-off-by: Ian Jackson 

Acked-by: Wei Liu 

Upgrade note is needed.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 16/17] x86: introduce new defconfigs for PV and HVM

2018-10-12 Thread Jan Beulich
>>> On 12.10.18 at 17:34,  wrote:
> On Fri, Oct 12, 2018 at 09:19:45AM -0600, Jan Beulich wrote:
>> >>> On 04.10.18 at 17:43,  wrote:
>> > They will be used by build tests.
>> 
>> And is it difficult for the build tests to set these up themselves?
> 
> I specifically said "build tests" but not "CI systems".
> 
> It wouldn't be very difficult for a CI system to generate these files,
> but my idea is that providing these in tree can make things more
> convenient for humans -- developers and maintainers.
> 
> Suppose you want a contributor to test their changes, instead of writing
> "first please generate the following files, and then ...", you just give
> them a rune / some runes to reference these files directly. There
> doesn't need explaining and no ambiguity will arise. And if there is
> disagreement on what work or what doesn't, it would be easy to
> reproduce.
> 
> To me this is a clear win: a small investment that comes with long term
> benefit.

Depends: At least for now I don't see it as an important goal to
avoid breaking the no-PV and no-HVM case (breaking the individual
ones is another thing). Furthermore, saying to someone "Set
HVM=n in your .config" is not a big deal at all.

What I want to avoid is that, once we gain further non-expert
options, (almost) all possible combinations of options end up as
defconfig-s. We don't have no-shadow and bigmem defconfigs,
yet I think we (should) care about not breaking those builds (I
routinely check them every once in a while).

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/6] x86/dom0: switch parse_dom0_param to use parse_boolean

2018-10-12 Thread Jan Beulich
>>> On 09.10.18 at 11:42,  wrote:
> No functional change expected.
> 
> Signed-off-by: Roger Pau Monné 

Reviewed-by: Jan Beulich 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 3/6] x86/pvh: allow PVH Dom0 to use the debug IO port console

2018-10-12 Thread Jan Beulich
>>> On 09.10.18 at 11:42,  wrote:
> --- a/docs/misc/xen-command-line.markdown
> +++ b/docs/misc/xen-command-line.markdown
> @@ -681,6 +681,17 @@ Flag that makes a dom0 boot in PVHv2 mode.
>  Flag that makes a dom0 use shadow paging. Only works when "pvh" is
>  enabled.
>  
> +> `debug-ioport`
> +
> +> Default: `false`
> +
> +Flag that enables the HVM debug console for a PVH Dom0. Xen will trap 
> accesses
> +to IO port 0xe9 so that Dom0 kernel can print output using this IO port 
> before
> +setting up the hypercall page.
> +
> +Note this option is not enabled by default because it might clash with 
> hardware
> +on the system using IO port 0xe9 and should only be used for debug purposes.

You also want to extend the "List of" at the top of the option.

> --- a/xen/arch/x86/dom0_build.c
> +++ b/xen/arch/x86/dom0_build.c
> @@ -212,12 +212,16 @@ struct vcpu *__init alloc_dom0_vcpu0(struct domain 
> *dom0)
>  bool __initdata opt_dom0_shadow;
>  #endif
>  bool __initdata dom0_pvh;
> +#ifdef CONFIG_HVM
> +bool __initdata opt_dom0_debug_ioport;
> +#endif

And dom0_pvh does not go into this same conditional?

> @@ -236,6 +240,10 @@ static int __init parse_dom0_param(const char *s)
>  #ifdef CONFIG_SHADOW_PAGING
>  else if ( (val = parse_boolean("shadow", s, ss)) >= 0 )
>  opt_dom0_shadow = val;
> +#endif
> +#ifdef CONFIG_HVM
> +else if ( (val = parse_boolean("debug-ioport", s, ss)) >= 0 )
> +opt_dom0_debug_ioport = val;
>  #endif
>  else
>  rc = -EINVAL;
> @@ -433,6 +441,11 @@ int __init dom0_setup_permissions(struct domain *d)
>  rc |= ioports_deny_access(d, pmtmr_ioport, pmtmr_ioport + 3);
>  /* PCI configuration space (NB. 0xcf8 has special treatment). */
>  rc |= ioports_deny_access(d, 0xcfc, 0xcff);
> +#ifdef CONFIG_HVM
> +if ( is_hvm_domain(d) && opt_dom0_debug_ioport )
> +/* HVM debug console IO port. */
> +rc |= ioports_deny_access(d, 0xe9, 0xe9);

I think we finally need a manifest constant for this port number.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] x86/HVM: adjust hvm_interrupt_blocked()

2018-10-12 Thread Jan Beulich
First of all, hvm_intsrc_mce was not considered here at all, yet nothing
blocks #MC (other than an already in-progress #MC, but dealing with this
is not the purpose of this patch).

Additionally STI-shadow only blocks maskable interrupts, but not NMI.

Signed-off-by: Jan Beulich 

--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3771,19 +3771,24 @@ enum hvm_intblk hvm_interrupt_blocked(st
 return intr;
 }
 
-if ( (intack.source != hvm_intsrc_nmi) &&
- !(guest_cpu_user_regs()->eflags & X86_EFLAGS_IF) )
-return hvm_intblk_rflags_ie;
+if ( intack.source == hvm_intsrc_mce )
+return hvm_intblk_none;
 
 intr_shadow = hvm_funcs.get_interrupt_shadow(v);
 
-if ( intr_shadow & (HVM_INTR_SHADOW_STI|HVM_INTR_SHADOW_MOV_SS) )
+if ( intr_shadow & HVM_INTR_SHADOW_MOV_SS )
 return hvm_intblk_shadow;
 
 if ( intack.source == hvm_intsrc_nmi )
 return ((intr_shadow & HVM_INTR_SHADOW_NMI) ?
 hvm_intblk_nmi_iret : hvm_intblk_none);
 
+if ( intr_shadow & HVM_INTR_SHADOW_STI )
+return hvm_intblk_shadow;
+
+if ( !(guest_cpu_user_regs()->eflags & X86_EFLAGS_IF) )
+return hvm_intblk_rflags_ie;
+
 if ( intack.source == hvm_intsrc_lapic )
 {
 uint32_t tpr = vlapic_get_reg(vcpu_vlapic(v), APIC_TASKPRI) & 0xF0;





___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 13/17] x86: don't create Dom0 if neither PV nor HVM is available

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:28:21PM +0100, Andrew Cooper wrote:
> On 12/10/18 16:12, Jan Beulich wrote:
>  On 04.10.18 at 17:43,  wrote:
> >> This will give us a Xen setting in idle loop. This doesn't have
> >> practical use except for debugging purpose.
> > Hmm, that's an acceptable alternative to the panic() variant, but
> > I'd still prefer that one. Again - let's see if Andrew has an opinion
> > either way.
> 
> I think that, for the purpose of keeping our interfaces clean, being
> able to compile without PV and without HVM is a useful property
> (especially as randconfig should hit it one in every 4 cases).
> 
> I've specifically needed to have no dom0 for certain debugging in the
> past.  (The HPET series to fix the broken MSI handler, which I realise
> still hasn't made its way upstream yet.)
> 
> I'm less certain however if implementing it like this is wise.  I
> implemented it with a "nodom0" command line parameter, and left the rest
> of the functionality intact.
> 
> In particular, one fuzzing idea I have is to have a tiny AFL stub on the
> end of the serial port, at which point, having no dom0 but PV and HVM
> fully compiled in would be the ideal position.
> 
> I'm sorry if this isn't necessarily a very helpful answer.  I'd be
> tempted to drop this patch for now, and let the first person with a
> concrete usecase to decide exactly how a no-dom0 system would look.

Sure. I don't mind dropping this patch.

Wei.

> 
> ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86: deal with firmware setting bogus TSC_ADJUST values

2018-10-12 Thread Wei Liu
On Mon, Oct 01, 2018 at 07:42:12AM -0600, Jan Beulich wrote:
> The system Intel have handed me for AVX512 emulator work ("Gigabyte
> Technology Co., Ltd. X299 AORUS Gaming 3 Pro/X299 AORUS Gaming 3
> Pro-CF, BIOS F3 12/28/2017") would not come up under Xen - it hung in
> the middle of Dom0 PCI initialization. As it turned out, Xen's time
> management did not work because of the firmware setting (only) the boot
> CPU's TSC_ADJUST MSR to a large negative value (on the order of -2^50).
> 
> Follow Linux (also shamelessly stealing their comments) in

Is there a specific commit or a range of commits in Linux that you can
put here?

> - clearing the register for the boot CPU (we don't have a need for
>   exceptions here yet, as the only exception in Linux is a class of
>   systems Xen doesn't work on anyway as far as I'm aware),
> - forcing non-negative values uniformly,
> - syncing the registers within sockets.
> Linux caps at 0x7fff as well, but their comment saying "as those
> wreckage the timer as well" does, to me at least, neither really explain

I tried to pin down what Linux does by searching the comment here but
nothing showed up -- searching "wreckage" on Linux master only yielded
three results, none of which matched the one you wrote here.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 10/12] tools/libfsimage: Add `xen' to .h names and principal .so name

2018-10-12 Thread Ian Jackson
Wei Liu writes ("Re: [PATCH 10/12] tools/libfsimage: Add `xen' to .h names and 
principal .so name"):
> On Fri, Oct 12, 2018 at 04:12:13PM +0100, Ian Jackson wrote:
> > `fsimage' is rather general.  And we do not expect this library to be
> > very useful out of tree because of its unstable ABI.
> > 
> > So add the word `xen'.  This will avoid naming conflicts with anyone
> > else's fsimage library.
> > 
> > Signed-off-by: Ian Jackson 
> 
> Acked-by: Wei Liu 
> 
> I think we will need another upgrade note for 4.12.

Sure, if you think so.

Juergen, something like:

 * The fsimage library used by pygrub, which does not have a stable
   ABI, and the corresponding python module, have been renamed to
   `xenfsimage' (to reduce namespace pollution).  Any out-of-tree
   users of this library will have to be updated.

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen optimization

2018-10-12 Thread Julien Grall
Hi,

Sorry for the formatting.

On Fri, 12 Oct 2018, 17:36 Milan Boberic,  wrote:

> Hi Stefano, glad to have you back :D,
> this is my setup:
> - dom0 is PetaLinux, has 1 vCPU and it's pinned for pCPU0
> - there is only one domU and this is my bare-metal app that also
> have one vCPU and it's pinned for pCPU1
> so yeah, there is only dom0 and bare-metal app on the board.
>
> Jitter is the same with and without Dario's patch.
>
> I'm still not sure about timer's passthrough because there is no mention
> of triple timer counter is device tree so I added:
>
> &ttc0 {
>xen,passthrough = <0x1>;
> };
>

Would you mind to explain what is the triple timer counter?



> at the end of the xen-overlay.dtsi file which I included in attachment.
>
> About patch you sent, I can't find this funcion void vgic_inject_irq in
> /xen/arch/arm/vgic.c file, this is link of git repository from where I
> build my xen so you can take a look if that printk can be put somewhere
> else.
>

There was some vGIC rework in Xen 4.11. There was also a new vGIC added
(selectable using NEW_VGIC). It might be worth to look at it.


> https://github.com/Xilinx/xen/
>

This is not the official Xen repository and look like patches have been
applied on top. I am afraid, I am not going to be able help here. Could you
do the same experiment with Xen 4.11?


>
> I ran some more testing and realized that results are the same with or
> without vwfi=native, which I think again points out that passthrough that I
> need to provide in device tree isn't valid.
>

This could also means that wfi is not used by the guest or you never go to
the idle vCPU.


>  And of course, higher the frequency of interrupts results in higher
> jitter. I'm still battling with Xilinx SDK and triple timer counter that's
> why I can't figure out what is the exact frequency set (I'm just rising it
> and lowering it), I'll give my best to solve that ASAP because we need to
> know exact value of frequency set.
>
> Thanks in advance!
>
> Milan
>
>
>
> On Fri, Oct 12, 2018 at 12:29 AM Stefano Stabellini <
> stefano.stabell...@xilinx.com> wrote:
>
>> On Thu, 11 Oct 2018, Milan Boberic wrote:
>> > On Wed, Oct 10, 2018 at 6:41 PM Meng Xu  wrote:
>> > >
>> > > The jitter may come from Xen or the OS in dom0.
>> > > It will be useful to know what is the jitter if you run the test on
>> PetaLinux.
>> > > (It's understandable the jitter is gone without OS. It is also common
>> > > that OS introduces various interferences.)
>> >
>> > Hi Meng,
>> > well... I'm using bare-metal application and I need it exclusively to
>> > be ran on one CPU as domU (guest) without OS (and I'm not sure how
>> > would I make the same app to be ran on PetaLinux dom0 :D haha).
>> > Is there a chance that PetaLinux as dom0 is creating this jitter and
>> > how? Is there a way of decreasing it?
>> >
>> > Yes, there are no prints.
>> >
>> > I'm not sure about this timer interrupt passthrough because I didn't
>> > find any example of it, in attachment I included xen-overlay.dtsi file
>> > which I edited to add passthrough, in earlier replies there are
>> > bare-metal configuration file. It would be helpful to know if those
>> > setting are correct. If they are not correct it would explain the
>> > jitter.
>> >
>> > Thanks in advance, Milan Boberic!
>>
>> Hi Milan,
>>
>> Sorry for taking so long to go back to this thread. But I am here now :)
>>
>> First, let me ask a couple of questions to understand the scenario
>> better: is there any interference from other virtual machines while you
>> measure the jitter? Or is the baremetal app the only thing actively
>> running on the board?
>>
>> Second, it would be worth double-checking that Dario's patch to fix
>> sched=null is not having unexpected side effects. I don't think so, it
>> would be worth testing with it and without it to be sure.
>>
>> I gave a look at your VM configuration. The configuration looks correct.
>> There is no dtdev settings, but given that none of the devices you are
>> assigning to the guest does any DMA, it should be OK. You want to make
>> sure that Dom0 is not trying to use those same devices -- make sure to
>> add "xen,passthrough;" to each corresponding node on the host device
>> tree.
>>
>> The error messages "No valid vCPU found" are due to the baremetal
>> applications trying to configure as target cpu for the interrupt cpu1
>> (the second cpu in the system), while actually only 1 vcpu is assigned
>> to the VM. Hence, only cpu0 is allowed. I don't think it should cause
>> any jitter issues, because the request is simply ignored. Just to be
>> safe, you might want to double check that the physical interrupt is
>> delivered to the right physical cpu, which would be cpu1 in your
>> configuration, the one running the only vcpu of the baremetal app. You
>> can do that by adding a printk to xen/arch/arm/vgic.c:vgic_inject_irq,
>> for example:
>>
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index 5a4f082..208fde7 

Re: [Xen-devel] [PATCH] x86/HVM: adjust hvm_interrupt_blocked()

2018-10-12 Thread Andrew Cooper
On 12/10/18 16:58, Jan Beulich wrote:
> First of all, hvm_intsrc_mce was not considered here at all, yet nothing
> blocks #MC (other than an already in-progress #MC, but dealing with this
> is not the purpose of this patch).

I don't believe we've got sufficient infrastructure to fix this
reasonably yet, but for the record, the real behaviour for MCEs is:

If intel
    broadcast to every thread covered by the MCE bank
else if AMD
    sent to the thread with the lowest id covered by the MCE bank

When trying to inject:

if !CR4.MCE or MCG_STATUS.MCIP
    shutdown

Furthermore, I believe even #MC is blocked by the MOVSS shadow, because
the purpose of the shadow is to indicate "my stack is not safe to take
an exception".

> Additionally STI-shadow only blocks maskable interrupts, but not NMI.

This has been discussed on LKML in the past, but `STI; HLT` will
deadlock if NMIs don't respect the STI shadow.

An NMI which hits that instruction boundary will IRET with IF clear, at
which point the core will halt and never wake up.

I believe the input from the vendor architects was that some very old
cores suffer from this problem, but anything you can get yours hand on
today will respect the STI shadow.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] amd-iommu: use correct constants in amd_iommu_get_next_table_from_pte()...

2018-10-12 Thread Woods, Brian
On Wed, Sep 26, 2018 at 02:44:07PM +0100, Paul Durrant wrote:
> ...and change the name to amd_iommu_get_address_from_pte() since the
> address read is not necessarily the address of a next level page table.
> (If the 'next level' field is not 1 - 6 then the address is a page
> address).
> 
> The constants in use prior to this patch relate to device table entries
> rather than page table entries. Although they do have the same value, it
> makes the code confusing to read.
> 
> This patch also changes the PDE/PTE pointer argument to void *, and
> removes any u32/uint32_t casts in the call sites. Unnecessary casts
> surrounding call sites are also removed.
> 
> No functional change.
> 
> NOTE: The patch also adds emacs boilerplate to iommu_map.c
> 
> Signed-off-by: Paul Durrant 

Not thrilled about adding the emacs part but it's allowed in the coding
style guide so.

Reviewd-by: Brian Woods 

-- 
Brian Woods

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2] amd-iommu: use correct constants in amd_iommu_get_next_table_from_pte()...

2018-10-12 Thread Paul Durrant
> -Original Message-
> From: Woods, Brian [mailto:brian.wo...@amd.com]
> Sent: 12 October 2018 17:59
> To: Paul Durrant 
> Cc: xen-devel@lists.xenproject.org; Suthikulpanit, Suravee
> ; Woods, Brian 
> Subject: Re: [PATCH v2] amd-iommu: use correct constants in
> amd_iommu_get_next_table_from_pte()...
> 
> On Wed, Sep 26, 2018 at 02:44:07PM +0100, Paul Durrant wrote:
> > ...and change the name to amd_iommu_get_address_from_pte() since the
> > address read is not necessarily the address of a next level page table.
> > (If the 'next level' field is not 1 - 6 then the address is a page
> > address).
> >
> > The constants in use prior to this patch relate to device table entries
> > rather than page table entries. Although they do have the same value, it
> > makes the code confusing to read.
> >
> > This patch also changes the PDE/PTE pointer argument to void *, and
> > removes any u32/uint32_t casts in the call sites. Unnecessary casts
> > surrounding call sites are also removed.
> >
> > No functional change.
> >
> > NOTE: The patch also adds emacs boilerplate to iommu_map.c
> >
> > Signed-off-by: Paul Durrant 
> 
> Not thrilled about adding the emacs part but it's allowed in the coding
> style guide so.

Not pretty but it's helpful to those of us who are constantly flipping between 
4 different source bases with 4 different coding styles.

> 
> Reviewd-by: Brian Woods 
> 

Thanks,

  Paul

> --
> Brian Woods

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v1 2/6] x86/vvmx: correct vmfail() usage for vmptrld and vmclear

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:27:55PM +0100, Sergey Dyasli wrote:
> Calling vmfail_valid() is correct only if vvmcx is valid. Modify
> functions to use vmfail() instead which performs the necessary check.
> 
> Signed-off-by: Sergey Dyasli 

Reviewed-by: Wei Liu 

I think the code is a bit convoluted because what fields get access is
hidden behind layers of macros and functions. In order to catch future
issues, may I suggest you add some assertions to vmfail_{in,}valid?

Like

   ASSERT(!cpu_has_vmx_vmcs_shadowing && vvmcx_invalid(v));

in vmfail_invalid. Something similar can be added in vmfail_valid as
well.

> ---
>  xen/arch/x86/hvm/vmx/vvmx.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
> index 8eee6d0ea8..26b7d72660 100644
> --- a/xen/arch/x86/hvm/vmx/vvmx.c
> +++ b/xen/arch/x86/hvm/vmx/vvmx.c
> @@ -1744,7 +1744,7 @@ int nvmx_handle_vmptrld(struct cpu_user_regs *regs)
>   !map_io_bitmap_all(v) ||
>   !_map_msr_bitmap(v) )
>  {
> -vmfail_valid(regs, VMX_INSN_VMPTRLD_INVALID_PHYADDR);
> +vmfail(regs, VMX_INSN_VMPTRLD_INVALID_PHYADDR);
>  goto out;
>  }
>  }
> @@ -1828,7 +1828,7 @@ int nvmx_handle_vmclear(struct cpu_user_regs *regs)
>  if ( rc == VMSUCCEED )
>  vmsucceed(regs);
>  else if ( rc == VMFAIL_VALID )
> -vmfail_valid(regs, VMX_INSN_VMCLEAR_INVALID_PHYADDR);
> +vmfail(regs, VMX_INSN_VMCLEAR_INVALID_PHYADDR);
>  else
>  vmfail_invalid(regs);
>  
> -- 
> 2.17.1
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v1 1/6] x86/vvmx: introduce vvmcx_valid()

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 04:27:54PM +0100, Sergey Dyasli wrote:
> As a convenient helper function and refactor the code to use it.
> 
> No functional change.
> 
> Signed-off-by: Sergey Dyasli 

Reviewed-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] amd-iommu: get rid of pointless IOMMU_PAGING_MODE_LEVEL_X definitions

2018-10-12 Thread Woods, Brian
On Thu, Sep 27, 2018 at 01:46:01PM +0100, Paul Durrant wrote:
> The levels are absolute numbers such that IOMMU_PAGING_MODE_LEVEL_X
> evaluates to X (for the valid range of 0 - 7) so simply use numbers in
> the code.
> 
> No functional change.
> 
> NOTE: This patch also adds emacs boilerplate to amd-iommu-defs.h
> 
> Signed-off-by: Paul Durrant 

Is there a strong reason to get rid of these?  Some of examples below
create seemingly magic numbers in the code.  While if you're familiar
with the functions this isn't a big deal, otherwise you have to dig
further to tell.

> +pte = table + pfn_to_pde_idx(gfn, 1);

> +need_flush = set_iommu_pde_present(pde, next_mfn, 0, iw, ir);

If there's a general consensus that getting rid of these is better, I
don't mind and will agree to it.

-- 
Brian Woods

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 03/12] gdbsx: Honour LDFLAGS when linking

2018-10-12 Thread Elena Ufimtseva
On Fri, Oct 12, 2018 at 04:12:06PM +0100, Ian Jackson wrote:
> From: Ian Jackson 
> 
> This command does the link, so it needs LDFLAGS.
> 
> Signed-off-by: Ian Jackson 

Acked-by: Elena Ufimtseva 

> ---
>  tools/debugger/gdbsx/Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/debugger/gdbsx/Makefile b/tools/debugger/gdbsx/Makefile
> index 723a2743cc..8d7cd94a31 100644
> --- a/tools/debugger/gdbsx/Makefile
> +++ b/tools/debugger/gdbsx/Makefile
> @@ -26,7 +26,7 @@ uninstall:
>   rm -f $(DESTDIR)$(sbindir)/gdbsx
>  
>  gdbsx: gx/gx_all.a xg/xg_all.a 
> - $(CC) -o $@ $^
> + $(CC) $(LDFLAGS) -o $@ $^
>  
>  xg/xg_all.a:
>   $(MAKE) -C xg
> -- 
> 2.11.0
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 16/17] x86: introduce new defconfigs for PV and HVM

2018-10-12 Thread Wei Liu
On Fri, Oct 12, 2018 at 09:43:43AM -0600, Jan Beulich wrote:
> >>> On 12.10.18 at 17:34,  wrote:
> > On Fri, Oct 12, 2018 at 09:19:45AM -0600, Jan Beulich wrote:
> >> >>> On 04.10.18 at 17:43,  wrote:
> >> > They will be used by build tests.
> >> 
> >> And is it difficult for the build tests to set these up themselves?
> > 
> > I specifically said "build tests" but not "CI systems".
> > 
> > It wouldn't be very difficult for a CI system to generate these files,
> > but my idea is that providing these in tree can make things more
> > convenient for humans -- developers and maintainers.
> > 
> > Suppose you want a contributor to test their changes, instead of writing
> > "first please generate the following files, and then ...", you just give
> > them a rune / some runes to reference these files directly. There
> > doesn't need explaining and no ambiguity will arise. And if there is
> > disagreement on what work or what doesn't, it would be easy to
> > reproduce.
> > 
> > To me this is a clear win: a small investment that comes with long term
> > benefit.
> 
> Depends: At least for now I don't see it as an important goal to
> avoid breaking the no-PV and no-HVM case (breaking the individual
> ones is another thing). Furthermore, saying to someone "Set
> HVM=n in your .config" is not a big deal at all.
> 
> What I want to avoid is that, once we gain further non-expert
> options, (almost) all possible combinations of options end up as
> defconfig-s. We don't have no-shadow and bigmem defconfigs,
> yet I think we (should) care about not breaking those builds (I
> routinely check them every once in a while).

I think I will just move these two configs somewhere else, say, under
automation directory, since they will be used there. We can also add the
configs you care about there.

This satisfies my requirement as well as yours.

Wei.

> 
> Jan
> 
> 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] amd-iommu: get rid of pointless IOMMU_PAGING_MODE_LEVEL_X definitions

2018-10-12 Thread Paul Durrant
> -Original Message-
> From: Woods, Brian [mailto:brian.wo...@amd.com]
> Sent: 12 October 2018 18:14
> To: Paul Durrant 
> Cc: xen-devel@lists.xenproject.org; Suthikulpanit, Suravee
> ; Woods, Brian 
> Subject: Re: [PATCH] amd-iommu: get rid of pointless
> IOMMU_PAGING_MODE_LEVEL_X definitions
> 
> On Thu, Sep 27, 2018 at 01:46:01PM +0100, Paul Durrant wrote:
> > The levels are absolute numbers such that IOMMU_PAGING_MODE_LEVEL_X
> > evaluates to X (for the valid range of 0 - 7) so simply use numbers in
> > the code.
> >
> > No functional change.
> >
> > NOTE: This patch also adds emacs boilerplate to amd-iommu-defs.h
> >
> > Signed-off-by: Paul Durrant 
> 
> Is there a strong reason to get rid of these?  Some of examples below
> create seemingly magic numbers in the code.  While if you're familiar
> with the functions this isn't a big deal, otherwise you have to dig
> further to tell.
> 

The numbers aren't magic though. The spec refers to levels by number rather 
than any sort of name. If the levels were named then it would be absolutely 
right to #define  , but that is not the case. Thus 
IMO getting rid of the definitions actually makes the code clearer for those 
(like myself) reading the spec.

> > +pte = table + pfn_to_pde_idx(gfn, 1);
> 
> > +need_flush = set_iommu_pde_present(pde, next_mfn, 0, iw, ir);
> 
> If there's a general consensus that getting rid of these is better, I
> don't mind and will agree to it.
> 

Anyone else care to comment?

  Paul

> --
> Brian Woods

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Xen optimization

2018-10-12 Thread Stefano Stabellini
On Fri, 12 Oct 2018, Milan Boberic wrote:
> Hi Stefano, glad to have you back :D,
> this is my setup:
>         - dom0 is PetaLinux, has 1 vCPU and it's pinned for pCPU0
>         - there is only one domU and this is my bare-metal app that also have 
> one vCPU and it's pinned for pCPU1
> so yeah, there is only dom0 and bare-metal app on the board.
> 
> Jitter is the same with and without Dario's patch.
> 
> I'm still not sure about timer's passthrough because there is no mention of 
> triple timer counter is device tree so I added:
> 
> &ttc0 {
>    xen,passthrough = <0x1>;
> };
> 
> at the end of the xen-overlay.dtsi file which I included in attachment.

This is definitely wrong. Can you please also post the full host device
tree with your modifications that you are using for Xen and Dom0?  You
should have something like:


timer@ff11 {
compatible = "cdns,ttc";
interrupt-parent = <0x2>;
interrupts = <0x0 0x24 0x4 0x0 0x25 0x4 0x0 0x26 0x4>;
reg = <0x0 0xff11 0x0 0x1000>;
timer-width = <0x20>;
power-domains = <0x3b>;
xen,passthrough;
};

For each of the nodes of the devices you are assigning to the DomU.


> About patch you sent, I can't find this funcion void vgic_inject_irq in 
> /xen/arch/arm/vgic.c file, this is link of git repository
> from where I build my xen so you can take a look if that printk can be put 
> somewhere else.
> 
> https://github.com/Xilinx/xen/

It's here: 
https://github.com/Xilinx/xen/blob/xilinx/stable-4.9/xen/arch/arm/vgic.c#L462

BTW you are using a pretty old branch, I suggest you moving to:

https://github.com/Xilinx/xen/tree/xilinx/versal/xen/arch/arm

It will work on your board too and it is based on the much newer Xen
4.11.


> I ran some more testing and realized that results are the same with or 
> without vwfi=native, which I think again points out that
> passthrough that I need to provide in device tree isn't valid.

In reality, the results are the same with and without vwfi=native only
if the baremetal app never issues any wfi instructions.


>  And of course, higher the frequency of interrupts results in higher jitter. 
> I'm still battling with Xilinx SDK and triple timer
> counter that's why I can't figure out what is the exact frequency set (I'm 
> just rising it and lowering it), I'll give my best to
> solve that ASAP because we need to know exact value of frequency set. 

Yep, that's important :-)


> 
> Thanks in advance!
> 
> Milan
>  
> 
> 
> On Fri, Oct 12, 2018 at 12:29 AM Stefano Stabellini 
>  wrote:
>   On Thu, 11 Oct 2018, Milan Boberic wrote:
>   > On Wed, Oct 10, 2018 at 6:41 PM Meng Xu  wrote:
>   > >
>   > > The jitter may come from Xen or the OS in dom0.
>   > > It will be useful to know what is the jitter if you run the test on 
> PetaLinux.
>   > > (It's understandable the jitter is gone without OS. It is also 
> common
>   > > that OS introduces various interferences.)
>   >
>   > Hi Meng,
>   > well... I'm using bare-metal application and I need it exclusively to
>   > be ran on one CPU as domU (guest) without OS (and I'm not sure how
>   > would I make the same app to be ran on PetaLinux dom0 :D haha).
>   > Is there a chance that PetaLinux as dom0 is creating this jitter and
>   > how? Is there a way of decreasing it?
>   >
>   > Yes, there are no prints.
>   >
>   > I'm not sure about this timer interrupt passthrough because I didn't
>   > find any example of it, in attachment I included xen-overlay.dtsi file
>   > which I edited to add passthrough, in earlier replies there are
>   > bare-metal configuration file. It would be helpful to know if those
>   > setting are correct. If they are not correct it would explain the
>   > jitter.
>   >
>   > Thanks in advance, Milan Boberic!
> 
>   Hi Milan,
> 
>   Sorry for taking so long to go back to this thread. But I am here now :)
> 
>   First, let me ask a couple of questions to understand the scenario
>   better: is there any interference from other virtual machines while you
>   measure the jitter? Or is the baremetal app the only thing actively
>   running on the board?
> 
>   Second, it would be worth double-checking that Dario's patch to fix
>   sched=null is not having unexpected side effects. I don't think so, it
>   would be worth testing with it and without it to be sure.
> 
>   I gave a look at your VM configuration. The configuration looks correct.
>   There is no dtdev settings, but given that none of the devices you are
>   assigning to the guest does any DMA, it should be OK. You want to make
>   sure that Dom0 is not trying to use those same devices -- make sure to
>   add "xen,passthrough;" to each corresponding node on the host device
>   tree.
> 
>   The error messages "No valid vCPU found" are due to the barem

[Xen-devel] [RFC PATCH v1 0/8] Series short description

2018-10-12 Thread Dario Faggioli
Hello,

Here it comes, core-scheduling for Credit2 as well. Well, this time,
it's actually group-scheduling (see below).

 git://xenbits.xen.org/people/dariof/xen.git 
rel/sched/credit2/group-scheduling-RFCv1
 
http://xenbits.xen.org/gitweb/?p=people/dariof/xen.git;a=shortlog;h=refs/heads/rel/sched/credit2/group-scheduling-RFCv1

 (Or 
https://github.com/fdario/xen/tree/rel/sched/credit2/group-scheduling-RFCv1 ,
  Or 
https://gitlab.com/dfaggioli/xen/tree/rel/sched/credit2/group-scheduling-RFCv1 )

An RFC series implementing the same feature for Credit1 is here:
https://lists.xenproject.org/archives/html/xen-devel/2018-08/msg02164.html

The two series, however, are completely independent, and I'd recommend
focusing on this one first. In fact, implementing the feature here in
Credit2 was waaay simpler, and the result is, IMO, already a lot better.

Therefore, I expect that the amount of effort required for making this
very series upstreamable to be much smaller than for the Credit1 one.
When this is in, we'll have one scheduler that supports
group-scheduling, and we can focus on what to do with the others.

Let me also point out, that there is some discussion (in the thread of
the Credit1 RFC series [1]), about whether a different approach toward
implementing core/group-scheduling wouldn't be better. I had this code
almost ready already, and so I decided to send it out anyway. If it then
turns out that we have to throw it away, then fine. But, so far, I'm all
but convinced that the way things are done in this series is not our
current best solution to deal with the problems we have at hand.

So, what's in here? Well, we have a generic group scheduling
implementation which seems to me to work reasonably well... For an
RFC. ;-P

I call it generic because, although the main aim is core-scheduling, it
can be made to work (and in fact, it already kind of does) with
different grouping (like node, socket, or arbitrary sets of CPUs).

I does not have the fairness and starvation issues that the RFC series
for Credit1 liked above has. I.e., it already sort-of works. :-D

Some improvements are necessary, mostly because Credit2 is not a fully
work conserving scheduler, and this hurts when we do things like group
scheduling. So we need to add logic for doing some quick load-balancing,
or work stealing, when a CPU goes idle, but that is not that much of a
big deal (I was already thinking to add it anyway).

Finding a way of considering group-scheduling while doing proper load
balancing is also on my todo list. It is less easy than the work
conserving-ification described above, but also less important, IMO.

What's not there? Well, mainly, we're missing updating the docs, and
tracing. About the latter, I have an unfinished patch which adds
tracepoints that will be useful to observe, understand and debug whether
the code behave as we expect. And I'll send out that soon too.

Some notes on the actual patches:
- patches 1 and 2 have been submitted already, but they're necessary
  for testing this series, so I've included them;
- credit2_group_sched=core has been not only boot tested, but I've also
  thrown at him a couple of (basic) workloads.
  credit2_group_sched=no has been also tested not to break things (but,
  e.g., I haven't measured the overhead the series introduces).
  credit2_group_sched=node has been boot tested, but not much else;
- cpupool and CPU hotplug have _not_ been tested;

Thanks and Regards,
Dario

[1] https://lists.xenproject.org/archives/html/xen-devel/2018-09/msg00707.html
https://lists.xenproject.org/archives/html/xen-devel/2018-10/msg01010.html

PS. I'm Cc-ing a few people with which we've discussed these issues, but
only to this cover letter, to avoid spamming you with scheduling code.
Find the patches on the list/git, or ping me, and I'll mail them to
you... :-)
---
Dario Faggioli (8):
  xen: sched: Credit2: during scheduling, update the idle mask before using 
it
  xen: sched: Credit2: avoid looping too much (over runqueues) during load 
balancing
  xen: sched: Credit2: show runqueue id during runqueue dump
  xen: sched: Credit2: generalize topology related bootparam handling
  xen: sched: Credit2 group-scheduling: data structures
  xen: sched: Credit2 group-scheduling: selecting next vcpu to run
  xen: sched: Credit2 group-scheduling: tickling
  xen: sched: Credit2 group-scheduling: anti-starvation measures


 xen/common/sched_credit2.c |  492 +++-
 1 file changed, 439 insertions(+), 53 deletions(-)
--
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v1 3/8] xen: sched: Credit2: show runqueue id during runqueue dump

2018-10-12 Thread Dario Faggioli
Instead than just a sequence number, and consistently
with what's shown when the runqueues are created.

Also add some more pretty printing here and there.

No functional change intended.

Signed-off-by: Dario Faggioli 
---
Cc: George Dunlap 
---
 xen/common/sched_credit2.c |5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 06b45725fa..617a7ece6e 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -3658,7 +3658,7 @@ dump_pcpu(const struct scheduler *ops, int cpu)
 #define cpustr keyhandler_scratch
 
 cpumask_scnprintf(cpustr, sizeof(cpustr), per_cpu(cpu_sibling_mask, cpu));
-printk("CPU[%02d] runq=%d, sibling=%s, ", cpu, c2r(cpu), cpustr);
+printk(" CPU[%02d] runq=%d, sibling=%s, ", cpu, c2r(cpu), cpustr);
 cpumask_scnprintf(cpustr, sizeof(cpustr), per_cpu(cpu_core_mask, cpu));
 printk("core=%s\n", cpustr);
 
@@ -3760,7 +3760,8 @@ csched2_dump(const struct scheduler *ops)
 /* We need the lock to scan the runqueue. */
 spin_lock(&rqd->lock);
 
-printk("Runqueue %d:\n", i);
+printk("Runqueue %d:\n", rqd->id);
+printk("CPUs:\n");
 
 for_each_cpu(j, &rqd->active)
 dump_pcpu(ops, j);


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v1 2/8] xen: sched: Credit2: avoid looping too much (over runqueues) during load balancing

2018-10-12 Thread Dario Faggioli
For doing load balancing between runqueues, we check the load of each
runqueue, select the one more "distant" than our own load, and then take
the proper runq lock and attempt vcpu migrations.

If we fail to take such lock, we try again, and the idea was to give up
and bail if, during the checking phase, we can't take the lock of any
runqueue (check the comment near to the 'goto retry;', in the middle of
balance_load())

However, the variable that controls the "give up and bail" part, is not
reset upon retries. Therefore, provided we did manage to check the load of
at least one runqueue during the first pass, if we can't get any runq lock,
we don't bail, but we try again taking the lock of that same runqueue
(and that may even more than once).

Signed-off-by: Dario Faggioli 
---
Cc: George Dunlap 
---
 xen/common/sched_credit2.c |3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 72fed2dd18..06b45725fa 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -2554,7 +2554,7 @@ static bool vcpu_is_migrateable(struct csched2_vcpu *svc,
 static void balance_load(const struct scheduler *ops, int cpu, s_time_t now)
 {
 struct csched2_private *prv = csched2_priv(ops);
-int i, max_delta_rqi = -1;
+int i, max_delta_rqi;
 struct list_head *push_iter, *pull_iter;
 bool inner_load_updated = 0;
 
@@ -2573,6 +2573,7 @@ static void balance_load(const struct scheduler *ops, int 
cpu, s_time_t now)
 update_runq_load(ops, st.lrqd, 0, now);
 
 retry:
+max_delta_rqi = -1;
 if ( !read_trylock(&prv->lock) )
 return;
 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v1 1/8] xen: sched: Credit2: during scheduling, update the idle mask before using it

2018-10-12 Thread Dario Faggioli
Load balancing, when happening, at the end of a "scheduler epoch", can
trigger vcpu migration, which in its turn may call runq_tickle(). If the
cpu where this happens was idle, but we're now going to schedule a vcpu
on it, let's update the runq's idle cpus mask accordingly _before_ doing
load balancing.

Not doing that, in fact, may cause runq_tickle() to think that the cpu
is still idle, and tickle it to go pick up a vcpu from the runqueue,
which might be wrong/unideal.

Signed-off-by: Dario Faggioli 
---
Cc: George Dunlap 
---
 xen/common/sched_credit2.c |   14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 2b16bcea21..72fed2dd18 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -3554,6 +3554,13 @@ csched2_schedule(
 __set_bit(__CSFLAG_scheduled, &snext->flags);
 }
 
+/* Clear the idle mask if necessary */
+if ( cpumask_test_cpu(cpu, &rqd->idle) )
+{
+__cpumask_clear_cpu(cpu, &rqd->idle);
+smt_idle_mask_clear(cpu, &rqd->smt_idle);
+}
+
 /*
  * The reset condition is "has a scheduler epoch come to an end?".
  * The way this is enforced is checking whether the vcpu at the top
@@ -3574,13 +3581,6 @@ csched2_schedule(
 balance_load(ops, cpu, now);
 }
 
-/* Clear the idle mask if necessary */
-if ( cpumask_test_cpu(cpu, &rqd->idle) )
-{
-__cpumask_clear_cpu(cpu, &rqd->idle);
-smt_idle_mask_clear(cpu, &rqd->smt_idle);
-}
-
 snext->start_time = now;
 snext->tickled_cpu = -1;
 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v1 4/8] xen: sched: Credit2: generalize topology related bootparam handling

2018-10-12 Thread Dario Faggioli
Right now, runqueue organization is the only bit of the scheduler that
use such topology related information. But that may not be true forever,
i.e., there may be other boot parameters which takes the same "core",
"socket", etc, strings as argument.

In fact, this is the case of the credit2_group_sched parameter,
introduced in later patches.

Therefore, let's:
- make the #define-s more general, i.e., RUNQUEUE -> TOPOLOGY;
- do the parsing outside of the specific function handling the
  credit2_runqueue param.

While there, we also move "node" before "socket", so that we have them
ordered from the smallest to the largest, and we can do math with their
values. (Yes, I know, relationship between node and socket is not always
that clear, but, I've found boxes, like EPYC, with more than one node in
one socket, and I've never found one where two socket are in the same
node, so...)

No functional change intended.

Signed-off-by: Dario Faggioli 
---
Cc: George Dunlap 
---
 xen/common/sched_credit2.c |   61 +---
 1 file changed, 35 insertions(+), 26 deletions(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 617a7ece6e..9550503b5b 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -434,34 +434,43 @@ integer_param("credit2_cap_period_ms", opt_cap_period);
  * either the same physical core, the same physical socket, the same NUMA
  * node, or just all of them, will be put together to form runqueues.
  */
-#define OPT_RUNQUEUE_CPU0
-#define OPT_RUNQUEUE_CORE   1
-#define OPT_RUNQUEUE_SOCKET 2
-#define OPT_RUNQUEUE_NODE   3
-#define OPT_RUNQUEUE_ALL4
-static const char *const opt_runqueue_str[] = {
-[OPT_RUNQUEUE_CPU] = "cpu",
-[OPT_RUNQUEUE_CORE] = "core",
-[OPT_RUNQUEUE_SOCKET] = "socket",
-[OPT_RUNQUEUE_NODE] = "node",
-[OPT_RUNQUEUE_ALL] = "all"
+#define OPT_TOPOLOGY_CPU0
+#define OPT_TOPOLOGY_CORE   1
+#define OPT_TOPOLOGY_NODE   2
+#define OPT_TOPOLOGY_SOCKET 3
+#define OPT_TOPOLOGY_ALL4
+static const char *const opt_topospan_str[] = {
+[OPT_TOPOLOGY_CPU] = "cpu",
+[OPT_TOPOLOGY_CORE] = "core",
+[OPT_TOPOLOGY_NODE] = "node",
+[OPT_TOPOLOGY_SOCKET] = "socket",
+[OPT_TOPOLOGY_ALL] = "all"
 };
-static int __read_mostly opt_runqueue = OPT_RUNQUEUE_SOCKET;
 
-static int __init parse_credit2_runqueue(const char *s)
+static int __init parse_topology_span(const char *s)
 {
 unsigned int i;
 
-for ( i = 0; i < ARRAY_SIZE(opt_runqueue_str); i++ )
+for ( i = 0; i < ARRAY_SIZE(opt_topospan_str); i++ )
 {
-if ( !strcmp(s, opt_runqueue_str[i]) )
-{
-opt_runqueue = i;
-return 0;
-}
+if ( !strcmp(s, opt_topospan_str[i]) )
+return i;
 }
 
-return -EINVAL;
+return -1;
+}
+
+static int __read_mostly opt_runqueue = OPT_TOPOLOGY_SOCKET;
+
+static int __init parse_credit2_runqueue(const char *s)
+{
+opt_runqueue = parse_topology_span(s);
+
+if ( opt_runqueue < 0 )
+return -EINVAL;
+
+ASSERT(opt_runqueue <= OPT_TOPOLOGY_ALL );
+return 0;
 }
 custom_param("credit2_runqueue", parse_credit2_runqueue);
 
@@ -883,12 +892,12 @@ cpu_to_runqueue(struct csched2_private *prv, unsigned int 
cpu)
 BUG_ON(cpu_to_socket(cpu) == XEN_INVALID_SOCKET_ID ||
cpu_to_socket(peer_cpu) == XEN_INVALID_SOCKET_ID);
 
-if (opt_runqueue == OPT_RUNQUEUE_CPU)
+if (opt_runqueue == OPT_TOPOLOGY_CPU)
 continue;
-if ( opt_runqueue == OPT_RUNQUEUE_ALL ||
- (opt_runqueue == OPT_RUNQUEUE_CORE && same_core(peer_cpu, cpu)) ||
- (opt_runqueue == OPT_RUNQUEUE_SOCKET && same_socket(peer_cpu, 
cpu)) ||
- (opt_runqueue == OPT_RUNQUEUE_NODE && same_node(peer_cpu, cpu)) )
+if ( opt_runqueue == OPT_TOPOLOGY_ALL ||
+ (opt_runqueue == OPT_TOPOLOGY_CORE && same_core(peer_cpu, cpu)) ||
+ (opt_runqueue == OPT_TOPOLOGY_SOCKET && same_socket(peer_cpu, 
cpu)) ||
+ (opt_runqueue == OPT_TOPOLOGY_NODE && same_node(peer_cpu, cpu)) )
 break;
 }
 
@@ -4021,7 +4030,7 @@ csched2_init(struct scheduler *ops)
opt_load_window_shift,
opt_underload_balance_tolerance,
opt_overload_balance_tolerance,
-   opt_runqueue_str[opt_runqueue],
+   opt_topospan_str[opt_runqueue],
opt_cap_period);
 
 printk(XENLOG_INFO "load tracking window length %llu ns\n",


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v1 8/8] xen: sched: Credit2 group-scheduling: anti-starvation measures

2018-10-12 Thread Dario Faggioli
With group scheduling enabled, if a vcpu of, say, domain A, is already
running on a CPU, the other CPUs of the group can only run vcpus of
that same domain. And in fact, we scan the runqueue and look for one.

But then what can happen is that vcpus of domain A takes turns at
switching between idle/blocked and running, and manage to keep every
other (vcpus of the other) domains out of a group of CPUs for long time,
or even indefinitely (impacting fairness, or causing starvation).

To avoid this, let's limit how deep we go along the runqueue in search
of a vcpu of domain A. That is, if we don't find any that have at least
a certain amount of credits less than what the vcpu at the top of the
runqueue has, give up and keep the CPU idle.

Signed-off-by: Dario Faggioli 
---
Cc: George Dunlap 
---
TODO:
- for now, CSCHED2_MIN_TIMER is what's used as threshold, but this can
  use some tuning (e.g., it probably wants to be adaptive, depending on
  how wide the coscheduling group of CPUs is, etc.)
---
 xen/common/sched_credit2.c |   32 +++-
 1 file changed, 31 insertions(+), 1 deletion(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index d2b4c907dc..a23c8f18d6 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -3476,7 +3476,7 @@ runq_candidate(struct csched2_runqueue_data *rqd,
unsigned int *skipped)
 {
 struct list_head *iter, *temp;
-struct csched2_vcpu *snext = NULL;
+struct csched2_vcpu *first_svc, *snext = NULL;
 struct csched2_private *prv = csched2_priv(per_cpu(scheduler, cpu));
 struct csched2_grpsched_data *gscd = c2gscd(cpu);
 bool yield = false, soft_aff_preempt = false;
@@ -3568,11 +3568,28 @@ runq_candidate(struct csched2_runqueue_data *rqd,
  * Of course, we also default to idle also if scurr is not runnable.
  */
 if ( vcpu_runnable(scurr->vcpu) && !soft_aff_preempt )
+
 snext = scurr;
 else
 snext = csched2_vcpu(idle_vcpu[cpu]);
 
  check_runq:
+/*
+ * To retain fairness, and avoid starvation issues, we don't let
+ * group scheduling make us run vcpus which are too far behing (i.e.,
+ * have less credits) than what is currently in the runqueue.
+ *
+ * XXX Just use MIN_TIMER as the threshold, for now.
+ */
+first_svc = list_entry(&rqd->runq, struct csched2_vcpu, runq_elem);
+if ( grpsched_enabled() && !is_idle_vcpu(scurr->vcpu) &&
+ !list_empty(&rqd->runq) )
+{
+ASSERT(gscd->sdom != NULL);
+if ( scurr->credit < first_svc->credit - CSCHED2_MIN_TIMER )
+snext = csched2_vcpu(idle_vcpu[cpu]);
+}
+
 list_for_each_safe( iter, temp, &rqd->runq )
 {
 struct csched2_vcpu * svc = list_entry(iter, struct csched2_vcpu, 
runq_elem);
@@ -3637,6 +3654,19 @@ runq_candidate(struct csched2_runqueue_data *rqd,
 continue;
 }
 
+/*
+ * As stated above, let's not go too far and risk picking up
+ * a vcpu which has too much lower credits than the one we would
+ * have picked if group scheduling was not enabled.
+ *
+ * There's a risk that this means leaving the CPU idle (if we don't
+ * find vcpus that satisfy this rule, and also the group scheduling
+ * constraints)... but that's what coscheduling is all about!
+ */
+if ( grpsched_enabled() && gscd->sdom != NULL &&
+ svc->credit < first_svc->credit - CSCHED2_MIN_TIMER )
+break;
+
 /*
  * If the one in the runqueue has more credit than current (or idle,
  * if current is not runnable), or if current is yielding, and also


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v1 6/8] xen: sched: Credit2 group-scheduling: selecting next vcpu to run

2018-10-12 Thread Dario Faggioli
When chosing which vcpu to run next, on a CPU which is in a group where
other vcpus are running already, only consider vcpus of the same domain
(of those vcpus that are running already!).

This is as easy as, in runq_candidate(), while traversing the runqueue,
skipping the vcpus that do not satisfy the group-scheduling constraints.

And now that such constraints are actually enforced, also add an ASSERT()
that checks that we really respect them.

Signed-off-by: Dario Faggioli 
---
Cc: George Dunlap 
---
TODO:
- Consider better the interactions between group-scheduling and
  soft-affinity (in runq_candidate() @3481);
---
 xen/common/sched_credit2.c |   44 +++-
 1 file changed, 43 insertions(+), 1 deletion(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index b11713e244..052e050394 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -3414,7 +3414,7 @@ csched2_runtime(const struct scheduler *ops, int cpu,
 /*
  * Find a candidate.
  */
-static struct csched2_vcpu *
+static noinline struct csched2_vcpu *
 runq_candidate(struct csched2_runqueue_data *rqd,
struct csched2_vcpu *scurr,
int cpu, s_time_t now,
@@ -3423,8 +3423,19 @@ runq_candidate(struct csched2_runqueue_data *rqd,
 struct list_head *iter, *temp;
 struct csched2_vcpu *snext = NULL;
 struct csched2_private *prv = csched2_priv(per_cpu(scheduler, cpu));
+struct csched2_grpsched_data *gscd = c2gscd(cpu);
 bool yield = false, soft_aff_preempt = false;
 
+/*
+ * Some more sanity checking. With group scheduling enabled, either:
+ * - the whole coscheduling group is currently idle. Or,
+ * - this CPU is currently idle. Or,
+ * - this CPU is running a vcpu from the same domain of all the
+ *   other one that are running in the group (if any).
+ */
+ASSERT(!grpsched_enabled() || gscd->sdom == NULL ||
+   scurr->sdom == NULL || gscd->sdom == scurr->sdom);
+
 *skipped = 0;
 
 if ( unlikely(is_idle_vcpu(scurr->vcpu)) )
@@ -3473,6 +3484,8 @@ runq_candidate(struct csched2_runqueue_data *rqd,
 {
 cpumask_t *online = cpupool_domain_cpumask(scurr->vcpu->domain);
 
+/* XXX deal with grpsched_enabled() == true */
+
 /* Ok, is any of the pcpus in scurr soft-affinity idle? */
 cpumask_and(cpumask_scratch, cpumask_scratch, &rqd->idle);
 cpumask_andnot(cpumask_scratch, cpumask_scratch, &rqd->tickled);
@@ -3528,6 +3541,23 @@ runq_candidate(struct csched2_runqueue_data *rqd,
 continue;
 }
 
+/*
+ * If groups scheduling is enabled, only consider svc if:
+ * - the whole group is idle. Or,
+ * - one or more other svc->sdom's vcpus are running already in the
+ *   pCPUs of the coscheduling group. Or,
+ * - there is only one vcpu running in the whole coscheduling group,
+ *   and it is running here on this CPU (and svc would preempt it).
+ */
+if ( grpsched_enabled() &&
+ gscd->sdom != NULL && gscd->sdom != svc->sdom &&
+ !(gscd->nr_running == 1 && scurr->sdom != NULL) )
+{
+ASSERT(gscd->nr_running != 0);
+(*skipped)++;
+continue;
+}
+
 /*
  * If a vcpu is meant to be picked up by another processor, and such
  * processor has not scheduled yet, leave it in the runqueue for him.
@@ -3715,6 +3745,18 @@ csched2_schedule(
 runq_remove(snext);
 __set_bit(__CSFLAG_scheduled, &snext->flags);
 
+/*
+ * If group scheduling is enabled, and we're switching to
+ * a non-idle vcpu, either:
+ * - they're from the same domain,
+ * - the whole coscheduling group was idle,
+ * - there was only 1 vcpu running in the whole scheduling group,
+ *   and it was running on this CPU (i.e., this CPU was not idle).
+ */
+ASSERT(!grpsched_enabled() || gscd->sdom == snext->sdom ||
+   (gscd->nr_running == 0 && gscd->sdom == NULL) ||
+   (gscd->nr_running == 1 && !is_idle_vcpu(scurr->vcpu)));
+
 /* Track which domain is running in the coscheduling group */
 gscd->sdom = snext->sdom;
 if ( is_idle_vcpu(scurr->vcpu) )


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v1 5/8] xen: sched: Credit2 group-scheduling: data structures

2018-10-12 Thread Dario Faggioli
Group scheduling is, for us, when a certain group of CPUs can only
execute the vcpus of one domain, at any given time. What CPUs form the
groups can be defined pretty much arbitrarily, but they're usually build
after the system topology. E.g., core-scheduling is a pretty popular
form of group scheduling, where the CPUs that are SMT sibling threads
within one core are in the same group.

So, basically, core-scheduling means that, if we have one core with two
threads, we will never run dAv0 (i.e., vcpu 0 of domain A) and dBv2, on
these two threads. In fact, we either run dAv0 and dAv3 on them, or, if
there's only one dA's vcpu that can run, then one of the thread stays
idle.

Making Credit2 support core-scheduling is the main aim of this patch
series, but the implementation is general, and allows the user to chose
a different granularity/arrangement of the groups (such as, per-NUMA
node groups).

As per this commit only, just the boot command line parameter (to
enable, disable and configure the feature), the data structures and
the domain tracking logic are implemented.

This means that, until we implement the group scheduling logic, in
later commits, the result of such "what domain is running in this group"
logic (which can be seen via `xl debug-keys r') is not to be considered
correct.

Signed-off-by: Dario Faggioli 
---
Cc: George Dunlap 
---
TODO:
- document credit2_group_sched in docs/misc/xen-command-line.markdown;
---
 xen/common/sched_credit2.c |  262 +++-
 1 file changed, 255 insertions(+), 7 deletions(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 9550503b5b..b11713e244 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -171,6 +171,36 @@
  *   pool, must occur only when holding the 'budget_lock'.
  */
 
+/*
+ * Group scheduling:
+ *
+ * A group of physical cpus are said to be coscheduling a domain if only
+ * virtual cpus of that same domain are running on any of the cpus. If there
+ * are not enough (ready to run) vcpus from the domain, some of the pCPUs in
+ * the coscheduling group stay idle.
+ *
+ * So, basically, after we've divided the cpus in coscheduling groups, each
+ * group will run a domain at a time. For instance, the cpus of coscheduling
+ * group A, at any given time, either run the vcpus of a specific guest, or are
+ * idle.
+ *
+ * Typically, coscheduling group are formed after topology consideration, e.g.,
+ * the SMT topology. I.e., all the cpus that share a core live  in the same
+ * coscheduling group. This has started to go under the name of
+ * 'core-scheduling'.
+ *
+ * Enabling a group scheduling like behavior may, depending on a number of
+ * factors, bring benefits from a performance or security point of view.
+ * E.g., core-scheduling could be of help in limiting information leak through
+ * side channel attacks on some SMT systems.
+ *
+ * NB: the term 'gang scheduling' also exist and is used, sometimes as
+ * synonym of coscheduling and group scheduling. However, strictly speaking,
+ * what gang scheduling means is that a certain set of vcpus (typically the
+ * vcpus of a guest) either run together, each on one cpu, or they don't run
+ * at all. And we therefore do not use this term here.
+ */
+
 /*
  * Locking:
  *
@@ -184,6 +214,9 @@
  *  + protects runqueue-wide data in csched2_runqueue_data;
  *  + protects vcpu parameters in csched2_vcpu for the vcpu in the
  *runqueue.
+ *  + protects group-scheduling wide data in csched2_grpsched_data. This
+ *is because we force cpus that are in the same coscheduling group, to
+ *also share the same runqueue.
  *
  * - Private scheduler lock
  *  + protects scheduler-wide data in csched2_private, such as:
@@ -474,6 +507,60 @@ static int __init parse_credit2_runqueue(const char *s)
 }
 custom_param("credit2_runqueue", parse_credit2_runqueue);
 
+/*
+ * Group scheduling.
+ *
+ * We support flexible coscheduling grouping strategies, such as:
+ *
+ * - cpu: meaning no group scheduling happens (i.e., this is how group
+ *scheduling is disabled);
+ *
+ * - core: pCPUs are grouped at the core-level. This means pCPUs that are
+ * sibling hyperthreads within the same core, are made part of
+ * the same group. Therefore, each core only executes one domain at
+ * a time. The number of vCPUs of such domain running on each core
+ * depends on how many threads the core itself has (typically 2, but
+ * systems with 4 threads per-core exists already);
+ *
+ * - node: pCPUs are grouped at the NUMA nodes level. This means all the pCPUs
+ * within a NUMA node, are made part of one group, and hence execute
+ * the vCPUs of one domain at a time. On an SMT systems, this of course
+ * means that all the threads of all the cores inside a node are in
+ * the same group.
+ *
+ * Per-socket --which often is the same than per-node, but not always-- and
+ * even global g

[Xen-devel] [RFC PATCH v1 7/8] xen: sched: Credit2 group-scheduling: tickling

2018-10-12 Thread Dario Faggioli
When chosing which CPU should be poked to go pick up a vcpu from the
runqueue, take group-scheduling into account, if it is enabled.

Basically, we avoid tickling CPUs that, even if they are idle, are part
of coscheduling groups where vcpus of other domains (wrt the one waking
up) are already running. Instead, we actively try to tickle the idle
CPUs within the coscheduling groups where vcpus of the same domain are
currently running.

Signed-off-by: Dario Faggioli 
---
Cc: George Dunlap 
---
TODO:
- deal with sched_smt_power_savings==true;
- optimize the search of appropriate CPUs to be tickled, most likely
  using a per-domain data structure. That will spare us having to do
  a loop.
---
 xen/common/sched_credit2.c |   73 +++-
 1 file changed, 64 insertions(+), 9 deletions(-)

diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c
index 052e050394..d2b4c907dc 100644
--- a/xen/common/sched_credit2.c
+++ b/xen/common/sched_credit2.c
@@ -1558,6 +1558,7 @@ runq_tickle(const struct scheduler *ops, struct 
csched2_vcpu *new, s_time_t now)
 s_time_t max = 0;
 unsigned int bs, cpu = new->vcpu->processor;
 struct csched2_runqueue_data *rqd = c2rqd(ops, cpu);
+struct csched2_grpsched_data *gscd = c2gscd(cpu);
 cpumask_t *online = cpupool_domain_cpumask(new->vcpu->domain);
 cpumask_t mask;
 
@@ -1593,10 +1594,19 @@ runq_tickle(const struct scheduler *ops, struct 
csched2_vcpu *new, s_time_t now)
   cpumask_test_cpu(cpu, &rqd->idle) &&
   !cpumask_test_cpu(cpu, &rqd->tickled)) )
 {
-ASSERT(cpumask_cycle(cpu, new->vcpu->cpu_hard_affinity) == cpu);
-SCHED_STAT_CRANK(tickled_idle_cpu_excl);
-ipid = cpu;
-goto tickle;
+/*
+ * If group scheduling is enabled, what's running on the
+ * other pCPUs of the coscheduling group also matters.
+ */
+if ( !grpsched_enabled() || gscd->sdom == NULL ||
+ gscd->sdom == new->sdom )
+{
+ASSERT(gscd->nr_running < cpumask_weight(&gscd->cpus));
+ASSERT(cpumask_cycle(cpu, new->vcpu->cpu_hard_affinity) == cpu);
+SCHED_STAT_CRANK(tickled_idle_cpu_excl);
+ipid = cpu;
+goto tickle;
+}
 }
 
 for_each_affinity_balance_step( bs )
@@ -1617,6 +1627,8 @@ runq_tickle(const struct scheduler *ops, struct 
csched2_vcpu *new, s_time_t now)
  */
 if ( unlikely(sched_smt_power_savings) )
 {
+/* XXX deal with grpsched_enabled() == true */
+
 cpumask_andnot(&mask, &rqd->idle, &rqd->smt_idle);
 cpumask_and(&mask, &mask, online);
 }
@@ -1626,6 +1638,15 @@ runq_tickle(const struct scheduler *ops, struct 
csched2_vcpu *new, s_time_t now)
 i = cpumask_test_or_cycle(cpu, &mask);
 if ( i < nr_cpu_ids )
 {
+struct csched2_grpsched_data *igscd = c2gscd(i);
+
+ASSERT(igscd->nr_running < cpumask_weight(&igscd->cpus));
+/*
+ * If we're doing core-scheduling, the CPU being in smt_idle also
+ * means that there are no other vcpus running in the group.
+ */
+ASSERT(opt_grpsched != OPT_TOPOLOGY_CORE ||
+   (igscd->sdom == NULL && igscd->nr_running == 0));
 SCHED_STAT_CRANK(tickled_idle_cpu);
 ipid = i;
 goto tickle;
@@ -1640,11 +1661,36 @@ runq_tickle(const struct scheduler *ops, struct 
csched2_vcpu *new, s_time_t now)
 cpumask_and(cpumask_scratch_cpu(cpu), cpumask_scratch_cpu(cpu), 
online);
 cpumask_and(&mask, &mask, cpumask_scratch_cpu(cpu));
 i = cpumask_test_or_cycle(cpu, &mask);
-if ( i < nr_cpu_ids )
+/*
+ * If we don't have group scheduling enabled, any CPU in the mask
+ * is fine. And in fact, during the very first iteration, we take
+ * the 'if', and go to tickling.
+ *
+ * If, OTOH, that is enabled, we want to tickle CPUs that are in
+ * groups where other vcpus of new's domain are running already.
+ *
+ * XXX Potential optimization: if we use a data structure where we
+ * keep track, for each domain, on what pCPUs the vcpus of the
+ * domain itself are currently running, we can probably avoid
+ * the loop.
+ */
+while ( !cpumask_empty(&mask) )
 {
-SCHED_STAT_CRANK(tickled_idle_cpu);
-ipid = i;
-goto tickle;
+struct csched2_grpsched_data *igscd = c2gscd(i);
+
+ASSERT(i < nr_cpu_ids);
+ASSERT(is_idle_vcpu(curr_on_cpu(i)) &&
+   csched2_vcpu(curr_on_cpu(i))->sdom == NULL);
+ASSERT(igscd->nr_running < cpumask_weight(&igscd->cpus));
+if ( !grpsched_enabled() || igscd->sdom == NULL ||
+ igscd->sdom == new->sdom)
+{
+ 

[Xen-devel] [xen-unstable test] 128614: regressions - FAIL

2018-10-12 Thread osstest service owner
flight 128614 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/128614/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-xtf-amd64-amd64-4   11 xtf-fep  fail REGR. vs. 128523
 test-xtf-amd64-amd64-1   11 xtf-fep  fail REGR. vs. 128523
 test-xtf-amd64-amd64-4   12 xtf-run  fail REGR. vs. 128523
 test-xtf-amd64-amd64-1   12 xtf-run  fail REGR. vs. 128523
 test-amd64-i386-freebsd10-i386 14 guest-saverestore  fail REGR. vs. 128523
 test-amd64-amd64-xl-pvhv2-intel 15 guest-saverestore fail REGR. vs. 128523
 test-amd64-i386-qemuu-rhel6hvm-intel 10 redhat-install   fail REGR. vs. 128523
 test-amd64-i386-qemut-rhel6hvm-intel 10 redhat-install   fail REGR. vs. 128523
 test-amd64-amd64-xl-pvshim   15 guest-saverestorefail REGR. vs. 128523
 test-armhf-armhf-xl-multivcpu  6 xen-install fail REGR. vs. 128523
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 10 debian-hvm-install fail REGR. 
vs. 128523
 test-amd64-amd64-qemuu-nested-intel 10 debian-hvm-install fail REGR. vs. 128523
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail REGR. vs. 128523
 test-amd64-i386-xl-qemuu-debianhvm-amd64 10 debian-hvm-install fail REGR. vs. 
128523
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail REGR. vs. 128523
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail REGR. 
vs. 128523
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 10 debian-hvm-install 
fail REGR. vs. 128523
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail 
REGR. vs. 128523
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 debian-hvm-install fail 
REGR. vs. 128523
 test-amd64-i386-xl-qemuu-ovmf-amd64 10 debian-hvm-install fail REGR. vs. 128523
 test-amd64-i386-xl-qemuu-win7-amd64 10 windows-install   fail REGR. vs. 128523
 test-amd64-amd64-xl-qemuu-ws16-amd64 10 windows-install  fail REGR. vs. 128523
 test-amd64-amd64-xl-qemut-ws16-amd64 10 windows-install  fail REGR. vs. 128523
 test-amd64-i386-xl-qemut-ws16-amd64 10 windows-install   fail REGR. vs. 128523
 test-amd64-i386-xl-qemuu-ws16-amd64 10 windows-install   fail REGR. vs. 128523

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 128523
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 128523
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 128523
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 128523
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 128523
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkf

Re: [Xen-devel] [PATCH v5 1/3] xen/xsm: remove unnecessary #define

2018-10-12 Thread Daniel De Graaf

On 10/09/2018 05:33 AM, Xin Li wrote:

this #define is unnecessary since XSM_INLINE is redefined in
xsm/dummy.h, it's a risk of build breakage, so remove it.

Signed-off-by: Xin Li 
Reviewed-by: Jan Beulich 


Acked-by: Daniel De Graaf 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf baseline-only test] 75402: trouble: blocked/broken

2018-10-12 Thread Platform Team regression test user
This run is configured for baseline tests only.

flight 75402 ovmf real [real]
http://osstest.xensource.com/osstest/logs/75402/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm  broken
 build-i386   broken
 build-amd64-pvopsbroken
 build-i386-xsm   broken
 build-amd64  broken
 build-i386-pvops broken

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 build-amd64   4 host-install(4)   broken baseline untested
 build-i386-pvops  4 host-install(4)   broken baseline untested
 build-i3864 host-install(4)   broken baseline untested
 build-amd64-xsm   4 host-install(4)   broken baseline untested
 build-i386-xsm4 host-install(4)   broken baseline untested
 build-amd64-pvops 4 host-install(4)   broken baseline untested

version targeted for testing:
 ovmf 7177be0bd8d372ef7eaa86df538bd45ec841ed23
baseline version:
 ovmf 8122c6bc8b6f1fde31f2af6c1560ec552204980d

Last test of basis75398  2018-10-12 00:20:37 Z0 days
Testing same since75402  2018-10-12 13:20:41 Z0 days1 attempts


People who touched revisions under test:
  Jim Dailey 
  jim.dai...@dell.com 

jobs:
 build-amd64-xsm  broken  
 build-i386-xsm   broken  
 build-amd64  broken  
 build-i386   broken  
 build-amd64-libvirt  blocked 
 build-i386-libvirt   blocked 
 build-amd64-pvopsbroken  
 build-i386-pvops broken  
 test-amd64-amd64-xl-qemuu-ovmf-amd64 blocked 
 test-amd64-i386-xl-qemuu-ovmf-amd64  blocked 



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xensource.com/osstest/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary

broken-job build-amd64-xsm broken
broken-job build-i386 broken
broken-job build-amd64-pvops broken
broken-job build-i386-xsm broken
broken-job build-amd64 broken
broken-job build-i386-pvops broken
broken-step build-amd64 host-install(4)
broken-step build-i386-pvops host-install(4)
broken-step build-i386 host-install(4)
broken-step build-amd64-xsm host-install(4)
broken-step build-i386-xsm host-install(4)
broken-step build-amd64-pvops host-install(4)

Push not applicable.


commit 7177be0bd8d372ef7eaa86df538bd45ec841ed23
Author: jim.dai...@dell.com 
Date:   Thu Oct 4 23:03:28 2018 +0800

MdePkg-BaseLib: Fix PathCleanUpDirectories() error involving "\..\.."

MdePkg-BaseLib: Fix PathCleanUpDirectories() error involving "\..\.."

The loop that removes "\..\" errs when multiple "\.." sequences are
in the path.  Before this change the code would modify a path like
"FS0:\efi\tools\..\.." to "FS0:\efi\\.." and then to "FS0:\efi\", but
the correct path is "FS0:\".

You can test the effect of this change in the shell by setting the
current directory to something like FS0:\efi\boot and then executing
the command "ls ..\..".  Before the change you will see the files in
the FS0:\efi directory; after the change, you will see the files in
the root directory of FS0:.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jim Dailey 
Reviewed-by: Ruiyu Ni 

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [xen-4.11-testing test] 128616: tolerable FAIL - PUSHED

2018-10-12 Thread osstest service owner
flight 128616 xen-4.11-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/128616/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install 
fail never pass
 test-amd64-i386-xl-pvshim12 guest-start  fail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  13 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  14 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-arm64-arm64-xl  13 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass

version targeted for testing:
 xen  33664f9a05401fac8f2c0be0bb7ee8a1851e4dcf
baseline version:
 xen  a2e35a759249bd8b6ffeeebc0a3bc96d9cca2fba

Last test of basis   128503  2018-10-08 12:36:53 Z4 days
Testing same since   128526  2018-10-09 13:11:25 Z3 days2 attempts


People who touched revisions under test:
  Ian Jackson 
  Wei Liu 

jobs:
 build-amd64-xsm  pass

Re: [Xen-devel] [PATCH 07/12] tools/xenstore: Re-introduce (fake) xs_restrict call to preserve ABI

2018-10-12 Thread Hans van Kranenburg
On 10/12/2018 05:12 PM, Ian Jackson wrote:
> From: Hans van Kranenburg 

No, this was in the changes that I copied back from Ubuntu, it was
written by Stefan Bader:

 >8 

Description: Re-introduce fake xs_restrict API call
 libxenstore cannot remove an API function without changing its version
 number. As long as we want to remain with 3.0 we have to keep it around.
 Debian might decide to increment the version at some point but we do not
 know how and when. So for now keep the version stable.

Author: Stefan Bader 

 >8 

> libxenstore3.0 in Xen 4.8 had this function.  We don't really want to
> bump the ABI version (soname) just for this, since we don't think
> there are actual callers anywhere.  But tools complain about the
> symbol going away.
> 
> So, provide a function xs_restrict which conforms to the original
> semantics, although it always fails.
> 
> Gbp-Pq: Topic xenstore
> Gbp-Pq: Name tools-fake-xs-restrict.patch
> Signed-off-by: Ian Jackson 
> ---
> v2: New in this version of the series
> ---
>  tools/xenstore/include/xenstore.h | 5 +
>  tools/xenstore/xs.c   | 6 ++
>  2 files changed, 11 insertions(+)
> 
> diff --git a/tools/xenstore/include/xenstore.h 
> b/tools/xenstore/include/xenstore.h
> index f460b8c5e5..0d95bb0e5c 100644
> --- a/tools/xenstore/include/xenstore.h
> +++ b/tools/xenstore/include/xenstore.h
> @@ -132,6 +132,11 @@ bool xs_mkdir(struct xs_handle *h, xs_transaction_t t,
>  bool xs_rm(struct xs_handle *h, xs_transaction_t t,
>  const char *path);
>  
> +/* Fake function which will always return false (required to let
> + * libxenstore remain at 3.0 version.
> + */
> +bool xs_restrict(struct xs_handle *h, unsigned domid);
> +
>  /* Get permissions of node (first element is owner, first perms is "other").
>   * Returns malloced array, or NULL: call free() after use.
>   */
> diff --git a/tools/xenstore/xs.c b/tools/xenstore/xs.c
> index 77700bff2b..cbcebb2bce 100644
> --- a/tools/xenstore/xs.c
> +++ b/tools/xenstore/xs.c
> @@ -796,6 +796,12 @@ unwind:
>   return false;
>  }
>  
> +/* Always return false a functionality has been removed in Xen 4.9 */
> +bool xs_restrict(struct xs_handle *h, unsigned domid)
> +{
> + return false;
> +}
> +
>  /* Watch a node for changes (poll on fd to detect, or call read_watch()).
>   * When the node (or any child) changes, fd will become readable.
>   * Token is returned when watch is read, to allow matching.
> 


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [ovmf test] 128670: all pass - PUSHED

2018-10-12 Thread osstest service owner
flight 128670 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/128670/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf bbce001515bbfcad24c216b1c9c25057e8c461e9
baseline version:
 ovmf 7177be0bd8d372ef7eaa86df538bd45ec841ed23

Last test of basis   128655  2018-10-12 04:10:52 Z0 days
Testing same since   128670  2018-10-12 13:40:57 Z0 days1 attempts


People who touched revisions under test:
  Dandan Bi 
  Jeff Brasen 
  Jim Dailey 
  jim.dai...@dell.com 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   7177be0bd8..bbce001515  bbce001515bbfcad24c216b1c9c25057e8c461e9 -> 
xen-tested-master

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

  1   2   >