Re: [Xen-devel] [PATCHv3] 03/28] arm: drop now redefined CONFIG_64BIT

[Jan Beulich]

>>> On 12.11.15 at 23:54,  wrote:
> The switch to Kconfig provides variables prefixed with CONFIG_ with
> results in a redefinition of this variable.

So I can't see how this can be a separate patch: Either the
redefinition causes a build failure after the previous patch (if
CONFIG_64BIT gets introduced before the one here) or the
symbol is now undefined (if the new instance gets introduced
later).

Jan



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Reproducible hang with cstate disabled

[Jan Beulich]

>>> On 13.11.15 at 06:47,  wrote:
> I saw xen hang after
> 
> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> (
> 
> The line that's supposed to be there is
> 
> (XEN) Brought up 24 CPUs
> 
> After power cycling I went into the BIOS. In the BIOS, C-STATE was disabled. 
> I changed it to
> 
> * Intel(R) C-STATE tech  [Enabled]
> * C3 State   [ACPI C2]
> * C6 State   [Enabled]
> * C State package limit setting  [Auto]
> * C1 Auto Demotion   [Disabled]
> * C3 Auto Demotion   [Disabled]
> 
> After making the above changes it booted up correctly.
> I turned C-STATE off again and it hung at the same place.
> I turned C-STATE back on and it worked again.

Interesting, but first of all this smells like a hardware or firmware
issue. Did you try booting with "no-mwait-idle" or one of its
equivalents? I ask because that driver is independent of any
BIOS settings.

Did you further try fiddling with any of the other settings you
quote above? Or limiting the maximum C state on the Xen
command line?

And finally, does "sync_console watchdog" end up in anything
more useful than output stopping in the middle of a line?

> I have not tested on another machine yet, but I expect the same behavior.

On another identical machine probably yes. On a different one I
would doubt it.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Uniform commands for booting xen

[Andrei Borzenkov]

On Fri, Nov 13, 2015 at 10:48 AM, Jan Beulich  wrote:
 On 12.11.15 at 18:09,  wrote:
>> On Thu, 2015-11-12 at 08:44 -0700, Jan Beulich wrote:
>>> > > > On 12.11.15 at 14:41,  wrote:
>>> > Hello, all. I'd like to have set of commands that would boot xen on all
>>> > platforms. I thought of following set:
>>> >
>>> > xen_hypervisor FILE XEN_OPTIONS
>>> > xen_kernel FILE KERNEL_OPTIONS
>>> > xen_initrd INITRD INITRD INITRD
>>> > all initrds are concatenated.
>>> > xen_xsm ???
>>>
>>> xen_ucode (and we might add more going forward). I don't see
>>> why the multiboot mechanism (kernel plus any number of modules)
>>> can't be used, without adding any Xen-specific directives.
>>
>> You likely aren't aware that on ARM Xen doesn't boot via multiboot, but via
>> a protocol which involves passing modules in an fdt[0].
>>
>> I had originally hoped that this would use the same command names in the
>> grub cfg, such that things would just work, however the grub maintainers
>> didn't like that (and I appreciate why).
>>
>> Hence on grub/ARM we already have xen_{hypervisor,kernel,initrd,...}.
>>
>> The question then is what grub-mkconfig (or more precisely
>> /etc/grub.d/20_linux_xen) ought to emit so that things just work on all
>> architectures.
>>
>> The author of the grub/ARM/Xen patches initially made it generate the xen_*
>> namas for arm and the multiboot names for x86, here is Vladimir's feedback
>> on that: http://lists.gnu.org/archive/html/grub-devel/2015-10/msg00133.html
>>
>> Which I think gets us to approximately today and Vladimir's question.
>
> Now that makes the situation really ugly (and supports my
> reservations regarding grub2 as a uniform solution for everything).

Well, if you want uniform solution for everything why not simply
implement uniform boot protocol everywhere?

> How do you express modules other than kernel+initrd in that
> scheme, without grub needing to be aware of any new addition we
> may find necessary going forward?
>

Are modules used by Xen self-identifying? Is it enough to simply pass
Xen kernel list of binary blobs or Xen kernel must be told what these
binary blobs are? If they are self identifying, why arm needs to be
passed module type in the first place?

> I think any architecture following a well defined, cross-arch
> protocol (like multiboot) should not require any special xen_*
> directives. If ARM64 needs Xen to be treated specially, special
> directives are maybe warranted for this particular case, but I don't
> see why all architectures supporting Xen should then automatically
> have to use those too.

So we can have common grub.cfg that does not depend on each platform
peculiarities and can be reused everywhere. Exactly so that grub could
be uniform solution for everything.

Let's face it. Bootloader obviously must be aware of any changes in
boot protocol requires for a kernel. If (Xen) boot protocol does any
addition, bootloader (GRUB) obviously must implement this addition. As
soon as it needs to implement it for at least one supported arch - it
will automatically be available on every other Xen platform. So it
does not really cost anything (and you can continue to use "module" on
multiboot complying platforms anyway).

Because we already have precedent of using incompatible Xen boot
protocol on different architectures, we try to find uniform solution
that covers everything. Give us protocol that does not require
knowledge of module type everywhere and we will use. But so far you
just shoot the messenger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC] libxl: relax readonly check introduced by XSA-142 fix

[Ian Campbell]

On Thu, 2015-11-12 at 10:53 -0700, Jim Fehlig wrote:
> Ian Campbell wrote:
> > On Thu, 2015-11-12 at 08:45 -0700, Jim Fehlig wrote:
> > >  
> > > > The commit message doesn't say anything about AHCI. Are AHCI disks
> > > > actually emulated correctly by QEMU with readonly=on?
> > > I just double checked, and good thing since AHCI + readonly is
> > > another
> > > rejected
> > > combination
> > > 
> > > /usr/lib/xen/bin/qemu-system-i386 -device ahci,id=ahci0 \
> > >  -drive file=/tmp/disk.raw,if=none,id=ahcidisk-
> > > 0,format=raw,readonly=on \
> > >  -device ide-hd,bus=ahci0.0,unit=0,drive=ahcidisk-0
> > > qemu-system-i386: -device ide-hd,bus=ahci0.0,unit=0,drive=ahcidisk-0:
> > > Can't use
> > > a read-only drive
> > > 
> > > So IDE/SATA/AHCI are all incompatible with readonly=on. I'll fix this
> > > and
> > > ammend
> > > the commit message in V2.
> > 
> > Just to clarify when you say "rejected" and "incompatible" do you mean
> > that
> > qemu will fail to start if you try, or that it will ignore you and give
> > a
> > writeable disk?
> 
> qemu will fail to start.

OK, that's good, I was a bit worried it might fail open.

> > If, as I think, it will fail, why don't we just always ask and rely on
> > qemu
> > to reject, instead of trying to whitelist the ones we know work in the
> > libxl code?
> 
> That would be possible, but makes it more difficult to track down why the 
> domain
> failed to start.[...]

Indeed.

> libxl: error: libxl_create.c:1340:domcreate_devmodel_started: device model did
> not start: -6

At a minimum this ought to do as the bootloader failed message does and say
"look in /var/log/xen/qemu-dm-sles12-hvm.log for more info". Ideally error
reporting from qemu back to the toolstack would be able to actually report
back what was going on somehow (which I appreciate might be rather
difficult to arrange).

Anyway, none of that is on you and since qemu fails safe if libxl gets it
wrong I don't think it should block this patch.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 1/9] libxc: reorganize domain builder guest memory allocator

[Ian Campbell]

On Thu, 2015-11-12 at 15:55 +, Wei Liu wrote:
> 
> Note that my Wheezy  PV installation is using grub2 so pvgrub doesn't
> seem to be able to parse its config file

FYI you can workaround this by installing the "pv-grub-menu" package (in
wheezy-backports and from Jessie onwards) which provides a grub1 compatible
menu.lst. I think (bu I'm not sure) that it can be coinstalled with the
regular grub2 packages for HVM booting purposes.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Uniform commands for booting xen

[Ian Campbell]

On Fri, 2015-11-13 at 00:48 -0700, Jan Beulich wrote:
> > > > On 12.11.15 at 18:09,  wrote:
> > On Thu, 2015-11-12 at 08:44 -0700, Jan Beulich wrote:
> > > > > > On 12.11.15 at 14:41,  wrote:
> > > > Hello, all. I'd like to have set of commands that would boot xen on
> > > > all
> > > > platforms. I thought of following set:
> > > > 
> > > > xen_hypervisor FILE XEN_OPTIONS
> > > > xen_kernel FILE KERNEL_OPTIONS
> > > > xen_initrd INITRD INITRD INITRD
> > > > all initrds are concatenated.
> > > > xen_xsm ???
> > > 
> > > xen_ucode (and we might add more going forward). I don't see
> > > why the multiboot mechanism (kernel plus any number of modules)
> > > can't be used, without adding any Xen-specific directives.
> > 
> > You likely aren't aware that on ARM Xen doesn't boot via multiboot, but
> > via
> > a protocol which involves passing modules in an fdt[0].
> > 
> > I had originally hoped that this would use the same command names in
> > the
> > grub cfg, such that things would just work, however the grub
> > maintainers
> > didn't like that (and I appreciate why).
> > 
> > Hence on grub/ARM we already have xen_{hypervisor,kernel,initrd,...}.
> > 
> > The question then is what grub-mkconfig (or more precisely
> > /etc/grub.d/20_linux_xen) ought to emit so that things just work on all
> > architectures.
> > 
> > The author of the grub/ARM/Xen patches initially made it generate the
> > xen_*
> > namas for arm and the multiboot names for x86, here is Vladimir's
> > feedback
> > on that: http://lists.gnu.org/archive/html/grub-devel/2015-10/msg00133.
> > html 
> > 
> > Which I think gets us to approximately today and Vladimir's question.
> 
> Now that makes the situation really ugly (and supports my
> reservations regarding grub2 as a uniform solution for everything).
> How do you express modules other than kernel+initrd in that
> scheme, without grub needing to be aware of any new addition we
> may find necessary going forward?

When I initially designed[0] my intention was that grub.cfg would be
unchanged between x86 and arm and therefore that the usual "multiboot" and
"module" commands would be used, but would be backed by the FDT protocol
not actual multiboot (my logic was that multiboot1 would never be added to
ARM and multiboot2 uses different command names so there was no clash, grub
upstream had good reasons for objecting to that though).

However at that time Xen/ARM didn't do as Xen/x86 does and assume something
from the ordering of the modules (first==dom0 kernel, second==dom0 initrd,
mechanisms to scan for other types, etc). Which led to the stuff at [1]
which put the onus for this inference into the bootloader, in a rather
complex way.

I since got convinced that this was madness and implemented in Xen/ARM
similar logic to x86 (i.e. inference based on the module ordering) so that
the bootloader could just present the modules in the order the cfg has them
and Xen/ARM would DTRT for the same set of common cases as Xen/x86 would.

At this point I should have updated [0] to much simplify things, since now
a single module command would have been much simpler. But it looks like I
neglected to do so and so the complex module type inference described in
[1] morphed during review (i.e. objections to the complicated type handling
in [1]) into the current xen_{kernel,initrd,foo} stuff we have now. Sorry
for a) not updating that wiki page in a timely manner and b) not noticing
this discrepancy was occurring during review.

I think at this point for the ARM stuff we could now ditch all these
xen_{kernel,initrd,xsm} etc in favour of a single xen_module command which
doesn't automatically attempt to infer or specify the type, and just lets
Xen figure it out.

AFAICT on the grub side this would mean exposing grub_cmd_xen_module
directly as a command and dropping the uses of set_module_type and the
other aliases.

Aside: Ideally I'd like to see a simplified variant of the --type stuff
added, which just takes a raw fdt compat value to use for flexibility in
the future, but that's not critical and orthogonal to this discussion.

> I think any architecture following a well defined, cross-arch
> protocol (like multiboot) should not require any special xen_*
> directives. If ARM64 needs Xen to be treated specially, special
> directives are maybe warranted for this particular case, but I don't
> see why all architectures supporting Xen should then automatically
> have to use those too. But yes, it's not my call decide this...

So the above would IMHO make arm64 a) more rational and b) closer to how
multiboot/x86 behaves, but I don't really think it addresses your core
concern.

In [2] Vladimir proposed three options to avoid assumption about the
machine which runs grub-mkconfig not being the one which is going to
execute grub.cfg:
- Check arch on boot time
- Check that new xen commands are supported (define a new feature)

Both of which are alternatives to the proposal for uniform commands made
here. Both have t

Re: [Xen-devel] Uniform commands for booting xen

[Ian Campbell]

On Fri, 2015-11-13 at 12:04 +0300, Andrei Borzenkov wrote:
> > How do you express modules other than kernel+initrd in that
> > scheme, without grub needing to be aware of any new addition we
> > may find necessary going forward?
> > 
> 
> Are modules used by Xen self-identifying? Is it enough to simply pass
> Xen kernel list of binary blobs or Xen kernel must be told what these
> binary blobs are? If they are self identifying, why arm needs to be
> passed module type in the first place?

At first Xen/ARM required the bootloader to identify, but that was since
identified as causing madness and fixed by having Xen/ARM do as Xen/x86
does and figure things out for itself, but I failed to communicate this
clearly and things got implemented on the grub side under the old
assumptions.

I just replied in more detail about that to Jan's mail, so I won't repeat
myself further here.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH] xen: sched: fix (ACPI S3) resume with cpupools with different schedulers.

[Dario Faggioli]

In fact, with 2 cpupools, one (the default) Credit and
one Credit2 (with at least 1 pCPU in the latter), trying
a (e.g., ACPI S3) suspend/resume crashes like this:

(XEN) [  150.587779] [ Xen-4.7-unstable  x86_64  debug=y  Not tainted ]
(XEN) [  150.587783] CPU:6
(XEN) [  150.587786] RIP:e008:[] 
sched_credit.c#csched_schedule+0xf2/0xc3d
(XEN) [  150.587796] RFLAGS: 00010086   CONTEXT: hypervisor
(XEN) [  150.587801] rax: 83031fa3c020   rbx: 830322c1b4b0   rcx: 

(XEN) [  150.587806] rdx: 83031fa78000   rsi: 000a   rdi: 
82d0802a9788
(XEN) [  150.587811] rbp: 83031fa7fe20   rsp: 83031fa7fd30   r8:  
83031fa8
(XEN) [  150.587815] r9:  0006   r10: 0008f7f2   r11: 
0006
(XEN) [  150.587819] r12: 8300dbdf3000   r13: 830322c1b4b0   r14: 
0006
(XEN) [  150.587823] r15:    cr0: 8005003b   cr4: 
26e0
(XEN) [  150.587827] cr3: dbaa8000   cr2: 
(XEN) [  150.587830] ds:    es:    fs:    gs:    ss:    cs: 
e008
(XEN) [  150.587835] Xen stack trace from rsp=83031fa7fd30:
... ... ...
(XEN) [  150.587962] Xen call trace:
(XEN) [  150.587966][] 
sched_credit.c#csched_schedule+0xf2/0xc3d
(XEN) [  150.587974][] schedule.c#schedule+0x128/0x635
(XEN) [  150.587979][] softirq.c#__do_softirq+0x82/0x8d
(XEN) [  150.587983][] do_softirq+0x13/0x15
(XEN) [  150.587988][] domain.c#idle_loop+0x5b/0x6b
(XEN) [  151.272182]
(XEN) [  151.274174] 
(XEN) [  151.279624] Panic on CPU 6:
(XEN) [  151.282915] Xen BUG at sched_credit.c:655
(XEN) [  151.287415] 

During suspend, the pCPUs are not removed from their
pools with the standard procedure (which would involve
schedule_cpu_switch(). During resume, they:
 1) are assigned to the default cpupool (CPU_UP_PREPARE
phase);
 2) are moved to the pool they were in before suspend,
via schedule_cpu_switch() (CPU_ONLINE phase)

During resume, scheduling (even if just the idle loop)
can happen right after the CPU_STARTING phase(before
CPU_ONLINE), i.e., before the pCPU is put back in its
pool. In this case, it is the default pool'sscheduler
that is invoked (Credit1, in the example above). But,
during suspend, the Credit2 specific vCPU data is not
being freed, and Credit1 specific vCPU data is not
allocated, during resume.

Therefore, Credit1 schedules on pCPUs whose idle vCPU's
sched_priv points to Credit2 vCPU data, and we crash.

Fix things by properly deallocating scheduler specific
data of the pCPU's pool scheduler during pCPU teardown,
and re-allocating them --always for &ops-- during pCPU
bringup.

Signed-off-by: Dario Faggioli 
---
Cc: George Dunlap 
Cc: Juergen Gross 
---
 xen/common/schedule.c |   23 +++
 1 file changed, 23 insertions(+)

diff --git a/xen/common/schedule.c b/xen/common/schedule.c
index 20f5f56..55fc32a 100644
--- a/xen/common/schedule.c
+++ b/xen/common/schedule.c
@@ -1381,6 +1381,25 @@ static int cpu_schedule_up(unsigned int cpu)
 if ( idle_vcpu[cpu] == NULL )
 return -ENOMEM;
 
+if ( idle_vcpu[cpu]->sched_priv == NULL )
+{
+struct vcpu *idle = idle_vcpu[cpu];
+
+/*
+ * During (ACPI?) suspend the scheduler specific data (what's in
+ * sched_priv) of the idle vCPU for this cpu's cpupool's scheduler
+ * is freed. At this stage of the resuming path, we treat the pCPU
+ * as it belongs to the default pool. Hence, let's allocate the
+ * default scheduler's data of the idle vCPU of this pCPU. It is
+ * schedule_cpu_switch() (invoked later,  if necessary) that sets
+ * things up as they were before suspend.
+ */
+idle->sched_priv = SCHED_OP(&ops, alloc_vdata, idle,
+idle->domain->sched_priv);
+if ( idle->sched_priv == NULL )
+return -ENOMEM;
+}
+
 if ( (ops.alloc_pdata != NULL) &&
  ((sd->sched_priv = ops.alloc_pdata(&ops, cpu)) == NULL) )
 return -ENOMEM;
@@ -1393,9 +1412,13 @@ static void cpu_schedule_down(unsigned int cpu)
 struct schedule_data *sd = &per_cpu(schedule_data, cpu);
 struct scheduler *sched = per_cpu(scheduler, cpu);
 
+SCHED_OP(sched, free_vdata, idle_vcpu[cpu]->sched_priv);
 if ( sd->sched_priv != NULL )
 SCHED_OP(sched, free_pdata, sd->sched_priv, cpu);
 
+idle_vcpu[cpu]->sched_priv = NULL;
+sd->sched_priv = NULL;
+
 kill_timer(&sd->s_timer);
 }
 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] HVM domains crash after upgrade from XEN 4.5.1 to 4.5.2

[Andrew Cooper]

On 13/11/15 07:25, Jan Beulich wrote:
 On 13.11.15 at 00:00,  wrote:
>> Am 12.11.15 um 17:43 schrieb Andrew Cooper:
>>> On 12/11/15 14:29, Atom2 wrote:
 Hi Andrew,
 thanks for your reply. Answers are inline further down.

 Am 12.11.15 um 14:01 schrieb Andrew Cooper:
> On 12/11/15 12:52, Jan Beulich wrote:
> On 12.11.15 at 02:08,  wrote:
>>> After the upgrade HVM domUs appear to no longer work - regardless
>>> of the
>>> dom0 kernel (tested with both 3.18.9 and 4.1.7 as the dom0 kernel); PV
>>> domUs, however, work just fine as before on both dom0 kernels.
>>>
>>> xl dmesg shows the following information after the first crashed HVM
>>> domU which is started as part of the machine booting up:
>>> [...]
>>> (XEN) Failed vm entry (exit reason 0x8021) caused by invalid guest
>>> state (0).
>>> (XEN) * VMCS Area **
>>> (XEN) *** Guest State ***
>>> (XEN) CR0: actual=0x0039, shadow=0x0011,
>>> gh_mask=
>>> (XEN) CR4: actual=0x2050, shadow=0x,
>>> gh_mask=
>>> (XEN) CR3: actual=0x0080, target_count=0
>>> (XEN)  target0=, target1=
>>> (XEN)  target2=, target3=
>>> (XEN) RSP = 0x6fdc (0x6fdc)  RIP =
>>> 0x0001 (0x0001)
>> Other than RIP looking odd for a guest still in non-paged protected
>> mode I can't seem to spot anything wrong with guest state.
> odd? That will be the source of the failure.
>
> Out of long mode, the upper 32bit of %rip should all be zero, and it
> should not be possible to set any of them.
>
> I suspect that the guest has exited for emulation, and there has been a
> bad update to %rip.  The alternative (which I hope is not the case) is
> that there is a hardware errata which allows the guest to accidentally
> get it self into this condition.
>
> Are you able to rerun with a debug build of the hypervisor?
 [snip]
 Another question is whether prior to enabling the debug USE flag it
 might make sense to re-compile with gcc-4.8.5 (please see my previous
 list reply) to rule out any compiler related issues. Jan, Andrew -
 what are your thoughts?
>>> First of all, check whether the compiler makes a difference on 4.5.2
>> Hi Andrew,
>> I changed the compiler and there was no change to the better: 
>> Unfortunately the HVM domU is still crashing with a similar error 
>> message as soon as it is being started.
>>> If both compiles result in a guest crashing in that manner, test a debug
>>> Xen to see if any assertions/errors are encountered just before the
>>> guest crashes.
>>>
>> As the compiler did not make any difference, I enabled the debug USE 
>> flag, re-compiled (using gcc-4.9.3), and rebooted using a serial console 
>> to capture output. Unfortunately I did not get very far and things 
>> become even stranger: This time the system did not even finnish the boot 
>> process, but rather hard-stopped pretty early with a message reading 
>> "Panic on CPU 3: DOUBLE FAULT -- system shutdown". The captured logfile 
>> is attached as "serial log.txt".
>>
>> As this happened immediately after the CPU microcode update, I thought 
>> there might be a connection and disabled the microcode update. After the 
>> next reboot it seemed as if the boot process got a bit further as 
>> evidenced by a few more lines in the log file (those between lines 136 
>> and 197 in the second log file named "serial log no ucode.txt"), but in 
>> the end it finnished off with an identical error message (only the CPU # 
>> was different this time, but that number seems to change between boots 
>> anyways).
>>
>> I hope that makes some sense to you.
> Not really, other than now even more suspecting bad hardware or
> something fundamentally wrong with your build. Did you retry with
> a freshly built 4.5.1? Could you alternatively try with a known good
> build of 4.5.2 (e.g. from osstest)?

Agreed.  Double faults indicate that the exception handing entry points
are not set up in an appropriate state.  Something is definitely wrong
with either the compiled binary or the hardware.

Several questions and lines of investigation:

Is this straight Xen 4.5.1 and 2, or do Gentoo have their own patches on
top?

On repeated attempts, are the details of the double fault identical
(other than the cpu), or does it move around (i.e. always do_IRQ+0x15)

Can you boot with console_timestamps=boot on the command line in the
future.  This will put Linux-sytle timestamps on log messages.

Can you also compile in the attached patch? I haven't quite got it
suitable for inclusion upstream yet, but it will also dump the
instruction stream under the fault.

Finally, can you disassemble the xen-syms which results from the debug
b

[Xen-devel] [distros-debian-jessie test] 38274: tolerable all pass

[Platform Team regression test user]

flight 38274 distros-debian-jessie real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/38274/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-armhf-jessie-netboot-pygrub 12 saverestore-support-check fail 
never pass
 test-armhf-armhf-armhf-jessie-netboot-pygrub 11 migrate-support-check fail 
never pass

baseline version:
 flight   38254

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-amd64-jessie-netboot-pvgrub pass
 test-amd64-i386-i386-jessie-netboot-pvgrub   pass
 test-amd64-i386-amd64-jessie-netboot-pygrub  pass
 test-armhf-armhf-armhf-jessie-netboot-pygrub pass
 test-amd64-amd64-i386-jessie-netboot-pygrub  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Osstest nested patch v15 (was RE: [OSSTEST PATCH v14 PART 2 10-26/26] Nested HVM testing)

[Ian Jackson]

Hu, Robert writes ("RE: Osstest nested patch v15 (was RE: [OSSTEST PATCH v14 
PART 2 10-26/26] Nested HVM testing)"):
> Yes. In last test step, ts-logs-capture host, I can see
> 
> 2015-11-12 03:22:06 Z fetching 
> /var/log/xen/osstest-serial-l1.guest.osstest.log to 
> osstest-host2---var-log-xen-osstest-serial-l1.guest.osstest.log
> 2015-11-12 03:22:06 Z executing scp ... 
> root@192.168.199.70:/var/log/xen/osstest-serial-l1.guest.osstest.log 
> logs/standalone/test-amd64-amd64-qemuu-nested/osstest-host2---var-log-xen-osstest-serial-l1.guest.osstest.log

Good.

> See attached. Seems no debug keys in it.

Actually:

(XEN) '0' pressed -> dumping Dom0's registers

...
(XEN) 'H' pressed -> dumping heap info (now-0x119:584209D7)

...

So it is working.

Thanks.  I will send out a final version for the record (and for Ian C
to hopefully ack).

Regards,
Ian.
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [linux-linus test] 64147: regressions - FAIL

[osstest service owner]

flight 64147 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/64147/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-rumpuserxen-i386  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-qemut-rhel6hvm-intel  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-qemuu-rhel6hvm-intel  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-xsm6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 6 xen-boot fail REGR. vs. 
59254
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-debianhvm-amd64  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-qemut-rhel6hvm-amd  6 xen-bootfail REGR. vs. 59254
 test-armhf-armhf-xl-xsm   6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-freebsd10-amd64  6 xen-boot   fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-debianhvm-amd64  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-ovmf-amd64  6 xen-boot   fail REGR. vs. 59254
 test-amd64-i386-qemuu-rhel6hvm-amd  6 xen-bootfail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-win7-amd64  6 xen-boot   fail REGR. vs. 59254
 test-amd64-i386-pair 10 xen-boot/dst_host fail REGR. vs. 59254
 test-amd64-i386-pair  9 xen-boot/src_host fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-win7-amd64  6 xen-boot   fail REGR. vs. 59254
 test-amd64-i386-freebsd10-i386  6 xen-bootfail REGR. vs. 59254
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  6 xen-boot  fail REGR. vs. 59254
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 16 
guest-localmigrate/x10 fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-winxpsp3  6 xen-boot fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-winxpsp3  6 xen-boot fail REGR. vs. 59254

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-libvirt-xsm   6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-libvirt   6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-xl-rtds 11 guest-start   fail REGR. vs. 59254
 test-amd64-i386-xl-raw6 xen-bootfail baseline untested
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 6 xen-boot fail baseline 
untested
 test-amd64-i386-libvirt-pair 10 xen-boot/dst_host   fail baseline untested
 test-amd64-i386-libvirt-pair  9 xen-boot/src_host   fail baseline untested
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail like 59254

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 14 guest-saverestorefail  never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw  9 debian-di-installfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-libvirt-qcow2  9 debian-di-installfail never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-vhd   9 debian-di-installfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass

version targeted for testing:
 linuxc5a37883f42be712a989e54d5d

Re: [Xen-devel] [raisin][PATCHv3] Handle unsupported distros with a prettier message

[Stefano Stabellini]

On Thu, 12 Nov 2015, Doug Goldstein wrote:
> Handle unknown distros by saying "unknown" instead of an empty string
> and for Gentoo users actually mention it.
> 
> Signed-off-by: Doug Goldstein 

Acked-by: Stefano Stabellini 

and applied


>  lib/common-functions.sh | 5 +
>  1 file changed, 5 insertions(+)
> 
> diff --git a/lib/common-functions.sh b/lib/common-functions.sh
> index 27f6434..19434c2 100644
> --- a/lib/common-functions.sh
> +++ b/lib/common-functions.sh
> @@ -98,6 +98,8 @@ function get_tests() {
>  }
>  
>  function get_distro() {
> +os_VENDOR="unknown"
> +
>  if [[ -x "`which lsb_release 2>/dev/null`" ]]
>  then
>  os_VENDOR=`lsb_release -i -s`
> @@ -150,6 +152,9 @@ function get_distro() {
>   sed -r 's/\"|\(|\)//g' | awk '{print $2}'`
>  os_RELEASE=`awk '/VERSION_ID=/' /etc/os-release | sed 
> 's/VERSION_ID=//' \
>  | sed 's/\"//g'`
> +elif [[ -f /etc/gentoo-release ]]
> +then
> +os_VENDOR="Gentoo"
>  fi
>  
>  # Simply distro version string
> -- 
> 2.4.10
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V2] libxl: relax readonly check introduced by XSA-142 fix

[Stefano Stabellini]

On Thu, 12 Nov 2015, Jim Fehlig wrote:
> The fix for XSA-142 is quite a big hammer, rejecting readonly
> disk configuration even when the requested backend is known to
> support readonly. While it is true that qemu doesn't support
> readonly for emulated IDE or AHCI disks
> 
> $ /usr/lib/xen/bin/qemu-system-i386 \
>  -drive file=/tmp/disk.raw,if=ide,media=disk,format=raw,readonly=on
> qemu-system-i386: Can't use a read-only drive
> 
> $ /usr/lib/xen/bin/qemu-system-i386 -device ahci,id=ahci0 \
>  -drive file=/tmp/disk.raw,if=none,id=ahcidisk-0,format=raw,readonly=on \
>  -device ide-hd,bus=ahci0.0,unit=0,drive=ahcidisk-0
> qemu-system-i386: -device ide-hd,bus=ahci0.0,unit=0,drive=ahcidisk-0:
> Can't use a read-only drive
> 
> It does support readonly SCSI disks
> 
> $ /usr/lib/xen/bin/qemu-system-i386 \
>  -drive file=/tmp/disk.raw,if=scsi,media=disk,format=raw,readonly=on
> [ok]
> 
> Inside a guest using such a disk, the SCSI kernel driver sees write
> protect on
> 
> [   7.339232] sd 2:0:1:0: [sdb] Write Protect is on
> 
> Also, PV drivers support readonly, but the patch rejects such
> configuration even when PV drivers (vdev=xvd*) have been explicitly
> specified and creation of an emulated twin is skiped.
> 
> This follow-up patch loosens the restriction to reject readonly when
> creating and emulated IDE or AHCI disk, but allows it when the backend
> is known to support readonly.
> 
> Signed-off-by: Jim Fehlig 

Acked-by: Stefano Stabellini 



> V2: Along with IDE+readonly, blacklist AHCI+readonly since it is not
> supported by qemu either.
> 
>  tools/libxl/libxl_dm.c | 29 -
>  1 file changed, 16 insertions(+), 13 deletions(-)
> 
> diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
> index 9c9eaa3..cb6deec 100644
> --- a/tools/libxl/libxl_dm.c
> +++ b/tools/libxl/libxl_dm.c
> @@ -1185,29 +1179,38 @@ static int 
> libxl__build_device_model_args_new(libxl__gc *gc,
>   * For other disks we translate devices 0..3 into
>   * hd[a-d] and ignore the rest.
>   */
> -if (strncmp(disks[i].vdev, "sd", 2) == 0)
> +if (strncmp(disks[i].vdev, "sd", 2) == 0) {
>  drive = libxl__sprintf
> -(gc, 
> "file=%s,if=scsi,bus=0,unit=%d,format=%s,cache=writeback",
> - pdev_path, disk, format);
> -else if (strncmp(disks[i].vdev, "xvd", 3) == 0)
> +(gc, 
> "file=%s,if=scsi,bus=0,unit=%d,format=%s,readonly=%s,cache=writeback",
> + pdev_path, disk, format, disks[i].readwrite ? "off" 
> : "on");
> +} else if (strncmp(disks[i].vdev, "xvd", 3) == 0) {
>  /*
>   * Do not add any emulated disk when PV disk are
>   * explicitly asked for.
>   */
>  continue;
> -else if (disk < 6 && b_info->u.hvm.hdtype == 
> LIBXL_HDTYPE_AHCI) {
> +} else if (disk < 6 && b_info->u.hvm.hdtype == 
> LIBXL_HDTYPE_AHCI) {
> +if (!disks[i].readwrite) {
> +LOG(ERROR, "qemu-xen doesn't support read-only AHCI 
> disk drivers");
> +return ERROR_INVAL;
> +}
>  flexarray_vappend(dm_args, "-drive",
>  
> GCSPRINTF("file=%s,if=none,id=ahcidisk-%d,format=%s,cache=writeback",
>  pdev_path, disk, format),
>  "-device", 
> GCSPRINTF("ide-hd,bus=ahci0.%d,unit=0,drive=ahcidisk-%d",
>  disk, disk), NULL);
>  continue;
> -} else if (disk < 4)
> +} else if (disk < 4) {
> +if (!disks[i].readwrite) {
> +LOG(ERROR, "qemu-xen doesn't support read-only IDE 
> disk drivers");
> +return ERROR_INVAL;
> +}
>  drive = libxl__sprintf
>  (gc, 
> "file=%s,if=ide,index=%d,media=disk,format=%s,cache=writeback",
>   pdev_path, disk, format);
> -else
> +} else {
>  continue; /* Do not emulate this disk */
> +}
>  }
>  
>  flexarray_append(dm_args, "-drive");
> -- 
> 1.8.0.1
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 7/7] xen/x86: support XENPF_settime64

[David Vrabel]

On 12/11/15 17:30, Stefano Stabellini wrote:
> Try XENPF_settime64 first, if it is not available fall back to
> XENPF_settime32.
> 
> No need to call __current_kernel_time() when all the info needed are
> already passed via the struct timekeeper * argument.
> 
> Return NOTIFY_BAD in case of errors.
[...]
> @@ -123,9 +124,13 @@ static int xen_pvclock_gtod_notify(struct notifier_block 
> *nb,
>   static struct timespec next_sync;
>  
>   struct xen_platform_op op;
> - struct timespec now;
> + struct timespec64 now;
> + struct timekeeper *tk = priv;
> + static bool settime64_supported = true;
> + int ret;
>  
> - now = __current_kernel_time();
> + now.tv_sec = tk->xtime_sec;
> + now.tv_nsec = (long)(tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift);

I think you should introduce __current_kernel_time64() or make
tk_xtime() available.

John, what do you think?

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] xen: sched: fix (ACPI S3) resume with cpupools with different schedulers.

[Jan Beulich]

>>> On 13.11.15 at 11:08,  wrote:
> Fix things by properly deallocating scheduler specific
> data of the pCPU's pool scheduler during pCPU teardown,
> and re-allocating them --always for &ops-- during pCPU
> bringup.

Avoiding this was done for a reason iirc: What if one such allocation
fails (e.g. due to heap fragmentation resulting from the allocation
order not being the exact inverse of the freeing order)?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 7/7] xen/x86: support XENPF_settime64

[Stefano Stabellini]

On Fri, 13 Nov 2015, David Vrabel wrote:
> On 12/11/15 17:30, Stefano Stabellini wrote:
> > Try XENPF_settime64 first, if it is not available fall back to
> > XENPF_settime32.
> > 
> > No need to call __current_kernel_time() when all the info needed are
> > already passed via the struct timekeeper * argument.
> > 
> > Return NOTIFY_BAD in case of errors.
> [...]
> > @@ -123,9 +124,13 @@ static int xen_pvclock_gtod_notify(struct 
> > notifier_block *nb,
> > static struct timespec next_sync;
> >  
> > struct xen_platform_op op;
> > -   struct timespec now;
> > +   struct timespec64 now;
> > +   struct timekeeper *tk = priv;
> > +   static bool settime64_supported = true;
> > +   int ret;
> >  
> > -   now = __current_kernel_time();
> > +   now.tv_sec = tk->xtime_sec;
> > +   now.tv_nsec = (long)(tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift);
> 
> I think you should introduce __current_kernel_time64() or make
> tk_xtime() available.
> 
> John, what do you think?

We already had this discussion:

http://marc.info/?l=linux-kernel&m=144717103112014&w=2
http://marc.info/?l=linux-kernel&m=144724877203864&w=2

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 0/2] Fix xc_domain_config usage

[Roger Pau Monne]

Hello,

Due to the HVMlite patches now x86 always requires a valid arch domain 
config. This is currently causing problems to out-of-tree toolstacks that 
are based on the Ocaml or the python bindings, since those bindings have lost 
the ability to create HVM guests because a zeroed arch domain config is 
always used regardless of the guest type.

In order to fix this add a new parameter to xc_domain_create that's a 
pointer to a arch domain config that can be null (the function itself will 
set a sensible default based on the guest type). Since 
xc_domain_create_config is meaningless now just remove it.

The in-tree callers are fixed in the first patch, while Qemu is fixed in the 
second patch. I'm not sure how this is going to work with the push gate, 
since both patches should be applied and tested together, or else Qemu build 
is going to fail.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH QEMU 2/2] xen: fix usage of xc_domain_create in domain builder

[Roger Pau Monne]

Due to the addition of HVMlite and the requirement to always provide a valid
xc_domain_configuration_t, xc_domain_create now always takes an arch domain
config, which can be NULL in order to mimic previous behaviour.

Signed-off-by: Roger Pau Monné 
---
Cc: Stefano Stabellini 
Cc: qemu-de...@nongnu.org
---
 hw/xenpv/xen_domainbuild.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/xenpv/xen_domainbuild.c b/hw/xenpv/xen_domainbuild.c
index c0ab753..a737908 100644
--- a/hw/xenpv/xen_domainbuild.c
+++ b/hw/xenpv/xen_domainbuild.c
@@ -234,7 +234,7 @@ int xen_domain_build_pv(const char *kernel, const char 
*ramdisk,
 int rc;
 
 memcpy(uuid, qemu_uuid, sizeof(uuid));
-rc = xc_domain_create(xen_xc, ssidref, uuid, flags, &xen_domid);
+rc = xc_domain_create(xen_xc, ssidref, uuid, flags, &xen_domid, NULL);
 if (rc < 0) {
 fprintf(stderr, "xen: xc_domain_create() failed\n");
 goto err;
-- 
1.9.5 (Apple Git-50.3)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH XEN 1/2] x86/libxc: add an arch domain config parameter to xc_domain_create

[Roger Pau Monne]

With the addition of HVMlite the hypervisor now always requires a non-null
arch domain config, which is different between HVM and PV guests.

Add a new parameter to xc_domain_create that contains a pointer to an arch
domain config. If the pointer is null, create a default arch domain config
based on guest type.

Fix all the in-tree callers to provide a null arch domain config in order to
mimic previous behaviour.

Signed-off-by: Roger Pau Monné 
---
 tools/libxc/include/xenctrl.h | 14 +++---
 tools/libxc/xc_domain.c   | 51 +++
 tools/libxl/libxl_create.c|  5 ++--
 tools/ocaml/libs/xc/xenctrl_stubs.c   |  2 +-
 tools/python/xen/lowlevel/xc/xc.c |  2 +-
 tools/xenstore/init-xenstore-domain.c |  2 +-
 6 files changed, 29 insertions(+), 47 deletions(-)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 2fec1fb..01a6dda 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -502,17 +502,9 @@ typedef union
 
 
 typedef struct xen_arch_domainconfig xc_domain_configuration_t;
-int xc_domain_create_config(xc_interface *xch,
-uint32_t ssidref,
-xen_domain_handle_t handle,
-uint32_t flags,
-uint32_t *pdomid,
-xc_domain_configuration_t *config);
-int xc_domain_create(xc_interface *xch,
- uint32_t ssidref,
- xen_domain_handle_t handle,
- uint32_t flags,
- uint32_t *pdomid);
+int xc_domain_create(xc_interface *xch, uint32_t ssidref,
+ xen_domain_handle_t handle, uint32_t flags,
+ uint32_t *pdomid, xc_domain_configuration_t *config);
 
 
 /* Functions to produce a dump of a given domain
diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
index e7278dd..0839e03 100644
--- a/tools/libxc/xc_domain.c
+++ b/tools/libxc/xc_domain.c
@@ -26,16 +26,31 @@
 #include 
 #include 
 
-int xc_domain_create_config(xc_interface *xch,
-uint32_t ssidref,
-xen_domain_handle_t handle,
-uint32_t flags,
-uint32_t *pdomid,
-xc_domain_configuration_t *config)
+int xc_domain_create(xc_interface *xch, uint32_t ssidref,
+ xen_domain_handle_t handle, uint32_t flags,
+ uint32_t *pdomid, xc_domain_configuration_t *config)
 {
+xc_domain_configuration_t lconfig;
 int err;
 DECLARE_DOMCTL;
 
+if ( config == NULL )
+{
+memset(&lconfig, 0, sizeof(lconfig));
+
+#if defined (__i386) || defined(__x86_64__)
+if ( flags & XEN_DOMCTL_CDF_hvm_guest )
+lconfig.emulation_flags = XEN_X86_EMU_ALL;
+#elif defined (__arm__) || defined(__aarch64__)
+lconfig.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
+lconfig.nr_spis = 0;
+#else
+#error Architecture not supported
+#endif
+
+config = &lconfig;
+}
+
 domctl.cmd = XEN_DOMCTL_createdomain;
 domctl.domain = (domid_t)*pdomid;
 domctl.u.createdomain.ssidref = ssidref;
@@ -52,30 +67,6 @@ int xc_domain_create_config(xc_interface *xch,
 return 0;
 }
 
-int xc_domain_create(xc_interface *xch,
- uint32_t ssidref,
- xen_domain_handle_t handle,
- uint32_t flags,
- uint32_t *pdomid)
-{
-xc_domain_configuration_t config;
-
-memset(&config, 0, sizeof(config));
-
-#if defined (__i386) || defined(__x86_64__)
-/* No arch-specific configuration for now */
-#elif defined (__arm__) || defined(__aarch64__)
-config.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
-config.nr_spis = 0;
-#else
-errno = ENOSYS;
-return -1;
-#endif
-
-return xc_domain_create_config(xch, ssidref, handle,
-   flags, pdomid, &config);
-}
-
 int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
  xen_pfn_t start_pfn, xen_pfn_t nr_pfns)
 {
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index f0fee00..4fbe7ac 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -528,9 +528,8 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config 
*d_config,
 
 /* Valid domid here means we're soft resetting. */
 if (!libxl_domid_valid_guest(*domid)) {
-ret = xc_domain_create_config(ctx->xch, info->ssidref,
-  handle, flags, domid,
-  xc_config);
+ret = xc_domain_create(ctx->xch, info->ssidref, handle, flags, domid,
+   xc_config);
 if (ret < 0) {
 LOGE(ERROR, "domain creation fail");
 rc = ERROR_FAIL;
diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c 
b/tools/oca

Re: [Xen-devel] [PATCH QEMU 2/2] xen: fix usage of xc_domain_create in domain builder

[Stefano Stabellini]

On Fri, 13 Nov 2015, Roger Pau Monne wrote:
> Due to the addition of HVMlite and the requirement to always provide a valid
> xc_domain_configuration_t, xc_domain_create now always takes an arch domain
> config, which can be NULL in order to mimic previous behaviour.
>
> Signed-off-by: Roger Pau Monné 

Give a look at include/hw/xen/xen_common.h and add a compatibility shim
there. Keep in mind that QEMU needs to build against any version of Xen
from 4.0 onward.


> Cc: Stefano Stabellini 
> Cc: qemu-de...@nongnu.org
> ---
>  hw/xenpv/xen_domainbuild.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/hw/xenpv/xen_domainbuild.c b/hw/xenpv/xen_domainbuild.c
> index c0ab753..a737908 100644
> --- a/hw/xenpv/xen_domainbuild.c
> +++ b/hw/xenpv/xen_domainbuild.c
> @@ -234,7 +234,7 @@ int xen_domain_build_pv(const char *kernel, const char 
> *ramdisk,
>  int rc;
>
>  memcpy(uuid, qemu_uuid, sizeof(uuid));
> -rc = xc_domain_create(xen_xc, ssidref, uuid, flags, &xen_domid);
> +rc = xc_domain_create(xen_xc, ssidref, uuid, flags, &xen_domid, NULL);
>  if (rc < 0) {
>  fprintf(stderr, "xen: xc_domain_create() failed\n");
>  goto err;
> --
> 1.9.5 (Apple Git-50.3)
> ___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 1/9] libxc: reorganize domain builder guest memory allocator

[Wei Liu]

On Fri, Nov 13, 2015 at 09:28:11AM +, Ian Campbell wrote:
> On Thu, 2015-11-12 at 15:55 +, Wei Liu wrote:
> > 
> > Note that my Wheezy  PV installation is using grub2 so pvgrub doesn't
> > seem to be able to parse its config file
> 
> FYI you can workaround this by installing the "pv-grub-menu" package (in
> wheezy-backports and from Jessie onwards) which provides a grub1 compatible
> menu.lst. I think (bu I'm not sure) that it can be coinstalled with the
> regular grub2 packages for HVM booting purposes.
> 

Thanks for the information.

I installed this package. Pvgrub worked fine. So I think this series
should be at least be able to pass our own push-gate.

Wei.

> Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 1/6] xen/arm: vgic-v2: Implement correctly ITARGETSR0 - ITARGETSR7 read-only

[Stefano Stabellini]

On Mon, 9 Nov 2015, Julien Grall wrote:
> Each ITARGETSR register are 4-byte wide and the offset is in byte.
> 
> The current implementation is computing the end of the range wrongly
> resulting to emulate only ITARGETSR{0,1} read-only. The rest will be
> treated as read-write.
> 
> As 8 registers should be read-only, the end of the range should be
> ITARGETSR + (4 * 8) - 1.
> 
> For convenience introduce ITARGETSR7 and ITARGETSR8.
> 
> Signed-off-by: Julien Grall 

Reviewed-by: Stefano Stabellini 


> This would be a good candidate to backport. Without it a guest could
> modify ITARGETSR{0-7} and redirect the interrupt to the wrong vCPU.
> 
> Spotted while testing to boot FreeBSD guest with this series.
> FreeBSD is writing in ITARGETSR{0 - 7} and will therefore crash xen
> due to the valid ASSERT in vgic_store_itargetsr.
> 
> Note that the emulation is not properly emulated the last register
> of each range. I'm planning to fix it in a follow-up series.
> 
> Changes in v5:
> - Patch added
> ---
>  xen/arch/arm/vgic-v2.c| 4 ++--
>  xen/include/asm-arm/gic.h | 2 ++
>  2 files changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index f7d784b..041291c 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -338,11 +338,11 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v, 
> mmio_info_t *info,
> v, r, gicd_reg - GICD_ICACTIVER);
>  return 0;
>  
> -case GICD_ITARGETSR ... GICD_ITARGETSR + 7:
> +case GICD_ITARGETSR ... GICD_ITARGETSR7:
>  /* SGI/PPI target is read only */
>  goto write_ignore_32;
>  
> -case GICD_ITARGETSR + 8 ... GICD_ITARGETSRN:
> +case GICD_ITARGETSR8 ... GICD_ITARGETSRN:
>  {
>  /* unsigned long needed for find_next_bit */
>  unsigned long target;
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 0116481..3064d1c 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -42,6 +42,8 @@
>  #define GICD_IPRIORITYR (0x400)
>  #define GICD_IPRIORITYRN (0x7F8)
>  #define GICD_ITARGETSR  (0x800)
> +#define GICD_ITARGETSR7 (0x81C)
> +#define GICD_ITARGETSR8 (0x820)
>  #define GICD_ITARGETSRN (0xBF8)
>  #define GICD_ICFGR  (0xC00)
>  #define GICD_ICFGRN (0xCFC)
> -- 
> 2.1.4
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCHv3] 26/28] build: convert HAS_GICV3 use to Kconfig

[Julien Grall]

Hi Doug,

On 12/11/15 22:54, Doug Goldstein wrote:
> Use the Kconfig generated CONFIG_HAS_GICV3 defines in the code base.
> 
> CC: Ian Campbell 
> CC: Stefano Stabellini 
> Signed-off-by: Doug Goldstein 
> ---
>  xen/arch/arm/Kconfig | 5 +
>  xen/arch/arm/Makefile| 2 +-
>  xen/arch/arm/Rules.mk| 2 --
>  xen/arch/arm/vgic.c  | 2 +-
>  xen/include/asm-arm/domain.h | 3 ++-
>  xen/include/asm-arm/gic.h| 4 ++--
>  xen/include/asm-arm/vgic.h   | 2 +-

I was expecting you to drop variable HAS_GICV3 in config/arm64.mk.

BTW, this remark is also valid for most of the patch in this series.
Configuration variable may live either in arch/*/Rules.mk or in config/*.mk.

>  7 files changed, 12 insertions(+), 8 deletions(-)

[..]

> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index e7e40da..1ce5e0b 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -102,7 +102,8 @@ struct arch_domain
>  struct pending_irq *pending_irqs;
>  /* Base address for guest GIC */
>  paddr_t dbase; /* Distributor base address */
> -#ifdef HAS_GICV3
> +paddr_t cbase; /* CPU base address */
> +#ifdef CONFIG_HAS_GICV3

As already said this on v1, can you please make sure that you series
don't re-introduce code or change it.

This should be pretty easy to check with grep. I.e any changes in *.c
and *.h files but in lines containing ifdef/endif are likely wrong.

>  /* GIC V3 addressing */
>  /* List of contiguous occupied by the redistributors */
>  struct vgic_rdist_region {

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V8 3/7] libxl: add pvusb API

[Olaf Hering]

On Wed, Oct 21, Chunyan Liu wrote:

> Add pvusb APIs, including:

> @@ -635,6 +664,8 @@ libxl_domain_config = Struct("domain_config", [
>  ("pcidevs", Array(libxl_device_pci, "num_pcidevs")),
>  ("rdms", Array(libxl_device_rdm, "num_rdms")),
>  ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")),
> +("usbctrls", Array(libxl_device_usbctrl, "num_usbctrls")),
> +("usbs", Array(libxl_device_usb, "num_usbs")),

Should that perhaps be "usbctrls" + "usbdevs" if the latter really
represents a device below one of the "usbctrls"? I dont know if it does,
just wondering.


Olaf

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCHv3] 00/28] Kconfig conversion

[Julien Grall]

Hi Doug,

On 12/11/15 22:54, Doug Goldstein wrote:
> The following series is a follow on to the Kconfig conversion patch series.
> There are still more components to convert however this is the bare minimal
> to get everything working and get the options out of the existing makefiles.
> 
> The CONFIG_HAS_ variables are there to match the behavior of the Linux
> CONFIG_HAVE_ variables. The purpose is to say that this hardware/profile/env
> supports this option while the CONFIG_ variable states that this option was
> requested on/off by user intervention.
> 
> Ultimately my goal is to allow for more parts of the hypervisor to be turned
> off at compile time and potentially make it easier to include more
> experimental features by others which can be turned off by default. Also to
> provide the one true location for all possible knobs in the source code.
> 
> The patch series can be grabbed at: https://github.com/cardoe/xen.git
> The branch is: kconfig_v3
> 
> Changes since v2:
> - drop x86_32 support (patch 2)
> - fix make defconfig (patch 2)
> - fix 'make -C xen' vs 'cd xen && make' behaving differently (patch 2)
> - fix for ARM64 builds (added patch 3)
> - At this point all targets are tested on x86_64, arm32, and arm64 with
>   fresh clones and rebuilds.

After this series, the resulting binary won't be the same. For instance
on ARM64 all the UART drivers are disabled.

The user/test system should be able to get the same options enabled by
default with and without your series.

I wasn't able to find any documentation how to use your Kconfig with
Xen, so I've tried different things which don't work as I was expected.

1) If I modify myself xen/.config to remove/add an option, the config
won't be recheck and ignored
2) make menuconfig doesn't expose any options => No possibility to
select any UART on ARM.

The latter is because how you define the option in the Kconfig:

config HAS_EXYNOS4210
bool

Without any text, it's not possible for the user to select this option.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 0/9] xen/arm: Bunch of fixes for the vGIC emulation

[Julien Grall]

Hi all,

The main point of this series is to fix the access to any register when the
user doesn't write at the base offset of the registers.

At the same, I took the opportunity to re-arrange the emulation and dropping
any registers which doesn't exists or not required by the spec.

This series is based on "xen/arm: vgic: Support 32-bit access for 64-bit
register" [1]. I've provided a branch with the 2 series applied:

git://xenbits.xen.org/people/julieng/xen-unstable.git branch gic-emulation-v1

Sincerely yours,

[1] http://lists.xen.org/archives/html/xen-devel/2015-11/msg00782.html

Julien Grall (9):
  xen/arm: vgic-v3: Use the correct offset GICR_IGRPMODR0
  xen/arm: vgic-v3: Only emulate identification registers requested by
the spec
  xen/arm: vgic: Properly emulate the full register
  xen/arm: vgic-v3: Remove GICR_MOVALLR and GICR_MOVLPIR
  xen/arm: vgic: Re-order the register emulations to match the memory
map
  xen/arm: vgic-v3: Emulate read to GICD_ICACTIVER
  xen/arm: vgic-v3: Remove spurious return in GICR_INVALLR
  xen/arm: vgic-v3: Don't implement write-only register read as zero
  xen/arm: vgic-v3: Make clear that GICD_*SPI_* registers are reserved

 xen/arch/arm/vgic-v2.c| 252 +++--
 xen/arch/arm/vgic-v3.c| 720 +-
 xen/include/asm-arm/gic_v3_defs.h |  16 +-
 xen/include/asm-arm/vgic-emul.h   |  24 ++
 4 files changed, 654 insertions(+), 358 deletions(-)
 create mode 100644 xen/include/asm-arm/vgic-emul.h

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 9/9] xen/arm: vgic-v3: Make clear that GICD_*SPI_* registers are reserved

[Julien Grall]

Our vGIC emulation have GICD_TYPER.MBIS set to 0 which means that
GICD_*SPI_* registers are reserved. Implement them using the *_reserved
labels.

Also, implement thoses registers for the read part.

Signed-off-by: Julien Grall 
---
 xen/arch/arm/vgic-v3.c | 24 
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 44e926a..985e866 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -951,15 +951,31 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 case VRANGE32(0x0020, 0x003C):
 goto read_impl_defined;
 
+case VREG32(GICD_SETSPI_NSR):
+/* Message based SPI is not implemented */
+goto read_reserved;
+
 case VREG32(0x0044):
 goto read_reserved;
 
+case VREG32(GICD_CLRSPI_NSR):
+/* Message based SPI is not implemented */
+goto read_reserved;
+
 case VREG32(0x004C):
 goto read_reserved;
 
+case VREG32(GICD_SETSPI_SR):
+/* Message based SPI is not implemented */
+goto read_reserved;
+
 case VREG32(0x0054):
 goto read_reserved;
 
+case VREG32(GICD_CLRSPI_SR):
+/* Message based SPI is not implemented */
+goto read_reserved;
+
 case VRANGE32(0x005C, 0x007C):
 goto read_reserved;
 
@@ -1125,28 +1141,28 @@ static int vgic_v3_distr_mmio_write(struct vcpu *v, 
mmio_info_t *info,
 
 case VREG32(GICD_SETSPI_NSR):
 /* Message based SPI is not implemented */
-goto write_ignore_32;
+goto write_reserved;
 
 case VREG32(0x0044):
 goto write_reserved;
 
 case VREG32(GICD_CLRSPI_NSR):
 /* Message based SPI is not implemented */
-goto write_ignore_32;
+goto write_reserved;
 
 case VREG32(0x004C):
 goto write_reserved;
 
 case VREG32(GICD_SETSPI_SR):
 /* Message based SPI is not implemented */
-goto write_ignore_32;
+goto write_reserved;
 
 case VREG32(0x0054):
 goto write_reserved;
 
 case VREG32(GICD_CLRSPI_SR):
 /* Message based SPI is not implemented */
-goto write_ignore_32;
+goto write_reserved;
 
 case VRANGE32(0x005C, 0x007C):
 goto write_reserved;
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 7/9] xen/arm: vgic-v3: Remove spurious return in GICR_INVALLR

[Julien Grall]

Signed-off-by: Julien Grall 
---
 xen/arch/arm/vgic-v3.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 98104ba..c68afdb 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -248,7 +248,6 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 case VREG64(GICR_INVALLR):
 /* WO. Read as zero */
 goto read_as_zero_64;
-return 0;
 
 case 0x00B8:
 goto read_reserved;
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 6/9] xen/arm: vgic-v3: Emulate read to GICD_ICACTIVER

[Julien Grall]

The GICD_ICACTIVER registers are missing in the read emulation of the
distributor.

Call the common emulation for the whole range.

Signed-off-by: Julien Grall 
---
 xen/arch/arm/vgic-v3.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index f9d8ecb..98104ba 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -966,6 +966,7 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 case VRANGE32(GICD_ISPENDR, GICD_ISPENDRN):
 case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
 case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
+case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
 case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
 case VRANGE32(GICD_ICFGR, GICD_ICFGRN):
 /*
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 1/9] xen/arm: vgic-v3: Use the correct offset GICR_IGRPMODR0

[Julien Grall]

The offset is 0x0D00 and not 0x0F80.

Also re-order the definition to keep all the definitions ordered.

Signed-off-by: Julien Grall 
---
 xen/include/asm-arm/gic_v3_defs.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/asm-arm/gic_v3_defs.h 
b/xen/include/asm-arm/gic_v3_defs.h
index 89a3548..34e8b0a 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -96,7 +96,6 @@
 /* GICR for SGI's & PPI's */
 
 #define GICR_IGROUPR0(0x0080)
-#define GICR_IGRPMODR0   (0x0F80)
 #define GICR_ISENABLER0  (0x0100)
 #define GICR_ICENABLER0  (0x0180)
 #define GICR_ISPENDR0(0x0200)
@@ -107,6 +106,7 @@
 #define GICR_IPRIORITYR7 (0x041C)
 #define GICR_ICFGR0  (0x0C00)
 #define GICR_ICFGR1  (0x0C04)
+#define GICR_IGRPMODR0   (0x0D00)
 #define GICR_NSACR   (0x0E00)
 
 #define GICR_TYPER_PLPIS (1U << 0)
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 8/9] xen/arm: vgic-v3: Don't implement write-only register read as zero

[Julien Grall]

A read to a write only register is unknown. Use a memorable value to
differentiate from an actual RAZ register.

Signed-off-by: Julien Grall 
---
 xen/arch/arm/vgic-v3.c | 20 
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index c68afdb..44e926a 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -217,12 +217,12 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 goto read_impl_defined;
 
 case VREG64(GICR_SETLPIR):
-/* WO. Read as zero */
-goto read_as_zero_64;
+/* WO. Read unknown */
+goto read_unknown;
 
 case VREG64(GICR_CLRLPIR):
-/* WO. Read as zero */
-goto read_as_zero_64;
+/* WO. Read unknown */
+goto read_unknown;
 
 case 0x0050:
 goto read_reserved;
@@ -239,15 +239,15 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 goto read_reserved;
 
 case VREG64(GICR_INVLPIR):
-/* WO. Read as zero */
-goto read_as_zero_64;
+/* WO. Read unknown */
+goto read_unknown;
 
 case 0x00A8:
 goto read_reserved;
 
 case VREG64(GICR_INVALLR):
-/* WO. Read as zero */
-goto read_as_zero_64;
+/* WO. Read unknown */
+goto read_unknown;
 
 case 0x00B8:
 goto read_reserved;
@@ -324,6 +324,10 @@ read_reserved:
v, gicr_reg);
 *r = 0;
 return 1;
+
+read_unknown:
+*r = vgic_reg64_extract(0xdeadbeafdeadbeaf, info);
+return 1;
 }
 
 static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 2/9] xen/arm: vgic-v3: Only emulate identification registers requested by the spec

[Julien Grall]

Most of the identification registers space contains implementation
defined registers (see 8.1.13 in ARM IHI 0069A) and only GIC{D,R}_PIDR2
is required to be implemented.

Currently the emulation of those registers mimic the ARM implementation,
but it's untrue to say that we properly emulate a such implementation.

Keep only GIC{D,R}_PIDR2 implemented with the "implementationd defined
bits" to zero and the ArchRev field (bits[7:4]) to 0x3 as we emulate a
GICv3.

Note that the emulation of the range wasn't valid anyway because the
registers are split in 2 sets (PIDR4-PIDR7 and PIDR0-PIDR2).

Signed-off-by: Julien Grall 
---
 xen/arch/arm/vgic-v3.c| 127 +++---
 xen/include/asm-arm/gic_v3_defs.h |  12 
 2 files changed, 76 insertions(+), 63 deletions(-)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 902f64a..0f6cb95 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -31,17 +31,15 @@
 #include 
 #include 
 
-/* GICD_PIDRn register values for ARM implementations */
-#define GICV3_GICD_PIDR0  0x92
-#define GICV3_GICD_PIDR1  0xb4
-#define GICV3_GICD_PIDR2  0x3b
-#define GICV3_GICD_PIDR4  0x04
-
-/* GICR_PIDRn register values for ARM implementations */
-#define GICV3_GICR_PIDR0  0x93
-#define GICV3_GICR_PIDR1  GICV3_GICD_PIDR1
+/*
+ * PIDR2: Only bits[7:4] are not implementation defined. We are
+ * emulating a GICv3 ([7:4] = 0x3).
+ *
+ * We don't emulate a specific registers scheme so implement the others
+ * bits as RES0 as recommended by the spec (see 8.1.13 in ARM IHI 0069A).
+ */
+#define GICV3_GICD_PIDR2  0x30
 #define GICV3_GICR_PIDR2  GICV3_GICD_PIDR2
-#define GICV3_GICR_PIDR4  GICV3_GICD_PIDR4
 
 /*
  * GICD_CTLR default value:
@@ -237,28 +235,20 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 case GICR_MOVALLR:
 /* WO Read as zero */
 goto read_as_zero_64;
-case GICR_PIDR0:
-if ( dabt.size != DABT_WORD ) goto bad_width;
-*r = vgic_reg32_extract(GICV3_GICR_PIDR0, info);
- return 1;
-case GICR_PIDR1:
-if ( dabt.size != DABT_WORD ) goto bad_width;
-*r = vgic_reg32_extract(GICV3_GICR_PIDR1, info);
- return 1;
+
+case 0xFFD0 ... 0xFFE4:
+/* Implementation defined identification registers */
+   goto read_impl_defined;
+
 case GICR_PIDR2:
 if ( dabt.size != DABT_WORD ) goto bad_width;
 *r = vgic_reg32_extract(GICV3_GICR_PIDR2, info);
  return 1;
-case GICR_PIDR3:
-/* Manufacture/customer defined */
-goto read_as_zero_32;
-case GICR_PIDR4:
-if ( dabt.size != DABT_WORD ) goto bad_width;
-*r = vgic_reg32_extract(GICV3_GICR_PIDR4, info);
- return 1;
-case GICR_PIDR5 ... GICR_PIDR7:
-/* Reserved0 */
-goto read_as_zero_32;
+
+case 0xFFEC ... 0xFFFC:
+ /* Implementation defined identification registers */
+ goto read_impl_defined;
+
 default:
 printk(XENLOG_G_ERR
"%pv: vGICR: unhandled read r%d offset %#08x\n",
@@ -280,6 +270,13 @@ read_as_zero_32:
 if ( dabt.size != DABT_WORD ) goto bad_width;
 *r = 0;
 return 1;
+
+read_impl_defined:
+printk(XENLOG_G_DEBUG
+   "%pv: vGICR: RAZ on implemention defined register offset %#08x\n",
+   v, gicr_reg);
+*r = 0;
+return 1;
 }
 
 static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info,
@@ -332,9 +329,19 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, 
mmio_info_t *info,
 case GICR_MOVALLR:
 /* LPI is not implemented */
 goto write_ignore_64;
-case GICR_PIDR7... GICR_PIDR0:
+
+case 0xFFD0 ... 0xFFE4:
+/* Implementation defined identification registers */
+   goto write_impl_defined;
+
+case GICR_PIDR2:
 /* RO */
 goto write_ignore_32;
+
+case 0xFFEC ... 0xFFFC:
+ /* Implementation defined identification registers */
+ goto write_impl_defined;
+
 default:
 printk(XENLOG_G_ERR "%pv: vGICR: unhandled write r%d offset %#08x\n",
v, dabt.reg, gicr_reg);
@@ -354,6 +361,12 @@ write_ignore_64:
 write_ignore_32:
 if ( dabt.size != DABT_WORD ) goto bad_width;
 return 1;
+
+write_impl_defined:
+printk(XENLOG_G_DEBUG
+   "%pv: vGICR: WI on implementation defined register offset %#08x\n",
+   v, gicr_reg);
+return 1;
 }
 
 static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
@@ -835,32 +848,21 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
 /* Replaced with GICR_ISPENDR0. So ignore write */
 goto read_as_zero_32;
-case GICD_PIDR0:
-/* GICv3 identification value */
-if ( dabt.size != DABT_WORD ) goto bad_width;
-*r = vgic_reg32_extract(GICV3_GICD_PIDR0, info);
-return 1;
-case GICD_PI

[Xen-devel] [PATCH 5/9] xen/arm: vgic: Re-order the register emulations to match the memory map

[Julien Grall]

It helps to find quickly whether we forgot to emulate a register or not.

At the same time add the missing reserved/implementation defined
registers. All other missing registers will be added in a follow-up if
necessary.

Note that only the distributor register map explicitely say the
size of a register (see 8.8 in ARM IHI 0069A). When the size is not
known, the implementation defined/reserved may not be emulated
correctly.

Signed-off-by: Julien Grall 

---

Note that the reserved range after IROUTER is overlapping in the
spec. It has been fixup in the code to avoid a build error.
---
 xen/arch/arm/vgic-v2.c | 177 ++---
 xen/arch/arm/vgic-v3.c | 298 +++--
 2 files changed, 328 insertions(+), 147 deletions(-)

diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index d1860a9..2c73133 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -210,9 +210,14 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 *r = vgic_reg32_extract(0x043b, info);
 return 1;
 
-/* Implementation defined -- read as zero */
-case 0x020 ... 0x03c:
-goto read_as_zero;
+case VRANGE32(0x00C, 0x01C):
+goto read_reserved;
+
+case VRANGE32(0x020, 0x03C):
+goto read_impl_defined;
+
+case VRANGE32(0x040, 0x07C):
+goto read_reserved;
 
 case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
 /* We do not implement security extensions for guests, read zero */
@@ -246,21 +251,6 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
 goto read_as_zero;
 
-case VRANGE32(GICD_ITARGETSR, GICD_ITARGETSRN):
-{
-uint32_t itargetsr;
-
-if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
-rank = vgic_rank_offset(v, 8, gicd_reg - GICD_ITARGETSR, DABT_WORD);
-if ( rank == NULL) goto read_as_zero;
-vgic_lock_rank(v, rank, flags);
-itargetsr = vgic_fetch_itargetsr(rank, gicd_reg - GICD_ITARGETSR);
-vgic_unlock_rank(v, rank, flags);
-*r = vgic_reg32_extract(itargetsr, info);
-
-return 1;
-}
-
 case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
 {
 uint32_t ipriorityr;
@@ -279,6 +269,27 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 return 1;
 }
 
+case VREG32(0x7FC):
+goto read_reserved;
+
+case VRANGE32(GICD_ITARGETSR, GICD_ITARGETSRN):
+{
+uint32_t itargetsr;
+
+if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+rank = vgic_rank_offset(v, 8, gicd_reg - GICD_ITARGETSR, DABT_WORD);
+if ( rank == NULL) goto read_as_zero;
+vgic_lock_rank(v, rank, flags);
+itargetsr = vgic_fetch_itargetsr(rank, gicd_reg - GICD_ITARGETSR);
+vgic_unlock_rank(v, rank, flags);
+*r = vgic_reg32_extract(itargetsr, info);
+
+return 1;
+}
+
+case VREG32(0xBFC):
+goto read_reserved;
+
 case VRANGE32(GICD_ICFGR, GICD_ICFGRN):
 {
 uint32_t icfgr;
@@ -295,6 +306,9 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 return 1;
 }
 
+case VRANGE32(0xD00, 0xDFC):
+goto read_impl_defined;
+
 case VRANGE32(GICD_NSACR, GICD_NSACRN):
 /* We do not implement security extensions for guests, read zero */
 goto read_as_zero_32;
@@ -305,32 +319,27 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 *r = 0xdeadbeef;
 return 1;
 
+case VRANGE32(0xF04, 0xF0C):
+goto read_reserved;
+
 /* Setting/Clearing the SGI pending bit via GICD is not supported */
 case VRANGE32(GICD_CPENDSGIR, GICD_CPENDSGIRN):
 case VRANGE32(GICD_SPENDSGIR, GICD_SPENDSGIRN):
 goto read_as_zero;
 
-/* Implementation defined -- read as zero */
-case 0xfd0 ... 0xfe4:
-goto read_as_zero;
+case VRANGE32(0xF30, 0xFCC):
+goto read_reserved;
+
+case VRANGE32(0xFD0, 0xFE4):
+goto read_impl_defined;
 
 case VREG32(GICD_ICPIDR2):
 if ( dabt.size != DABT_WORD ) goto bad_width;
 printk(XENLOG_G_ERR "%pv: vGICD: unhandled read from ICPIDR2\n", v);
 return 0;
 
-/* Implementation defined -- read as zero */
-case 0xfec ... 0xffc:
-goto read_as_zero;
-
-/* Reserved -- read as zero */
-case 0x00c ... 0x01c:
-case 0x040 ... 0x07c:
-case 0x7fc:
-case 0xbfc:
-case 0xf04 ... 0xf0c:
-case 0xf30 ... 0xfcc:
-goto read_as_zero;
+case VRANGE32(0xFEC, 0xFFC):
+goto read_impl_defined;
 
 default:
 printk(XENLOG_G_ERR "%pv: vGICD: unhandled read r%d offset %#08x\n",
@@ -349,6 +358,20 @@ read_as_zero_32:
 read_as_zero:
 *r = 0;
 return 1;
+
+read_impl_defined:
+printk(XENLOG_G_DEBUG
+   "%pv: vGICD: RAZ o

[Xen-devel] [PATCH 4/9] xen/arm: vgic-v3: Remove GICR_MOVALLR and GICR_MOVLPIR

[Julien Grall]

The 2 registers are not described in the software spec (ARM IHI 0069A)
and their offsets are marked "implementation defined".

Signed-off-by: Julien Grall 

---

Note that I didn't combine the 2 cases because a follow-up patch
will add reserved registers between the 2 cases.
---
 xen/arch/arm/vgic-v3.c| 20 
 xen/include/asm-arm/gic_v3_defs.h |  2 --
 2 files changed, 8 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 892104d..28f075a 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -241,13 +241,11 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 *r = vgic_reg32_extract(GICR_SYNCR_NOT_BUSY, info);
 return 1;
 
-case VREG64(GICR_MOVLPIR):
-/* WO Read as zero */
-goto read_as_zero_64;
+case VREG64(0x0100):
+goto read_impl_defined;
 
-case VREG64(GICR_MOVALLR):
-/* WO Read as zero */
-goto read_as_zero_64;
+case VREG64(0x0110):
+goto read_impl_defined;
 
 case 0xFFD0 ... 0xFFE4:
 /* Implementation defined identification registers */
@@ -348,13 +346,11 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, 
mmio_info_t *info,
 /* RO */
 goto write_ignore_32;
 
-case VREG64(GICR_MOVLPIR):
-/* LPI is not implemented */
-goto write_ignore_64;
+case VREG64(0x0100):
+goto write_impl_defined;
 
-case VREG64(GICR_MOVALLR):
-/* LPI is not implemented */
-goto write_ignore_64;
+case VREG64(0x0110):
+goto write_impl_defined;
 
 case 0xFFD0 ... 0xFFE4:
 /* Implementation defined identification registers */
diff --git a/xen/include/asm-arm/gic_v3_defs.h 
b/xen/include/asm-arm/gic_v3_defs.h
index 5a6938c..6d98491 100644
--- a/xen/include/asm-arm/gic_v3_defs.h
+++ b/xen/include/asm-arm/gic_v3_defs.h
@@ -77,8 +77,6 @@
 #define GICR_INVLPIR (0x00A0)
 #define GICR_INVALLR (0x00B0)
 #define GICR_SYNCR   (0x00C0)
-#define GICR_MOVLPIR (0x100)
-#define GICR_MOVALLR (0x0110)
 #define GICR_PIDR2   GICD_PIDR2
 
 /* GICR for SGI's & PPI's */
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 3/9] xen/arm: vgic: Properly emulate the full register

[Julien Grall]

The offset in the emulation is based on byte. As most of the registers
are 64/32 bits, they will span over multiple bytes.

However, the current emulation only care about the first offset. This
will result to not emulate properly any access on the register with
other offset.

Introduce new macros to help implementing access on multiple byte and
use them over the vGIC emulation.

Note that I didn't convert the reserved/implementation defined
registers. It will be done in a follow-up.

Signed-off-by: Julien Grall 
---
 xen/arch/arm/vgic-v2.c  |  81 ++--
 xen/arch/arm/vgic-v3.c  | 287 
 xen/include/asm-arm/vgic-emul.h |  24 
 3 files changed, 240 insertions(+), 152 deletions(-)
 create mode 100644 xen/include/asm-arm/vgic-emul.h

diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index 4fb954b..d1860a9 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -31,6 +31,7 @@
 #include 
 #include 
 #include 
+#include 
 
 static struct {
 bool_t enabled;
@@ -177,13 +178,14 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 
 switch ( gicd_reg )
 {
-case GICD_CTLR:
+case VREG32(GICD_CTLR):
 if ( dabt.size != DABT_WORD ) goto bad_width;
 vgic_lock(v);
 *r = vgic_reg32_extract(v->domain->arch.vgic.ctlr, info);
 vgic_unlock(v);
 return 1;
-case GICD_TYPER:
+
+case VREG32(GICD_TYPER):
 {
 uint32_t typer;
 
@@ -198,7 +200,8 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 
 return 1;
 }
-case GICD_IIDR:
+
+case VREG32(GICD_IIDR):
 if ( dabt.size != DABT_WORD ) goto bad_width;
 /*
  * XXX Do we need a JEP106 manufacturer ID?
@@ -211,11 +214,11 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 case 0x020 ... 0x03c:
 goto read_as_zero;
 
-case GICD_IGROUPR ... GICD_IGROUPRN:
+case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
 /* We do not implement security extensions for guests, read zero */
 goto read_as_zero_32;
 
-case GICD_ISENABLER ... GICD_ISENABLERN:
+case VRANGE32(GICD_ISENABLER, GICD_ISENABLERN):
 if ( dabt.size != DABT_WORD ) goto bad_width;
 rank = vgic_rank_offset(v, 1, gicd_reg - GICD_ISENABLER, DABT_WORD);
 if ( rank == NULL) goto read_as_zero;
@@ -224,7 +227,7 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 vgic_unlock_rank(v, rank, flags);
 return 1;
 
-case GICD_ICENABLER ... GICD_ICENABLERN:
+case VRANGE32(GICD_ICENABLER, GICD_ICENABLERN):
 if ( dabt.size != DABT_WORD ) goto bad_width;
 rank = vgic_rank_offset(v, 1, gicd_reg - GICD_ICENABLER, DABT_WORD);
 if ( rank == NULL) goto read_as_zero;
@@ -234,16 +237,16 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 return 1;
 
 /* Read the pending status of an IRQ via GICD is not supported */
-case GICD_ISPENDR ... GICD_ISPENDRN:
-case GICD_ICPENDR ... GICD_ICPENDRN:
+case VRANGE32(GICD_ISPENDR, GICD_ISPENDRN):
+case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
 goto read_as_zero;
 
 /* Read the active status of an IRQ via GICD is not supported */
-case GICD_ISACTIVER ... GICD_ISACTIVERN:
-case GICD_ICACTIVER ... GICD_ICACTIVERN:
+case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
+case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
 goto read_as_zero;
 
-case GICD_ITARGETSR ... GICD_ITARGETSRN:
+case VRANGE32(GICD_ITARGETSR, GICD_ITARGETSRN):
 {
 uint32_t itargetsr;
 
@@ -258,7 +261,7 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 return 1;
 }
 
-case GICD_IPRIORITYR ... GICD_IPRIORITYRN:
+case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
 {
 uint32_t ipriorityr;
 
@@ -276,7 +279,7 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 return 1;
 }
 
-case GICD_ICFGR ... GICD_ICFGRN:
+case VRANGE32(GICD_ICFGR, GICD_ICFGRN):
 {
 uint32_t icfgr;
 
@@ -292,26 +295,26 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, 
mmio_info_t *info,
 return 1;
 }
 
-case GICD_NSACR ... GICD_NSACRN:
+case VRANGE32(GICD_NSACR, GICD_NSACRN):
 /* We do not implement security extensions for guests, read zero */
 goto read_as_zero_32;
 
-case GICD_SGIR:
+case VREG32(GICD_SGIR):
 if ( dabt.size != DABT_WORD ) goto bad_width;
 /* Write only -- read unknown */
 *r = 0xdeadbeef;
 return 1;
 
 /* Setting/Clearing the SGI pending bit via GICD is not supported */
-case GICD_CPENDSGIR ... GICD_CPENDSGIRN:
-case GICD_SPENDSGIR ... GICD_SPENDSGIRN:
+case VRANGE32(GICD_CPENDSGIR, GICD_CPENDSGIRN):
+case VRANGE32(GICD_SPENDSGIR, GICD_SPENDSGIRN):
 goto read_as_zer

[Xen-devel] [xen-unstable test] 64149: regressions - FAIL

[osstest service owner]

flight 64149 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/64149/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-arndale   6 xen-boot  fail REGR. vs. 64035
 test-armhf-armhf-xl-multivcpu 17 guest-destroyfail REGR. vs. 64035

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 11 guest-start   fail REGR. vs. 64035
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 16 
guest-localmigrate/x10 fail blocked in 64035
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 9 debian-hvm-install fail 
like 64035
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail like 64035
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail like 64035

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-xl-vhd   9 debian-di-installfail   never pass
 test-armhf-armhf-libvirt-raw  9 debian-di-installfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-libvirt-qcow2  9 debian-di-installfail never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass

version targeted for testing:
 xen  4fa72bd15564c59cbd54d4db31a3060293e782ca
baseline version:
 xen  22a1fbb575df3a3a7726cdeb5ddf19cc8f60827c

Last test of basis64035  2015-11-10 08:01:11 Z3 days
Testing same since64149  2015-11-11 19:15:29 Z1 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Aravind Gopalakrishnan 
  Jan Beulich 
  Riku Voipio 
  Roger Pau Monné 
  Shannon Zhao 
  Wei Liu 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-oldkern  pass
 build-i386-oldkern   pass
 build-amd64-prev pass
 build-i386-prev  pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  pass
 build-i386-rumpuserxen   pass
 test-amd64-amd64-xl  pass
 test-armh

[Xen-devel] [PATCH 0/5] libbxl: add support for pvscsi, iteration 6

[Olaf Hering]

Note: just a rebase close to "RELEASE-4.6.0", with no substantial changes

Port vscsi=[] and scsi-{attach,detach,list} commands from xend to libxl.

libvirt uses its existing SCSI support:
http://lists.xenproject.org/archives/html/xen-devel/2015-04/msg02963.html

targetcli/rtslib has to be aware of xen-scsiback (upstream unresponsive):
http://article.gmane.org/gmane.linux.scsi.target.devel/8146

TODO:
 - check if idl names should contain "ctrl" and "device"
 - check if "detach " can be done with single host/single device instead
   of the ->remove field
 - check if a transaction should be used in libxl__device_vscsi_add
 - maybe use events instead of polling for "state" changes in reconfigure
   (libxl__wait_for_backend vs. libxl__ev_devstate_wait)
 - comply with tools/libxl/CODING_STYLE everywhere
 - document pvscsi in xen wiki

Changes between v5 and v6:
 - rebase to 'staging' (a7b39c8 and b7e7ad8)
 - Fix off-by-one in xlu__vscsi_compare_udev
 - xl.cfg: use options instead of option
 - xl.cfg: fix grammar for pdev/options
 - xl.cfg: fix Usually typo
 - Remove next_vscsi_dev_id from libxl_device_vscsi
 - Use XLU_WWN_LEN also in libxl_vscsi.c

Changes between v4 and v5:
 - vscsiif.h: refer to backend_domid
 - Set update_json in libxl_device_vscsi_remove
 - Remove comment from libxl__device_vscsi_add
 - Remove debug LOG from libxl__device_vscsi_reconfigure
 - Move local nb variable in libxl__device_vscsi_reconfigure
 - Make be_path const in libxl__device_vscsi_reconfigure
 - Adjust libxl__device_vscsi_dev_backend_set to avoid long lines
 - Adjust libxl__device_vscsi_dev_backend_rm to avoid long lines
 - Use CTX in libxl__device_vscsi_dev_backend_rm
 - Make be_path const in libxl__device_vscsi_dev_backend_rm
 - xl.cfg: Its typo
 - xl.cfg: Use persistent instead of persistant
 - Rename feature_host to scsi_raw_cmds
 - target-create-xen-scsiback.sh: detect pvops and xenlinux
 - Wrap long lines in main_vscsilist
 - Call libxl_vscsiinfo_dispose unconditional
 - Let scsi-list print p-dev instead of p-devname
 - Handle broken vscsi device entry in xenstore
 - Split libxl__vscsi_fill_host from libxl_device_vscsi_list
 - Make xlu_vscsi_append_dev static
 - Remove reference to pvscsi.txt from xenstore-paths.markdown
 - xl.cfg: update Linux and xenlinux
 - xl.cfg: refer to backend domain instead of dom0
 - xl.cfg: be more verbose what persistant format is
 - return if libxl__device_vscsi_dev_backend_set fails in 
libxl__device_vscsi_new_backend
 - target-create-xen-scsiback.sh: set also alias for libvirt
http://lists.xenproject.org/archives/html/xen-devel/2015-05/msg00523.html

Changes between v3 and v4:
 - Use libxl__device_nextid in libxl__device_vscsi_add
 - Remove check for duplicate pdev assignment from libxl_device_vscsi_get_host
 - Caller provides libxl_device_vscsi to libxl_device_vscsi_get_host
 - Define LIBXL_HAVE_VSCSI
 - Remove init_val from libxl_vscsi_pdev_type
 - Move some functions from libxl to libxlu
 - Introduce libxl_device_vscsi->next_vscsi_dev_id to handle holes
 - Use Struct in KeyedUnion for ocaml idl
 - docs: Mention pvscsi in xl and xl.cfg
 - Turn feature_host into a defbool and add checking
 - Support pvops and /dev/ nodes in config
 - Wrap entire libxlu_vscsi.c with ifdef linux
 - Set remove flag in libxl_device_vscsi_list
 - Fix vscsiif path in xenstore-paths.markdown
 - vscsiif.h: add some notes about xenstore layout
 - Add copyright to libxlu_vscsi.c and libxl_vscsi.c
 - Scripts to create and delete xen-scsiback nodes in Linux target framework
 - Remove pvscsi.txt
http://lists.xenproject.org/archives/html/xen-devel/2015-04/msg01949.html

Changes between v2 and v3:
 - Adjust change for vscsiif.h
 - Support "naa.wwn:lun" notation in pvops kernel
 - Add example for pvops kernel using targetcli
   patch required for python-rtslib:
   http://article.gmane.org/gmane.linux.scsi.target.devel/8146
 - Use vdev variable in libxl_device_vscsi_parse
http://lists.xenproject.org/archives/html/xen-devel/2015-03/msg00734.html

Changes between v1 and v2:
 - ported to current staging
http://lists.xenproject.org/archives/html/xen-devel/2015-03/msg00030.html

Initial attempt was sent a year ago:
http://lists.xenproject.org/archives/html/xen-devel/2014-04/msg03958.html
Most comments are addressed.

This version has been tested with SLES as backend and frontend.
This version has been tested with pvops as backend and SLES as frontend.



Olaf Hering (5):
  vscsiif.h: fix WWN notation for p-dev property
  docs: add vscsi to xenstore-paths.markdown
  libxl: add support for vscsi
  vscsiif.h: add some notes about xenstore layout
  Scripts to create and delete xen-scsiback nodes in Linux target
framework

 docs/man/xl.cfg.pod.5|  55 +++
 docs/man/xl.pod.1|  18 +
 docs/misc/xenstore-paths.markdown|  10 +
 tools/libxl/Makefile |   2 +
 tools/libxl/libxl.c  | 440 ++
 tools/libxl/libxl.h  

[Xen-devel] [PATCH 3/5] libxl: add support for vscsi

[Olaf Hering]

Port pvscsi support from xend to libxl:

 vscsi=['pdev,vdev{,options}']
 xl scsi-attach
 xl scsi-detach
 xl scsi-list

Signed-off-by: Olaf Hering 
Cc: Ian Jackson 
Cc: Stefano Stabellini 
Cc: Ian Campbell 
Cc: Wei Liu 
---
 docs/man/xl.cfg.pod.5|  55 +++
 docs/man/xl.pod.1|  18 +
 tools/libxl/Makefile |   2 +
 tools/libxl/libxl.c  | 440 
 tools/libxl/libxl.h  |  27 ++
 tools/libxl/libxl_create.c   |   1 +
 tools/libxl/libxl_device.c   |   2 +
 tools/libxl/libxl_internal.h |  16 +
 tools/libxl/libxl_types.idl  |  55 +++
 tools/libxl/libxl_types_internal.idl |   1 +
 tools/libxl/libxl_vscsi.c| 273 +
 tools/libxl/libxlu_vscsi.c   | 749 +++
 tools/libxl/libxlutil.h  |  18 +
 tools/libxl/xl.h |   3 +
 tools/libxl/xl_cmdimpl.c | 205 +-
 tools/libxl/xl_cmdtable.c|  15 +
 16 files changed, 1879 insertions(+), 1 deletion(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index b63846a..9b785cc 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -517,6 +517,61 @@ value is optional if this is a guest domain.
 
 =back
 
+=item B
+
+Specifies the PVSCSI devices to be provided to the guest. PVSCSI passes
+SCSI devices from the backend domain to the guest.
+
+Each VSCSI_SPEC_STRING consists of "pdev,vdev[,options]".
+'pdev' describes the physical device, preferable in a persistent format such 
as /dev/disk/by-*/*.
+'vdev' is the domU device in vHOST:CHANNEL:TARGET:LUN notation, all integers.
+'options' lists additional flags which a backend may recognize.
+
+The supported values for "pdev" and "options" depends on the backend driver 
used:
+
+=over 4
+
+=item B
+
+=over 4
+
+=item C
+
+The backend driver in the pvops kernel is part of the Linux-IO Target framework
+(LIO). As such the SCSI devices have to be configured first with the tools
+provided by this framework, such as a xen-scsiback aware targetcli. The "pdev"
+in domU.cfg has to refer to a config item in that framework instead of the raw
+device. Usually this is a WWN in the form of "na.WWN:LUN".
+
+=item C
+
+No options recognized.
+
+=back
+
+=item B
+
+=over 4
+
+=item C
+
+The dom0 device in either /dev/scsidev or pHOST:CHANNEL:TARGET:LUN notation.
+
+It's recommended to use persistent names "/dev/disk/by-*/*" to refer to a 
"pdev".
+The toolstack will translate this internally to "h:c:t:l" notation, which is 
how
+the backend driver will access the device. Using the "h:c:t:l" notation for
+"pdev" in domU.cfg is discouraged because this value will change across 
reboots,
+depending on the detection order in the OS.
+
+=item C
+
+Currently only the option value "feature-host" is recognized. SCSI command
+emulation in backend driver is bypassed when "feature-host" is specified.
+
+=back
+
+=back
+
 =item B
 
 Specifies the paravirtual framebuffer devices which should be supplied
diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
index 4279c7c..0674149 100644
--- a/docs/man/xl.pod.1
+++ b/docs/man/xl.pod.1
@@ -1293,6 +1293,24 @@ List virtual trusted platform modules for a domain.
 
 =back
 
+=head2 PVSCSI DEVICES
+
+=over 4
+
+=item B I I I,I<[feature-host]>
+
+Creates a new vscsi device in the domain specified by I.
+
+=item B I I
+
+Removes the vscsi device from domain specified by I.
+
+=item B I I<[domain-id] ...>
+
+List vscsi devices for the domain specified by I.
+
+=back
+
 =head1 PCI PASS-THROUGH
 
 =over 4
diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index 6ff5bee..2b2b859 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -97,6 +97,7 @@ endif
 LIBXL_LIBS += -lyajl
 
 LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \
+   libxl_vscsi.o \
libxl_dom.o libxl_exec.o libxl_xshelp.o libxl_device.o \
libxl_internal.o libxl_utils.o libxl_uuid.o \
libxl_json.o libxl_aoutils.o libxl_numa.o libxl_vnuma.o 
\
@@ -139,6 +140,7 @@ AUTOINCS= libxlu_cfg_y.h libxlu_cfg_l.h _libxl_list.h 
_paths.h \
 AUTOSRCS= libxlu_cfg_y.c libxlu_cfg_l.c
 AUTOSRCS += _libxl_save_msgs_callout.c _libxl_save_msgs_helper.c
 LIBXLU_OBJS = libxlu_cfg_y.o libxlu_cfg_l.o libxlu_cfg.o \
+   libxlu_vscsi.o \
libxlu_disk_l.o libxlu_disk.o libxlu_vif.o libxlu_pci.o
 $(LIBXLU_OBJS): CFLAGS += $(CFLAGS_libxenctrl) # For xentoollog.h
 
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 854e957..81928a8 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -2063,6 +2063,436 @@ static int libxl__resolve_domid(libxl__gc *gc, const 
char *name,
 }
 
 
/**/
+
+static void libxl__device_vscsi_dev_backend_rm(libxl__gc *gc,
+  libxl_vscsi_d

[Xen-devel] [PATCH 4/5] vscsiif.h: add some notes about xenstore layout

[Olaf Hering]

Signed-off-by: Olaf Hering 
Acked-by: Ian Campbell 
Cc: Ian Campbell 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Keir Fraser 
Cc: Tim Deegan 
---
 xen/include/public/io/vscsiif.h | 68 +
 1 file changed, 68 insertions(+)

diff --git a/xen/include/public/io/vscsiif.h b/xen/include/public/io/vscsiif.h
index e8e38a9..2c5f04a 100644
--- a/xen/include/public/io/vscsiif.h
+++ b/xen/include/public/io/vscsiif.h
@@ -104,6 +104,74 @@
  *  response structures.
  */
 
+/*
+ * Xenstore format in practice
+ * ===
+ * 
+ * The backend driver uses a single_host:many_devices notation to manage domU
+ * devices. Everything is stored in 
/local/domain//backend/vscsi/.
+ * The xenstore layout looks like this (dom0 is assumed to be the 
backend_domid):
+ * 
+ * //feature-host = "0"
+ * //frontend = "/local/domain//device/vscsi/0"
+ * //frontend-id = ""
+ * //online = "1"
+ * //state = "4"
+ * //vscsi-devs/dev-0/p-dev = "8:0:2:1" or "naa.wwn:lun"
+ * //vscsi-devs/dev-0/state = "4"
+ * //vscsi-devs/dev-0/v-dev = "0:0:0:0"
+ * //vscsi-devs/dev-1/p-dev = "8:0:2:2"
+ * //vscsi-devs/dev-1/state = "4"
+ * //vscsi-devs/dev-1/v-dev = "0:0:1:0"
+ * 
+ * The frontend driver maintains its state in
+ * /local/domain//device/vscsi/.
+ * 
+ * /backend = "/local/domain/0/backend/vscsi//"
+ * /backend-id = "0"
+ * /event-channel = "20"
+ * /ring-ref = "43"
+ * /state = "4"
+ * /vscsi-devs/dev-0/state = "4"
+ * /vscsi-devs/dev-1/state = "4"
+ * 
+ * In addition to the entries for backend and frontend these flags are stored
+ * for the toolstack:
+ * 
+ * //vscsi-devs/dev-1/p-devname = "/dev/$device"
+ * 
+ * 
+ * Backend/frontend protocol
+ * =
+ * 
+ * To create a vhost along with a device:
+ * //feature-host = "0"
+ * //frontend = "/local/domain//device/vscsi/0"
+ * //frontend-id = ""
+ * //online = "1"
+ * //state = "1"
+ * //vscsi-devs/dev-0/p-dev = "8:0:2:1"
+ * //vscsi-devs/dev-0/state = "1"
+ * //vscsi-devs/dev-0/v-dev = "0:0:0:0"
+ * Wait for //state + //vscsi-devs/dev-0/state 
become 4
+ * 
+ * To add another device to a vhost:
+ * //state = "7"
+ * //vscsi-devs/dev-1/p-dev = "8:0:2:2"
+ * //vscsi-devs/dev-1/state = "1"
+ * //vscsi-devs/dev-1/v-dev = "0:0:1:0"
+ * Wait for //state + //vscsi-devs/dev-1/state 
become 4
+ * 
+ * To remove a device from a vhost:
+ * //state = "7"
+ * //vscsi-devs/dev-1/state = "5"
+ * Wait for //state to become 4
+ * Wait for //vscsi-devs/dev-1/state become 6
+ * Remove //vscsi-devs/dev-1/{state,p-dev,v-dev,p-devname}
+ * Remove //vscsi-devs/dev-1/
+ *
+ */
+
 /* Requests from the frontend to the backend */
 
 /*

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 2/5] docs: add vscsi to xenstore-paths.markdown

[Olaf Hering]

Signed-off-by: Olaf Hering 
Acked-by: Ian Campbell 
Cc: Ian Campbell 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Keir Fraser 
Cc: Tim Deegan 
---
 docs/misc/xenstore-paths.markdown | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/docs/misc/xenstore-paths.markdown 
b/docs/misc/xenstore-paths.markdown
index d94ea9d..b1c68ce 100644
--- a/docs/misc/xenstore-paths.markdown
+++ b/docs/misc/xenstore-paths.markdown
@@ -232,6 +232,11 @@ A virtual keyboard device frontend. Described by
 A virtual network device frontend. Described by
 [xen/include/public/io/netif.h][NETIF]
 
+ ~/device/vscsi/$DEVID/* []
+
+A virtual scsi device frontend. Described by
+[xen/include/public/io/vscsiif.h][SCSIIF]
+
  ~/console/* []
 
 The primary PV console device. Described in [console.txt](console.txt)
@@ -302,6 +307,10 @@ A virtual keyboard device backend. Described by
 A virtual network device backend. Described by
 [xen/include/public/io/netif.h][NETIF]
 
+ ~/backend/vscsi/$DOMID/$DEVID/* []
+
+A PV SCSI backend.
+
  ~/backend/console/$DOMID/$DEVID/* []
 
 A PV console backend. Described in [console.txt](console.txt)
@@ -403,6 +412,7 @@ ifb device used by Remus to buffer network output from the 
associated vif.
 [KBDIF]: 
http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,kbdif.h.html
 [LIBXLMEM]: http://xenbits.xen.org/docs/unstable/misc/libxl_memory.txt
 [NETIF]: 
http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,netif.h.html
+[SCSIIF]: 
http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,vscsiif.h.html
 [SI]: 
http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#Struct_start_info
 [VCPU]: 
http://xenbits.xen.org/docs/unstable/hypercall/include,public,vcpu.h.html
 [XSWIRE]: 
http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,xs_wire.h.html

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH 5/5] Scripts to create and delete xen-scsiback nodes in Linux target framework

[Olaf Hering]

Just to make them public, not meant for merging:
The scripts used during development to create a bunch of SCSI devices in
dom0 using the Linux target framework. targetcli3 and rtslib3 is used.

A patch is required for python-rtslib:
http://article.gmane.org/gmane.linux.scsi.target.devel/8146

Signed-off-by: Olaf Hering 
---
 tools/misc/Makefile  |   4 +
 tools/misc/target-create-xen-scsiback.sh | 135 +++
 tools/misc/target-delete-xen-scsiback.sh |  41 ++
 3 files changed, 180 insertions(+)

diff --git a/tools/misc/Makefile b/tools/misc/Makefile
index c4490f3..3d5d75f 100644
--- a/tools/misc/Makefile
+++ b/tools/misc/Makefile
@@ -34,6 +34,8 @@ INSTALL_SBIN += $(INSTALL_SBIN-y)
 
 # Everything to be installed in a private bin/
 INSTALL_PRIVBIN+= xenpvnetboot
+INSTALL_PRIVBIN+= target-create-xen-scsiback.sh
+INSTALL_PRIVBIN+= target-delete-xen-scsiback.sh
 
 # Everything to be installed
 TARGETS_ALL := $(INSTALL_BIN) $(INSTALL_SBIN) $(INSTALL_PRIVBIN)
@@ -44,6 +46,8 @@ TARGETS_COPY += xen-ringwatch
 TARGETS_COPY += xencons
 TARGETS_COPY += xencov_split
 TARGETS_COPY += xenpvnetboot
+TARGETS_COPY += target-create-xen-scsiback.sh
+TARGETS_COPY += target-delete-xen-scsiback.sh
 
 # Everything which needs to be built
 TARGETS_BUILD := $(filter-out $(TARGETS_COPY),$(TARGETS_ALL))
diff --git a/tools/misc/target-create-xen-scsiback.sh 
b/tools/misc/target-create-xen-scsiback.sh
new file mode 100755
index 000..96d4c39
--- /dev/null
+++ b/tools/misc/target-create-xen-scsiback.sh
@@ -0,0 +1,135 @@
+#!/usr/bin/env bash
+unset LANG
+unset ${!LC_*}
+set -x
+set -e
+
+modprobe --version
+targetcli --version
+udevadm --version
+blockdev --version
+parted --version
+sfdisk --version
+mkswap --version
+
+configfs=/sys/kernel/config
+target_path=$configfs/target
+
+num_luns=4
+num_hosts=4
+
+case "$1" in
+   -p)
+   backend="pvops"
+   ;;
+   -x)
+   backend="xenlinux"
+   ;;
+   *)
+   : "usage: $0 [-p|-x]"
+   if grep -qw xenfs$ /proc/filesystems
+   then
+   backend="pvops"
+   else
+   backend="xenlinux"
+   fi
+   ;;
+esac
+
+get_wwn() {
+   sed '
+   s@-@@g
+   s@^\(.\{16\}\)\(.*\)@\1@
+   ' /proc/sys/kernel/random/uuid
+}
+
+if test ! -d "${target_path}"
+then
+   modprobe -v configfs
+   mount -vt configfs configfs $configfs
+   modprobe -v target_core_mod
+fi
+if test "${backend}" = "pvops"
+then
+   modprobe -v xen-scsiback
+fi
+
+host=0
+while test $host -lt $num_hosts
+do
+   host=$(( $host + 1 ))
+   lun=0
+   loopback_wwn="naa.`get_wwn`"
+   pvscsi_wwn="naa.`get_wwn`"
+   targetcli /loopback create ${loopback_wwn}
+   if test "${backend}" = "pvops"
+   then
+   targetcli /xen-pvscsi create ${pvscsi_wwn}
+   fi
+   while test $lun -lt $num_luns
+   do
+   : h $host l $lun
+   f_file=/dev/shm/Fileio.${host}.${lun}.file
+   f_uuid=/dev/shm/Fileio.${host}.${lun}.uuid
+   f_link=/dev/shm/Fileio.${host}.${lun}.link
+   fileio_name="fio_${host}.${lun}"
+   pscsi_name="ps_${host}.${lun}"
+
+   targetcli /backstores/fileio create name=${fileio_name} 
"file_or_dev=${f_file}" size=$((1024*1024 * 8 )) sparse=true
+   targetcli /loopback/${loopback_wwn}/luns create 
/backstores/fileio/${fileio_name} $lun
+
+   udevadm settle --timeout=4
+
+   vpd_uuid="`sed -n '/^T10 VPD Unit Serial 
Number:/s@^[^:]\+:[[:blank:]]\+@@p' 
/sys/kernel/config/target/core/fileio_*/${fileio_name}/wwn/vpd_unit_serial`"
+   if test -z "${vpd_uuid}"
+   then
+   exit 1
+   fi
+   echo "${vpd_uuid}" > "${f_uuid}"
+   by_id="`echo ${vpd_uuid} | sed 
's@-@@g;s@^\(.\{25\}\)\(.*\)@scsi-36001405\1@'`"
+   ln -sfvbn "/dev/disk/by-id/${by_id}" "${f_link}"
+
+   f_major=$((`stat --dereference --format=0x%t "${f_link}"`))
+   f_minor=$((`stat --dereference --format=0x%T "${f_link}"`))
+   if test -z "${f_major}" || test -z "${f_minor}"
+   then
+   exit 1
+   fi
+   f_alias=`ls -d 
/sys/dev/block/${f_major}:${f_minor}/device/scsi_device/*:*:*:*`
+   if test -z "${f_alias}"
+   then
+   exit 1
+   fi
+   f_alias=${f_alias##*/}
+
+   blockdev --rereadpt "${f_link}"
+   udevadm settle --timeout=4
+   echo 1,12,S | sfdisk "${f_link}"
+   udevadm settle --timeout=4
+   blockdev --rereadpt "${f_link}"
+   udevadm settle --timeout=4
+   parted -s "${f_link}" unit s print
+
+   d_link="`readlink \"${f_link}\"`"
+   if test -n "${d_link

[Xen-devel] [PATCH 1/5] vscsiif.h: fix WWN notation for p-dev property

[Olaf Hering]

The pvops kernel expects either "naa.WWN:LUN" or "h:c:t:l" in the p-dev
property. Add the missing :LUN part to the comment.

Signed-off-by: Olaf Hering 
Acked-by: Ian Campbell 
Cc: Ian Campbell 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Keir Fraser 
Cc: Tim Deegan 
---
 xen/include/public/io/vscsiif.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/public/io/vscsiif.h b/xen/include/public/io/vscsiif.h
index 7a1db05..e8e38a9 100644
--- a/xen/include/public/io/vscsiif.h
+++ b/xen/include/public/io/vscsiif.h
@@ -60,7 +60,7 @@
  *
  *  A string specifying the backend device: either a 4-tuple "h:c:t:l"
  *  (host, controller, target, lun, all integers), or a WWN (e.g.
- *  "naa.60014054ac780582").
+ *  "naa.60014054ac780582:0").
  *
  * v-dev
  *  Values: string

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 2/3] arm: export platform_op XENPF_settime64

[Julien Grall]

Hi Stefano,

On 12/11/15 17:46, Stefano Stabellini wrote:
> Call update_domain_wallclock_time at domain initialization.
> Set time_offset_seconds to the number of seconds between physical boot
> and domain initialization: it is going to be used to get/set the
> wallclock time.
> Add time_offset_seconds to system_time when before calling do_settime,
> so that system_time actually accounts for all the time in nsec between
> machine boot and when the wallclock was set.
> 
> Expose xsm_platform_op to ARM.
> 
> Signed-off-by: Stefano Stabellini 

Reviewed-by: Julien Grall 

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 3/3] xen/arm: move ticks conversions function declarations to the header file

[Julien Grall]

Hi Stefano,

On 12/11/15 17:46, Stefano Stabellini wrote:
> This is just a cleanup, not required at the moment.
> 
> Signed-off-by: Stefano Stabellini 

Reviewed-by: Julien Grall 

> ---
>  xen/arch/arm/vtimer.c  |3 ---
>  xen/include/asm-arm/time.h |3 +++
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/arm/vtimer.c b/xen/arch/arm/vtimer.c
> index c8e2a12..629feb4 100644
> --- a/xen/arch/arm/vtimer.c
> +++ b/xen/arch/arm/vtimer.c
> @@ -29,9 +29,6 @@
>  #include 
>  #include 
>  
> -extern s_time_t ticks_to_ns(uint64_t ticks);
> -extern uint64_t ns_to_ticks(s_time_t ns);
> -
>  /*
>   * Check if regs is allowed access, user_gate is tail end of a
>   * CNTKCTL_EL1_ bit name which gates user access
> diff --git a/xen/include/asm-arm/time.h b/xen/include/asm-arm/time.h
> index d755f36..f99d6e8 100644
> --- a/xen/include/asm-arm/time.h
> +++ b/xen/include/asm-arm/time.h
> @@ -37,6 +37,9 @@ extern void __cpuinit init_timer_interrupt(void);
>  /* Counter value at boot time */
>  extern uint64_t boot_count;
>  
> +extern s_time_t ticks_to_ns(uint64_t ticks);
> +extern uint64_t ns_to_ticks(s_time_t ns);
> +
>  void preinit_xen_time(void);
>  
>  #endif /* __ARM_TIME_H__ */
> 


-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 19/24] Osstest/Testsupport.pm: change target's default kernkind to 'pvops'

[Ian Jackson]

This is safe only if no existing flights would be affected.  (That is,
the meaning of no existing sets of runvars would be changed.)

To check whether this would make any difference I did some database
searches.  Since any time target_kernkind_check is called it sets a
corresponding `console' runvar, I can search for `console' without a
corresponding `kernkind'.  I ran this query:

  select * from (select *, (select name from runvars r2 where
  r2.flight=r1.flight and r2.job=r1.job and r2.name=
  replace(r1.name,'console','kernkind')) kk from runvars r1 where
  r1.name like '%console') iq where kk is null order by flight desc;

and it found nothing since flight 7682.  So I think we can change the
default.

Signed-off-by:  Ian Jackson 
Signed-off-by:  Robert Ho 
Tested-by:  Robert Ho 
Acked-by: Ian Campbell 
---
v15: New patch
---
 Osstest/TestSupport.pm |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index b507ada..cb496d2 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2065,7 +2065,7 @@ sub target_var ($$) {
 sub target_kernkind_check ($) {
 my ($gho) = @_;
 my $pfx= target_var_prefix($gho);
-my $kernkind= $r{$pfx."kernkind"};
+my $kernkind= $r{$pfx."kernkind"} // 'pvops';
 my $isguest= exists $gho->{Guest};
 if ($kernkind eq 'pvops') {
 store_runvar($pfx."rootdev", 'xvda') if $isguest;
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 15/24] Nested HVM: Provide test-nested recipe

[Ian Jackson]

From: Robert Ho 

Signed-off-by: Robert Ho 
Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v14: ts-nested-setup command line syntax updated.
v15: (Robert Ho) remove the unnecessary l1 destroy; as it will
 implicitly powered off by framework as a nested host.
 This change hasn't been confirmed by Ian Jackson yet; I
 plan to separate it as fix patch but by mistake squashed
 it in. Ian Jackson may want to revert this if he dosn't
 agree.
---
 sg-run-job |9 +
 1 file changed, 9 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index 172c87e..a2527ca 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -348,6 +348,15 @@ proc run-job/test-pair-oneway {} {
 run-ts . =  ts-guest-stop  dst_host  + debian
 }
 
+proc need-hosts/test-nested {} {return host}
+proc run-job/test-nested {} {
+run-ts . = ts-debian-hvm-install + host l1
+run-ts . = ts-nested-setup + --define l1=host:l1
+nested-layer-descend l1
+run-ts . = ts-debian-hvm-install l1 l2
+run-ts . = ts-guest-stop l1 l2
+}
+
 proc test-guest-migr {g} {
 set to_reap [spawn-ts . = ts-migrate-support-check + host $g 1]
 set can_migrate [reap-ts $to_reap]
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 23/24] Serial: Add new serial method object for `guest' type

[Ian Jackson]

From: Robert Ho 

L1 guests' serial ports are owned by qemu in L0.  We can send them
debug keys by writing to the qemu pipe.

(xl debug-key looks like it would be useful but it actually sends
debug keys to the hypervisor of the host it is running on.  We want to
send the debug keys to the hypervisor and kernel from the outside.)

Log fetching is not needed because from the POV of the L0 the L1 is a
guest, so the L0's log capture will already fetch the L1's serial
console output.

Signed-off-by: Robert Ho 
Signed-off-by: Ian Jackson 

---
v17: Get FIFO path right.
 Log the ssh command.
 Put quotes around the redirection, as required.
 Export sshuho from TestSupport.
 Make ::guest->fetch_logs actually use @_ so that it works.
v16: Mostly rewritten: now uses new keys_real base class, and
 uses the qemu pipe rather than xl debug-keys.
v15: New patch.
---
 Osstest/Serial/guest.pm |   96 +++
 Osstest/TestSupport.pm  |4 +-
 2 files changed, 98 insertions(+), 2 deletions(-)
 create mode 100644 Osstest/Serial/guest.pm

diff --git a/Osstest/Serial/guest.pm b/Osstest/Serial/guest.pm
new file mode 100644
index 000..2511556
--- /dev/null
+++ b/Osstest/Serial/guest.pm
@@ -0,0 +1,96 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Intel Inc.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+# Send debug keys to nested host (L1).
+
+package Osstest::Serial::guest;
+
+use strict;
+use warnings;
+
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::Serial::keys_real;
+
+BEGIN {
+use Exporter ();
+our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
+$VERSION = 1.00;
+@ISA = qw(Exporter Osstest::Serial::keys_real);
+@EXPORT  = qw();
+%EXPORT_TAGS = ( );
+
+@EXPORT_OK   = qw();
+}
+
+sub new {
+my ($class, $l1ho, $methname, @args) = @_;
+my $mo = { Target => $l1ho, Parent => $l1ho->{Host} };
+
+logm("serial method $methname $mo->{Target}{Name}: @args");
+return bless $mo, $class;
+}
+
+sub keys_prepare {
+my ($mo) = @_;
+
+my $gho = $mo->{Target};
+my $pho = $mo->{Parent};
+my $puho = sshuho('root',$pho);
+my $domname = $r{"$gho->{Guest}_domname"};
+
+my $fifo = "/root/$flight.$job.$domname.serial.in";
+
+# NB this by-hand construction and execution of an
+# ssh command line bypasses the usual timeout arrangements.
+my ($sshopts) = sshopts();
+my $cmd = "ssh @$sshopts $puho 'cat >$fifo'";
+logm("spawning $cmd");
+
+# timeouts: open will carry on regardless even if the command hangs
+open SERIALWRITE, "|$cmd" or die $!;
+autoflush SERIALWRITE 1;
+}
+
+sub keys_write {
+my ($mo, $what,$str,$pause) = @_;
+logm("xenuse sending $what");
+
+# timeouts: we are going to write much less than any plausible
+# PIPE_MAX so there is no risk that we will block on write
+print SERIALWRITE $str or die $!;
+sleep($pause);
+}
+
+sub keys_shutdown {
+my ($mo) = @_;
+
+# timeouts: close waits for the child to exit, so set an alarm
+alarm(15);
+$!=0; $?=0; close SERIALWRITE or die "$? $!";
+alarm(0);
+}
+
+sub fetch_logs {
+my ($mo) = @_;
+
+logm("$mo->{Target}{Name} (nested host) serial console logs".
+" will be found in guest logs from $mo->{Parent}{Name} (parent)");
+return;
+}
+
+1;
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 47fc1b1..6be50e3 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -50,7 +50,7 @@ BEGIN {
 
   target_cmd_root target_cmd target_cmd_build
   target_cmd_output_root target_cmd_output
-  target_cmd_inputfh_root
+  target_cmd_inputfh_root sshuho
   target_getfile target_getfile_root
   target_putfile target_putfile_root
   target_putfilecontents_stash
@@ -874,7 +874,7 @@ sub selecthost ($) {
$child->{Power} = 'guest';
power_cycle_host_setup($child);
 
-   $child->{Properties}{Serial} = 'noop'; # todo
+   $child->{Properties}{Serial} = 'guest';
serial_host_setup($child);
 
my $msg = "L$child->{NestingLevel} host $child->{Ident}:";
-- 
1.7.10.4


__

[Xen-devel] [OSSTEST PATCH 09/24] await_tcp(): Run check_ip on each loop iteration

[Ian Jackson]

From: Robert Ho 

await_tcp is often invoked after a reboot.

In this situation the target's IP address may change.  If this happens
while await_tcp is running, we would continue to poll the old IP address.
Fix this by running target_check_ip on each iteration.

Signed-off-by: Robert Ho 
Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v14: Dropped change to selecthost, which was in code which is no
  longer present in this version of the series.
 Rewritten to use target_check_ip.
 Dropped IMO-unnecessary comment.
---
 Osstest/TestSupport.pm |5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 325e2e3..b70f19f 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2029,9 +2029,10 @@ sub await_tcp ($$$) {
 my ($maxwait,$interval,$ho) = @_;
 target_adjust_timeout($ho,\$maxwait);
 poll_loop($maxwait,$interval,
-  "await tcp $ho->{Name} $ho->{TcpCheckPort}",
+  "await tcp $ho->{Name} $ho->{Ip} $ho->{TcpCheckPort}",
   sub {
-return target_tcp_check($ho,$interval);
+   return target_check_ip($ho) //
+   target_tcp_check($ho,$interval);
 });
 }
 
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 06/24] Nested hosts: Provide PDU power method

[Ian Jackson]

From: Robert Ho 

This `guest' power method uses VM create/destroy.  It is automatically
used for nested hosts.  It would not make much sense to configure it
manually.

For nested host/guest, its power on/off method shall be
its host invoke $(toolstack)->create/destroy method.

Signed-off-by: Robert Ho 
Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v14: Mostly rewritten by iwj
---
 Osstest/PDU/guest.pm   |   59 
 Osstest/TestSupport.pm |2 +-
 2 files changed, 60 insertions(+), 1 deletion(-)
 create mode 100755 Osstest/PDU/guest.pm

diff --git a/Osstest/PDU/guest.pm b/Osstest/PDU/guest.pm
new file mode 100755
index 000..b6bf9a1
--- /dev/null
+++ b/Osstest/PDU/guest.pm
@@ -0,0 +1,59 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Intel.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+
+package Osstest::PDU::guest;
+
+use strict;
+use warnings;
+use Switch;
+
+use Osstest;
+use Osstest::TestSupport;
+use IO::File;
+
+BEGIN {
+use Exporter ();
+our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
+$VERSION = 1.00;
+@ISA = qw(Exporter);
+@EXPORT  = qw();
+%EXPORT_TAGS = ( );
+
+@EXPORT_OK   = qw();
+}
+
+sub new {
+my ($class, $ho) = @_;
+return bless { Target => $ho }, $class;
+}
+
+sub pdu_power_state {
+my ($mo, $on) = @_;
+
+my $child = $mo->{Target};
+my $parent = $child->{Host};
+die "$child->{Name} ?" unless $parent;
+
+logm("power $child->{Name} nested on $parent->{Name} ".($on+0));
+if ($on) {
+   toolstack($parent)->create($child);
+} else {
+   toolstack($parent)->destroy($child);
+}
+}
+
+1;
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 3145040..9a256a9 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -861,7 +861,7 @@ sub selecthost ($) {
$child->{Info} = [ "in", $parent->{Name}, @{ $parent->{Info} } ];
$child->{NestingLevel} = $parent->{NestingLevel}+1;
 
-   # $child->{Power} = 'guest';   todo
+   $child->{Power} = 'guest';
power_cycle_host_setup($child);
 
$child->{Properties}{Serial} = 'noop'; # todo
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 14/24] Nested HVM: Provide ts-nested-setup to help make L1 usable as a host

[Ian Jackson]

From: Robert Ho 

* Provide the L1 with some storage for its own guests' disks
* Install some packages in the L1
* Optionally, set a runvar defining the L1 for the rest of the job

The recipe is going to run ts-xen-install etc.

Signed-off-by: longtao.pang 
Signed-off-by: Robert Ho 
Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v14: Use target_check_ip (renamed, earlier, in a new patch)
 ts-nested-setup now has a default gueststorage_size of 20G
  and this is implicitly used by make-flight.
 Adjusted for new selecthost nested host syntax and correspondingly
  completely changed invocation syntax.
 Only optionally sets the runvar, if you pass --define.  (This
  will make it easier to play around with interactively.)
 Broken the pieces of work out into subroutines for clarity.
 Comment about guest storage slightly edited and rewrapped.
 Guest storage runvars and perl variable names etc. renamed to
  `gueststorage' rather than `guest_storage'.
 LVM and VG names and perl variable names changed to be clearer.
 Install `ed' too.
 Use lv_create and toolstack()->block_attach.
 Dropped ack from Ian Campbell.
---
 ts-nested-setup |   99 +++
 1 file changed, 99 insertions(+)
 create mode 100755 ts-nested-setup

diff --git a/ts-nested-setup b/ts-nested-setup
new file mode 100755
index 000..f7e0ebd
--- /dev/null
+++ b/ts-nested-setup
@@ -0,0 +1,99 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Intel Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use Osstest::Debian;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our $dodefine;
+if (@ARGV && $ARGV[0] eq '--define') { $dodefine=1; shift @ARGV; }
+
+our ($l1_identspec) = @ARGV;
+
+my $l1 = selecthost($l1_identspec);
+my $l0 = $l1->{Host};
+
+die unless $l0;
+
+our $def_gueststorage_size = 2; # Mby
+
+sub packages () {
+# Faster to install packages while L1 is still running PVHVM
+target_install_packages_norec($l1, qw(lvm2 rsync ed genisoimage));
+}
+
+sub guest_storage () {
+# We need to attach an extra disk to the L1 guest to be used as L2
+# guest storage:
+#
+# When running in a nested HVM environment the L1 domain is acting
+# as both a guest to L0 and a host to L2 guests and therefore
+# potentially sees connections to two independent xenstore
+# instances, one provided by the L0 host and one which is provided
+# by the L1 instance of xenstore.
+#
+# Unfortunately the kernel is not capable of dealing with this and
+# is only able to cope with a single xenstore connection. Since
+# the L1 toolstack and L2 guests absolutely require xenstore to
+# function we therefore cannot use the L0 xenstore and therefore
+# cannot use PV devices (xvdX etc) in the L1 guest and must use
+# emulated devices (sdX etc).
+#
+# However at the moment we have not yet rebooted L1 into Xen and
+# so it does have PV devices available and sdb actually appears as
+# xvdb.  We could disable the Xen platform device and use emulated
+# devices for the install phase too but that would be needlessly
+# slow.
+# 
+# Instead, to avoid needing to even mention the name of xvdb or
+# sdb, we create the vg in l0.  When the l1 reboots it will
+# automatically find the empty vg we have created for it, and
+# target_choose_vg on l1 (which is used by all the guest creation
+# ts-* scripts) will use it since it has plenty of space.
+
+my $size = guest_var($l1,'gueststorage_size',$def_gueststorage_size);
+die "gueststorage_size is undefined" unless $size;
+
+my $outer_vg = target_choose_vg($l0, $size);
+my $outer_lv = "$l1->{Ident}_gueststorage_outer_lv";
+my $inner_vg = "$l1->{Ident}_gueststorage_vg";
+
+target_cmd_root($l0, "vgremove -f $inner_vg ||:");
+my $outer_lvdev = lv_create($l0, $outer_vg, $outer_lv, $size);
+
+target_cmd_root($l0, 

[Xen-devel] [OSSTEST PATCH 16/24] Nested HVM: Add test job to appropriate flights

[Ian Jackson]

From: Robert Ho 

Signed-off-by: longtao.pang 
Signed-off-by: Robert Ho 
Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
---
v17: Use usual_debianhvm_image
v14: Use default gueststorage_size, rather than setting runvar.
 Dropped acked from Ian Campbell.
---
 make-flight |   28 
 1 file changed, 28 insertions(+)

diff --git a/make-flight b/make-flight
index f908017..b59b87d 100755
--- a/make-flight
+++ b/make-flight
@@ -249,6 +249,33 @@ do_hvm_win7_x64_tests () {
 all_hostflags=$most_hostflags,hvm
 }
 
+do_hvm_debian_nested_tests () {
+  bios=$1
+
+  if [ $xenarch != amd64 -o $dom0arch != amd64 \
+  -o "x$qemuu_suffix" != "x-qemuu" ]; then
+return
+  fi
+
+  case $xenbranch in
+xen-3.*-testing)  return;;
+xen-4.0-testing)  return;;
+xen-4.1-testing)  return;;
+xen-4.2-testing)  return;;
+xen-4.3-testing)  return;;
+  esac
+
+  job_create_test test-$xenarch$kern-$dom0arch$qemuu_suffix-nested \
+test-nested xl $xenarch $dom0arch $qemuu_runvar\
+l1_image=$(usual_debianhvm_image amd64)\
+l1_vifmodel='e1000'\
+l1_memsize='3072'  \
+l1_enable_nestedhvm='true' \
+l2_image=$(usual_debianhvm_image amd64)\
+bios=$bios \
+all_hostflags=$most_hostflags,hvm
+}
+
 branch_debianhvm_arch () {
   case $branch in
 xen-unstable-smoke) echo i386;;
@@ -592,6 +619,7 @@ test_matrix_do_one () {
 do_hvm_rhel6_tests
 
 do_hvm_debian_tests
+do_hvm_debian_nested_tests seabios
 
   done # qemuu_suffix
 
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 20/24] Osstest/Testsupport.pm: use get_target_property() for some host setup

[Ian Jackson]

From: Robert Ho 

For nested cases, nested host can inherit its host's property for
dhcp watch setup and ether_prefix property setup.

Signed-off-by: Robert Ho 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v15: New patch
---
 Osstest/TestSupport.pm |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index cb496d2..32f2956 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -735,7 +735,7 @@ sub lv_dev_mapper ($$) {
 sub dhcp_watch_setup ($$) {
 my ($ho,$gho) = @_;
 
-my $meth = get_host_property($ho,'dhcp-watch-method',undef);
+my $meth = get_target_property($ho,'dhcp-watch-method',undef);
 $gho->{DhcpWatch} = get_host_method_object($ho, 'DhcpWatch', $meth);
 }
 
@@ -1583,7 +1583,7 @@ sub target_choose_vg ($$) {
 
 sub ether_prefix($) {
 my ($ho) = @_;
-my $prefix = get_host_property($ho, 'gen-ether-prefix-base');
+my $prefix = get_target_property($ho, 'gen-ether-prefix-base');
 $prefix =~ m/^(\w+:\w+):(\w+):(\w+)$/ or die "$prefix ?";
 my $lhs = $1;
 my $pv = (hex($2)<<8) | (hex($3));
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH v17 00/24] Nested HVM

[Ian Jackson]

This is, I think, a fully-working series to support Nested HVM testing
in osstest.  It can be found here:

git://xenbits.xen.org/people/iwj/osstest.git
  http://xenbits.xen.org/git-http/people/iwj/osstest.git
in base.nested-hvm.v17..wip.nested-hvm.v17

 a  01  cs-adjust-flight: Add some missing doc comment info
*   02  cs-adjust-flight: Allow adjusting "this" flight
 aT 03  selecthost: Minor cleanups
+   04  make-flight: Break out usual_debianhvm_image and honour ...
 aT 05  selecthost: Support nested hosts (guests which are also hosts)
 aT 06  Nested hosts: Provide PDU power method
 aT 07  DhcpWatch::leases: Fix a reporting message
 aT 08  target_check_ip: Rename and improve from guest_check_ip
 aT 09  await_tcp(): Run check_ip on each loop iteration
 aT 10  LVM: Break out lv_create
 aT 11  Toolstack::xl: Provide block_attach method
 at 12  sg-run-job: Break out per-host-prep and per-host-finish
 AT 13  sg-run-job: Provide infrastructure for layers of nesting
 aT 14  Nested HVM: Provide ts-nested-setup to help make L1 usable as a host
 aT 15  Nested HVM: Provide test-nested recipe
* T 16  Nested HVM: Add test job to appropriate flights
 aT 17  ts-xen-install: Properly handle hosts without a static IP address
 aT 18  ts-xen-install: networking: Rename `nodhcp' to `ensurebridge'
 aT 19  Osstest/Testsupport.pm: change target's default kernkind to 'pvops'
 aT 20  Osstest/Testsupport.pm: use get_target_property() for some host setup
 at 21  HVM guests: Use qemu "pipe:" for serial output logging
 at 22  Serial: Factor out Osstest::Serial::keys_real
* t 23  Serial: Add new serial method object for `guest' type
+   24  Serial::xenuse: Send xenuse output to /dev/null

Key:
  a   acked by Ian Campbell
  A   acked by Ian Jackson
  T   formally Tested-by Robert Hu
  t   informal test report in email from Robert Hu

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 13/24] sg-run-job: Provide infrastructure for layers of nesting

[Ian Jackson]

Provides nested-layer-descend, which can be called in an individual
test job at the appropriate point (after the L1 has been set up).

The inner host is a guest of the outer host; powering it off means
destroying it.  Putting the poweroff at this point in the loop, rather
than in per-host-finish, avoids powering off physical servers.  The
use of `.'  rather than `!.' for iffail means we do not power off
after failures (as we might want to preserve the state for debugging
etc).

Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Signed-off-by: Robert Ho 
Acked-by: Ian Jackson 
---
v14: Squash syntax fix from Robert Ho into this patch
v15: Remove spurious "=" from final-poweroff step invocation
---
 sg-run-job |   21 -
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/sg-run-job b/sg-run-job
index 884a21d..172c87e 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -39,6 +39,7 @@ proc per-host-finish {} {
 
 proc run-job {job} {
 global jobinfo builds flight ok need_xen_hosts anyfailed
+global nested_layers_hosts
 
 set ok 1
 set anyfailed 0
@@ -52,6 +53,7 @@ proc run-job {job} {
 set need_xen_hosts $nh
 set need_build_host 0
 }
+set nested_layers_hosts {}
 
 catching-otherwise blockedcheck-not-blocked
 if {!$ok} return
@@ -70,7 +72,15 @@ proc run-job {job} {
 
 if {$ok} { catching-otherwise fail  run-job/$jobinfo(recipe)  }
 
-per-host-finish
+while 1 {
+   per-host-finish
+   
+   if {![llength $nested_layers_hosts]} break
+
+   per-host-ts . final-poweroff {ts-host-powercycle --power=0}
+
+set need_xen_hosts [lunappend nested_layers_hosts]
+}
 
 if {$need_build_host && $anyfailed} {
run-ts  !broken capture-logs  ts-logs-capture + host
@@ -247,6 +257,15 @@ proc per-host-ts {iffail ident script args} {
 }
 }
 
+proc nested-layer-descend {nested_hosts} {
+# We save need_xen_hosts on a stack in nested_layers_hosts
+# It gets popped again during the cleanup part of run-job
+global nested_layers_hosts need_xen_hosts
+lappend nested_layers_hosts $need_xen_hosts
+set need_xen_hosts $nested_hosts
+per-host-prep
+}
+
 #-- test recipes --
 
 proc need-hosts/test-debian-nomigr {} { return host }
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 03/24] selecthost: Minor cleanups

[Ian Jackson]

Document the syntax for $ident.

Log the ident as well as the selected hostname.

Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v17: Fix typo in doc message.
v14: New patch
---
 Osstest/TestSupport.pm |   16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index aa41952..8c3612c 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -810,6 +810,20 @@ sub power_state ($$) {
 sub selecthost ($) {
 my ($ident) = @_;
 # must be run outside transaction
+
+# $ident is 
+#
+#  can be , typically "host" or "xxx_host"
+# which means look up the runvar $r{$ident} which
+# contains 
+# OR
+#  can be =
+# which means ignore  except for logging purposes etc.
+# and use 
+#
+#  is  which means use that host (and all
+# its flags and properties from the configuration and database)
+
 my $name;
 if ($ident =~ m/=/) {
 $ident= $`;
@@ -917,7 +931,7 @@ sub selecthost ($) {
 
 $mjobdb->host_check_allocated($ho);
 
-logm("host: selected $ho->{Name} ".
+logm("host $ho->{Ident}: selected $ho->{Name} ".
 (defined $ho->{Ether} ? $ho->{Ether} : '').
 " $ho->{Ip}".
  (!$ho->{Shared} ? '' :
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 22/24] Serial: Factor out Osstest::Serial::keys_real

[Ian Jackson]

The sympathy and xenuse serial modules had too much in common.  Factor
out the common code, which is now responsible for
  - knowledge of the Xen console switch
  - splitting strings up into individual keys
  - timing decisions
  - error trapping and logging

This new class is an abstract base class for the concrete serial
method classes, and calls back to its derived class to prepare, send
each actual key, and shut down.

There is some functional change: notably, after failure to send the
first debug key, sending the remainder will not be attempted.

While we're here, fix a typo `dettach' to `detach'.

Signed-off-by: Ian Jackson 
Acked-by: Ian Campbell 
---
v17: Move `force attach' and `force detach' writes into xenuse.pm
  where they belong.
 Add typo fix.
---
 Osstest/Serial/keys_real.pm |   65 +++
 Osstest/Serial/sympathy.pm  |   55 
 Osstest/Serial/xenuse.pm|   57 ++---
 3 files changed, 104 insertions(+), 73 deletions(-)
 create mode 100644 Osstest/Serial/keys_real.pm

diff --git a/Osstest/Serial/keys_real.pm b/Osstest/Serial/keys_real.pm
new file mode 100644
index 000..80a159d
--- /dev/null
+++ b/Osstest/Serial/keys_real.pm
@@ -0,0 +1,65 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+
+package Osstest::Serial::keys_real;
+
+# Base class providing debug keys for real serial ports.
+# Derived class is expected to provide:
+#   $mo->keys_prepare();
+#   $mo->keys_prepare($what,$str,$pause);
+#   $mo->keys_shutdown();
+
+
+use strict;
+use warnings;
+
+use Osstest::TestSupport;
+
+sub request_debug {
+my ($mo,$conswitch,$xenkeys,$guestkeys) = @_;
+
+if (!eval {
+   local ($SIG{'PIPE'}) = 'IGNORE';
+
+   $mo->keys_prepare();
+
+   my $debugkeys= sub {
+   my ($what, $keys) = @_;
+   foreach my $k (split //, $keys) {
+   $mo->keys_write("$what debug info request, debug key $k",
+   $k, 2);
+   }
+   };
+
+   $mo->keys_write('request for input to Xen', $conswitch, 1);
+   $debugkeys->('Xen', $xenkeys);
+   sleep(10);
+   $debugkeys->('guest', $guestkeys);
+   sleep(10);
+   $mo->keys_write("RET to dom0","$conswitch\r", 5);
+
+   $mo->keys_shutdown();
+
+   1;
+}) {
+   warn "failed to send debug key(s): $@\n";
+   return 0;
+}
+return 1;
+}
+
+1;
diff --git a/Osstest/Serial/sympathy.pm b/Osstest/Serial/sympathy.pm
index d6bf425..fa34143 100644
--- a/Osstest/Serial/sympathy.pm
+++ b/Osstest/Serial/sympathy.pm
@@ -22,12 +22,13 @@ use warnings;
 
 use Osstest;
 use Osstest::TestSupport;
+use Osstest::Serial::keys_real;
 
 BEGIN {
 use Exporter ();
 our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
 $VERSION = 1.00;
-@ISA = qw(Exporter);
+@ISA = qw(Exporter Osstest::Serial::keys_real);
 @EXPORT  = qw();
 %EXPORT_TAGS = ( );
 
@@ -57,48 +58,28 @@ sub new {
 return bless $mo, $class;
 }
 
-sub request_debug {
-my ($mo,$conswitch,$xenkeys,$guestkeys) = @_;
+sub keys_prepare {
+}
 
-my $targhost= $mo->{Server};
+sub keys_write {
+my ($mo, $what,$str,$pause) = @_;
 
+my $targhost= $mo->{Server};
 my ($sshopts) = sshopts();
-my $sympwrite= sub {
-my ($what,$str,$pause) = @_;
-logm("sympathy sending $what");
-if (!eval {
-local ($SIG{'PIPE'}) = 'IGNORE';
-my $sock= $mo->{Socket};
-my $rcmd= "sympathy -c -k $sock -N >/dev/null";
-$rcmd= "alarm 5 $rcmd";
-open SYMPWRITE, "|ssh @$sshopts root\@$targhost '$rcmd'" or die $!;
-autoflush SYMPWRITE 1;
-print SYMPWRITE $str or die $!;
-sleep($pause);
-close SYMPWRITE or die "$? $!";
-1;
-}) {
-warn "failed to send $what: $@\n";
-return 0;
-}
-return 1;
-};
 
-my $debugkeys= sub {
-   my ($what, $keys) = @_;
-   foreach my $k (split //, $keys) {
-   $sympwrite->("$what debug info request, debug key $k", $k, 2);
-   }
-};
+logm("sympathy sending $what");
 
-$sympwrite->('request for i

[Xen-devel] [OSSTEST PATCH 01/24] cs-adjust-flight: Add some missing doc comment info

[Ian Jackson]

Signed-off-by: Ian Jackson 
Acked-by: Ian Campbell 
---
v14: New patch
---
 cs-adjust-flight |8 
 1 file changed, 8 insertions(+)

diff --git a/cs-adjust-flight b/cs-adjust-flight
index 4bfef48..d70abf7 100755
--- a/cs-adjust-flight
+++ b/cs-adjust-flight
@@ -24,6 +24,14 @@
 #   . means all jobs
 #   ^   means $foo =~ m/^/
 #   /   means $foo =~ m//
+#
+# :
+#   
+#   new:
+#
+# options:
+#   -v  verbose - list changes to stderr
+#   -D  debugging output to stderr
 
 # This is part of "osstest", an automated testing framework for Xen.
 # Copyright (C) 2009-2013 Citrix Inc.
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 02/24] cs-adjust-flight: Allow adjusting "this" flight

[Ian Jackson]

This allows cs-adjust-flight to be run by hand to adjust runvars, in a
flight being used with hand-invocation of ./ts-* scripts.

Signed-off-by: Ian Jackson 
Tested-by: Ian Jackson 
---
v17: Better documentation of running: flight status
v14: New patch
---
 cs-adjust-flight |9 +
 1 file changed, 9 insertions(+)

diff --git a/cs-adjust-flight b/cs-adjust-flight
index d70abf7..a72cd88 100755
--- a/cs-adjust-flight
+++ b/cs-adjust-flight
@@ -28,6 +28,8 @@
 # :
 #   
 #   new:
+#   running: uses OSSTEST_FLIGHT (flight referred to
+# must be blessed `running' or `constructing')
 #
 # options:
 #   -v  verbose - list changes to stderr
@@ -395,6 +397,13 @@ sub main () {
 verbose_discard();
 changes();
 });
+} elsif ($dstflightspec =~ m/^running:$/) {
+$dstflight = $ENV{OSSTEST_FLIGHT} // die;
+db_retry($dstflight,[qw(constructing running)],
+ $dbh_tests, [qw(flights)], sub {
+verbose_discard();
+changes();
+});
 } elsif ($dstflightspec =~ m/^new:/) {
 my $intended = $'; #';
 db_retry($dbh_tests, [qw(flights)], sub {
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 17/24] ts-xen-install: Properly handle hosts without a static IP address

[Ian Jackson]

From: Robert Ho 

Check IpStatic, and if it is not set, provide a dhcp stanza in
/etc/network/interfaces, rather than an `inet static' one.

This is necessary for L1 nested hosts, because they don't have a
static IP address.

In principle this makes matters more correct for physical hosts
without static IP addresses, but these are currently not supported
by selecthost().

Signed-off-by: Robert Ho 
Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v14: Only use `inet dhcp' if !$ho->{IpStatic}.
---
 ts-xen-install |5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/ts-xen-install b/ts-xen-install
index b511e2b..d9aa694 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -320,11 +320,14 @@ END
 /auto $iface\n/x;
 if (m/^\s* iface \s+ (?: $physif | xenbr0 ) \s+ inet \s /x) {
 $suppress= 1;
-print EO <{IpStatic} ? <{Ip}
 netmask $netmask
 gateway $gateway
+END
+iface $iface inet dhcp
+END
 $bridgex
 END
 }
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 10/24] LVM: Break out lv_create

[Ian Jackson]

We are going to want to reuse this.

lv_create doesn't (want to) take a $gho, but the $vg and $lv names
directly (so that callers can use it when they don't have a suitable
$gho whose $gho->{Lvdev} they want to use).

In the one existing call site we pass $gho->{Vg} and $gho->{Lv} so
that the effect is the same.

There is a minor functional change: $gho->{Lvdev} has been put through
lv_dev_mapper.  But we don't care about that in lv_create (since the
LVM operations, and dd, are perfectly happy to use the `real',
non-/dev/mapper, names).  So we can just use /dev/$vg/$lv.

Signed-off-by: Ian Jackson 
Signed-off-by: Robert Ho 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v17: Discuss change to /dev/$vg/$lv from $gho->{Lvdev}.
v14: New patch
v15: Change some trivial typo, so to resolve conflicts with
 production tree.
---
 Osstest/TestSupport.pm |   15 +++
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index b70f19f..b507ada 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -63,7 +63,7 @@ BEGIN {
   target_install_packages target_install_packages_norec
   target_jobdir target_extract_jobdistpath_subdir
   target_extract_jobdistpath
-  lv_dev_mapper
+  lv_create lv_dev_mapper
 
   poll_loop tcpconnect await_tcp
   contents_make_cpio file_simple_write_contents
@@ -713,6 +713,15 @@ sub poll_loop ($$$&) {
 logm("$what: ok. (${waited}s)");
 }
 
+sub lv_create () {
+my ($ho, $vg, $lv, $mb) = @_;
+my $lvdev = "/dev/$vg/$lv";
+target_cmd_root($ho, "lvremove -f $lvdev ||:");
+target_cmd_root($ho, "lvcreate -L ${mb}M -n $lv $vg");
+target_cmd_root($ho, "dd if=/dev/zero of=$lvdev count=10");
+return $lvdev;
+}
+
 sub lv_dev_mapper ($$) {
 my ($vg,$lv) = @_;
 $vg =~ s/-/--/g;
@@ -1692,9 +1701,7 @@ sub prepareguest ($$) {
 
 sub prepareguest_part_lvmdisk ($$$) {
 my ($ho, $gho, $disk_mb) = @_;
-target_cmd_root($ho, "lvremove -f $gho->{Lvdev} ||:");
-target_cmd_root($ho, "lvcreate -L ${disk_mb}M -n $gho->{Lv} $gho->{Vg}");
-target_cmd_root($ho, "dd if=/dev/zero of=$gho->{Lvdev} count=10");
+lv_create($ho, $gho->{Vg}, $gho->{Lv}, $disk_mb);
 }
 
 sub make_vhd ($$$) {
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 21/24] HVM guests: Use qemu "pipe:" for serial output logging

[Ian Jackson]

Modern qemu has the "pipe:/PATH" character driver.  This opens
/PATH.in for reading and /PATH.out for writing.  In my tests, I found
that:
  - contrary to the documentation, they do not need to be pipes
(at least, /PATH.out can be a file)
  - but they must both already exist
  - qemu will follow symlinks, so /PATH.out can be a symlink to
a file
  - if /PATH.in is a fifo, qemu will tolerate other processes opening
it for writing, and writing things, only occasionally.  (Probably,
qemu opens it O_RDWR; or perhaps it reopens it after EOF.)

Use this feature to achieve the following:
  - guest serial output ends up in /var/log/xen/osstest-serial-GUEST.log
(which is already captured by ts-logs-capture) rather than
interleaved with qemu's stderr output (in the libxl-created logfile)
  - guest serial input comes from a pipe in /root which we can open
and write to if we want to talk to the guest

We are mostly interested in the final bullet point, because that will
allow us to send debug keys to the emulated serial port of an L1
nested HVM guest.

Looking at the source code of qemu in 4.2 and 4.6 I think the above
approach will work with all relevant qemu-xen's.

If the device model version is qemu-xen-traditional, `pipe:' is not
supported.  If device_model_version is not set, we will be using
whatever the xen.git we used defaults to.  For Xen 4.1 and earlier
that is qemu-xen-traditional, and I'm slightly loathe to break osstest
for those earlier versions.  There doesn't seem to be anything else in
the runvars that would clue us in.  So be cautious and do not use the
new feature unless device_model_version is explicitly set.

The nested tests are all -qemuu so set device_model_version.

Signed-off-by: Ian Jackson 
Acked-by: Ian Campbell 
---
v17: Fix fifo mode to be 600, not 700
v16: New patch
---
 Osstest/TestSupport.pm |   12 +++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 32f2956..47fc1b1 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1884,8 +1884,18 @@ END
# fd pointing to mini-os's console. IOW any such path used
# here ends up in the host logs in /var/log/xen/qemu-dm-$guest.log
$cfg .= "serial='file:/var/log/dm-serial.log'\n";
-} else {
+} elsif (!(defined $devmodel) || $devmodel =~ /traditional/) {
$cfg .= "serial='file:/dev/stderr'\n";
+} else {
+   my $logpath = "/var/log/xen/osstest-serial-$gho->{Name}.log";
+   my $basepath = "/root/$flight.$job.$gho->{Name}.serial";
+   target_cmd_root($ho, 

[Xen-devel] [OSSTEST PATCH 12/24] sg-run-job: Break out per-host-prep and per-host-finish

[Ian Jackson]

No functional change.

We now call the per-host-ts finish steps unconditionally, rather than
only if !$need_build_host, per-host-ts is (complicated) no-op if
$need_build_host, since in that case $need_xen_hosts is {}.

Signed-off-by: Ian Jackson 
Tested by: Robert Ho 
Acked-by: Ian Campbell 
---
v14: Squash typo fix from Robert into this patch
---
 sg-run-job |   31 +++
 1 file changed, 19 insertions(+), 12 deletions(-)

diff --git a/sg-run-job b/sg-run-job
index 66145b8..884a21d 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -23,6 +23,20 @@ source ./tcl/osstestlib.tcl
 readconfig
 source-method JobDB
 
+proc per-host-prep {} {
+per-host-ts .   host-ping-check-native/@ ts-host-ping-check
+per-host-ts .   xen-install/@ ts-xen-install
+per-host-ts .   xen-boot/@ts-host-reboot
+
+per-host-ts .   host-ping-check-xen/@ ts-host-ping-check
+per-host-ts .   =(*) {ts-leak-check basis}
+}
+
+proc per-host-finish {} {
+per-host-ts .   ={ts-leak-check check}
+per-host-ts !broken capture-logs/@(*) ts-logs-capture
+}
+
 proc run-job {job} {
 global jobinfo builds flight ok need_xen_hosts anyfailed
 
@@ -51,22 +65,15 @@ proc run-job {job} {
 if {$ok} { setstatus running  }
 
 per-host-ts broken  host-install/@(*) ts-host-install-twice
-per-host-ts .   host-ping-check-native/@ ts-host-ping-check
-per-host-ts .   xen-install/@ ts-xen-install
-per-host-ts .   xen-boot/@ts-host-reboot
-per-host-ts .   host-ping-check-xen/@ ts-host-ping-check
 
-per-host-ts .   =(*) {ts-leak-check basis}
+per-host-prep
 
 if {$ok} { catching-otherwise fail  run-job/$jobinfo(recipe)  }
-per-host-ts .   ={ts-leak-check check}
 
-if {!$need_build_host} {
-per-host-ts !broken capture-logs/@(*) ts-logs-capture
-} else {
-if {$anyfailed} {
-run-ts  !broken capture-logs  ts-logs-capture + host
-}
+per-host-finish
+
+if {$need_build_host && $anyfailed} {
+   run-ts  !broken capture-logs  ts-logs-capture + host
 }
 
 if {$ok} { setstatus pass }
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 08/24] target_check_ip: Rename and improve from guest_check_ip

[Ian Jackson]

Make this function suitable for running on targets with static IP
addresses.  (Ie, on physical hosts.)  Accordingly, rename it and
adjust all call sites.

Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v14: New patch
---
 Osstest/TestSupport.pm |   11 ++-
 ts-guest-localmigrate  |2 +-
 ts-guest-migrate   |2 +-
 ts-guest-saverestore   |2 +-
 4 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 9a256a9..325e2e3 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -95,7 +95,7 @@ BEGIN {
   prepareguest_part_lvmdisk prepareguest_part_diskimg
   prepareguest_part_xencfg
   guest_umount_lv guest_await guest_await_dhcp_tcp
-  guest_checkrunning guest_check_ip guest_find_ether
+  guest_checkrunning target_check_ip guest_find_ether
   guest_find_domid guest_check_up guest_check_up_quick
   guest_get_state guest_await_reboot
   guest_await_shutdown guest_await_destroy guest_destroy
@@ -730,8 +730,9 @@ sub dhcp_watch_setup ($$) {
 $gho->{DhcpWatch} = get_host_method_object($ho, 'DhcpWatch', $meth);
 }
 
-sub guest_check_ip ($) {
-my ($gho) = @_;
+sub target_check_ip ($) {
+my ($gho) = @_; # returns error message or undef
+return undef if $gho->{IpStatic};
 guest_find_ether($gho);
 $gho->{DhcpWatch}->check_ip($gho);
 }
@@ -871,7 +872,7 @@ sub selecthost ($) {
$msg .= " guest $child->{Guest} (@{ $child->{Info} })";
$msg .= " $child->{Ether}";
 
-   my $err = guest_check_ip($child);
+   my $err = target_check_ip($child);
$msg .= " ".(defined $err ? " $err" : $child->{Ip});
 
logm($msg);
@@ -1956,7 +1957,7 @@ sub guest_await_dhcp_tcp ($$) {
  " $gho->{TcpCheckPort}".
   " link/ip/tcp",
   sub {
-my $err= guest_check_ip($gho);
+my $err= target_check_ip($gho);
 return $err if defined $err;
 
 return
diff --git a/ts-guest-localmigrate b/ts-guest-localmigrate
index 8fe986d..85a0887 100755
--- a/ts-guest-localmigrate
+++ b/ts-guest-localmigrate
@@ -40,7 +40,7 @@ sub migrate () {
 guest_await_dhcp_tcp($gho, 5);
 guest_check_up($gho);
 
-my $err= guest_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
+my $err= target_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
 
 for (my $rep=1; $rep<=$reps; $rep++) {
 logm("== rep $rep ==");
diff --git a/ts-guest-migrate b/ts-guest-migrate
index b77d0de..505fab2 100755
--- a/ts-guest-migrate
+++ b/ts-guest-migrate
@@ -30,7 +30,7 @@ our $gho = selectguest($ARGV[2],$sho);
 
 sub migrate () {
 guest_checkrunning($sho,$gho) or die $gho->{Name};
-my $err= guest_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
+my $err= target_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
 toolstack($sho)->migrate($gho, $dho, $timeout{Migrate});
 }
 
diff --git a/ts-guest-saverestore b/ts-guest-saverestore
index 7e13d90..73883ef 100755
--- a/ts-guest-saverestore
+++ b/ts-guest-saverestore
@@ -26,7 +26,7 @@ our ($ho,$gho) = ts_get_host_guest(@ARGV);
 
 sub save () {
 guest_checkrunning($ho,$gho) or die $gho->{Name};
-my $err= guest_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
+my $err= target_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
 toolstack($ho)->save($gho,"image",200);
 target_ping_check_down($gho);
 }
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 04/24] make-flight: Break out usual_debianhvm_image and honour DEBIAN_IMAGE_VERSION

[Ian Jackson]

No functional change.  (Verified for xen-unstable with
standalone-generate-dump-flight-runvars.)

Signed-off-by: Ian Jackson 
Tested-by: Ian Jackson 
---
v17: New patch.
---
 make-flight |2 +-
 mfi-common  |5 +
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/make-flight b/make-flight
index 28820f5..f908017 100755
--- a/make-flight
+++ b/make-flight
@@ -285,7 +285,7 @@ do_hvm_debian_test_one () {
 test-debianhvm $toolstack $xenarch $dom0arch $qemuu_runvar \
 enable_xsm=$xsm \
 $stubdom_runvar $testvars   \
-debianhvm_image=debian-7.2.0-$arch-CD-1.iso \
+debianhvm_image=$(usual_debianhvm_image $arch) \
 debianhvm_iso_kernel=/$iso_dir/vmlinuz \
 debianhvm_iso_ramdisk=/$iso_dir/initrd.gz \
 bios=$bios \
diff --git a/mfi-common b/mfi-common
index fab4aa3..5fbe195 100644
--- a/mfi-common
+++ b/mfi-common
@@ -351,6 +351,11 @@ job_create_test () {
 xenbuildjob=$xenbuildjob buildjob=$buildjob $tsbuildjob "$@"
 }
 
+usual_debianhvm_image () {
+  local arch=$1; shift
+  echo debian-${DEBIAN_IMAGE_VERSION-7.2.0}-$arch-CD-1.iso
+}
+
 # Iterate over xenarch, dom0arch and kernel calling test_matrix_do_one
 # for each combination.
 #
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 07/24] DhcpWatch::leases: Fix a reporting message

[Ian Jackson]

This talks about `guest_check_ip', but this code is now factored out
into a method.  Use the correct method name in reporting.

Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v14: New patch
---
 Osstest/DhcpWatch/leases.pm |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/DhcpWatch/leases.pm b/Osstest/DhcpWatch/leases.pm
index 9a74c40..b74ebf0 100644
--- a/Osstest/DhcpWatch/leases.pm
+++ b/Osstest/DhcpWatch/leases.pm
@@ -171,7 +171,7 @@ sub check_ip ($$) {
 }
 $gho->{Ip}= $best->{' addr'};
 
-report_once($gho, 'guest_check_ip', 
+report_once($gho, 'leases::check_ip', 
"guest $gho->{Name}: $gho->{Ether} $gho->{Ip}");
 return undef;
 }
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 24/24] Serial::xenuse: Send xenuse output to /dev/null

[Ian Jackson]

Like sympathy, attaching via xenuse causes xenuse to send output from
the host to its own stdout.

But we don't want the ts-logs-capture stdout to contain this serial
output, interleaved with its own log messages.  We'll capture the
whole serial log from the xenuse logfile.  So redirect it to /dev/null.

I have checked that xenuse does (at least sometimes, eg when given a
nonexistent hostname) use stderr when something actually goes wrong.

Signed-off-by: Ian Jackson 
Tested-by: Ian Jackson 
---
v17: New patch
---
 Osstest/Serial/xenuse.pm |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Serial/xenuse.pm b/Osstest/Serial/xenuse.pm
index e1270e0..c79d986 100644
--- a/Osstest/Serial/xenuse.pm
+++ b/Osstest/Serial/xenuse.pm
@@ -54,7 +54,7 @@ sub keys_prepare {
 
 my $xenuse= $c{XenUsePath} || "xenuse";
 
-open XENUSEWRITE, "|$xenuse -t $ho->{Name}" or die $!;
+open XENUSEWRITE, "|$xenuse -t $ho->{Name} >/dev/null" or die $!;
 autoflush XENUSEWRITE 1;
 
 $mo->keys_write('force attach', "\x05cf", 1); # ^E c f == force attach
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 18/24] ts-xen-install: networking: Rename `nodhcp' to `ensurebridge'

[Ian Jackson]

This function does not (now) always undo the DHCP configuration.
Sometimes it leaves it.  Its main function is to ensure that we have
a bridge for use by guests.

So rename the function.

Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v14: This patch was previously 4/4 of a miniature series containing
  a different way of dealing with the Nested HVM L1 DHCP problem.
---
 ts-xen-install |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ts-xen-install b/ts-xen-install
index d9aa694..3d0f394 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -247,7 +247,7 @@ sub hosts () {
 });
 }
 
-sub nodhcp () {
+sub ensurebridge () {
 target_editfile_root($ho, "/etc/network/interfaces",
  "etc-network-interfaces", sub {
 my $physif= get_host_property($ho,'interface force',undef);
@@ -370,6 +370,6 @@ if ($checkmode) {
 adjustconfig();
 setupboot();
 setupinitd();
-nodhcp();
+ensurebridge();
 hosts();
 }
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 11/24] Toolstack::xl: Provide block_attach method

[Ian Jackson]

It is possible that this may work some of the time with xm, so I have
taken no measures to prevent it running then.

Signed-off-by: Ian Jackson 
Signed-off-by: Robert Hu 
Tested-by: Robert Hu 
Acked-by: Ian Campbell 

v14: New patch
v15: Fix missing $ho assignment in block-attach method
---
 Osstest/Toolstack/xl.pm |9 +
 1 file changed, 9 insertions(+)

diff --git a/Osstest/Toolstack/xl.pm b/Osstest/Toolstack/xl.pm
index 0f8abed..d956bcd 100644
--- a/Osstest/Toolstack/xl.pm
+++ b/Osstest/Toolstack/xl.pm
@@ -109,4 +109,13 @@ sub restore () {
." $f", $timeout);
 }
 
+sub block_attach () {
+my ($self,$gho,$xldiskspec) = @_;
+die "quotes in $xldiskspec ?" if $xldiskspec =~ m/'/;
+my $ho = $self->{Host};
+my $gn = $gho->{Name};
+my $cmd = $self->{_VerboseCommand}." block-attach $gn '$xldiskspec'";
+target_cmd_root($ho, $cmd, 100);
+}
+
 1;
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH 05/24] selecthost: Support nested hosts (guests which are also hosts)

[Ian Jackson]

We introduce a new syntax: instead of a hostname (which might appear
in a command line argument to a ts-* script and hence be passed to
selecthost, or which might be in a runvar), we now support
:.

Such `hosts' (let us refer to such a thing as an L1, although in
principle further nesting may be possible) are expected to be
dynamically created.  So they do not have flags and properties in the
configuration (or in an Executive instance's database).

The IP address is determined dynamically from the leases file.  If the
L1 is not running, then no IP address may be found.  This is not an
error.  Users of this facility will need to make sure that ts-*
scripts which are unaware of the L1's special status are only invoked
when it is known that the L1 is up and has obtained its IP address.

`Power cycling' the L1 will be done by VM control operations in the
L0; this will come in a subsequent patch.

`Serial access' to the L1 guest will likewise need to be done via the
console arrangements in L0.

Signed-off-by: Ian Jackson 
Tested-by: Robert Ho 
Acked-by: Ian Campbell 
---
v14: New patch
---
 Osstest/TestSupport.pm |   37 -
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 8c3612c..3145040 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -807,6 +807,7 @@ sub power_state ($$) {
 
 #-- host selection and properties --
 
+sub selecthost ($);
 sub selecthost ($) {
 my ($ident) = @_;
 # must be run outside transaction
@@ -821,8 +822,12 @@ sub selecthost ($) {
 # which means ignore  except for logging purposes etc.
 # and use 
 #
-#  is  which means use that host (and all
+#  can be  which means use that host (and all
 # its flags and properties from the configuration and database)
+# OR
+#  can be : meaning use the
+# Xen domain name  on the host specified by
+# , which is an  as above.
 
 my $name;
 if ($ident =~ m/=/) {
@@ -838,6 +843,7 @@ sub selecthost ($) {
 Ident => $ident,
 Name => $name,
 TcpCheckPort => 22,
+NestingLevel => 0,
 Info => [],
 };
 if (defined $job) {
@@ -845,6 +851,35 @@ sub selecthost ($) {
  $c{DebianSuite});
 }
 
+#- handle hosts which are themselves guests (nested) -
+
+if ($name =~ s/^(.*)://) {
+   my $parentname = $1;
+   my $parent = selecthost($parentname);
+   my $child = selectguest($name,$parent);
+   $child->{Ident} = $ho->{Ident};
+   $child->{Info} = [ "in", $parent->{Name}, @{ $parent->{Info} } ];
+   $child->{NestingLevel} = $parent->{NestingLevel}+1;
+
+   # $child->{Power} = 'guest';   todo
+   power_cycle_host_setup($child);
+
+   $child->{Properties}{Serial} = 'noop'; # todo
+   serial_host_setup($child);
+
+   my $msg = "L$child->{NestingLevel} host $child->{Ident}:";
+   $msg .= " guest $child->{Guest} (@{ $child->{Info} })";
+   $msg .= " $child->{Ether}";
+
+   my $err = guest_check_ip($child);
+   $msg .= " ".(defined $err ? " $err" : $child->{Ip});
+
+   logm($msg);
+
+   # all the rest of selecthost is wrong for this case
+   return $child;
+}
+
 #- calculation of the host's properties -
 
 $ho->{Properties} = { };
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 2/6] xen/arm: vgic-v2: Handle correctly byte write in ITARGETSR

[Stefano Stabellini]

On Mon, 9 Nov 2015, Julien Grall wrote:
> During a store, the byte is always in the low part of the register (i.e
> [0:7]).
> 
> We are incorrectly masking the register by using a shift of the byte
> offset in the ITARGETSR while the byte is alwasy in r[0:7]. This will
> result in a target list equal to 0 which is ignored by the emulation.
> 
> Because of that the guest will only be able to modify the first byte in
> each ITARGETSR.
> 
> Furthermore, the body of the loop is retrieving the old target list
> using the index of the byte.
> 
> To avoid modifying too much the loop, shift the byte stored to the correct
> offset.
> 
> Signed-off-by: Julien Grall 

Acked-by: Stefano Stabellini 



> This change used to be embedded in "xen/arm: vgic: Optimize the way
> to store the target vCPU in the rank". It has been moved out to
> avoid having too much functional changes in a single patch.
> 
> This patch is a good candidate to backport to Xen 4.6 and Xen 4.5.
> Without it a guest won't be able migrate an IRQ from one vCPU to
> another if it's using byte access to write in ITARGETSR.
> 
> Note that if we backport this patch alone, the resulting code in
> earlier version of Xen will be complex to read. As the last patch
> of this serie should also be backported, I'm planning to request
> backport for the whole series.
> 
> Changes in v5:
> - Update commit message based on Ian's suggestion
> 
> Changes in v4:
> - Patch added
> ---
>  xen/arch/arm/vgic-v2.c | 12 ++--
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index 041291c..486e497 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -353,11 +353,11 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v, 
> mmio_info_t *info,
>  /* 8-bit vcpu mask for this domain */
>  BUG_ON(v->domain->max_vcpus > 8);
>  target = (1 << v->domain->max_vcpus) - 1;
> -if ( dabt.size == 2 )
> -target = target | (target << 8) | (target << 16) | (target << 
> 24);
> +target = target | (target << 8) | (target << 16) | (target << 24);
> +if ( dabt.size == DABT_WORD )
> +target &= r;
>  else
> -target = (target << (8 * (gicd_reg & 0x3)));
> -target &= r;
> +target &= (r << (8 * (gicd_reg & 0x3)));
>  /* ignore zero writes */
>  if ( !target )
>  goto write_ignore;
> @@ -381,7 +381,7 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v, 
> mmio_info_t *info,
>  
>  if ( new_target != old_target )
>  {
> -irq = gicd_reg - GICD_ITARGETSR + (i / 8);
> +irq = (gicd_reg & ~0x3) - GICD_ITARGETSR + (i / 8);
>  v_target = v->domain->vcpu[new_target];
>  v_old = v->domain->vcpu[old_target];
>  vgic_migrate_irq(v_old, v_target, irq);
> @@ -393,7 +393,7 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v, 
> mmio_info_t *info,
>   DABT_WORD)] = target;
>  else
>  vgic_byte_write(&rank->v2.itargets[REG_RANK_INDEX(8,
> -  gicd_reg - GICD_ITARGETSR, DABT_WORD)], target, 
> gicd_reg);
> +  gicd_reg - GICD_ITARGETSR, DABT_WORD)], r, gicd_reg);
>  vgic_unlock_rank(v, rank, flags);
>  return 1;
>  }
> -- 
> 2.1.4
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [OSSTEST PATCH 02/24] cs-adjust-flight: Allow adjusting "this" flight

[Ian Campbell]

On Fri, 2015-11-13 at 12:03 +, Ian Jackson wrote:
> This allows cs-adjust-flight to be run by hand to adjust runvars, in a
> flight being used with hand-invocation of ./ts-* scripts.
> 
> Signed-off-by: Ian Jackson 
> Tested-by: Ian Jackson 

Acked-by: Ian Campbell 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [OSSTEST PATCH 04/24] make-flight: Break out usual_debianhvm_image and honour DEBIAN_IMAGE_VERSION

[Ian Campbell]

On Fri, 2015-11-13 at 12:03 +, Ian Jackson wrote:
> No functional change.  (Verified for xen-unstable with
> standalone-generate-dump-flight-runvars.)
> 
> Signed-off-by: Ian Jackson 
> Tested-by: Ian Jackson 

Acked-by: Ian Campbell 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [OSSTEST PATCH 23/24] Serial: Add new serial method object for `guest' type

[Ian Campbell]

On Fri, 2015-11-13 at 12:03 +, Ian Jackson wrote:
> From: Robert Ho 
> 
> L1 guests' serial ports are owned by qemu in L0.  We can send them
> debug keys by writing to the qemu pipe.
> 
> (xl debug-key looks like it would be useful but it actually sends
> debug keys to the hypervisor of the host it is running on.  We want to
> send the debug keys to the hypervisor and kernel from the outside.)
> 
> Log fetching is not needed because from the POV of the L0 the L1 is a
> guest, so the L0's log capture will already fetch the L1's serial
> console output.
> 
> Signed-off-by: Robert Ho 
> Signed-off-by: Ian Jackson 

Acked-by: Ian Campbell 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [OSSTEST PATCH 24/24] Serial::xenuse: Send xenuse output to /dev/null

[Ian Campbell]

On Fri, 2015-11-13 at 12:03 +, Ian Jackson wrote:
> Like sympathy, attaching via xenuse causes xenuse to send output from
> the host to its own stdout.
> 
> But we don't want the ts-logs-capture stdout to contain this serial
> output, interleaved with its own log messages.  We'll capture the
> whole serial log from the xenuse logfile.  So redirect it to /dev/null.
> 
> I have checked that xenuse does (at least sometimes, eg when given a
> nonexistent hostname) use stderr when something actually goes wrong.
> 
> Signed-off-by: Ian Jackson 
> Tested-by: Ian Jackson 

Acked-by: Ian Campbell 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH QEMU v2 2/2] xen: fix usage of xc_domain_create in domain builder

[Roger Pau Monne]

Due to the addition of HVMlite and the requirement to always provide a valid
xc_domain_configuration_t, xc_domain_create now always takes an arch domain
config, which can be NULL in order to mimic previous behaviour.

Add a small stub called xen_domain_create that encapsulates the correct call
to xc_domain_create depending on the libxc version detected.

Signed-off-by: Roger Pau Monné 
---
Cc: Stefano Stabellini 
Cc: qemu-de...@nongnu.org
---
Changes since v1:
 - Add a compat layer to support previous libxc versions.
 - Add machinery to detect current Xen unstable (4.7).
---
 configure   | 17 +
 hw/xenpv/xen_domainbuild.c  |  2 +-
 include/hw/xen/xen_common.h | 12 
 3 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/configure b/configure
index 779623a..001982f 100755
--- a/configure
+++ b/configure
@@ -1863,6 +1863,23 @@ EOF
   elif
   cat > $TMPC <
+#include 
+int main(void) {
+  xc_interface *xc = NULL;
+  xen_domain_handle_t handle;
+  xc_domain_create(xc, 0, handle, 0, NULL, NULL);
+  return 0;
+}
+EOF
+  compile_prog "" "$xen_libs"
+then
+xen_ctrl_version=470
+xen=yes
+
+  # Xen 4.6
+  elif
+  cat > $TMPC <
 #include 
 #include 
 #include 
diff --git a/hw/xenpv/xen_domainbuild.c b/hw/xenpv/xen_domainbuild.c
index c0ab753..07814f6 100644
--- a/hw/xenpv/xen_domainbuild.c
+++ b/hw/xenpv/xen_domainbuild.c
@@ -234,7 +234,7 @@ int xen_domain_build_pv(const char *kernel, const char 
*ramdisk,
 int rc;
 
 memcpy(uuid, qemu_uuid, sizeof(uuid));
-rc = xc_domain_create(xen_xc, ssidref, uuid, flags, &xen_domid);
+rc = xen_domain_create(xen_xc, ssidref, uuid, flags, &xen_domid, NULL);
 if (rc < 0) {
 fprintf(stderr, "xen: xc_domain_create() failed\n");
 goto err;
diff --git a/include/hw/xen/xen_common.h b/include/hw/xen/xen_common.h
index 5923290..e5ba732 100644
--- a/include/hw/xen/xen_common.h
+++ b/include/hw/xen/xen_common.h
@@ -417,4 +417,16 @@ static inline int xen_set_ioreq_server_state(XenXC xc, 
domid_t dom,
 
 #endif
 
+static inline int xen_domain_create(XenXC xc, uint32_t ssidref,
+xen_domain_handle_t handle, uint32_t flags,
+uint32_t *pdomid,
+xc_domain_configuration_t *config)
+{
+#if CONFIG_XEN_CTRL_INTERFACE_VERSION < 470
+return xc_domain_create(xc, ssidref, handle, flags, pdomid);
+#else
+return xc_domain_create(xc, ssidref, handle, flags, pdomid, config);
+#endif
+}
+
 #endif /* QEMU_HW_XEN_COMMON_H */
-- 
1.9.5 (Apple Git-50.3)


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] xen: sched: fix (ACPI S3) resume with cpupools with different schedulers.

[Dario Faggioli]

On Fri, 2015-11-13 at 03:41 -0700, Jan Beulich wrote:
> > > > On 13.11.15 at 11:08,  wrote:
> > Fix things by properly deallocating scheduler specific
> > data of the pCPU's pool scheduler during pCPU teardown,
> > and re-allocating them --always for &ops-- during pCPU
> > bringup.
> 
> Avoiding this was done for a reason iirc: What if one such allocation
> fails (e.g. due to heap fragmentation resulting from the allocation
> order not being the exact inverse of the freeing order)?
> 
I don't think I'm getting the point.

cpu_schedule_up() allocates, with the alloc_pdata hook, stuff that was
freed by cpu_schedule_down (with the free_pdata hook) already.

It actually allocates two things, the other being the entire idle vCPU
of the pCPU, but that is only if idle_vcpu[cpu] is NULL, which should
not be happening during suspend/resume.

In any case, if one of these allocations fails, it already returns
ENOMEM. What happens then is, of course, caller's call. If it happens
during boot for pCPU 0, we BUG(). If it happens with non-boot pCPUs
(either at boot, during hotplug or during suspend/resume), we fail the
bringup of that pCPU at the CPU_UP_PREPARE phase.

This patch introduces one more occasion for the failure to occur, sure,
if there's no (proper) memory. Without this patch the system crashes
already, and even if there is enough (proper) memory, thuogh.

Furthermore, at a later point on the pCPU bringup path,
schedule_cpu_switch() is called. In the scenario described in the
changelog, what it will do is deallocating the existing scheduler
specific per-vCPU data, and re-allocating a new one. If the allocation
fails at that point, what happens is that we fail the bringup at the
CPU_ONLINE stage and (in cpu_up()) we BUG(). This does not really look
better than what happens with the patch applied to me... Quite the
opposite, TBH.

Actually, re-looking at schedule_cpu_switch(), the situation is
probably even worse than how I described it in the changelog! In fact,
still in the scenario introduced there, when we actually get to call
schedule_cpu_switch() (during CPU_ONLINE), the pCPU --let it be pCPU
6-- is in the following state:
 - it's still formally in cpupool1, but it's scheduler is set to
   &ops, i.e., to Credit1,
 - idle_vcpu[6].sched_priv points to Credit2 vCPU data,
 - it is being switched to pool1, with ops == Credit2.

So, schedule_cpu_switch() itself will:
 - allocate a new Credit2 per vCPU data structure,
 - make idle_vcpu[6].sched_priv point such new structure,
 - try to deallocate the still existing Credit2 per vCPU data 
   structure with Credit1's (old_ops in the function) free_vdata hook!!
 - bad things (or so I expect!)

Note, in particular, the the next to last item: we'd be calling
Credit1's csched_free_vdata() to deallocate something that was
allocated with Credit2's csched_alloc_vdata()... Which looks like
another rather interesting bug to me (I haven't been able to trigger
it, most likely because the one shown in the changelog manifests first,
but I can try, if we're curios :-D).

So, in summary, this patch fixes two bugs, one that is showing and one
latent... Not bad! :-)

Fact is, there is an inconsistency between the pCPU's scheduler (set in
cpu_schedule_up(), to Credit1, in the example) and the pCPU's idle
vCPU's private data (Credit2 data in the example), which, if left in
place, could explode somewhere else, at some future point. In my
opinion, the best solution is to remove such inconsistency, and that's
what the patch tries to do.

Another idea could be trying to completely and fully initialize the
_proper_ scheduler for the pCPU in cpu_schedule_up(), but it's not
guaranteed that this will be doable for all schedulers; e.g., it is
impossible for Credit2. Maybe, but I'd have to try, since it's only the
inconsistency in the status of the idle vCPU that triggers the bug, we
can at least set that straight in cpu_schedule_up()... As I said, I'd
have to check, and I'm fine with trying, if it is deemed worthwhile.

Let me know what you think.

Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH XEN 1/2] x86/libxc: add an arch domain config parameter to xc_domain_create

[Andrew Cooper]

On 13/11/15 11:05, Roger Pau Monne wrote:
> With the addition of HVMlite the hypervisor now always requires a non-null
> arch domain config, which is different between HVM and PV guests.
>
> Add a new parameter to xc_domain_create that contains a pointer to an arch
> domain config. If the pointer is null, create a default arch domain config
> based on guest type.
>
> Fix all the in-tree callers to provide a null arch domain config in order to
> mimic previous behaviour.
>
> Signed-off-by: Roger Pau Monné 

Reviewed-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] xen: sched: fix (ACPI S3) resume with cpupools with different schedulers.

[Jan Beulich]

>>> On 13.11.15 at 13:30,  wrote:
> On Fri, 2015-11-13 at 03:41 -0700, Jan Beulich wrote:
>> > > > On 13.11.15 at 11:08,  wrote:
>> > Fix things by properly deallocating scheduler specific
>> > data of the pCPU's pool scheduler during pCPU teardown,
>> > and re-allocating them --always for &ops-- during pCPU
>> > bringup.
>> 
>> Avoiding this was done for a reason iirc: What if one such allocation
>> fails (e.g. due to heap fragmentation resulting from the allocation
>> order not being the exact inverse of the freeing order)?
>> 
> I don't think I'm getting the point.
> 
> cpu_schedule_up() allocates, with the alloc_pdata hook, stuff that was
> freed by cpu_schedule_down (with the free_pdata hook) already.

If there is no change in the allocation pattern (i.e. including the
order of frees and allocs), then I'm sorry for the noise. I understood
the description of the change differently then.

> It actually allocates two things, the other being the entire idle vCPU
> of the pCPU, but that is only if idle_vcpu[cpu] is NULL, which should
> not be happening during suspend/resume.
> 
> In any case, if one of these allocations fails, it already returns
> ENOMEM. What happens then is, of course, caller's call. If it happens
> during boot for pCPU 0, we BUG(). If it happens with non-boot pCPUs
> (either at boot, during hotplug or during suspend/resume), we fail the
> bringup of that pCPU at the CPU_UP_PREPARE phase.

Except that after resume the system should get into the same state
it was before suspend, i.e. in particular same number of CPUs. Imo
there simply shouldn't be any allocations on the resume path -
everything should be possible to be picked up as it was left during
suspend. Along the lines of e.g. the handling of affinities we once
had (and maybe still have), getting saved away and restored instead
of trashed.

> So, in summary, this patch fixes two bugs, one that is showing and one
> latent... Not bad! :-)

Agreed.

> Fact is, there is an inconsistency between the pCPU's scheduler (set in
> cpu_schedule_up(), to Credit1, in the example) and the pCPU's idle
> vCPU's private data (Credit2 data in the example), which, if left in
> place, could explode somewhere else, at some future point. In my
> opinion, the best solution is to remove such inconsistency, and that's
> what the patch tries to do.

Removing the inconsistency is certainly a requirement. The question
just is how to do so without risking system health in other ways.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCHv3] 00/28] Kconfig conversion

[Jan Beulich]

>>> On 13.11.15 at 12:41,  wrote:
> 2) make menuconfig doesn't expose any options => No possibility to
> select any UART on ARM.
> 
> The latter is because how you define the option in the Kconfig:
> 
> config HAS_EXYNOS4210
>   bool
> 
> Without any text, it's not possible for the user to select this option.

This part at least is intended at this point in time (see past
discussions) - we don't want any user configurable items just yet.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 7/8] libxl: implement virDomainGetJobInfo

[Joao Martins]

Introduce support for domainGetJobInfo to get info about the
ongoing job. If the job is active it will update the
timeElapsed which is computed with the "started" field added to
struct libxlDomainJobObj.  For now we support just the very basic
info and all jobs have VIR_DOMAIN_JOB_UNBOUNDED (i.e. no completion
time estimation) plus timeElapsed computed.

Openstack Kilo uses the Job API to monitor live-migration
progress which is currently nonexistent in libxl driver and
therefore leads to a crash in the nova compute node. Right
now, migration doesn't use jobs in the source node and will
return VIR_DOMAIN_JOB_NONE. Though nova handles this case and
will migrate it properly instead of crashing.

Signed-off-by: Joao Martins 
---
Changes since v1:
 - s/inexistent/nonexistent/g in commit message
 - s/estimed/estimated/g
 - Bump version to 1.2.22
---
 src/libxl/libxl_domain.c | 28 
 src/libxl/libxl_domain.h |  6 ++
 src/libxl/libxl_driver.c | 38 ++
 3 files changed, 72 insertions(+)

diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c
index adb4563..baa9617 100644
--- a/src/libxl/libxl_domain.c
+++ b/src/libxl/libxl_domain.c
@@ -74,6 +74,10 @@ libxlDomainObjInitJob(libxlDomainObjPrivatePtr priv)
 if (virCondInit(&priv->job.cond) < 0)
 return -1;
 
+if (VIR_ALLOC(priv->job.current) < 0)
+return -1;
+
+memset(priv->job.current, 0, sizeof(*(priv->job.current)));
 return 0;
 }
 
@@ -90,6 +94,7 @@ static void
 libxlDomainObjFreeJob(libxlDomainObjPrivatePtr priv)
 {
 ignore_value(virCondDestroy(&priv->job.cond));
+VIR_FREE(priv->job.current);
 }
 
 /* Give up waiting for mutex after 30 seconds */
@@ -131,6 +136,8 @@ libxlDomainObjBeginJob(libxlDriverPrivatePtr driver 
ATTRIBUTE_UNUSED,
 VIR_DEBUG("Starting job: %s", libxlDomainJobTypeToString(job));
 priv->job.active = job;
 priv->job.owner = virThreadSelfID();
+priv->job.started = now;
+priv->job.current->type = VIR_DOMAIN_JOB_UNBOUNDED;
 
 return 0;
 
@@ -179,6 +186,27 @@ libxlDomainObjEndJob(libxlDriverPrivatePtr driver 
ATTRIBUTE_UNUSED,
 return virObjectUnref(obj);
 }
 
+int
+libxlDomainJobUpdateTime(struct libxlDomainJobObj *job)
+{
+virDomainJobInfoPtr jobInfo = job->current;
+unsigned long long now;
+
+if (!job->started)
+return 0;
+
+if (virTimeMillisNow(&now) < 0)
+return -1;
+
+if (now < job->started) {
+job->started = 0;
+return 0;
+}
+
+jobInfo->timeElapsed = now - job->started;
+return 0;
+}
+
 static void *
 libxlDomainObjPrivateAlloc(void)
 {
diff --git a/src/libxl/libxl_domain.h b/src/libxl/libxl_domain.h
index 44b3e0b..1c1eba3 100644
--- a/src/libxl/libxl_domain.h
+++ b/src/libxl/libxl_domain.h
@@ -53,6 +53,8 @@ struct libxlDomainJobObj {
 virCond cond;   /* Use to coordinate jobs */
 enum libxlDomainJob active; /* Currently running job */
 int owner;  /* Thread which set current job */
+unsigned long long started; /* When the job started */
+virDomainJobInfoPtr current;/* Statistics for the current job */
 };
 
 typedef struct _libxlDomainObjPrivate libxlDomainObjPrivate;
@@ -88,6 +90,10 @@ libxlDomainObjEndJob(libxlDriverPrivatePtr driver,
  virDomainObjPtr obj)
 ATTRIBUTE_RETURN_CHECK;
 
+int
+libxlDomainJobUpdateTime(struct libxlDomainJobObj *job)
+ATTRIBUTE_RETURN_CHECK;
+
 void
 libxlDomainEventQueue(libxlDriverPrivatePtr driver,
   virObjectEventPtr event);
diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c
index 8db6536..b0b6ea7 100644
--- a/src/libxl/libxl_driver.c
+++ b/src/libxl/libxl_driver.c
@@ -5236,6 +5236,43 @@ libxlDomainMemoryStats(virDomainPtr dom,
 return ret;
 }
 
+static int
+libxlDomainGetJobInfo(virDomainPtr dom,
+  virDomainJobInfoPtr info)
+{
+libxlDomainObjPrivatePtr priv;
+virDomainObjPtr vm;
+int ret = -1;
+
+if (!(vm = libxlDomObjFromDomain(dom)))
+goto cleanup;
+
+if (virDomainGetJobInfoEnsureACL(dom->conn, vm->def) < 0)
+goto cleanup;
+
+priv = vm->privateData;
+if (!priv->job.active) {
+memset(info, 0, sizeof(*info));
+info->type = VIR_DOMAIN_JOB_NONE;
+ret = 0;
+goto cleanup;
+}
+
+/* In libxl we don't have an estimated completion time
+ * thus we always set to unbounded and update time
+ * for the active job. */
+if (libxlDomainJobUpdateTime(&priv->job) < 0)
+goto cleanup;
+
+memcpy(info, priv->job.current, sizeof(virDomainJobInfo));
+ret = 0;
+
+ cleanup:
+if (vm)
+virObjectUnlock(vm);
+return ret;
+}
+
 #undef LIBXL_SET_MEMSTAT
 
 #define LIBXL_RECORD_UINT(error, key, value, ...) \
@@ -6096,6 +6133,7 @@ static virHypervisorDriver libxlHypervisorDriver = {
 #endif
 .nodeGetFreeMemory = libxlNodeGetFreeMemo

[Xen-devel] [PATCH v3 2/8] libxl: implement virDomainMemorystats

[Joao Martins]

Introduce support for domainMemoryStats API call, which
consequently enables the use of `virsh dommemstat` command to
query for memory statistics of a domain. We support
the following statistics: balloon info, available and currently
in use. swap-in, swap-out, major-faults, minor-faults require
cooperation of the guest and thus currently not supported.

We build on the data returned from libxl_domain_info and deliver
it in the virDomainMemoryStat format.

Signed-off-by: Joao Martins 
---
Changes since v1:
 - Cleanup properly after error fetching domain stats
 - Dispose libxl_dominfo after succesfull call to
 libxl_domain_info()
 - Bump version to 1.2.22
---
 src/libxl/libxl_driver.c | 70 
 1 file changed, 70 insertions(+)

diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c
index 50f6e34..f4fc7bc 100644
--- a/src/libxl/libxl_driver.c
+++ b/src/libxl/libxl_driver.c
@@ -4749,6 +4749,75 @@ libxlDomainGetCPUStats(virDomainPtr dom,
 return ret;
 }
 
+#define LIBXL_SET_MEMSTAT(TAG, VAL) \
+if (i < nr_stats) { \
+stats[i].tag = TAG; \
+stats[i].val = VAL; \
+i++; \
+}
+
+static int
+libxlDomainMemoryStats(virDomainPtr dom,
+   virDomainMemoryStatPtr stats,
+   unsigned int nr_stats,
+   unsigned int flags)
+{
+libxlDriverPrivatePtr driver = dom->conn->privateData;
+libxlDriverConfigPtr cfg = libxlDriverConfigGet(driver);
+virDomainObjPtr vm;
+libxl_dominfo d_info;
+unsigned mem, maxmem;
+size_t i = 0;
+int ret = -1;
+
+virCheckFlags(0, -1);
+
+if (!(vm = libxlDomObjFromDomain(dom)))
+goto cleanup;
+
+if (virDomainMemoryStatsEnsureACL(dom->conn, vm->def) < 0)
+goto cleanup;
+
+if (libxlDomainObjBeginJob(driver, vm, LIBXL_JOB_QUERY) < 0)
+goto cleanup;
+
+if (!virDomainObjIsActive(vm)) {
+virReportError(VIR_ERR_OPERATION_INVALID,
+   "%s", _("domain is not running"));
+goto endjob;
+}
+
+if (libxl_domain_info(cfg->ctx, &d_info, vm->def->id) != 0) {
+virReportError(VIR_ERR_INTERNAL_ERROR,
+   _("libxl_domain_info failed for domain '%d'"),
+   vm->def->id);
+goto endjob;
+}
+mem = d_info.current_memkb;
+maxmem = d_info.max_memkb;
+
+LIBXL_SET_MEMSTAT(VIR_DOMAIN_MEMORY_STAT_ACTUAL_BALLOON, maxmem - mem);
+LIBXL_SET_MEMSTAT(VIR_DOMAIN_MEMORY_STAT_AVAILABLE, maxmem);
+LIBXL_SET_MEMSTAT(VIR_DOMAIN_MEMORY_STAT_RSS, mem);
+
+ret = i;
+
+libxl_dominfo_dispose(&d_info);
+
+ endjob:
+if (!libxlDomainObjEndJob(driver, vm)) {
+virObjectUnlock(vm);
+vm = NULL;
+}
+
+ cleanup:
+if (vm)
+virObjectUnlock(vm);
+return ret;
+}
+
+#undef LIBXL_SET_MEMSTAT
+
 static int
 libxlConnectDomainEventRegisterAny(virConnectPtr conn, virDomainPtr dom, int 
eventID,
virConnectDomainEventGenericCallback 
callback,
@@ -5342,6 +5411,7 @@ static virHypervisorDriver libxlHypervisorDriver = {
 #endif
 .nodeGetFreeMemory = libxlNodeGetFreeMemory, /* 0.9.0 */
 .nodeGetCellsFreeMemory = libxlNodeGetCellsFreeMemory, /* 1.1.1 */
+.domainMemoryStats = libxlDomainMemoryStats, /* 1.2.22 */
 .domainGetCPUStats = libxlDomainGetCPUStats, /* 1.2.22 */
 .connectDomainEventRegister = libxlConnectDomainEventRegister, /* 0.9.0 */
 .connectDomainEventDeregister = libxlConnectDomainEventDeregister, /* 
0.9.0 */
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 8/8] libxl: implement virDomainGetJobStats

[Joao Martins]

Introduces support for domainGetJobStats which has the same
info as domainGetJobInfo but in a slightly different format.
Another difference is that virDomainGetJobStats can also
retrieve info on the most recently completed job. Though so
far this is only used in the source node to know if the
migration has been completed. But because we don't support
completed jobs we will deliver an error.

Signed-off-by: Joao Martins 
---
Changes since v1:
 - Fixed indentation on libxlDomainGetJobStats()
 - s/estimed/estimated/g
 - Bump version to 1.2.22
---
 src/libxl/libxl_driver.c | 53 
 1 file changed, 53 insertions(+)

diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c
index b0b6ea7..dda14c2 100644
--- a/src/libxl/libxl_driver.c
+++ b/src/libxl/libxl_driver.c
@@ -5273,6 +5273,58 @@ libxlDomainGetJobInfo(virDomainPtr dom,
 return ret;
 }
 
+static int
+libxlDomainGetJobStats(virDomainPtr dom,
+   int *type,
+   virTypedParameterPtr *params,
+   int *nparams,
+   unsigned int flags)
+{
+libxlDomainObjPrivatePtr priv;
+virDomainObjPtr vm;
+virDomainJobInfoPtr jobInfo;
+int ret = -1;
+int maxparams = 0;
+
+/* VIR_DOMAIN_JOB_STATS_COMPLETED not supported yet */
+virCheckFlags(0, -1);
+
+if (!(vm = libxlDomObjFromDomain(dom)))
+goto cleanup;
+
+if (virDomainGetJobStatsEnsureACL(dom->conn, vm->def) < 0)
+goto cleanup;
+
+priv = vm->privateData;
+jobInfo = priv->job.current;
+if (!priv->job.active) {
+*type = VIR_DOMAIN_JOB_NONE;
+*params = NULL;
+*nparams = 0;
+ret = 0;
+goto cleanup;
+}
+
+/* In libxl we don't have an estimated completion time
+ * thus we always set to unbounded and update time
+ * for the active job. */
+if (libxlDomainJobUpdateTime(&priv->job) < 0)
+goto cleanup;
+
+if (virTypedParamsAddULLong(params, nparams, &maxparams,
+VIR_DOMAIN_JOB_TIME_ELAPSED,
+jobInfo->timeElapsed) < 0)
+goto cleanup;
+
+*type = jobInfo->type;
+ret = 0;
+
+ cleanup:
+if (vm)
+virObjectUnlock(vm);
+return ret;
+}
+
 #undef LIBXL_SET_MEMSTAT
 
 #define LIBXL_RECORD_UINT(error, key, value, ...) \
@@ -6134,6 +6186,7 @@ static virHypervisorDriver libxlHypervisorDriver = {
 .nodeGetFreeMemory = libxlNodeGetFreeMemory, /* 0.9.0 */
 .nodeGetCellsFreeMemory = libxlNodeGetCellsFreeMemory, /* 1.1.1 */
 .domainGetJobInfo = libxlDomainGetJobInfo, /* 1.2.22 */
+.domainGetJobStats = libxlDomainGetJobStats, /* 1.2.22 */
 .domainBlockStats = libxlDomainBlockStats, /* 1.2.22 */
 .domainBlockStatsFlags = libxlDomainBlockStatsFlags, /* 1.2.22 */
 .domainInterfaceStats = libxlDomainInterfaceStats, /* 1.2.22 */
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 6/8] libxl: implement virConnectGetAllDomainStats

[Joao Martins]

Introduce support for connectGetAllDomainStats call that
allow us to _all_ domain(s) statistics including network, block,
cpus and memory. Changes are rather mechanical and mostly
take care of the format to export the data.

Signed-off-by: Joao Martins 
---
Changes since v1:
 - Rework flags checking on libxlDomainGetStats
 for VIR_DOMAIN_STATS_{VCPU,INTERFACE,BLOCK}
 - Removed path since we are reusing .ifname
 - Init dominfo and dispose it on cleanup.
 - Fixed VIR_FREE issue that was reported with make syntax-check"
 - Bump version to 1.2.22
---
 src/libxl/libxl_driver.c | 266 +++
 1 file changed, 266 insertions(+)

diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c
index ba1d67b..8db6536 100644
--- a/src/libxl/libxl_driver.c
+++ b/src/libxl/libxl_driver.c
@@ -5238,6 +5238,271 @@ libxlDomainMemoryStats(virDomainPtr dom,
 
 #undef LIBXL_SET_MEMSTAT
 
+#define LIBXL_RECORD_UINT(error, key, value, ...) \
+do { \
+char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; \
+snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, \
+ key, ##__VA_ARGS__); \
+if (virTypedParamsAddUInt(&tmp->params, \
+  &tmp->nparams, \
+  &maxparams, \
+  param_name, \
+  value) < 0) \
+goto error;   \
+} while (0)
+
+#define LIBXL_RECORD_LL(error, key, value, ...) \
+do { \
+char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; \
+snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, \
+ key, ##__VA_ARGS__); \
+if (virTypedParamsAddLLong(&tmp->params, \
+   &tmp->nparams, \
+   &maxparams, \
+   param_name, \
+   value) < 0) \
+goto error; \
+} while (0)
+
+#define LIBXL_RECORD_ULL(error, key, value, ...) \
+do { \
+char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; \
+snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, \
+ key, ##__VA_ARGS__); \
+if (virTypedParamsAddULLong(&tmp->params, \
+&tmp->nparams, \
+&maxparams, \
+param_name, \
+value) < 0) \
+goto error; \
+} while (0)
+
+#define LIBXL_RECORD_STR(error, key, value, ...) \
+do { \
+char param_name[VIR_TYPED_PARAM_FIELD_LENGTH]; \
+snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH, \
+ key, ##__VA_ARGS__); \
+if (virTypedParamsAddString(&tmp->params, \
+&tmp->nparams, \
+&maxparams, \
+param_name, \
+value) < 0) \
+goto error; \
+} while (0)
+
+static int
+libxlDomainGetStats(virConnectPtr conn,
+virDomainObjPtr dom,
+unsigned int stats,
+virDomainStatsRecordPtr *record)
+{
+libxlDriverPrivatePtr driver = conn->privateData;
+libxlDriverConfigPtr cfg = libxlDriverConfigGet(driver);
+virDomainStatsRecordPtr tmp;
+libxl_dominfo d_info;
+libxl_vcpuinfo *vcpuinfo = NULL;
+int maxcpu, hostcpus;
+unsigned long long mem, maxmem;
+int maxparams = 0;
+int ret = -1;
+size_t i, state;
+unsigned int domflags = stats & (VIR_DOMAIN_STATS_BALLOON |
+ VIR_DOMAIN_STATS_CPU_TOTAL);
+
+if (VIR_ALLOC(tmp) < 0)
+return ret;
+
+libxl_dominfo_init(&d_info);
+
+mem = virDomainDefGetMemoryInitial(dom->def);
+maxmem = virDomainDefGetMemoryActual(dom->def);
+d_info.cpu_time = 0;
+
+if (domflags && virDomainObjIsActive(dom) &&
+!libxl_domain_info(cfg->ctx, &d_info, dom->def->id)) {
+mem = d_info.current_memkb;
+maxmem = d_info.max_memkb;
+}
+
+if (stats & VIR_DOMAIN_STATS_STATE) {
+LIBXL_RECORD_UINT(cleanup, "state.reason", dom->state.reason);
+LIBXL_RECORD_UINT(cleanup, "state.state", dom->state.state);
+}
+
+if (stats & VIR_DOMAIN_STATS_BALLOON) {
+LIBXL_RECORD_ULL(cleanup, "balloon.current", mem);
+LIBXL_RECORD_ULL(cleanup, "balloon.maximum", maxmem);
+}
+
+if (stats & VIR_DOMAIN_STATS_CPU_TOTAL)
+LIBXL_RECORD_ULL(cleanup, "cpu.time", d_info.cpu_time);
+
+if (stats & VIR_DOMAIN_STATS_VCPU) {
+vcpuinfo = libxl_list_vcpu(cfg->ctx, dom->def->id, &maxcpu, &hostcpus);
+
+for (i = 0; i < dom->def->vcpus; i++) {
+if (!vcpuinfo)
+state = VIR_VCPU_OFFLINE;
+else if (vcpuinfo[i].running)
+state = VIR_VCPU_RUNNING;
+else if (vcpuinfo[i].blocked)
+state = VIR_VCPU_BLOCKED;
+else
+state = VIR_VCPU_OFFLINE;
+

[Xen-devel] [PATCH v3 4/8] util: add virDiskNameParse to handle disk and partition idx

[Joao Martins]

Introduce a new helper function "virDiskNameParse" which extends
virDiskNameToIndex but handling both disk index and partition index.
Also rework virDiskNameToIndex to be based on virDiskNameParse.
A test is also added for this function testing both valid and
invalid disk names.

Signed-off-by: Joao Martins 
---
Changes since v2:
 - Sort the newly added symbol in the list
---
 src/libvirt_private.syms |  1 +
 src/util/virutil.c   | 41 +++
 src/util/virutil.h   |  1 +
 tests/utiltest.c | 56 
 4 files changed, 95 insertions(+), 4 deletions(-)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index a835f18..7e60d87 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -148,6 +148,7 @@ virDomainCapsNew;
 
 # conf/domain_conf.h
 virBlkioDeviceArrayClear;
+virDiskNameParse;
 virDiskNameToBusDeviceIndex;
 virDiskNameToIndex;
 virDomainActualNetDefFree;
diff --git a/src/util/virutil.c b/src/util/virutil.c
index cddc78a..76591be 100644
--- a/src/util/virutil.c
+++ b/src/util/virutil.c
@@ -537,14 +537,17 @@ const char *virEnumToString(const char *const*types,
 }
 
 /* Translates a device name of the form (regex) /^[fhv]d[a-z]+[0-9]*$/
- * into the corresponding index (e.g. sda => 0, hdz => 25, vdaa => 26)
- * Note that any trailing string of digits is simply ignored.
+ * into the corresponding index and partition number
+ * (e.g. sda0 => (0,0), hdz2 => (25,2), vdaa12 => (26,12))
  * @param name The name of the device
- * @return name's index, or -1 on failure
+ * @param disk The disk index to be returned
+ * @param partition The partition index to be returned
+ * @return 0 on success, or -1 on failure
  */
-int virDiskNameToIndex(const char *name)
+int virDiskNameParse(const char *name, int *disk, int *partition)
 {
 const char *ptr = NULL;
+char *rem;
 int idx = 0;
 static char const* const drive_prefix[] = {"fd", "hd", "vd", "sd", "xvd", 
"ubd"};
 size_t i;
@@ -573,6 +576,36 @@ int virDiskNameToIndex(const char *name)
 if (ptr[n_digits] != '\0')
 return -1;
 
+*disk = idx;
+
+/* Convert trailing digits into our partition index */
+if (partition) {
+*partition = 0;
+
+/* Shouldn't start by zero */
+if (n_digits > 1 && *ptr == '0')
+return -1;
+
+if (n_digits && virStrToLong_i(ptr, &rem, 10, partition) < 0)
+return -1;
+}
+
+return 0;
+}
+
+/* Translates a device name of the form (regex) /^[fhv]d[a-z]+[0-9]*$/
+ * into the corresponding index (e.g. sda => 0, hdz => 25, vdaa => 26)
+ * Note that any trailing string of digits is simply ignored.
+ * @param name The name of the device
+ * @return name's index, or -1 on failure
+ */
+int virDiskNameToIndex(const char *name)
+{
+int idx;
+
+if (virDiskNameParse(name, &idx, NULL))
+idx = -1;
+
 return idx;
 }
 
diff --git a/src/util/virutil.h b/src/util/virutil.h
index 53025f7..02387e0 100644
--- a/src/util/virutil.h
+++ b/src/util/virutil.h
@@ -67,6 +67,7 @@ int virDoubleToStr(char **strp, double number)
 char *virFormatIntDecimal(char *buf, size_t buflen, int val)
 ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK;
 
+int virDiskNameParse(const char *name, int *disk, int *partition);
 int virDiskNameToIndex(const char* str);
 char *virIndexToDiskName(int idx, const char *prefix);
 
diff --git a/tests/utiltest.c b/tests/utiltest.c
index 98c689d..a8f9060 100644
--- a/tests/utiltest.c
+++ b/tests/utiltest.c
@@ -23,6 +23,23 @@ static const char* diskNames[] = {
 "sdia", "sdib", "sdic", "sdid", "sdie", "sdif", "sdig", "sdih", "sdii", 
"sdij", "sdik", "sdil", "sdim", "sdin", "sdio", "sdip", "sdiq", "sdir", "sdis", 
"sdit", "sdiu", "sdiv", "sdiw", "sdix", "sdiy", "sdiz"
 };
 
+struct testDiskName
+{
+const char *name;
+int idx;
+int partition;
+};
+
+static struct testDiskName diskNamesPart[] = {
+{"sda0",  0,   0},
+{"sdb10", 1,  10},
+{"sdc2147483647", 2,  2147483647},
+};
+
+static const char* diskNamesInvalid[] = {
+"sda00", "sda01", "sdb-1"
+};
+
 static int
 testIndexToDiskName(const void *data ATTRIBUTE_UNUSED)
 {
@@ -79,6 +96,44 @@ testDiskNameToIndex(const void *data ATTRIBUTE_UNUSED)
 
 
 
+static int
+testDiskNameParse(const void *data ATTRIBUTE_UNUSED)
+{
+size_t i;
+int idx;
+int partition;
+struct testDiskName *disk = NULL;
+
+for (i = 0; i < ARRAY_CARDINALITY(diskNamesPart); ++i) {
+disk = &diskNamesPart[i];
+if (virDiskNameParse(disk->name, &idx, &partition))
+return -1;
+
+if (disk->idx != idx) {
+VIR_TEST_DEBUG("\nExpect [%d]\n", disk->idx);
+VIR_TEST_DEBUG("Actual [%d]\n", idx);
+return -1;
+}
+
+if (disk->partition != partition) {
+VIR_TEST_DEBUG("\nExpect [%d]\n", disk->partition);
+VIR_TEST_DEBUG("Actual [%d]\n",

[Xen-devel] [PATCH v3 0/8] libxl: domain statistics support

[Joao Martins]

Hey,

[ Apologies for resending this series before receiving comments, but I
  found two issues (namely on patch 3 and 4) and thus I am reposting it
  as v3 having those fixed. ]

This series bring support for various statistics about domains
regarding CPU, Memory, Network Interfaces, Block and Jobs. Not all of
the statistics are implemented: qdisk support is missing in this series
and some of the memory statistics aren't available.

With this series we further implement 7 more functions of libvirt APIs.
It is organized as follows:

 * Patch 1, 2: implements cpu/memory statistics.
 * Patch 3: implements network statistics
 * Patch 4: adds helper method virDiskNameParse as an extension
to virDiskNameToIndex (New)
 * Patch 5: VBD block statistics. QDisk will follow up in a separate
series regarding QEMU monitor integration.
 * Patch 6: implement fetching all domain statistics
 * Patch 7, 8: implements Job information statistics.

Overall it looks big but 70% of the patch is due to #5 and #6 but doesn't
add necessarily more complexity to the driver I believe. Patch #7 and #8
are of special importance because GetJobInfo and GetJobStats are now used
in Openstack Kilo to monitor live-migration progress. These two patches
together with the p2p migration I sent earlier let us sucessfully
live-migrate with Openstack Kilo. Furthermore with this series we get to
support nova diagnostics.

Tested this series on 4.4.3 and 4.5 setups plus Openstack Kilo.

Individual patches contain the changelog and comments addressed since v1.

Thanks!

Joao Martins (8):
  libxl: implement virDomainGetCPUStats
  libxl: implement virDomainMemorystats
  libxl: implement virDomainInterfaceStats
  util: add virDiskNameParse to handle disk and partition idx
  libxl: implement virDomainBlockStats
  libxl: implement virConnectGetAllDomainStats
  libxl: implement virDomainGetJobInfo
  libxl: implement virDomainGetJobStats

 configure.ac |   2 +-
 src/libvirt_private.syms |   1 +
 src/libxl/libxl_domain.c |  54 +++
 src/libxl/libxl_domain.h |   6 +
 src/libxl/libxl_driver.c | 960 +++
 src/util/virutil.c   |  41 +-
 src/util/virutil.h   |   1 +
 tests/utiltest.c |  56 +++
 8 files changed, 1116 insertions(+), 5 deletions(-)

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 1/8] libxl: implement virDomainGetCPUStats

[Joao Martins]

Introduce support for domainGetCPUStats API call and consequently
allow us to use `virsh cpu-stats`. The latter returns a more brief
output than the one provided by`virsh vcpuinfo`.

Signed-off-by: Joao Martins 
---
Changes since v1:
 - Remove libxl_vcpuinfo_dispose() in favor or using
 libxl_vcpuinfo_list_free(), and also removing VIR_FREE call
 - Dispose libxl_dominfo after succesfull call to
 libxl_domain_info()
 - Fixed identation of parameters
 - Bump version to 1.2.22
---
 src/libxl/libxl_driver.c | 110 +++
 1 file changed, 110 insertions(+)

diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c
index fcdcbdb..50f6e34 100644
--- a/src/libxl/libxl_driver.c
+++ b/src/libxl/libxl_driver.c
@@ -73,6 +73,8 @@ VIR_LOG_INIT("libxl.libxl_driver");
 #define LIBXL_CONFIG_FORMAT_XM "xen-xm"
 #define LIBXL_CONFIG_FORMAT_SEXPR "xen-sxpr"
 
+#define LIBXL_NB_TOTAL_CPU_STAT_PARAM 1
+
 #define HYPERVISOR_CAPABILITIES "/proc/xen/capabilities"
 #define HYPERVISOR_XENSTORED "/dev/xen/xenstored"
 
@@ -4641,6 +4643,113 @@ libxlDomainIsUpdated(virDomainPtr dom)
 }
 
 static int
+libxlDomainGetTotalCPUStats(libxlDriverPrivatePtr driver,
+virDomainObjPtr vm,
+virTypedParameterPtr params,
+unsigned int nparams)
+{
+libxlDriverConfigPtr cfg = libxlDriverConfigGet(driver);
+libxl_dominfo d_info;
+int ret = -1;
+
+if (nparams == 0)
+return LIBXL_NB_TOTAL_CPU_STAT_PARAM;
+
+if (libxl_domain_info(cfg->ctx, &d_info, vm->def->id) != 0) {
+virReportError(VIR_ERR_INTERNAL_ERROR,
+   _("libxl_domain_info failed for domain '%d'"),
+   vm->def->id);
+return ret;
+}
+
+if (virTypedParameterAssign(¶ms[0], VIR_DOMAIN_CPU_STATS_CPUTIME,
+VIR_TYPED_PARAM_ULLONG, d_info.cpu_time) < 0)
+nparams = ret;
+
+libxl_dominfo_dispose(&d_info);
+return nparams;
+}
+
+static int
+libxlDomainGetPerCPUStats(libxlDriverPrivatePtr driver,
+  virDomainObjPtr vm,
+  virTypedParameterPtr params,
+  unsigned int nparams,
+  int start_cpu,
+  unsigned int ncpus)
+{
+libxl_vcpuinfo *vcpuinfo;
+int maxcpu, hostcpus;
+size_t i;
+libxlDriverConfigPtr cfg = libxlDriverConfigGet(driver);
+int ret = -1;
+
+if (nparams == 0 && ncpus != 0)
+return LIBXL_NB_TOTAL_CPU_STAT_PARAM;
+else if (nparams == 0)
+return vm->def->maxvcpus;
+
+if ((vcpuinfo = libxl_list_vcpu(cfg->ctx, vm->def->id, &maxcpu,
+&hostcpus)) == NULL) {
+virReportError(VIR_ERR_INTERNAL_ERROR,
+   _("Failed to list vcpus for domain '%d' with 
libxenlight"),
+   vm->def->id);
+return ret;
+}
+
+for (i = start_cpu; i < maxcpu && i < ncpus; ++i) {
+if (virTypedParameterAssign(¶ms[(i-start_cpu)],
+VIR_DOMAIN_CPU_STATS_CPUTIME,
+VIR_TYPED_PARAM_ULLONG,
+vcpuinfo[i].vcpu_time) < 0)
+goto cleanup;
+}
+ret = nparams;
+
+ cleanup:
+libxl_vcpuinfo_list_free(vcpuinfo, maxcpu);
+return ret;
+}
+
+static int
+libxlDomainGetCPUStats(virDomainPtr dom,
+   virTypedParameterPtr params,
+   unsigned int nparams,
+   int start_cpu,
+   unsigned int ncpus,
+   unsigned int flags)
+{
+libxlDriverPrivatePtr driver = dom->conn->privateData;
+virDomainObjPtr vm = NULL;
+int ret = -1;
+
+virCheckFlags(VIR_TYPED_PARAM_STRING_OKAY, -1);
+
+if (!(vm = libxlDomObjFromDomain(dom)))
+goto cleanup;
+
+if (virDomainGetCPUStatsEnsureACL(dom->conn, vm->def) < 0)
+goto cleanup;
+
+if (!virDomainObjIsActive(vm)) {
+virReportError(VIR_ERR_OPERATION_INVALID,
+   "%s", _("domain is not running"));
+goto cleanup;
+}
+
+if (start_cpu == -1)
+ret = libxlDomainGetTotalCPUStats(driver, vm, params, nparams);
+else
+ret = libxlDomainGetPerCPUStats(driver, vm, params, nparams,
+  start_cpu, ncpus);
+
+ cleanup:
+if (vm)
+virObjectUnlock(vm);
+return ret;
+}
+
+static int
 libxlConnectDomainEventRegisterAny(virConnectPtr conn, virDomainPtr dom, int 
eventID,
virConnectDomainEventGenericCallback 
callback,
void *opaque, virFreeCallback freecb)
@@ -5233,6 +5342,7 @@ static virHypervisorDriver libxlHypervisorDriver = {
 #endif
 .nodeGetFreeMemory = libxlNodeGetFreeMemory, /* 0.9.0 */
 .nodeGetCellsFreeMemory = libxlNodeGetCellsFreeMemory, /* 1.1

[Xen-devel] [PATCH v3 5/8] libxl: implement virDomainBlockStats

[Joao Martins]

Introduce initial support for domainBlockStats API call that
allow us to query block device statistics. openstack nova
uses this API call to query block statistics, alongside
virDomainMemoryStats and virDomainInterfaceStats.  Note that
this patch only introduces it for VBD for starters. QDisk
will come in a separate patch series.

A new statistics data structure is introduced to fit common
statistics among others specific to the underlying block
backends. For the VBD statistics on linux these are exported
via sysfs on the path:

"/sys/bus/xen-backend/devices/vbd--/statistics"

To calculate the block devno libxlDiskPathToID is introduced.
Each backend implements its own function to extract statistics
and let there be handled the different platforms. An alternative
would be to reuse libvirt xen driver function.

VBD stats are exposed in reqs and number of sectors from
blkback, and it's up to us to convert it to sector sizes.
The sector size is gathered through xenstore in the device
backend entry "physical-sector-size". This adds up an extra
dependency namely of xenstore for doing the xs_read.

BlockStatsFlags variant is also implemented which has the
added benefit of getting the number of flush requests.

Signed-off-by: Joao Martins 
---
Changes since v1:
 - Fix identation issues
 - Set ret to LIBXL_VBD_SECTOR_SIZE
 - Reuse VIR_STRDUP error instead of doing virReportError
 when we fail to set stats->backend
 - Change virAsprintf(...) error checking
 - Change error to VIR_ERR_OPERATION_FAILED when xenstore path
 does not exist and when failing to read stat.
 - Resolve issues with 'make syntax-check' with cppi installed.
 - Remove libxlDiskPathMatches in favor of using virutil
 virDiskNameParse to fetch disk and partition index.
 - Rename libxlDiskPathParse to libxlDiskPathToID and rework
 function to just convert disk and partition index to devno.
 - Bump version to 1.2.22
---
 configure.ac |   2 +-
 src/libxl/libxl_driver.c | 375 +++
 2 files changed, 376 insertions(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index f481c50..10c56e5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -896,7 +896,7 @@ if test "$with_libxl" != "no" ; then
 LIBS="$LIBS $LIBXL_LIBS"
 AC_CHECK_LIB([xenlight], [libxl_ctx_alloc], [
 with_libxl=yes
-LIBXL_LIBS="$LIBXL_LIBS -lxenlight -lxenctrl"
+LIBXL_LIBS="$LIBXL_LIBS -lxenlight -lxenctrl -lxenstore"
 ],[
 if test "$with_libxl" = "yes"; then
 fail=1
diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c
index 9a5a74c..ba1d67b 100644
--- a/src/libxl/libxl_driver.c
+++ b/src/libxl/libxl_driver.c
@@ -59,6 +59,7 @@
 #include "network/bridge_driver.h"
 #include "locking/domain_lock.h"
 #include "virstats.h"
+#include 
 
 #define VIR_FROM_THIS VIR_FROM_LIBXL
 
@@ -75,6 +76,7 @@ VIR_LOG_INIT("libxl.libxl_driver");
 #define LIBXL_CONFIG_FORMAT_SEXPR "xen-sxpr"
 
 #define LIBXL_NB_TOTAL_CPU_STAT_PARAM 1
+#define LIBXL_NB_TOTAL_BLK_STAT_PARAM 6
 
 #define HYPERVISOR_CAPABILITIES "/proc/xen/capabilities"
 #define HYPERVISOR_XENSTORED "/dev/xen/xenstored"
@@ -103,6 +105,25 @@ struct _libxlOSEventHookInfo {
 int id;
 };
 
+/* Object used to store disk statistics across multiple xen backends */
+typedef struct _libxlBlockStats libxlBlockStats;
+typedef libxlBlockStats *libxlBlockStatsPtr;
+struct _libxlBlockStats {
+long long rd_req;
+long long rd_bytes;
+long long wr_req;
+long long wr_bytes;
+long long f_req;
+
+char *backend;
+union {
+struct {
+long long ds_req;
+long long oo_req;
+} vbd;
+} u;
+};
+
 /* Function declarations */
 static int
 libxlDomainManagedSaveLoad(virDomainObjPtr vm,
@@ -4644,6 +4665,358 @@ libxlDomainIsUpdated(virDomainPtr dom)
 }
 
 static int
+libxlDiskPathToID(const char *virtpath)
+{
+static char const* drive_prefix[] = {"xvd", "hd", "sd"};
+int disk, partition, chrused;
+int fmt, id;
+char *r;
+size_t i;
+
+fmt = id = -1;
+
+/* Find any disk prefixes we know about */
+for (i = 0; i < ARRAY_CARDINALITY(drive_prefix); i++) {
+if (STRPREFIX(virtpath, drive_prefix[i]) &&
+!virDiskNameParse(virtpath, &disk, &partition)) {
+fmt = i;
+break;
+}
+}
+
+/* Handle it same way as xvd */
+if (fmt < 0 &&
+(sscanf(virtpath, "d%ip%i%n", &disk, &partition, &chrused) >= 2
+ && chrused == strlen(virtpath)))
+fmt = 0;
+
+/* Test indexes ranges and calculate the device id */
+switch (fmt) {
+case 0: /* xvd */
+if (disk <= 15 && partition <= 15)
+id = (202 << 8) | (disk << 4) | partition;
+else if ((disk <= ((1<<20)-1)) || partition <= 255)
+id = (1 << 28) | (disk << 8) | partition;
+break;
+case 1: /* hd */
+if (disk <= 3 && partition <= 63)
+id = 

[Xen-devel] [PATCH v3 3/8] libxl: implement virDomainInterfaceStats

[Joao Martins]

Introduce support for domainInterfaceStats API call for querying
network interface statistics. Consequently it also enables the
use of `virsh domifstat  ` command.

After succesful guest creation we fill the network
interfaces names based on domain, device id and append suffix
if it's emulated in the following form: vif.[-emu]. Because
we need the devid from domain config (which is filled by libxl on domain
create)  we cannot do generate the names in console callback. On domain
cleanup we also clear ifname, in case it was set by libvirt (i.e.
being prefixed with "vif"). We also skip these two steps in case the name
of the interface was manually inserted by the adminstrator.

For getting the interface statistics we resort to virNetInterfaceStats
and let libvirt handle the platform specific nits. Note that the latter
is not yet supported in FreeBSD.

Signed-off-by: Joao Martins 
---
Changes since v2:
 - Clear ifname if it's autogenerated, since otherwise will persist
 on successive domain starts. Change commit message reflecting this
 change.

Changes since v1:
 - Fill .ifname after domain start with generated
 name from libxl  based on domain id and devid returned by libxl.
 After that path validation don interfaceStats is enterily based
 on ifname pretty much like the other drivers.
 - Modify commit message reflecting the changes mentioned in
 the previous item.
 - Bump version to 1.2.22
---
 src/libxl/libxl_domain.c | 26 ++
 src/libxl/libxl_driver.c | 48 
 2 files changed, 74 insertions(+)

diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c
index 40dcea1..adb4563 100644
--- a/src/libxl/libxl_domain.c
+++ b/src/libxl/libxl_domain.c
@@ -728,6 +728,17 @@ libxlDomainCleanup(libxlDriverPrivatePtr driver,
 }
 }
 
+if ((vm->def->nnets)) {
+ssize_t i;
+
+for (i = 0; i < vm->def->nnets; i++) {
+virDomainNetDefPtr net = vm->def->nets[i];
+
+if (STRPREFIX(net->ifname, "vif"))
+VIR_FREE(net->ifname);
+}
+}
+
 if (virAsprintf(&file, "%s/%s.xml", cfg->stateDir, vm->def->name) > 0) {
 if (unlink(file) < 0 && errno != ENOENT && errno != ENOTDIR)
 VIR_DEBUG("Failed to remove domain XML for %s", vm->def->name);
@@ -901,6 +912,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, 
virDomainObjPtr vm,
 virDomainDefPtr def = NULL;
 virObjectEventPtr event = NULL;
 libxlSavefileHeader hdr;
+ssize_t i;
 int ret = -1;
 uint32_t domid = 0;
 char *dom_xml = NULL;
@@ -1023,6 +1035,20 @@ libxlDomainStart(libxlDriverPrivatePtr driver, 
virDomainObjPtr vm,
  */
 vm->def->id = domid;
 
+for (i = 0; i < vm->def->nnets; i++) {
+virDomainNetDefPtr net = vm->def->nets[i];
+libxl_device_nic *x_nic = &d_config.nics[i];
+const char *suffix =
+x_nic->nictype != LIBXL_NIC_TYPE_VIF ? "-emu" : "";
+
+if (net->ifname)
+continue;
+
+if (virAsprintf(&net->ifname, "vif%d.%d%s",
+domid, x_nic->devid, suffix) < 0)
+continue;
+}
+
 /* Always enable domain death events */
 if (libxl_evenable_domain_death(cfg->ctx, vm->def->id, 0, &priv->deathW))
 goto cleanup_dom;
diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c
index f4fc7bc..9a5a74c 100644
--- a/src/libxl/libxl_driver.c
+++ b/src/libxl/libxl_driver.c
@@ -58,6 +58,7 @@
 #include "virhostdev.h"
 #include "network/bridge_driver.h"
 #include "locking/domain_lock.h"
+#include "virstats.h"
 
 #define VIR_FROM_THIS VIR_FROM_LIBXL
 
@@ -4643,6 +4644,52 @@ libxlDomainIsUpdated(virDomainPtr dom)
 }
 
 static int
+libxlDomainInterfaceStats(virDomainPtr dom,
+  const char *path,
+  virDomainInterfaceStatsPtr stats)
+{
+libxlDriverPrivatePtr driver = dom->conn->privateData;
+virDomainObjPtr vm;
+ssize_t i;
+int ret = -1;
+
+if (!(vm = libxlDomObjFromDomain(dom)))
+goto cleanup;
+
+if (virDomainInterfaceStatsEnsureACL(dom->conn, vm->def) < 0)
+goto cleanup;
+
+if (libxlDomainObjBeginJob(driver, vm, LIBXL_JOB_QUERY) < 0)
+goto cleanup;
+
+if (!virDomainObjIsActive(vm)) {
+virReportError(VIR_ERR_OPERATION_INVALID,
+   "%s", _("domain is not running"));
+goto endjob;
+}
+
+/* Check the path is one of the domain's network interfaces. */
+for (i = 0; i < vm->def->nnets; i++) {
+if (vm->def->nets[i]->ifname &&
+STREQ(vm->def->nets[i]->ifname, path)) {
+ret = virNetInterfaceStats(path, stats);
+break;
+}
+}
+
+ endjob:
+if (!libxlDomainObjEndJob(driver, vm)) {
+virObjectUnlock(vm);
+vm = NULL;
+}
+
+ cleanup:
+if (vm)
+virObjectUnlock(vm);
+return ret;
+}
+
+static int
 libxlDomainGetTotalCPUStats(libxlDriverPrivatePtr

[Xen-devel] Boot regression following c/s "x86/IO-APIC: fix setup of Xen internally used IRQs"

[Andrew Cooper]

(XEN) [0.00] IRQ limits: 72 GSI, 15304 MSI/MSI-X
(XEN) [0.00] Switched to APIC driver x2apic_cluster.

(XEN) [6.333211] I/O virtualisation enabled
(XEN) [6.339317]  - Dom0 mode: Relaxed
(XEN) [6.344720] Interrupt remapping enabled
(XEN) [6.350864] Enabled directed EOI with ioapic_ack_old on!
(XEN) [6.359325] ENABLING IO-APIC IRQs
(XEN) [6.364724]  -> Using old ACK method
(XEN) [6.370601] [ Xen-4.7.0-xs108373-d  x86_64  debug=y  Not
tainted ]
(XEN) [6.381270] CPU:0
(XEN) [6.385469] RIP:e008:[]
x2apic.c#cpu_mask_to_apicid_x2apic_cluster+0xa4/0x1b9
(XEN) [6.398376] RFLAGS: 00010246   CONTEXT: hypervisor
(XEN) [6.406171] rax: 82d08039d598   rbx: 83023ffee800  
rcx: 0005
(XEN) [6.417316] rdx: 80007d2f7fc63a00   rsi: 0050  
rdi: 83023ffee800
(XEN) [6.428464] rbp: 82d080317d68   rsp: 82d080317d38  
r8:  07ff
(XEN) [6.439682] r9:  0020   r10: 00100010  
r11: 
(XEN) [6.450841] r12: 0005   r13:   
r14: 0005
(XEN) [6.462068] r15: 0004   cr0: 8005003b  
cr4: 06e0
(XEN) [6.473220] cr3: 788b   cr2: 
(XEN) [6.481007] ds:    es:    fs:    gs:    ss:
   cs: e008
(XEN) [6.491572] Xen stack trace from rsp=82d080317d38:

(XEN) [6.724830] Xen call trace:
(XEN) [6.729658][]
x2apic.c#cpu_mask_to_apicid_x2apic_cluster+0xa4/0x1b9
(XEN) [6.741671][] setup_IO_APIC+0x998/0x124e
(XEN) [6.749868][] smp_prepare_cpus+0x241/0x26a
(XEN) [6.758251][] __start_xen+0x209b/0x25d1
(XEN) [6.766335][] __high_start+0x53/0x60

The #GP fault is from a poisoned per_cpu area, and exact fault is from
the calculation of dest in:

static unsigned int cpu_mask_to_apicid_x2apic_cluster(const cpumask_t
*cpumask)
{
unsigned int cpu = cpumask_any(cpumask);
unsigned int dest = per_cpu(cpu_2_logical_apicid, cpu);
const cpumask_t *cluster_cpus = per_cpu(cluster_cpus, cpu);


This is affecting multiple machines in the XenServer test pool, all of
which use the x2apic_cluster APIC driver.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCHv3] 00/28] Kconfig conversion

[Julien Grall]

On 13/11/15 12:54, Jan Beulich wrote:
 On 13.11.15 at 12:41,  wrote:
>> 2) make menuconfig doesn't expose any options => No possibility to
>> select any UART on ARM.
>>
>> The latter is because how you define the option in the Kconfig:
>>
>> config HAS_EXYNOS4210
>>  bool
>>
>> Without any text, it's not possible for the user to select this option.
> 
> This part at least is intended at this point in time (see past
> discussions) - we don't want any user configurable items just yet.

Ok, but in this case all the options set before *must* be set in the
KConfig.

Otherwise we end up as today where ARM32/ARM64 is not usable anymore.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 2/7] xen/arm: introduce HYPERVISOR_platform_op on arm and arm64

[Julien Grall]

Hi Stefano,

On 12/11/15 17:30, Stefano Stabellini wrote:
> Signed-off-by: Stefano Stabellini 
> 
> ---
> 
> Changes in v2:
> - rename dom0_op to platform_op
> ---
>  arch/arm/include/asm/xen/hypercall.h |2 ++
>  arch/arm/xen/enlighten.c |1 +
>  arch/arm/xen/hypercall.S |1 +
>  arch/arm64/xen/hypercall.S   |1 +
>  4 files changed, 5 insertions(+)
> 
> diff --git a/arch/arm/include/asm/xen/hypercall.h 
> b/arch/arm/include/asm/xen/hypercall.h
> index 712b50e..c3e00d0 100644
> --- a/arch/arm/include/asm/xen/hypercall.h
> +++ b/arch/arm/include/asm/xen/hypercall.h
> @@ -35,6 +35,7 @@
>  
>  #include 
>  #include 
> +#include 
>  
>  long privcmd_call(unsigned call, unsigned long a1,
>   unsigned long a2, unsigned long a3,
> @@ -49,6 +50,7 @@ int HYPERVISOR_memory_op(unsigned int cmd, void *arg);
>  int HYPERVISOR_physdev_op(int cmd, void *arg);
>  int HYPERVISOR_vcpu_op(int cmd, int vcpuid, void *extra_args);
>  int HYPERVISOR_tmem_op(void *arg);
> +int HYPERVISOR_platform_op(void *arg);

int HYPERVISOR_platform_op(struct xen_platform_op *platform_op) to allow
compiler type checking and match the x86 version.

Also, the implementation of the helper differ from x86. On x86, the
helper takes care of setting the interface_version while here you
request the caller to do it.

It's better if we have similar requirement across the architecture as
this helpers may be called from common code.

>  int HYPERVISOR_multicall(struct multicall_entry *calls, uint32_t nr);
>  
>  static inline int

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 4/7] arm: extend pvclock_wall_clock with sec_hi

[Julien Grall]

Hi Stefano,

On 12/11/15 17:30, Stefano Stabellini wrote:
> The hypervisor actually exposes an additional field to struct
> pvclock_wall_clock, with the high 32 bit seconds.
> 
> Signed-off-by: Stefano Stabellini 

Reviewed-by: Julien Grall 

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH QEMU v2 2/2] xen: fix usage of xc_domain_create in domain builder

[Stefano Stabellini]

On Fri, 13 Nov 2015, Roger Pau Monne wrote:
> Due to the addition of HVMlite and the requirement to always provide a valid
> xc_domain_configuration_t, xc_domain_create now always takes an arch domain
> config, which can be NULL in order to mimic previous behaviour.
>
> Add a small stub called xen_domain_create that encapsulates the correct call
> to xc_domain_create depending on the libxc version detected.
>
> Signed-off-by: Roger Pau Monné 

FYI this is going to conflict with Ian's series:

1447070487-31229-1-git-send-email-ian.campb...@citrix.com

Is the corresponding libxc change already in Xen?


> Cc: Stefano Stabellini 
> Cc: qemu-de...@nongnu.org
> ---
> Changes since v1:
>  - Add a compat layer to support previous libxc versions.
>  - Add machinery to detect current Xen unstable (4.7).
> ---
>  configure   | 17 +
>  hw/xenpv/xen_domainbuild.c  |  2 +-
>  include/hw/xen/xen_common.h | 12 
>  3 files changed, 30 insertions(+), 1 deletion(-)
>
> diff --git a/configure b/configure
> index 779623a..001982f 100755
> --- a/configure
> +++ b/configure
> @@ -1863,6 +1863,23 @@ EOF
>elif
>cat > $TMPC <  #include 
> +#include 
> +int main(void) {
> +  xc_interface *xc = NULL;
> +  xen_domain_handle_t handle;
> +  xc_domain_create(xc, 0, handle, 0, NULL, NULL);
> +  return 0;
> +}
> +EOF
> +  compile_prog "" "$xen_libs"
> +then
> +xen_ctrl_version=470
> +xen=yes
> +
> +  # Xen 4.6
> +  elif
> +  cat > $TMPC < +#include 
>  #include 
>  #include 
>  #include 
> diff --git a/hw/xenpv/xen_domainbuild.c b/hw/xenpv/xen_domainbuild.c
> index c0ab753..07814f6 100644
> --- a/hw/xenpv/xen_domainbuild.c
> +++ b/hw/xenpv/xen_domainbuild.c
> @@ -234,7 +234,7 @@ int xen_domain_build_pv(const char *kernel, const char 
> *ramdisk,
>  int rc;
>
>  memcpy(uuid, qemu_uuid, sizeof(uuid));
> -rc = xc_domain_create(xen_xc, ssidref, uuid, flags, &xen_domid);
> +rc = xen_domain_create(xen_xc, ssidref, uuid, flags, &xen_domid, NULL);
>  if (rc < 0) {
>  fprintf(stderr, "xen: xc_domain_create() failed\n");
>  goto err;
> diff --git a/include/hw/xen/xen_common.h b/include/hw/xen/xen_common.h
> index 5923290..e5ba732 100644
> --- a/include/hw/xen/xen_common.h
> +++ b/include/hw/xen/xen_common.h
> @@ -417,4 +417,16 @@ static inline int xen_set_ioreq_server_state(XenXC xc, 
> domid_t dom,
>
>  #endif
>
> +static inline int xen_domain_create(XenXC xc, uint32_t ssidref,
> +xen_domain_handle_t handle, uint32_t 
> flags,
> +uint32_t *pdomid,
> +xc_domain_configuration_t *config)
> +{
> +#if CONFIG_XEN_CTRL_INTERFACE_VERSION < 470
> +return xc_domain_create(xc, ssidref, handle, flags, pdomid);
> +#else
> +return xc_domain_create(xc, ssidref, handle, flags, pdomid, config);
> +#endif
> +}
> +
>  #endif /* QEMU_HW_XEN_COMMON_H */

Please follow the existing scheme by introducing two xen_domain_create
functions.

In addition, the patch causes a build failure against most Xen versions:

In file included from 
/local/scratch/sstabellini/qemu/include/hw/xen/xen_backend.h:4:0,
 from hw/block/xen_disk.c:38:
/local/scratch/sstabellini/qemu/include/hw/xen/xen_common.h:445:37: error: 
unknown type name ‘xc_domain_configuration_t’
  CChw/bt/hci-csr.o
make: *** [hw/block/xen_disk.o] Error 1___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH QEMU v2 2/2] xen: fix usage of xc_domain_create in domain builder

[Ian Campbell]

On Fri, 2015-11-13 at 13:46 +, Stefano Stabellini wrote:
> On Fri, 13 Nov 2015, Roger Pau Monne wrote:
> > Due to the addition of HVMlite and the requirement to always provide a
> > valid
> > xc_domain_configuration_t, xc_domain_create now always takes an arch
> > domain
> > config, which can be NULL in order to mimic previous behaviour.
> > 
> > Add a small stub called xen_domain_create that encapsulates the correct
> > call
> > to xc_domain_create depending on the libxc version detected.
> > 
> > Signed-off-by: Roger Pau Monné 
> 
> FYI this is going to conflict with Ian's series:
> 
> 1447070487-31229-1-git-send-email-ian.campb...@citrix.com

The bit being patched here is disabled by
"xen: make it possible to build without the Xen PV domain builder" in that
series.

In any case I think Roger's stuff would be better off going in first and I
can easily rebase over it.

> In addition, the patch causes a build failure against most Xen versions:
> 
> In file included from
> /local/scratch/sstabellini/qemu/include/hw/xen/xen_backend.h:4:0,
>  from hw/block/xen_disk.c:38:
> /local/scratch/sstabellini/qemu/include/hw/xen/xen_common.h:445:37:
> error: unknown type name ‘xc_domain_configuration_t’
>   CChw/bt/hci-csr.o
> make: *** [hw/block/xen_disk.o] Error 1

Given the only caller today passes NULL I'd suggest moving the NULL down
into the wrapper, i.e. dropping the argument from xen_domain_create.

If someone wants to resurrect the domain build in QEMU _and_ teach it to do
pvh, then they will surely be able to refactor this to suit their needs.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 6/7] xen/arm: set the system time in Xen via the XENPF_settime64 hypercall

[Julien Grall]

On 12/11/15 17:30, Stefano Stabellini wrote:
> +static int xen_pvclock_gtod_notify(struct notifier_block *nb,
> +unsigned long was_set, void *priv)
> +{
> + /* Protected by the calling core code serialization */
> + static struct timespec64 next_sync;
> +
> + struct xen_platform_op op;
> + struct timespec64 now, system_time;
> + struct timekeeper *tk = priv;
> +
> + now.tv_sec = tk->xtime_sec;
> + now.tv_nsec = (long)(tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift);
> + system_time = timespec64_add(now, tk->wall_to_monotonic);
> +
> + /*
> +  * We only take the expensive HV call when the clock was set
> +  * or when the 11 minutes RTC synchronization time elapsed.
> +  */
> + if (!was_set && timespec64_compare(&now, &next_sync) < 0)
> + return NOTIFY_OK;
> +
> + op.interface_version = XENPF_INTERFACE_VERSION;
> + op.cmd = XENPF_settime64;
> + op.u.settime64.mbz = 0;
> + op.u.settime64.secs = now.tv_sec;
> + op.u.settime64.nsecs = now.tv_nsec;
> + op.u.settime64.system_time = timespec64_to_ns(&system_time);
> + printk("GTOD: Setting to %llu.%09u at %llu\n",
> +op.u.settime64.secs,
> +op.u.settime64.nsecs,
> +op.u.settime64.system_time);

Is this printk really useful?

> + (void)HYPERVISOR_platform_op(&op);
> +
> + /*
> +  * Move the next drift compensation time 11 minutes
> +  * ahead. That's emulating the sync_cmos_clock() update for
> +  * the hardware RTC.
> +  */
> + next_sync = now;
> + next_sync.tv_sec += 11 * 60;
> +
> + return NOTIFY_OK;
> +}
> +
> +static struct notifier_block xen_pvclock_gtod_notifier = {
> + .notifier_call = xen_pvclock_gtod_notify,
> +};
> +
>  static void xen_percpu_init(void)
>  {
>   struct vcpu_register_vcpu_info info;
> @@ -313,7 +364,9 @@ static int __init xen_guest_init(void)
>  
>   pv_time_ops.steal_clock = xen_stolen_accounting;
>   static_key_slow_inc(¶virt_steal_enabled);
> -
> + if (xen_initial_domain())
> + pvclock_gtod_register_notifier(&xen_pvclock_gtod_notifier);
> + 

I think, you've introduced a trailing whitespace.

>   return 0;
>  }
>  early_initcall(xen_guest_init);
> 

Regards,


-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCHv3] 03/28] arm: drop now redefined CONFIG_64BIT

[Doug Goldstein]

On 11/13/15 2:07 AM, Jan Beulich wrote:
 On 12.11.15 at 23:54,  wrote:
>> The switch to Kconfig provides variables prefixed with CONFIG_ with
>> results in a redefinition of this variable.
> 
> So I can't see how this can be a separate patch: Either the
> redefinition causes a build failure after the previous patch (if
> CONFIG_64BIT gets introduced before the one here) or the
> symbol is now undefined (if the new instance gets introduced
> later).
> 
> Jan
> 
> 

I can squash this up. You are right it makes more sense.


-- 
Doug Goldstein



signature.asc
Description: OpenPGP digital signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1 02/11] xen/xsplice: Hypervisor implementation of XEN_XSPLICE_op

[Konrad Rzeszutek Wilk]

On Thu, Nov 12, 2015 at 09:28:36AM -0700, Jan Beulich wrote:
> >>> On 03.11.15 at 19:15,  wrote:
> > Signed-off-by: Ross Lagerwall 
> > ---
> > v2: Rebased on keyhandler: rework keyhandler infrastructure
> > v3: Fixed XSM.
> > v4: Removed REVERTED state.
> > Split status and error code.
> > Add REPLACE action.
> > Separate payload data from the payload structure.
> > s/XSPLICE_ID_../XSPLICE_NAME_../
> 
> Odd - the subject says v1.
> 
> > --- /dev/null
> > +++ b/xen/common/xsplice.c
> > @@ -0,0 +1,398 @@
> > +/*
> > + * Copyright (c) 2015 Oracle and/or its affiliates. All rights reserved.
> > + *
> > + */
> > +
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> 
> No use of this header except in code shared with the tools.
> 
> Also I'd like to encourage you to sort all xen/ headers in a file
> (and all public/ and asm/ ones, just that this doesn't apply here)
> alphabetically in new code.

I tried sorting it. I couldn't build the file. And then I put
it on the 'TODO yak-shaving pile' and hadn't yet touched it.

There has to be some automatic tool to help with figuring the
header dependencies for all the headers, not just the ones
I am uisng.
Not exactly sure how to make all of the head
> 
> > +}
> > +
> > +
> > +static int verify_payload(xen_sysctl_xsplice_upload_t *upload)
> 
> Double blank line above.
> 
> > +/*
> > + * We MUST be holding the spinlock.
> > + */
> 
> Which spinlock? Also this is a single line comment.
> 
> > +static void __free_payload(struct payload *data)
> 
> I see no reason for this function to have two underscores at the
> beginning of its name.
> 
> 
> > +err_raw:
> > +free_xenheap_pages(raw_data, get_order_from_bytes(upload->size));
> > +err_data:
> 
> Labels indented by at least one blank please.
> 
> > +static int xsplice_list(xen_sysctl_xsplice_list_t *list)
> > +{
> > +xen_xsplice_status_t status;
> > +struct payload *data;
> > +unsigned int idx = 0, i = 0;
> > +int rc = 0;
> > +unsigned int ver = payload_version;
> > +
> > +if ( list->nr > 1024 )
> > +return -E2BIG;
> > +
> > +if ( list->pad != 0 )
> > +return -EINVAL;
> > +
> > +if ( guest_handle_is_null(list->status) ||
> > + guest_handle_is_null(list->id) ||
> > + guest_handle_is_null(list->len) )
> > +return -EINVAL;
> 
> ???
> 
> > +if ( !guest_handle_okay(list->status, sizeof(status) * list->nr) ||
> > + !guest_handle_okay(list->id, XEN_XSPLICE_NAME_SIZE * list->nr) ||
> > + !guest_handle_okay(list->len, sizeof(uint32_t) * list->nr) )
> > +return -EINVAL;
> > +
> > +spin_lock(&payload_list_lock);
> > +if ( list->idx > payload_cnt )
> > +{
> > +spin_unlock(&payload_list_lock);
> > +return -EINVAL;
> > +}
> > +
> > +list_for_each_entry( data, &payload_list, list )
> > +{
> > +uint32_t len;
> > +
> > +if ( list->idx > i++ )
> > +continue;
> > +
> > +status.state = data->state;
> > +status.rc = data->rc;
> > +len = strlen(data->id);
> > +
> > +/* N.B. 'idx' != 'i'. */
> > +if ( copy_to_guest_offset(list->id, idx * XEN_XSPLICE_NAME_SIZE,
> > +  data->id, len) ||
> > + copy_to_guest_offset(list->len, idx, &len, 1) ||
> > + copy_to_guest_offset(list->status, idx, &status, 1) )
> 
> You having used guest_handle_okay() above, all of these can be
> __copy_to_guest_offset)() afaict.
> 
> > +{
> > +rc = -EFAULT;
> > +break;
> > +}
> > +idx ++;
> 
> Spurious blank.
> 
> > +if ( hypercall_preempt_check() || (idx + 1 > list->nr) )
> > +{
> > +break;
> > +}
> 
> Pointless braces.
> 
> Also - what about an input list->nr of zero?

Duh! Right.
> 
> > +}
> > +list->nr = payload_cnt - i; /* Remaining amount. */
> > +spin_unlock(&payload_list_lock);
> > +list->version = ver;
> > +
> > +/* And how many we have processed. */
> > +return rc ? rc : idx;
> 
> Please omit the middle expression in cases like this. To be honest I
> can't help myself thinking that I'v already made at least some of
> these comments.

You did. I hadn't had a chance to address them. Sorry about you
wasting your time and not addressing them in the code yet.

> 
> > --- a/xen/include/public/sysctl.h
> > +++ b/xen/include/public/sysctl.h
> > @@ -766,6 +766,161 @@ struct xen_sysctl_tmem_op {
> >  typedef struct xen_sysctl_tmem_op xen_sysctl_tmem_op_t;
> >  DEFINE_XEN_GUEST_HANDLE(xen_sysctl_tmem_op_t);
> >  
> > +/*
> > + * XEN_SYSCTL_XSPLICE_op
> > + *
> > + * Refer to the docs/misc/xsplice.markdown for the design details
> > + * of this hypercall.
> 
> To someone importing this header into another project, this
> reference may be quite odd. Don't these get translated to
> some html with a canonical place on the web?
> 
> > +struct x

Re: [Xen-devel] [PATCH] xen: sched: fix (ACPI S3) resume with cpupools with different schedulers.

[Dario Faggioli]

On Fri, 2015-11-13 at 05:52 -0700, Jan Beulich wrote:
>>> On 13.11.15 at 13:30,  wrote:

> > cpu_schedule_up() allocates, with the alloc_pdata hook, stuff that
> > was
> > freed by cpu_schedule_down (with the free_pdata hook) already.
> 
> If there is no change in the allocation pattern (i.e. including the
> order of frees and allocs), then I'm sorry for the noise. I
> understood
> the description of the change differently then.
> 
There are changes. So, here it is the situation.

 Right now:   After this patch:
 --
 * Boot / CPU switch pools:   * Boot / CPU switch pools:
cpu_schedule_up():   cpu_schedule_up():=
 v0 = alloc_vdata()   v0 = alloc_vdata()
 p0 = alloc_pdata()   p0 = alloc_pdata()

 * Suspend:   * Suspend:
cpu_schedule_down()  cpu_schedule_down():
 free_pdata(p0)   free_vdata(v0)
  free_pdata(p0) [x]

 * Resume:* Resume:
cpu_schedule_up()cpu_schedule_up():
 p1 = alloc_pdata()   v1 = alloc_vdata()
  p1 = alloc_pdata() [xx]
   [*]
schedule_cpu_switch()schedule_cpu_switch():
 p2 = alloc_pdata()   p2 = alloc_pdata()
 v1 = alloc_vdata()   v2 = alloc_vdata()
 free_vdata(v0) [**]  free_vdata(v1)
 free_pdata(p1)   free_pdata(p1)

[*] BUG, the one described in the changelog, if scheduling happens at 
this point (I can reproduce this with 100% reliability)

[*] LATENT BUG, as it is possible that the free_vdata() function 
   
called here is not the counterpart of the alloc_vdata() function 
   
used for allocating v0

So, there are allocations already in the resume path. With this patch,
there is one more free in the suspend path and one more alloc in the
resume path.

I can change the order of operations at [x] and/or [xx] (and send a
v2), e.g., make things look like this, if that helps,:

* Suspend:
   cpu_schedule_down():
free_pdata(p0)
free_vdata(v0)

* Resume:
   cpu_schedule_up():
v1 = alloc_vdata()
p1 = alloc_pdata()

Would that work?

I don't think I neither can easily get away with the bug, nor eliminate
the explained inconsistency, without that additional free-alloc pair.

> > In any case, if one of these allocations fails, it already returns
> > ENOMEM. What happens then is, of course, caller's call. If it
> > happens
> > during boot for pCPU 0, we BUG(). If it happens with non-boot pCPUs
> > (either at boot, during hotplug or during suspend/resume), we fail
> > the
> > bringup of that pCPU at the CPU_UP_PREPARE phase.
> 
> Except that after resume the system should get into the same state
> it was before suspend, i.e. in particular same number of CPUs.
>
And those same CPUs being in the same pools, which is where this all
springs from.

That being said, I totally agree. But if something unexpected happens
(which is what we are talking about), like being in lack of memory, is
there any value in "limiting the damage"? I think there is, and that is
why I looked at and reported the failure path in details, when
answering your "what if alloc fail" question.

If there is no point in that, why do we handle the error and rollback
the CPU bringup in cpu_up() at all? We could just, at least in case of
resume, BUG_ON() every time we don't get a NOTIFY_DONE result... 

>  Imo
> there simply shouldn't be any allocations on the resume path -
> everything should be possible to be picked up as it was left during
> suspend. 
>
Agreed again. I'm not sure to what extent we can apply this to
scheduling per pCPU and per vCPU data. For example, per vCPU data was
not being deallocated (before my patch, I mean), but per pCPU data was.

I'm not sure why exactly this was the case in the first place, but I
guess it has to do with the fact that we, OTOH, want to actually
deallocate as much stuff as possible in the case of CPU hot unplug, and
parts of this code is shared between the two.

But it is also, IMO, related to the above reasoning on error handling.
I mean, if we just don't kill the per pCPU data the scheduler keeps for
CPU x, and then, during resume, CPU x fails to come up, what happens?
We risk leaking that data or, worse, failing at making the schedule
aware that he does not really have CPU x available any longer.

So, again, if we value "graceful" error compensation, I see some of
these re-allocation necessary. Perhaps, the work I'm doing in another
series of separating scheduler per pCPU data allocation and
initialization can be helpful (not against the risk of leaks, though).

> > Fact is, there is an inconsistency between the pCPU's scheduler
> > (set in
> > cpu_schedule_up(), to Credit1, in the example) and the pCPU's idle
> > vCPU's private data (Credit2 data in the example), which, if left
> > in
> > place, could explode somewhere else, at some future point. In my
> > 

Re: [Xen-devel] [PATCH v5 3/6] xen/arm: vgic-v2: Don't ignore a write in ITARGETSR if one field is 0

[Stefano Stabellini]

On Mon, 9 Nov 2015, Julien Grall wrote:
> The current implementation ignores the whole write if one of the field is
> 0. Although, based on the spec (4.3.12 IHI 0048B.b), 0 is a valid value
> when:
> - The interrupt is not wired in the distributor. From the Xen
> point of view, it means that the corresponding bit is not set in
> d->arch.vgic.allocated_irqs.
> - The user wants to disable the IRQ forwarding in the distributor.
> I.e the IRQ stays pending in the distributor and never received by
> the guest.
> 
> Implementing the later will require more work in Xen because we always
> assume the interrupt is forwarded to a valid vCPU. So for now, ignore
> any field where the value is 0.
> 
> The emulation of the write access of ITARGETSR has been reworked and
> moved to a new function because it would have been difficult to
> implement properly the behavior with the current code.
> 
> The new implementation is breaking the register in 4 distinct bytes. For
> each byte, it will check the validity of the target list, find the new
> target, migrate the interrupt and store the value if necessary.
> 
> In the new implementation there is nearly no distinction of the access
> size to avoid having too many different path which is harder to test.
> 
> Signed-off-by: Julien Grall 
> 
> ---
> This change used to be embedded in "xen/arm: vgic: Optimize the way
> to store the target vCPU in the rank". It has been moved out to
> avoid having too much functional changes in a single patch.
> 
> I'm not sure if this patch should be backported to Xen 4.6. Without
> it any guest writing 0 in one the field won't be able to migrate
> other interrupts. Although, in all the use case I've seen, the guest
> is read ITARGETSR first and write-back the value with the
> corresponding byte changed.
> 
> Note that NR_BITS_PER_TARGET has been kept as a define because it
> will be use in another function in the next patch and I wanted to
> avoid having to duplicate const var = ...
> 
> Changes in v5:
> - s/NR_TARGET_PER_REG/NR_TARGETS_PER_ITARGETSR/
> - s/NR_BIT_PER_TARGET/NR_BITS_PER_TARGET/
> - Compute NR_BITS_PER_TARGET based on NR_TARGETS_PER_ITARGETSR
> - Typoes in the comments
> - Compute the shift for itargetsr on every loop
> - Use gprintk(XENLOG_WARNING,...)
> - Clarify commit message
> 
> Changes in v4:
> - Patch added
> ---
>  xen/arch/arm/vgic-v2.c | 145 
> ++---
>  1 file changed, 102 insertions(+), 43 deletions(-)
> 
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index 486e497..ad2ea0a 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -57,6 +57,98 @@ void vgic_v2_setup_hw(paddr_t dbase, paddr_t cbase, 
> paddr_t csize,
>  vgic_v2_hw.aliased_offset = aliased_offset;
>  }
>  
> +#define NR_TARGETS_PER_ITARGETSR4U
> +#define NR_BITS_PER_TARGET  (32U / NR_TARGETS_PER_ITARGETSR)
> +
> +/*
> + * Store an ITARGETSR register. This function only deals with ITARGETSR8
> + * and onwards.
> + *
> + * Note the offset will be aligned to the appropriate boundary.
> + */
> +static void vgic_store_itargetsr(struct domain *d, struct vgic_irq_rank 
> *rank,
> + unsigned int offset, uint32_t itargetsr)
> +{
> +unsigned int i;
> +unsigned int regidx = REG_RANK_INDEX(8, offset, DABT_WORD);
> +unsigned int virq;
> +
> +ASSERT(spin_is_locked(&rank->lock));
> +
> +/*
> + * The ITARGETSR0-7, used for SGIs/PPIs, are implemented RO in the
> + * emulation and should never call this function.
> + *
> + * They all live in the first rank.
> + */
> +BUILD_BUG_ON(NR_INTERRUPT_PER_RANK != 32);
> +ASSERT(rank->index >= 1);
> +
> +offset &= INTERRUPT_RANK_MASK;
> +offset &= ~(NR_TARGETS_PER_ITARGETSR - 1);
> +
> +virq = rank->index * NR_INTERRUPT_PER_RANK + offset;

The patch looks good, but these three lines I think could be replaced
with:

virq = offset & ~(NR_TARGETS_PER_ITARGETSR - 1);

isn't it right?



> +for ( i = 0; i < NR_TARGETS_PER_ITARGETSR; i++, offset++, virq++ )

offset is not needed in the loop


> +{
> +unsigned int new_target, old_target;
> +uint8_t new_mask, old_mask;
> +
> +/*
> + * Don't need to mask as we rely on new_mask to fit for only one
> + * target.
> + */
> +BUILD_BUG_ON((sizeof (new_mask) * 8) != NR_BITS_PER_TARGET);
> +
> +new_mask = itargetsr >> (i * NR_BITS_PER_TARGET);
> +old_mask = vgic_byte_read(rank->v2.itargets[regidx], i);
> +
> +/*
> + * SPIs are using the 1-N model (see 1.4.3 in ARM IHI 0048B).
> + * While the interrupt could be set pending to all the vCPUs in
> + * target list, it's not guaranteed by the spec.
> + * For simplicity, always route the vIRQ to the first interrupt
> +