[Xen-devel] [PATCH] x86/hvm: Allow guest_request vm_events coming from userspace

[Razvan Cojocaru]

Allow guest userspace code to request that a vm_event be sent out
via VMCALL. This functionality seems to be handy for a number of
Xen developers, as stated on the mailing list (thread "[Xen-devel]
HVMOP_guest_request_vm_event only works from guest in ring0").

Signed-off-by: Razvan Cojocaru 
---
 xen/arch/x86/hvm/hvm.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 202866a..2d38936 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -5349,8 +5349,14 @@ int hvm_do_hypercall(struct cpu_user_regs *regs)
 switch ( mode )
 {
 case 8:
+if ( eax == __HYPERVISOR_hvm_op &&
+ regs->rdi == HVMOP_guest_request_vm_event )
+break;
 case 4:
 case 2:
+if ( eax == __HYPERVISOR_hvm_op &&
+ regs->_ebx == HVMOP_guest_request_vm_event )
+break;
 hvm_get_segment_register(curr, x86_seg_ss, &sreg);
 if ( unlikely(sreg.attr.fields.dpl) )
 {
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 04/25] arm/altp2m: Move hostp2m init/teardown to individual functions.

[Sergej Proskurin]

Hi Julien,


On 08/03/2016 07:40 PM, Julien Grall wrote:
> Hello Sergej,
>
> Title: s/altp2m/p2m/ and please drop the full stop.

Ok.

>
> On 01/08/16 18:10, Sergej Proskurin wrote:
>> This commit pulls out generic init/teardown functionality out of
>> p2m_init and p2m_teardown into p2m_init_one, p2m_free_one, and
>> p2m_flush_table functions.  This allows our future implementation to
>> reuse existing code for the initialization/teardown of altp2m views.
>
> Please avoid to mix-up code movement and new additions. This makes the
> code more difficult to review.

I will remove unnecessary code movements.

>
> Also, you don't mention the new changes in the commit message.

Since this patch is new to the patch series (the patch that got split is
#07, where I have commented the changes), I did not add any but rather
described the patch without being specific to the patch version.

>
> After reading the patch, it should really be divided and explain why
> you split like that.
>

Ok.

>>
>> Signed-off-by: Sergej Proskurin 
>> ---
>> Cc: Stefano Stabellini 
>> Cc: Julien Grall 
>> ---
>> v2: Added the function p2m_flush_table to the previous version.
>> ---
>>  xen/arch/arm/p2m.c| 74
>> +--
>>  xen/include/asm-arm/p2m.h | 11 +++
>>  2 files changed, 70 insertions(+), 15 deletions(-)
>>
>> diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
>> index cbc64a1..17f3299 100644
>> --- a/xen/arch/arm/p2m.c
>> +++ b/xen/arch/arm/p2m.c
>> @@ -1360,50 +1360,94 @@ static void p2m_free_vmid(struct domain *d)
>>  spin_unlock(&vmid_alloc_lock);
>>  }
>>
>> -void p2m_teardown(struct domain *d)
>> +/* Reset this p2m table to be empty */
>> +void p2m_flush_table(struct p2m_domain *p2m)
>
> Any function exported should have its prototype in an header within
> the same patch.
>

I will change that.

>>  {
>> -struct p2m_domain *p2m = &d->arch.p2m;
>> -struct page_info *pg;
>> +struct page_info *page, *pg;
>> +unsigned int i;
>> +
>> +page = p2m->root;
>
>
> This function can be called with p2m->root equal to NULL. (see the
> check in p2m_free_one.
>

I will add the check, thank you.

>> +
>> +/* Clear all concatenated first level pages */
>> +for ( i = 0; i < P2M_ROOT_PAGES; i++ )
>> +clear_and_clean_page(page + i);
>>
>> +/* Free the rest of the trie pages back to the paging pool */
>>  while ( (pg = page_list_remove_head(&p2m->pages)) )
>>  free_domheap_page(pg);
>> +}
>> +
>> +static inline void p2m_free_one(struct p2m_domain *p2m)
>
> Why inline here? Also, it seems that you export the function later.
> Why don't you do it here?
>

I will do that. Thank you.

> Finally, I think this function should be rename p2m_teardown_one to
> match the callers' name.
>

Ok.

>> +{
>> +p2m_flush_table(p2m);
>> +
>> +/* Free VMID and reset VTTBR */
>> +p2m_free_vmid(p2m->domain);
>
> Why do you move the call to p2m_free_vmid?
>

When flushing a table, I did not want to free the associated VMID, as it
would need to be allocated right afterwards (see altp2m_propagate_change
and altp2m_reset). Since this would need to be done also in functions
like p2m_altp2m_reset, I moved this call to p2m_free_one. I believe
there is no need to free the VMIDs if the associated p2m is not freed as
well.

>> +p2m->vttbr.vttbr = INVALID_VTTBR;
>
> Why do you reset vttbr, the p2m will never be used afterwards.
>

Fair. I did that just for the sake of completeness.

>>
>>  if ( p2m->root )
>>  free_domheap_pages(p2m->root, P2M_ROOT_ORDER);
>>
>>  p2m->root = NULL;
>>
>> -p2m_free_vmid(d);
>> -
>>  radix_tree_destroy(&p2m->mem_access_settings, NULL);
>>  }
>>
>> -int p2m_init(struct domain *d)
>> +int p2m_init_one(struct domain *d, struct p2m_domain *p2m)
>
> Any function exported should have its prototype in an header within
> the same patch.
>

I will change that, thank you.

>>  {
>> -struct p2m_domain *p2m = &d->arch.p2m;
>>  int rc = 0;
>>
>>  rwlock_init(&p2m->lock);
>>  INIT_PAGE_LIST_HEAD(&p2m->pages);
>>
>> -p2m->vmid = INVALID_VMID;
>> -
>
> Why this is dropped?
>

This will be shown in patch #07. We reuse altp2m views and check whether
a p2m was flushed by checking for a valid VMID.

>>  rc = p2m_alloc_vmid(d);
>>  if ( rc != 0 )
>>  return rc;
>>
>> -p2m->max_mapped_gfn = _gfn(0);
>> -p2m->lowest_mapped_gfn = _gfn(ULONG_MAX);
>> -
>> -p2m->default_access = p2m_access_rwx;
>> +p2m->domain = d;
>> +p2m->access_required = false;
>>  p2m->mem_access_enabled = false;
>> +p2m->default_access = p2m_access_rwx;
>> +p2m->root = NULL;
>> +p2m->max_mapped_gfn = _gfn(0);
>> +p2m->lowest_mapped_gfn = INVALID_GFN;
>
> Please don't move code when it is not necessary. This make the code
> review more difficult to read.
>

Ok.

>> +p2m->vttbr.vttbr = INVALID_VTTBR;
>>  radix_tree_init(&p2m->mem_access_settings);
>>
>> -rc = p2m_alloc_

Re: [Xen-devel] [PATCH] x86/hvm: Allow guest_request vm_events coming from userspace

[Jan Beulich]

>>> On 05.08.16 at 09:08,  wrote:
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -5349,8 +5349,14 @@ int hvm_do_hypercall(struct cpu_user_regs *regs)
>  switch ( mode )
>  {
>  case 8:
> +if ( eax == __HYPERVISOR_hvm_op &&
> + regs->rdi == HVMOP_guest_request_vm_event )
> +break;
>  case 4:
>  case 2:
> +if ( eax == __HYPERVISOR_hvm_op &&
> + regs->_ebx == HVMOP_guest_request_vm_event )
> +break;

For one, Coverity will choke on there not being a fall-through
annotation. And then the resulting fall-through behavior is now
wrong: You don't want the 64-bit case to also do the 16-/32-bit
check.

Finally, as indicated before, such a special casing model doesn't
scale well, i.e. as soon as further exception would get suggested
the code would quickly become hard to understand and maintain
(as is already being supported by the issue pointed out above,
addressing of which will likely further convolute this code).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 04/25] arm/altp2m: Move hostp2m init/teardown to individual functions.

[Julien Grall]

On 05/08/16 08:26, Sergej Proskurin wrote:

Hi Julien,


Hello Sergej,



On 08/03/2016 07:40 PM, Julien Grall wrote:


[...]


+{
+p2m_flush_table(p2m);
+
+/* Free VMID and reset VTTBR */
+p2m_free_vmid(p2m->domain);


Why do you move the call to p2m_free_vmid?



When flushing a table, I did not want to free the associated VMID, as it
would need to be allocated right afterwards (see altp2m_propagate_change
and altp2m_reset). Since this would need to be done also in functions
like p2m_altp2m_reset, I moved this call to p2m_free_one. I believe
there is no need to free the VMIDs if the associated p2m is not freed as
well.


That does not answer my question. You moved p2m_free_vmid within 
p2m_free_one. It used to be at the end of the function and now it is at 
the beginning. See...



+p2m->vttbr.vttbr = INVALID_VTTBR;


[...]



 if ( p2m->root )
 free_domheap_pages(p2m->root, P2M_ROOT_ORDER);

 p2m->root = NULL;

-p2m_free_vmid(d);
-


here. So please don't move code unless there is a good reason. This 
series is already quite difficult to read.



 radix_tree_destroy(&p2m->mem_access_settings, NULL);
 }

-int p2m_init(struct domain *d)
+int p2m_init_one(struct domain *d, struct p2m_domain *p2m)


Any function exported should have its prototype in an header within
the same patch.



I will change that, thank you.


 {
-struct p2m_domain *p2m = &d->arch.p2m;
 int rc = 0;

 rwlock_init(&p2m->lock);
 INIT_PAGE_LIST_HEAD(&p2m->pages);

-p2m->vmid = INVALID_VMID;
-


Why this is dropped?



This will be shown in patch #07. We reuse altp2m views and check whether
a p2m was flushed by checking for a valid VMID.


Which is wrong. A flushed altp2m should not need to be reinitialized 
(i.e reseting the lock...).


A flushed altp2m only need to have max_mapped_gfn and lowest_mapped_gfn 
reset.


[...]


+p2m->vttbr.vttbr = INVALID_VTTBR;
 radix_tree_init(&p2m->mem_access_settings);

-rc = p2m_alloc_table(d);
-


The function p2m_init_one should fully initialize the p2m (i.e
allocate the table).



The function p2m_init_one is currently called also for initializing the
hostp2m, which is not dynamically allocated. Since we are sharing code
between the hostp2m and altp2m initialization, I solved it that way. We
could always allocate the hostp2m dynamically, that would solve it quite
easily without additional checks. The other solution can simply check,
whether the p2m is NULL and perform additional p2m allocation. What do
you think?


I don't understand your point here. It does not matter whether the 
hostp2m is allocated dynamically or not.


p2m_init_one should fully initialize a p2m and this does not depends on 
dynamic allocation.


[...]


diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
index 5c7cd1a..1f9c370 100644
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -18,6 +18,11 @@ struct domain;

 extern void memory_type_changed(struct domain *);

+typedef enum {
+p2m_host,
+p2m_alternate,
+} p2m_class_t;
+


This addition should really be in a separate patch.



The function p2m_init_hostp2m uses p2m_host for initialization. I can
introduce a patch before this one, however to make it easier for the
reviewer.


You did not get my point here. A patch should do one thing: moving code 
or adding new code. Not both at the same.


Adding p2m_class_t and setting p2m_host should really be done in a 
separate patch. This will ease the review this patch series.


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 07/25] arm/altp2m: Add altp2m init/teardown routines.

[Julien Grall]

On 05/08/16 07:53, Sergej Proskurin wrote:

Hi Julien,


Hello Sergej,


On 08/03/2016 08:12 PM, Julien Grall wrote:

On 01/08/16 18:10, Sergej Proskurin wrote:

+int altp2m_init(struct domain *d)
+{
+unsigned int i;
+
+spin_lock_init(&d->arch.altp2m_lock);
+
+for ( i = 0; i < MAX_ALTP2M; i++ )
+{
+d->arch.altp2m_p2m[i] = NULL;
+d->arch.altp2m_vttbr[i] = INVALID_VTTBR;


I don't think altp2m_vttbr is useful. There is no real performance
impact to free the whole altp2m if the altp2m is destroyed (see
altp2m_destroy_by_id) and re-allocated afterwards.

The code will actually much simpler. With this solution you will be
able to detect if an altp2m is available by testin altp2m_p2m[i] is NULL.



This is true. I did not want to free the entire domain got every time
the hostp2m got changed, while altp2m was active (see
altp2m_propagate_change). But it would not introduce much more overhead
when it does. Thank you.


I think you misunderstood my point. When you flush the altp2m you only 
need to reset lowest_mapped_gfn and max_mapped_gfn aside freeing 
intermediate page table.


The altp2m should be fully freed when it get destroyed, nobody will use 
it anyway. This will also simplify a lot the logic.


Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC 00/04] xen: arm64: Support VM save/restore on arm64

[Julien Grall]

On 05/08/16 03:54, Chenxiao Zhao wrote:
> Hi all,

Hello Chenxiao,

Thank you for posting your work on the ML. I have noticed
that your patches did not show up as a single thread in my
mailbox.

git-send-email should do it for you if you pass all the patches 
in one go. Below what the command I using to send on xen-devel:

git send-email *.patch --to xen-devel@lists.xen.org  

> This patch added support VM save/restore for arm64. It is based on
> previous work done by Ian Campbell [1] with some bug fixes to make it
> work on stable-4.7. You should apply Ian's patch first.
> 
> There still some known issues that have not fixed yet.
> 
> * GIC v2 not support
> * No live migration

Ian series did not implement live migration too. It is a bigger
task and I think is a not a requisite for this series.

> * does not support VM with multiple CPU cores
> 
> 
> [1]
> https://lists.xenproject.org/archives/html/xen-devel/2015-12/msg01053.html
> 

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [libvirt test] 99955: tolerable FAIL - PUSHED

[osstest service owner]

flight 99955 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99955/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass

version targeted for testing:
 libvirt  d5813d72ad2bd8f9735b96bafe2ba350f74b3994
baseline version:
 libvirt  e584615b81b5cabb252b1866171bde25b4f06d05

Last test of basis99936  2016-08-04 04:24:38 Z1 days
Testing same since99955  2016-08-05 04:21:10 Z0 days1 attempts


People who touched revisions under test:
  Andrea Bolognani 
  Boris Fiuczynski 
  Daniel P. Berrange 
  John Ferlan 
  Michal Privoznik 
  Peter Krempa 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmpass
 test-amd64-amd64-libvirt-xsm pass
 test-armhf-armhf-libvirt-xsm fail
 test-amd64-i386-libvirt-xsm  pass
 test-amd64-amd64-libvirt pass
 test-armhf-armhf-libvirt fail
 test-amd64-i386-libvirt  pass
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair pass
 test-armhf-armhf-libvirt-qcow2   fail
 test-armhf-armhf-libvirt-raw fail
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=libvirt
+ revision=d5813d72ad2bd8f9735b96bafe2ba350f74b3994
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push libvirt 
d5813d7

Re: [Xen-devel] [PATCH RFC 06/12] xen/x86: populate PVHv2 Dom0 physical memory map

[Roger Pau Monne]

On Thu, Aug 04, 2016 at 07:43:39PM +0100, Andrew Cooper wrote:
> On 02/08/16 10:19, Roger Pau Monne wrote:
> > On Fri, Jul 29, 2016 at 08:04:12PM +0100, Andrew Cooper wrote:
> >> On 29/07/16 17:29, Roger Pau Monne wrote:
> >>> +/* Calculate the biggest usable order given a size in bytes. */
> >>> +static inline unsigned int get_order(uint64_t size)
> >>> +{
> >>> +unsigned int order;
> >>> +uint64_t pg;
> >>> +
> >>> +ASSERT((size & ~PAGE_MASK) == 0);
> >>> +pg = PFN_DOWN(size);
> >>> +for ( order = 0; pg >= (1 << (order + 1)); order++ );
> >>> +
> >>> +return order;
> >>> +}
> >> We already have get_order_from_bytes() and get_order_from_pages(), the
> >> latter of which looks like it will suit your usecase.
> > Not really, or at least they don't do the same as get_order. This function 
> > calculates the maximum order you can use so that there are no pages left 
> > over, (ie: if you have a size of 3145728bytes (3MiB), this function will 
> > return order 9 (2MiB), while the other ones will return order 10 (4MiB)). I 
> > don't really understand while other places in code request bigger orders 
> > and 
> > then free the leftovers, isn't this also causing memory shattering?
> 
> Sounds like we want something like get_order_{floor,ceil}() which makes
> it obvious which way non-power-of-two get rounded.

Right, that makes sense, will rename the current one to ceil, and add the 
floor variant.

> >>> +if ( order == 0 && memflags )
> >>> +{
> >>> +/* Try again without any memflags. */
> >>> +memflags = 0;
> >>> +order = MAX_ORDER;
> >>> +continue;
> >>> +}
> >>> +if ( order == 0 )
> >>> +panic("Unable to allocate memory with order 0!\n");
> >>> +order--;
> >>> +continue;
> >>> +}
> >> It would be far more efficient to try and allocate only 1G and 2M
> >> blocks.  Most of memory is free at this point, and it would allow the
> >> use of HAP superpage mappings, which will be a massive performance boost
> >> if they aren't shattered.
> > That's what I'm trying to do, but we might have to use pages of lower order 
> > when filling the smaller gaps.
> 
> As a general principle, we should try not to have any gaps.  This also
> extends to guests using more intelligence when deciding now to mutate
> its physmap.

Yes, but in this case we are limited by the original e820 from the host.
A DomU (without passthrough) will have all it's memory contiguously.
 
> >  As an example, this are the stats when 
> > building a domain with 6048M of RAM:
> >
> > (XEN) Memory allocation stats:
> > (XEN) Order 18: 5GB
> > (XEN) Order 17: 512MB
> > (XEN) Order 15: 256MB
> > (XEN) Order 14: 128MB
> > (XEN) Order 12: 16MB
> > (XEN) Order 10: 8MB
> > (XEN) Order  9: 4MB
> > (XEN) Order  8: 2MB
> > (XEN) Order  7: 1MB
> > (XEN) Order  6: 512KB
> > (XEN) Order  5: 256KB
> > (XEN) Order  4: 128KB
> > (XEN) Order  3: 64KB
> > (XEN) Order  2: 32KB
> > (XEN) Order  1: 16KB
> > (XEN) Order  0: 4KB
> >
> > IMHO, they are quite good.
> 
> What are the RAM characteristics of the host?  Do you have any idea what
> the hap superpage characteristics are like after the guest has booted?

This is the host RAM map:

(XEN)   - 0009c800 (usable)
(XEN)  0009c800 - 000a (reserved)
(XEN)  000e - 0010 (reserved)
(XEN)  0010 - ad662000 (usable)
(XEN)  ad662000 - adb1f000 (reserved)
(XEN)  adb1f000 - b228b000 (usable)
(XEN)  b228b000 - b2345000 (reserved)
(XEN)  b2345000 - b236a000 (ACPI data)
(XEN)  b236a000 - b2c9a000 (ACPI NVS)
(XEN)  b2c9a000 - b2fff000 (reserved)
(XEN)  b2fff000 - b300 (usable)
(XEN)  b380 - b800 (reserved)
(XEN)  f800 - fc00 (reserved)
(XEN)  fec0 - fec01000 (reserved)
(XEN)  fed0 - fed04000 (reserved)
(XEN)  fed1c000 - fed2 (reserved)
(XEN)  fee0 - fee01000 (reserved)
(XEN)  ff00 - 0001 (reserved)
(XEN)  0001 - 00024700 (usable)

No idea about the HAP superpage characteristics, how can I fetch this 
information? (I know I can dump the guest EPT tables, but that just 
saturates the console).

> In a case like this, I think it would be entirely reasonable to round up
> to the nearest 2MB, and avoid all of those small page mappings.

Hm, but then we would be expanding the RAM region and we should either 
modify the guest e820 to reflect that (which I think it's a bad idea, given 
that we might be shadowing MMIO regions), or simply use a 2MB page to cover 
a 4KB hole, in which case we are throwing away memory and the computation of 
the required memory will be off.

Roger.

Re: [Xen-devel] [PATCH v5 0/4] tools: make xenstore domain/daemon configurable

[Wei Liu]

Pushed.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] libxl: return any serial tty path in libxl_console_get_tty

[Wei Liu]

On Thu, Aug 04, 2016 at 09:07:45AM +0100, Wei Liu wrote:
> On Thu, Aug 04, 2016 at 09:07:56AM +0800, Bob Liu wrote:
> > When specifying a serial list in domain config, users of
> > libxl_console_get_tty cannot get the tty path of a second specified pty 
> > serial,
> > since right now it always returns the tty path of serial 0.
> > 
> > Signed-off-by: Bob Liu 
> 
> Acked-by: Wei Liu 
> 
> Ian, I think this is a backport candidate.
> 

Pushed.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] xen-20160804-tag

[Peter Maydell]

On 4 August 2016 at 18:44, Stefano Stabellini  wrote:
> The following changes since commit 09704e6ded83fa0bec14baf32f800f6512156ca0:
>
>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into 
> staging (2016-08-04 10:24:27 +0100)
>
> are available in the git repository at:
>
>
>   git://xenbits.xen.org/people/sstabellini/qemu-dm.git tags/xen-20160804-tag
>
> for you to fetch changes up to 0968c91ce00f42487fb11de5da38e53b5dc6bc7f:
>
>   Xen PCI passthrough: fix passthrough failure when no interrupt pin 
> (2016-08-04 10:42:48 -0700)
>
> 
> Xen 2016/08/04
>
> 
> Bruce Rogers (1):
>   Xen PCI passthrough: fix passthrough failure when no interrupt pin

Applied, thanks.

-- PMM

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH] x86/time: also use rdtsc_ordered() in check_tsc_warp()

[Jan Beulich]

This really was meant to be added in a v2 of what became commit
fa74e70500 ("x86/time: introduce and use rdtsc_ordered()").

Signed-off-by: Jan Beulich 
---
Noticed while backporting.

--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -1146,16 +1146,13 @@ static void local_time_calibration(void)
  */
 static void check_tsc_warp(unsigned long tsc_khz, unsigned long *max_warp)
 {
-#define rdtsc_barrier() mb()
 static DEFINE_SPINLOCK(sync_lock);
 static cycles_t last_tsc;
 
 cycles_t start, now, prev, end;
 int i;
 
-rdtsc_barrier();
-start = get_cycles();
-rdtsc_barrier();
+start = rdtsc_ordered();
 
 /* The measurement runs for 20 msecs: */
 end = start + tsc_khz * 20ULL;
@@ -1170,9 +1167,7 @@ static void check_tsc_warp(unsigned long
  */
 spin_lock(&sync_lock);
 prev = last_tsc;
-rdtsc_barrier();
-now = get_cycles();
-rdtsc_barrier();
+now = rdtsc_ordered();
 last_tsc = now;
 spin_unlock(&sync_lock);
 



x86/time: also use rdtsc_ordered() in check_tsc_warp()

This really was meant to be added in a v2 of what became commit
fa74e70500 ("x86/time: introduce and use rdtsc_ordered()").

Signed-off-by: Jan Beulich 
---
Noticed while backporting.

--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -1146,16 +1146,13 @@ static void local_time_calibration(void)
  */
 static void check_tsc_warp(unsigned long tsc_khz, unsigned long *max_warp)
 {
-#define rdtsc_barrier() mb()
 static DEFINE_SPINLOCK(sync_lock);
 static cycles_t last_tsc;
 
 cycles_t start, now, prev, end;
 int i;
 
-rdtsc_barrier();
-start = get_cycles();
-rdtsc_barrier();
+start = rdtsc_ordered();
 
 /* The measurement runs for 20 msecs: */
 end = start + tsc_khz * 20ULL;
@@ -1170,9 +1167,7 @@ static void check_tsc_warp(unsigned long
  */
 spin_lock(&sync_lock);
 prev = last_tsc;
-rdtsc_barrier();
-now = get_cycles();
-rdtsc_barrier();
+now = rdtsc_ordered();
 last_tsc = now;
 spin_unlock(&sync_lock);
 
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] x86/time: also use rdtsc_ordered() in check_tsc_warp()

[Andrew Cooper]

On 05/08/16 11:23, Jan Beulich wrote:
> This really was meant to be added in a v2 of what became commit
> fa74e70500 ("x86/time: introduce and use rdtsc_ordered()").
>
> Signed-off-by: Jan Beulich 

Reviewed-by: Andrew Cooper 
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 14/23] acpi: Move ACPI code to tools/libacpi

[Jan Beulich]

>>> On 04.08.16 at 23:06,  wrote:
> Signed-off-by: Boris Ostrovsky 
> Acked-by: Jan Beulich 

There's a dependency this ack has which I came too think of only now:
The move here results in a maintainership change, which should
either be avoided (by updating ./MAINTAINERS at the same time), or
we should agree on new maintainership (this stuff could likely also go
under the ACPI entry).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 14/23] acpi: Move ACPI code to tools/libacpi

[Wei Liu]

On Fri, Aug 05, 2016 at 04:43:51AM -0600, Jan Beulich wrote:
> >>> On 04.08.16 at 23:06,  wrote:
> > Signed-off-by: Boris Ostrovsky 
> > Acked-by: Jan Beulich 
> 
> There's a dependency this ack has which I came too think of only now:
> The move here results in a maintainership change, which should
> either be avoided (by updating ./MAINTAINERS at the same time), or
> we should agree on new maintainership (this stuff could likely also go
> under the ACPI entry).
> 

If Andrew and you are happy to continue maintaining ACPI related code,
I'm happy with that, too.

It is mostly code movement after all.

Wei.

> Jan
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2] domctl: relax getdomaininfo permissions

[Jan Beulich]

Qemu needs access to this for the domain it controls, both due to it
being used by xc_domain_memory_mapping() (which qemu calls) and the
explicit use in hw/xenpv/xen_domainbuild.c:xen_domain_poll(). Extend
permissions to that of any "ordinary" domctl: A domain controlling the
targeted domain can invoke this operation for that target domain (which
is being achieved by no longer passing NULL to xsm_domctl()).

This at once avoids a for_each_domain() loop when the ID of an
existing domain gets passed in.

Reported-by: Marek Marczykowski-Górecki 
Signed-off-by: Jan Beulich 
---
v2: Add a comment. Clarify description as to what additional permission
is being granted.
---
I know there had been an alternative patch suggestion, but that one
doesn't seem have seen a formal submission so far, so here is my
original proposal.

I wonder what good the duplication of the returned domain ID does: I'm
tempted to remove the one in the command-specific structure. Does
anyone have insight into why it was done that way?

I further wonder why we have XSM_OTHER: The respective conversion into
other XSM_* values in xsm/dummy.h could as well move into the callers,
making intentions more obvious when looking at the actual code.

--- a/tools/flask/policy/modules/xen.if
+++ b/tools/flask/policy/modules/xen.if
@@ -149,7 +149,7 @@ define(`device_model', `
create_channel($2, $1, $2_channel)
allow $1 $2_channel:event create;
 
-   allow $1 $2_target:domain shutdown;
+   allow $1 $2_target:domain { getdomaininfo shutdown };
allow $1 $2_target:mmu { map_read map_write adjust physmap target_hack 
};
allow $1 $2_target:hvm { getparam setparam trackdirtyvram hvmctl 
irqlevel pciroute pcilevel cacheattr send_irq };
 ')
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -396,14 +396,13 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xe
 switch ( op->cmd )
 {
 case XEN_DOMCTL_createdomain:
-case XEN_DOMCTL_getdomaininfo:
 case XEN_DOMCTL_test_assign_device:
 case XEN_DOMCTL_gdbsx_guestmemio:
 d = NULL;
 break;
 default:
 d = rcu_lock_domain_by_id(op->domain);
-if ( d == NULL )
+if ( !d && op->cmd != XEN_DOMCTL_getdomaininfo )
 return -ESRCH;
 }
 
@@ -817,14 +816,22 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xe
 
 case XEN_DOMCTL_getdomaininfo:
 {
-domid_t dom = op->domain;
-
-rcu_read_lock(&domlist_read_lock);
+domid_t dom = DOMID_INVALID;
 
-for_each_domain ( d )
-if ( d->domain_id >= dom )
+if ( !d )
+{
+ret = -EINVAL;
+if ( op->domain >= DOMID_FIRST_RESERVED )
 break;
 
+rcu_read_lock(&domlist_read_lock);
+
+dom = op->domain;
+for_each_domain ( d )
+if ( d->domain_id >= dom )
+break;
+}
+
 ret = -ESRCH;
 if ( d == NULL )
 goto getdomaininfo_out;
@@ -839,6 +846,10 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xe
 copyback = 1;
 
 getdomaininfo_out:
+/* When d was non-NULL upon entry, no cleanup is needed. */
+if ( dom == DOMID_INVALID )
+break;
+
 rcu_read_unlock(&domlist_read_lock);
 d = NULL;
 break;
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -61,7 +61,12 @@ static always_inline int xsm_default_act
 return 0;
 case XSM_TARGET:
 if ( src == target )
+{
 return 0;
+case XSM_XS_PRIV:
+if ( src->is_xenstore )
+return 0;
+}
 /* fall through */
 case XSM_DM_PRIV:
 if ( target && src->target == target )
@@ -71,10 +76,6 @@ static always_inline int xsm_default_act
 if ( src->is_privileged )
 return 0;
 return -EPERM;
-case XSM_XS_PRIV:
-if ( src->is_xenstore || src->is_privileged )
-return 0;
-return -EPERM;
 default:
 LINKER_BUG_ON(1);
 return -EPERM;


domctl: relax getdomaininfo permissions

Qemu needs access to this for the domain it controls, both due to it
being used by xc_domain_memory_mapping() (which qemu calls) and the
explicit use in hw/xenpv/xen_domainbuild.c:xen_domain_poll(). Extend
permissions to that of any "ordinary" domctl: A domain controlling the
targeted domain can invoke this operation for that target domain (which
is being achieved by no longer passing NULL to xsm_domctl()).

This at once avoids a for_each_domain() loop when the ID of an
existing domain gets passed in.

Reported-by: Marek Marczykowski-Górecki 
Signed-off-by: Jan Beulich 
---
v2: Add a comment. Clarify description as to what additional permission
is being granted.
---
I know there had been an alternative patch suggestion, but that one
doesn't seem have seen a formal submission so far, so here is my
original proposal.

I wonder what good the duplication of the returned d

Re: [Xen-devel] monitor access to pages with a specific p2m_type_t

[sepanta s]

On Tue, Aug 2, 2016 at 8:23 PM, Tamas K Lengyel  wrote:

>
>
> On Tue, Aug 2, 2016 at 12:19 AM, sepanta s  wrote:
>
>>
>>
>> On Sat, Jul 23, 2016 at 3:49 PM, sepanta s  wrote:
>>
>>>
> Hi,
> Is there any sample code which I can undestand how to capture the
> events on the gfns which have p2m_ram_shared enabled ?
> I couldn't find any ... .
> I would be grateful if any help , as there is not any documents
> through net to use :(
>
>
 Should I just set the ring_page as the pages which are shared and mark
> them read-only, then capture the write events?
>

 Not sure what ring_page you are talking about, but if you mark the
 pages read-only with mem_access you will get notifications for events that
 lead to unsharing with p2m_ram_shared type pages as well.

>>>
>>> There was a function in mem-sharing.c which is intended to announce the
>>> failed unshared pages. It is "mem_sharing_notify_enomem" .
>>> I added "mem_sharing_notify_unshare" as a new function and call it in
>>> also XEN_DOMCTL_VM_EVENT_OP_UNSHARING and "HVM_PARAM_USHARING_RING_PFN".
>>> I also added the required codes in /xen/common/vm_event.c and
>>> /tools/libxc/xc_vm_event so as
>>> I have added a new event for the unsharing actions of a page.
>>> Then, I wrote a sample code line xen-access and create a ring for the
>>> pages of a domain and listen to unshared events of it.
>>>

> BTW, I added a function called mem_sharing_notify_unshare to
> mem_sharing.c and added it to __mem_sharing_unshare_page at this part:
>
> *if ( p2m_change_type_one(d, gfn, p2m_ram_shared, p2m_ram_rw) )*
> *{*
> *gdprintk(XENLOG_ERR, "Could not change p2m type d %hu gfn %lx.\n", *
> *d->domain_id, gfn);*
> *BUG();*
> *}else {*
>
>
> * mem_sharing_notify_unshare(d,gfn.0);*
> *}*
>
>
 IMHO this duplicates a lot of what mem_access does already, I don't
 think there is a need for a separate notification on another ring.


>>> You are right, xen-access should work but I couldn't change its code and
>>> couldn't get the mem-access events.
>>>  I just added the above function to be sure that unsharing a page
>>> happens and works fine. Because I couldn't get the access requests on
>>> shared-pages of a vm.
>>> In xen-access, Instead of setting all the pages' default access to rx ,
>>> I just call xc_set_mem_access for the pages with p2m_ram_shared and assign
>>> rx as the default access but there is no requests on this ring.
>>>

>>>
>>> So by having a vm event channel listening to unsharing event, I can see
> the notification in xen-access . To do so, I
> have used vm_event_enable which uses HVM_PARAM_SHARING_RING_PFN .
> But, when I used memshrtool to share all the pages of two vms - my vm1
> and its clone vm2 .
> About 900 MB of the ram is shared but there are a lot of unshared
> events happening.
>

 Yes, I would say that's expected.

>>>

> When I do the sharing one more time, there is not any pages unshared
> as the total number of shared pages stay the same.
>

 Well, if you let the domain run for a while after sharing, then you do
 the sharing like that again you are likely going to crash the VM.


> Seems no unsharing is done as the number of shared pages are the same.
> Does any page fault triggers when a write operation is done on a
> shared page among two vms ?
>

 I would guess the the VM crashed and that's why you don't see any more
 unsharing at that point.

>>> Yes it was crashed as I checked it.
>>> The scenario of sharing is I use:
>>> I pause the origin VM and then run memshrtool on origin VM and clone VM.
>>> After sharing all the pages between these two VMs,Clone VM seems to be
>>> inaccessible. The clone seems to work as the attached photo shows, its cpu
>>> time increases and it exceeds the dom0 cpu time but when I use gvncviewer
>>> to see the GUI of the Clone VM, the mouse or keyboard don't work. (origin
>>> VM is ubunut-64-1 and clone VM is ubuntu-64-clone-1). Is there anything I
>>> have missed in sharing between two VMs?
>>>
>>> [image: Inline image 2]
>>>
>>
>> Can someone help please ? still have problem with the machine .
>> is it because sharing is not based on content?
>>
>>
> Well, the emulated device contexts probably get all messed up when you
> just clone over the memory of the domain and that's why they don't work.
> The only way I got this to work is by doing a save/restore of the origin
> domain before doing the memory sharing.
>
> Tamas
>
I do so , as I use the following commands to create the clone :
xl save -c origin-domid /tmp/clone
xl restore -p -e  /etc/xen/clone.config  /tmp/clone
Also I keep the origin and clone both unpaused before sharing and after
that, I just unpause the clone , but I
can just run some commands and then the clone stops responding to mouse
operations.
My vms

[Xen-devel] [xen-unstable-smoke test] 99958: tolerable all pass - PUSHED

[osstest service owner]

flight 99958 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99958/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  35dbf099ac18924d40533c9d1b9bfbf1ecb818c9
baseline version:
 xen  97323702bf101615cfdcf329913094c4cc82bf99

Last test of basis99948  2016-08-04 20:01:24 Z0 days
Testing same since99958  2016-08-05 10:01:24 Z0 days1 attempts


People who touched revisions under test:
  Bob Liu 
  David Scott 
  Juergen Gross 
  Wei Liu 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=35dbf099ac18924d40533c9d1b9bfbf1ecb818c9
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
35dbf099ac18924d40533c9d1b9bfbf1ecb818c9
+ branch=xen-unstable-smoke
+ revision=35dbf099ac18924d40533c9d1b9bfbf1ecb818c9
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-4.7-testing
+ '[' x35dbf099ac18924d40533c9d1b9bfbf1ecb818c9 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src 
'[fetch=try]'
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local 'options=[fetch=try]'
 getconfig GitCacheProxy
 perl -e '
use Osstest;
readglobalconfig();
print $c{"GitCacheProxy"} or die $!;
'
+++ local cache=git://cache:9419/
+++ '[' xgit://cache:9419/ '!=' x ']'
+++ 

[Xen-devel] [xen-unstable test] 99952: regressions - trouble: blocked/broken/fail/pass

[osstest service owner]

flight 99952 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99952/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-debianhvm-amd64 15 guest-localmigrate/x10 fail REGR. 
vs. 99917
 build-armhf-pvops 4 host-build-prep   fail REGR. vs. 99917

Regressions which are regarded as allowable (not blocking):
 build-amd64-rumpuserxen   6 xen-buildfail   like 99917
 build-i386-rumpuserxen6 xen-buildfail   like 99917
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop  fail like 99917
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail like 99917
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 99917
 test-amd64-amd64-xl-rtds  9 debian-install   fail   like 99917
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 99917

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass

version targeted for testing:
 xen  97323702bf101615cfdcf329913094c4cc82bf99
baseline version:
 xen  e9522e4932aaa7f083688b6612b5897839409260

Last test of basis99917  2016-08-03 08:27:27 Z2 days
Failing since 99928  2016-08-03 16:14:26 Z1 days4 attempts
Testing same since99952  2016-08-05 01:44:14 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Anshul Makkar 
  Dario Faggioli 
  George Dunlap 
  Jacob Pan 
  Jan Beulich 
  Joao Martins 
  Juergen Gross 
  Julien Grall 
  Kevin Tian 
  Len Brown 
  Paul Durrant 
  Rafael J. Wysocki 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 
  Yu Zhang 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-oldkern  pass
 build-i386-oldkern   pass
 build-amd64-prev pass
 build-i386-prev  pass
 build-amd64-pvopspass
 build-armhf-pvopsbroken  
 build-i386-pvops pass
 build-amd64-rumpuserxen  fail
 build-i386-rumpuserxen   fail
 test-amd64-amd64-xl   

[Xen-devel] preparations for 4.5.4

[Jan Beulich]

All,

it was only earlier today when I realized that 4.5.4 would have been
due about two weeks back; I apologize for this oversight. In any event,
please indicate backports you find missing from its staging tree, but
which you deem necessary in the release. Please note that 4.5.4 is
expected to be the last xenproject.org managed release from its
branch.

Jan



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 0/4] x86/ioreq server: Introduce HVMMEM_ioreq_server mem type.

[George Dunlap]

On 05/08/16 03:44, Yu Zhang wrote:
> 
> On 7/12/2016 5:02 PM, Yu Zhang wrote:
>> XenGT leverages ioreq server to track and forward the accesses to GPU
>> I/O resources, e.g. the PPGTT(per-process graphic translation tables).
>> Currently, ioreq server uses rangeset to track the BDF/ PIO/MMIO ranges
>> to be emulated. To select an ioreq server, the rangeset is searched to
>> see if the I/O range is recorded. However, number of ram pages to be
>> tracked may exceed the upper limit of rangeset.
>>
>> Previously, one solution was proposed to refactor the rangeset, and
>> extend its upper limit. However, after 12 rounds discussion, we have
>> decided to drop this approach due to security concerns. Now this new
>> patch series introduces a new mem type, HVMMEM_ioreq_server, and added
>> hvm operations to let one ioreq server to claim its ownership of ram
>> pages with this type. Accesses to a page of this type will be handled
>> by the specified ioreq server directly.
>>
>> Yu Zhang (4):
>>x86/ioreq server: Rename p2m_mmio_write_dm to p2m_ioreq_server.
>>x86/ioreq server: Add new functions to get/set memory types.
>>x86/ioreq server: Add HVMOP to map guest ram with p2m_ioreq_server to
>>  an ioreq server.
>>x86/ioreq server: Reset outstanding p2m_ioreq_server entries when an
>>  ioreq server unmaps.
>>
>>   xen/arch/x86/hvm/emulate.c   |  33 +++-
>>   xen/arch/x86/hvm/hvm.c   | 395
>> ++-
>>   xen/arch/x86/hvm/ioreq.c |  41 
>>   xen/arch/x86/mm/hap/hap.c|   9 +
>>   xen/arch/x86/mm/hap/nested_hap.c |   2 +-
>>   xen/arch/x86/mm/p2m-ept.c|  14 +-
>>   xen/arch/x86/mm/p2m-pt.c |  30 ++-
>>   xen/arch/x86/mm/p2m.c|  77 
>>   xen/arch/x86/mm/shadow/multi.c   |   3 +-
>>   xen/include/asm-x86/hvm/ioreq.h  |   2 +
>>   xen/include/asm-x86/p2m.h|  36 +++-
>>   xen/include/public/hvm/hvm_op.h  |  34 +++-
>>   12 files changed, 528 insertions(+), 148 deletions(-)
>>
> 
> Hi Jan & George, any comments on this version? :)
> Sorry if this mail disturbs, but it has been more than 3 weeks since the
> post...

Yes, sorry for the delay -- I did take a look at it earlier this week
but of course there was quite a bit of new code, so it's not something I
can just give a once-over and ack. :-)

I should be able to get to it next week.

 -George



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.6.1 crash with altp2m enabled bydefault

[Kevin.Mayer]

According to the xen dmesg

(XEN) RIP:e008:[] vmx_vmenter_helper+0x27e/0x30a
(XEN) RFLAGS: 00010003   CONTEXT: hypervisor
(XEN) rax: 8005003b   rbx: 8300e72fc000   rcx: 
(XEN) rdx: 6c00   rsi: 830617fd7fc0   rdi: 8300e6fc
(XEN) rbp: 830617fd7c40   rsp: 830617fd7c30   r8:  
(XEN) r9:  830be8dc9310   r10:    r11: 3475e9cf85d0
(XEN) r12: 0006   r13: 830c14ee1000   r14: 8300e6fc
(XEN) r15: 830617fd   cr0: 8005003b   cr4: 26e0
(XEN) cr3: 0001bd665000   cr2: 0451
(XEN) ds:    es:    fs:    gs:    ss:    cs: e008

0x82d0801fa0c3 :mov$0x6c00,%edx
0x82d0801fa0c8 :vmwrite %rax,%rdx

The vmwrite tries to write 0x8005003b   to 0x6c00.
But the active VCPU has a 0-vmcs-pointer.



> -Ursprüngliche Nachricht-
> Von: Jan Beulich [mailto:jbeul...@suse.com]
> Gesendet: Donnerstag, 4. August 2016 17:36
> An: Mayer, Kevin 
> Cc: andrew.coop...@citrix.com; xen-devel@lists.xen.org
> Betreff: Re: AW: AW: [Xen-devel] Xen 4.6.1 crash with altp2m enabled by
> default
> 
> >>> On 04.08.16 at 17:08,  wrote:
> > crash> x /130x 0x830bd0da1000
> > 0x830bd0da1000: 0x000e  0x
> > 0x830bd0da1010: 0x  0x
> > 0x830bd0da1020: 0x  0x
> > 0x830bd0da1030: 0x  0x
> > 0x830bd0da1040: 0x  0x
> > 0x830bd0da1050: 0x  0x
> > 0x830bd0da1060: 0x  0x
> > 0x830bd0da1070: 0x  0x000bd0da3000
> > 0x830bd0da1080: 0x000c17e36000  0x
> > 0x830bd0da1090: 0x  0x
> > 0x830bd0da10a0: 0xe7512000  0xe7513000
> > 0x830bd0da10b0: 0x000bd0da  0x
> > 0x830bd0da10c0: 0x  0x
> > 0x830bd0da10d0: 0x  0x006fedea809b
> > 0x830bd0da10e0: 0x0001a379e000  0x000610f9101e
> > 0x830bd0da10f0: 0x  0x
> > 0x830bd0da1100: 0x  0x0007010600070106
> > 0x830bd0da1110: 0x  0x
> > 0x830bd0da1120: 0x006bb6a075fa  0x00060042003f
> > 0x830bd0da1130: 0x  0x000fefff
> > 0x830bd0da1140: 0x  0x51ff
> > 0x830bd0da1150: 0x0041  0x
> > 0x830bd0da1160: 0x  0x000c
> > 0x830bd0da1170: 0x  0x
> > 0x830bd0da1180: 0x0001  0x
> > 0x830bd0da1190: 0x0008  0x
> > 0x830bd0da11a0: 0x0001  0x0096
> > 0x830bd0da11b0: 0x82d0802bc208  0x806f6dbc
> > 0x830bd0da11c0: 0x  0x0400
> > 0x830bd0da11d0: 0x80550f34  0xf0e48161
> > 0x830bd0da11e0: 0x0246  0x
> > 0x830bd0da11f0: 0xf79c3000  0x804de6f0
> > 0x830bd0da1200: 0x0023  0x
> > 0x830bd0da1210: 0x00c0f300  0x0008
> > 0x830bd0da1220: 0x  0x00c09b00
> > 0x830bd0da1230: 0x0010  0x
> > 0x830bd0da1240: 0x00c09300  0x0023
> > 0x830bd0da1250: 0x  0x00c0f300
> > 0x830bd0da1260: 0x0030  0xffdff000
> > 0x830bd0da1270: 0x00c093001fff  0x
> > 0x830bd0da1280: 0x  0x01c0
> > 0x830bd0da1290: 0x  0x
> > 0x830bd0da12a0: 0x01c0  0x0028
> > 0x830bd0da12b0: 0x80042000  0x8b0020ab
> > 0x830bd0da12c0: 0x8003f000  0x8003f400
> > 0x830bd0da12d0: 0x07ff03ff  0x8001003b
> > 0x830bd0da12e0: 0x00039000  0x26d9
> > 0x830bd0da12f0: 0xdc3c  0x
> > 0x830bd0da1300: 0xe008  0x
> > 0x830bd0da1310: 0x  0xe040
> > 0x830bd0da1320: 0x050100070406  0x
> > 0x830bd0da1330: 0x  0x80050033
> > 0x830bd0da1340: 0x

[Xen-devel] [qemu-mainline test] 99953: regressions - FAIL

[osstest service owner]

flight 99953 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99953/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-qemuu-rhel6hvm-amd  5 xen-install fail REGR. vs. 99944

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 15 guest-start/debian.repeat fail REGR. vs. 99944
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 99944
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 99944
 test-amd64-amd64-xl-rtds  9 debian-install   fail   like 99944

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass

version targeted for testing:
 qemuu42e0d60f1615ef63d16e41bb1668805560c37870
baseline version:
 qemuu09704e6ded83fa0bec14baf32f800f6512156ca0

Last test of basis99944  2016-08-04 14:42:43 Z0 days
Testing same since99953  2016-08-05 02:01:01 Z0 days1 attempts


People who touched revisions under test:
  Peter Maydell 
  Riku Voipio 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-lib

Re: [Xen-devel] [PATCH v2] domctl: relax getdomaininfo permissions

[Andrew Cooper]

On 05/08/16 12:20, Jan Beulich wrote:
> Qemu needs access to this for the domain it controls, both due to it
> being used by xc_domain_memory_mapping() (which qemu calls) and the
> explicit use in hw/xenpv/xen_domainbuild.c:xen_domain_poll(). Extend
> permissions to that of any "ordinary" domctl: A domain controlling the
> targeted domain can invoke this operation for that target domain (which
> is being achieved by no longer passing NULL to xsm_domctl()).
>
> This at once avoids a for_each_domain() loop when the ID of an
> existing domain gets passed in.
>
> Reported-by: Marek Marczykowski-Górecki 
> Signed-off-by: Jan Beulich 
> ---
> v2: Add a comment. Clarify description as to what additional permission
> is being granted.
> ---
> I know there had been an alternative patch suggestion, but that one
> doesn't seem have seen a formal submission so far, so here is my
> original proposal.
>
> I wonder what good the duplication of the returned domain ID does: I'm
> tempted to remove the one in the command-specific structure. Does
> anyone have insight into why it was done that way?

I wonder whether the first incarnation of this hypercall lacked a domid
field in the returned structure?  It seems like the kind of thing which
would be omitted, until the sysctl list version got introduced.

>
> I further wonder why we have XSM_OTHER: The respective conversion into
> other XSM_* values in xsm/dummy.h could as well move into the callers,
> making intentions more obvious when looking at the actual code.
>
> --- a/xen/include/xsm/dummy.h
> +++ b/xen/include/xsm/dummy.h
> @@ -61,7 +61,12 @@ static always_inline int xsm_default_act
>  return 0;
>  case XSM_TARGET:
>  if ( src == target )
> +{
>  return 0;
> +case XSM_XS_PRIV:
> +if ( src->is_xenstore )
> +return 0;
> +}
>  /* fall through */
>  case XSM_DM_PRIV:
>  if ( target && src->target == target )
> @@ -71,10 +76,6 @@ static always_inline int xsm_default_act
>  if ( src->is_privileged )
>  return 0;
>  return -EPERM;
> -case XSM_XS_PRIV:
> -if ( src->is_xenstore || src->is_privileged )
> -return 0;
> -return -EPERM;
>  default:
>  LINKER_BUG_ON(1);
>  return -EPERM;

What is this change in relation to?  I can't see how it is related to
the XSM changes mentioned in the commit, as that is strictly for the use
of XSM_OTHER.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/3] xen: Have schedulers revise initial placement

[Jan Beulich]

>>> On 01.08.16 at 14:32,  wrote:
> On Mon, 2016-08-01 at 04:40 -0600, Jan Beulich wrote:
>> > > > On 15.07.16 at 20:02,  wrote:
>> > 
>> > To solve this, when inserting a vcpu, always call the per-scheduler
>> > "pick" function to revise the initial placement.  This will
>> > automatically take all knowledge the scheduler has into account.
>> > 
>> > Signed-off-by: George Dunlap 
>>
>> Should this and patch 3 be backported?
>> 
> Yes, I think they're good backporting candidates.

Well, they appear to work fine on 4.7, but putting them onto 4.5
causes an early boot crash (BUG_ON( cpu != svc->vcpu->processor )
in __runq_insert()). Pulling in e59321d154 ("credit: remove cpu
argument to __runq_insert()") obviously makes that crash go
away, just to, a little later, hit the similar one close to the top of
csched_load_balance().

I'd really like to have those backported, but I have to ask one
of you to identify which prereq-s are needed on 4.6 and 4.5
(I'll revert them from 4.5 right away, but I'll wait for an osstest
flight to confirm the same issue exists on 4.6). On 4.5 I'd then
additionally depend on you to indicate whether sedf needs
some additional adjustment.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC Design Doc v2] Add vNVDIMM support for Xen

[Konrad Rzeszutek Wilk]

> > > As above, if linux driver detects the signature "NVDIMM_PFN_INFO" and
> > > a matched checksum, it will know it's safe to write to the reserved
> > > area. Otherwise, it will treat the pmem namespace as a raw device and
> > > store page struct's in the normal RAM.
> > 
> > OK, so my worry is that we will have a divergence. Which is that
> > the system admin creates this under ndctl v0, boots Xen uses it.
> > Then moves the NVDIMM to another machine which has ndctl v1 and
> > he/she boots in Linux.
> > 
> > Linux gets all confused b/c the region has something it can't understand
> > and the user is very angry.
> > 
> > So it sounds like the size the ndctl reserves MUST be baked in an ABI
> > and made sure to expand if needed.
> >
> 
> ndctl is a management tool which passes all its requests to the driver
> via sysfs, so the compatibility across different versions of Linux
> would actual be introduced by the different versions of drivers.
> 
> All newer versions of drivers should provide backwards compatibility
> to previous versions (which is the current drivers'
> behavior). However, the forwards compatibility is hard to preserved,
> e.g.
>  - an old version w/o reserved area support (e.g. the one in linux
>kernel 4.2) recognizes a pmem namespace w/ reserved area as a raw
>device and may write to the reserved area. If it's a xen reserved
>area and the driver is in dom0, the dom0 kernel will crash.

Yikes!
>
>  - the same crash would happen if an old version driver w/ reserved
>area support but xen reserved area support (e.g. the one in linux
>kernel 4.7) is used for a pmem namespace w/ xen reserved area.
> 
> For the cross-OS compatibility, there is an effort to standardize the
> reservation. In the meantime, only linux is capable to handle such
> pmem namespaces with reserved area.

It may be good to mention these difficulties you enumerated in the design
doc so if somebody does end up in this position and they search for it - they
could find a reference.

> 
> > ..snip..
> > > > This "balloon out" is interesting. You are effectively telling Linux
> > > > to ignore a certain range of 'struct page_info', so that if somebody
> > > > uses /sys/debug/kernel/page_walk it won't blow up? (As the kerne
> > > > can't read the struct page_info anymore).
> > > >
> > > > How would you do this? Simulate an NVDIMM unplug?
> > > 
> > > s/page_info/page/ (struct page for linux, struct page_info for xen)
> > > 
> > > As in Jan's comment, "balloon out" is a confusing name here.
> > > Basically, it's to remove the reserved area from some resource struct
> > > in nvdimm driver to avoid it's accessed out of the driver via the
> > > resource struct. And the nvdimm driver does not map the reserved area,
> > > so I think it cannot be touched via page_walk.
> > 
> > OK, I need to read the Linux code more to make sure I am
> > not missing something.
> > 
> > Basically the question that keeps revolving in my head is:
> > 
> > Why is this even neccessary?
> > 
> > Let me expand - it feels like (and I think I am missing something
> > here) that we are crippling the Linux driver so that it won't
> > break - b/c if it tried to access the 'strut page_info' in this
> > reserved region it would crash. So we eliminate that, and make
> > the driver believe the region exists (is reserved), but it can't
> > use it. And instead use the normal RAM pages to keep track
> > of the NVDIMM SPAs.
> > 
> > Or perhaps not keep track at all and just treat the whole
> > NVDIMM as opaque MMIO that is inaccessible?
> >
> 
> If we trust the driver in dom0 kernel always does correct things (and
> we can trust it, right?), no crash will happen. However, as Jan
> comment 
> (https://lists.xenproject.org/archives/html/xen-devel/2016-08/msg00433.html):
> 
> | Right now Dom0 isn't allowed to access any memory in use by Xen
> | (and not explicitly shared), and I don't think we should deviate
> | from that model for pmem.
> 
> xen hypervisor must explicitly disallow dom0 from accessing the
> reserved area.

Right.
> 
> > But how will that work if there is a DAX filesystem on it?
> > The ext4 needs some mechanism to access the files that are there.
> > (Otherwise you couldn't use the fiemap ioctl to find the SPAs).
> >
> 
> No, the file system does not touch the reserved area. If a reserved

Ah, OK!
> area exists, the start SPA of /dev/pmem0 reported via sysfs is the
> start SPA of the reserved area, so fiemap can still work.
> 
> > [see below]
> > > 
> > > > 
> > > > But if you do that how will SMART tools work anymore? And
> > > > who would do the _DSM checks on the health of the NVDIMM?
> > > >
> > > 
> > > A userspace SMART tool cannot access the reserved area, so I think it
> > > can still work. I haven't look at the implementation of any SMART
> > > tools for NVDIMM, but I guess they would finally call the driver to
> > > evaluate the ARS _DSM which reports the bad blocks. As long as the
> > > driver does not return the bad blocks

Re: [Xen-devel] live migration to qemu.git fails

[Olaf Hering]

On Wed, Aug 03, Olaf Hering wrote:

> On Wed, Aug 03, Anthony PERARD wrote:
> 
> > Haven't you try to create a guest with Xen 4.5 and qemu-xen-4.5, and
> > then migrate to Xen 4.7 with QEMU-2.6/master?
> 
> In the end I tried xen-4.5/6/7/8 as source and their qemu-xen, and
> migrated to qemu#master. All failed.

Today I built qemu binaries from stable-2.4/stable-2.5/v2.6.0 to prepare
a bisect. Because 2.4/2.5 does not compile with staging-4.7 I went back
to staging-4.6. And to my surprise live migration works in every
combination I have tried. Not sure what went wrong with earlier testing.

I will see what I find..

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-mainline baseline-only test] 66921: tolerable FAIL

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 66921 qemu-mainline real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/66921/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail like 66917
 test-amd64-amd64-amd64-pvgrub 10 guest-start  fail  like 66917

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-midway   13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-midway   12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass

version targeted for testing:
 qemuu09704e6ded83fa0bec14baf32f800f6512156ca0
baseline version:
 qemuu6eac5f7bad6cd8f56b3514ac485dd35d79abff66

Last test of basis66917  2016-08-04 14:46:50 Z0 days
Testing same since66921  2016-08-05 02:14:33 Z0 days1 attempts


People who touched revisions under test:
  Cao jin 
  Daniel P. Berrange 
  Dave Hansen 
  Emilio G. Cota 
  Eric Blake 
  Fam Zheng 
  Gerd Hoffmann 
  Greg Kurz 
  Igor Mammedov 
  Markus Armbruster 
  Paolo Bonzini 
  Peter Maydell 
  Peter Xu 
  Robert Ho 
  Sergey Fedorov 
  Shmulik Ladkani 
  Stefan Hajnoczi 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-libvirt-qemuu-d

[Xen-devel] [distros-debian-jessie test] 66922: tolerable all pass

[Platform Team regression test user]

flight 66922 distros-debian-jessie real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/66922/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-armhf-jessie-netboot-pygrub 12 saverestore-support-check fail 
never pass
 test-armhf-armhf-armhf-jessie-netboot-pygrub 11 migrate-support-check fail 
never pass

baseline version:
 flight   66858

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-amd64-jessie-netboot-pvgrub pass
 test-amd64-i386-i386-jessie-netboot-pvgrub   pass
 test-amd64-i386-amd64-jessie-netboot-pygrub  pass
 test-armhf-armhf-armhf-jessie-netboot-pygrub pass
 test-amd64-amd64-i386-jessie-netboot-pygrub  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 1/3] xen/common: Avoid undefined behaviour by shifting into a sign bit

[Andrew Cooper]

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: George Dunlap 
CC: Konrad Rzeszutek Wilk 
CC: Stefano Stabellini 
CC: Tim Deegan 
---
 xen/common/domctl.c   | 2 +-
 xen/common/xmalloc_tlsf.c | 4 ++--
 xen/include/xen/sched.h   | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 8f25131..cf7928c 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -188,7 +188,7 @@ void getdomaininfo(struct domain *d, struct 
xen_domctl_getdomaininfo *info)
 (d->controller_pause_count > 0  ? XEN_DOMINF_paused: 0) |
 (d->debugger_attached   ? XEN_DOMINF_debugged  : 0) |
 (d->is_xenstore ? XEN_DOMINF_xs_domain : 0) |
-d->shutdown_code << XEN_DOMINF_shutdownshift;
+(unsigned int)d->shutdown_code << XEN_DOMINF_shutdownshift;
 
 switch ( d->guest_type )
 {
diff --git a/xen/common/xmalloc_tlsf.c b/xen/common/xmalloc_tlsf.c
index b13317e..6c1b882 100644
--- a/xen/common/xmalloc_tlsf.c
+++ b/xen/common/xmalloc_tlsf.c
@@ -177,7 +177,7 @@ static inline void MAPPING_INSERT(unsigned long r, int *fl, 
int *sl)
 static inline struct bhdr *FIND_SUITABLE_BLOCK(struct xmem_pool *p, int *fl,
int *sl)
 {
-u32 tmp = p->sl_bitmap[*fl] & (~0 << *sl);
+u32 tmp = p->sl_bitmap[*fl] & (~0u << *sl);
 struct bhdr *b = NULL;
 
 if ( tmp )
@@ -187,7 +187,7 @@ static inline struct bhdr *FIND_SUITABLE_BLOCK(struct 
xmem_pool *p, int *fl,
 }
 else
 {
-*fl = ffs(p->fl_bitmap & (~0 << (*fl + 1))) - 1;
+*fl = ffs(p->fl_bitmap & (~0u << (*fl + 1))) - 1;
 if ( likely(*fl > 0) )
 {
 *sl = ffs(p->sl_bitmap[*fl]) - 1;
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 888bc19..bb4ee4e 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -483,7 +483,7 @@ extern struct vcpu *idle_vcpu[NR_CPUS];
 #define is_idle_domain(d) ((d)->domain_id == DOMID_IDLE)
 #define is_idle_vcpu(v)   (is_idle_domain((v)->domain))
 
-#define DOMAIN_DESTROYED (1<<31) /* assumes atomic_t is >= 32 bits */
+#define DOMAIN_DESTROYED (1u << 31) /* assumes atomic_t is >= 32 bits */
 #define put_domain(_d) \
   if ( atomic_dec_and_test(&(_d)->refcnt) ) domain_destroy(_d)
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 3/3] x86/microcode: Avoid undefined behaviour from signed integer overflow

[Andrew Cooper]

The checksum should be calculated using unsigned 32bit integers, as it is
intended to overflow and end at 0.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Kevin Tian 
CC: Jun Nakajima 
---
 xen/arch/x86/microcode_intel.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index 6949c25..5b411b4 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -143,7 +143,8 @@ static int microcode_sanity_check(void *mc)
 struct extended_sigtable *ext_header = NULL;
 struct extended_signature *ext_sig;
 unsigned long total_size, data_size, ext_table_size;
-int sum, orig_sum, ext_sigcount = 0, i;
+uint32_t sum, orig_sum;
+int ext_sigcount = 0, i;
 
 total_size = get_totalsize(mc_header);
 data_size = get_datasize(mc_header);
@@ -201,7 +202,7 @@ static int microcode_sanity_check(void *mc)
 orig_sum = 0;
 i = (MC_HEADER_SIZE + data_size) / DWSIZE;
 while ( i-- )
-orig_sum += ((int *)mc)[i];
+orig_sum += ((uint32_t *)mc)[i];
 if ( orig_sum )
 {
 printk(KERN_ERR "microcode: aborting, bad checksum\n");
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 2/3] xen/x86: Avoid undefined behaviour by shifting into a sign bit

[Andrew Cooper]

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
---
 xen/arch/x86/apic.c   | 2 +-
 xen/arch/x86/cpu/common.c | 2 +-
 xen/arch/x86/x86_64/traps.c   | 2 +-
 xen/include/asm-x86/apicdef.h | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
index f3727cd..3fb9a82 100644
--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -571,7 +571,7 @@ void setup_local_APIC(void)
 for (i = APIC_ISR_NR - 1; i >= 0; i--) {
 value = apic_read(APIC_ISR + i*0x10);
 for (j = 31; j >= 0; j--) {
-if (value & (1<> ht_mask_width;
+   core_select_mask = (~(-1u << core_plus_mask_width)) >> ht_mask_width;
 
c->cpu_core_id = phys_pkg_id(initial_apicid, ht_mask_width)
& core_select_mask;
diff --git a/xen/arch/x86/x86_64/traps.c b/xen/arch/x86/x86_64/traps.c
index 19f58a1..1b1b61a 100644
--- a/xen/arch/x86/x86_64/traps.c
+++ b/xen/arch/x86/x86_64/traps.c
@@ -414,7 +414,7 @@ void subarch_percpu_traps_init(void)
 unmap_domain_page(stub_page);
 
 /* Common SYSCALL parameters. */
-wrmsr(MSR_STAR, 0, (FLAT_RING3_CS32<<16) | __HYPERVISOR_CS);
+wrmsr(MSR_STAR, 0, ((unsigned)FLAT_RING3_CS32 << 16) | __HYPERVISOR_CS);
 wrmsr(MSR_SYSCALL_MASK, XEN_SYSCALL_MASK, 0U);
 }
 
diff --git a/xen/include/asm-x86/apicdef.h b/xen/include/asm-x86/apicdef.h
index 8752287..da7f4d3 100644
--- a/xen/include/asm-x86/apicdef.h
+++ b/xen/include/asm-x86/apicdef.h
@@ -30,7 +30,7 @@
 #defineAPIC_EIO_ACK0x0 /* 
Write this to the EOI register */
 #defineAPIC_RRR0xC0
 #defineAPIC_LDR0xD0
-#defineAPIC_LDR_MASK   (0xFF<<24)
+#defineAPIC_LDR_MASK   (0xFFu<<24)
 #defineGET_xAPIC_LOGICAL_ID(x) (((x)>>24)&0xFF)
 #defineSET_xAPIC_LOGICAL_ID(x) (((x)<<24))
 #defineAPIC_ALL_CPUS   0xFF
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] domctl: relax getdomaininfo permissions

[Jan Beulich]

>>> On 05.08.16 at 15:10,  wrote:
> On 05/08/16 12:20, Jan Beulich wrote:
>> I wonder what good the duplication of the returned domain ID does: I'm
>> tempted to remove the one in the command-specific structure. Does
>> anyone have insight into why it was done that way?
> 
> I wonder whether the first incarnation of this hypercall lacked a domid
> field in the returned structure?  It seems like the kind of thing which
> would be omitted, until the sysctl list version got introduced.

Oh, good point - that makes clear why the field can't be dropped:
That sysctl would break then.

>> --- a/xen/include/xsm/dummy.h
>> +++ b/xen/include/xsm/dummy.h
>> @@ -61,7 +61,12 @@ static always_inline int xsm_default_act
>>  return 0;
>>  case XSM_TARGET:
>>  if ( src == target )
>> +{
>>  return 0;
>> +case XSM_XS_PRIV:
>> +if ( src->is_xenstore )
>> +return 0;
>> +}
>>  /* fall through */
>>  case XSM_DM_PRIV:
>>  if ( target && src->target == target )
>> @@ -71,10 +76,6 @@ static always_inline int xsm_default_act
>>  if ( src->is_privileged )
>>  return 0;
>>  return -EPERM;
>> -case XSM_XS_PRIV:
>> -if ( src->is_xenstore || src->is_privileged )
>> -return 0;
>> -return -EPERM;
>>  default:
>>  LINKER_BUG_ON(1);
>>  return -EPERM;
> 
> What is this change in relation to?  I can't see how it is related to
> the XSM changes mentioned in the commit, as that is strictly for the use
> of XSM_OTHER.

I don't see any XSM changes mentioned in the description, there
was only the XSM_OTHER related question outside the description.
Anyway - the change above is what guarantees the XSM_XS_PRIV
check, as invoked by xsm_domctl()'s XEN_DOMCTL_getdomaininfo
case, to fall through into XSM_DM_PRIV - after all that's what the
whole patch is about.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] CODING_STYLE: Allow single-sentence comments without full stops

[George Dunlap]

On Thu, Aug 4, 2016 at 11:25 AM, Ian Jackson  wrote:
> George Dunlap writes ("[PATCH] CODING_STYLE: Allow single-sentence comments 
> without full stops"):
>> One of the common ways in which contributors trip up over the
>> CODING_STYLE guides is by not putting a full stop at the end of a
>> comment when there is only a single sentence.  Calling these out is a
>> waste of everybody's time: The full stop at the end of a comment with
>> a single sentence (or a single phrase) adds absolutely nothing to the
>> legibility of the code.
>>
>> Modify CODING_STYLE to allow comments with a single sentence or
>> sentence fragment to either have a full stop or not, while making it
>> clear that comments with multiple sentences must have a full stop at
>> the end of each sentence.
>>
>> Signed-off-by: George Dunlap 
>
> Seriously ?
>
> Yes, the bike shed can be purple with occasional mauve.
>
> Acked-by: Ian Jackson 

Thanks Ian.

The other comments so far don't seem to be objections to this
relaxation, but about whether standards should be relaxed further.  If
I don't hear any objections by Monday I'll check this patch in and let
others submit follow-on patches if they want.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] live migration to qemu.git fails

[Anthony PERARD]

On Fri, Aug 05, 2016 at 03:37:32PM +0200, Olaf Hering wrote:
> On Wed, Aug 03, Olaf Hering wrote:
> 
> > On Wed, Aug 03, Anthony PERARD wrote:
> > 
> > > Haven't you try to create a guest with Xen 4.5 and qemu-xen-4.5, and
> > > then migrate to Xen 4.7 with QEMU-2.6/master?
> > 
> > In the end I tried xen-4.5/6/7/8 as source and their qemu-xen, and
> > migrated to qemu#master. All failed.
> 
> Today I built qemu binaries from stable-2.4/stable-2.5/v2.6.0 to prepare
> a bisect. Because 2.4/2.5 does not compile with staging-4.7 I went back
> to staging-4.6.

Well, 4.7 introduces new libraries and older QEMUs do not support it out
of the box. You'll need to add some CFLAGS and LDFLADS to have them
working, like does xen.git (staging-4.7) when compiling qemu-xen.

If you compile older qemu out if xen tree, you would need to at least
add:
--extra-cflags="-DXC_WANT_COMPAT_EVTCHN_API=1 -DXC_WANT_COMPAT_GNTTAB_API=1 
-DXC_WANT_COMPAT_MAP_FOREIGN_API=1"

> And to my surprise live migration works in every
> combination I have tried. Not sure what went wrong with earlier testing.

Good to hear.

Thanks,

-- 
Anthony PERARD

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/3] xen/common: Avoid undefined behaviour by shifting into a sign bit

[Jan Beulich]

>>> On 05.08.16 at 15:50,  wrote:
> --- a/xen/common/domctl.c
> +++ b/xen/common/domctl.c
> @@ -188,7 +188,7 @@ void getdomaininfo(struct domain *d, struct 
> xen_domctl_getdomaininfo *info)
>  (d->controller_pause_count > 0  ? XEN_DOMINF_paused: 0) |
>  (d->debugger_attached   ? XEN_DOMINF_debugged  : 0) |
>  (d->is_xenstore ? XEN_DOMINF_xs_domain : 0) |
> -d->shutdown_code << XEN_DOMINF_shutdownshift;
> +(unsigned int)d->shutdown_code << XEN_DOMINF_shutdownshift;

Is adding a cast here really the most suitable fix? The only two places
shutdown_code gets set (besides the -1 initialization) have their right
side a u8. Nothing ever checks for the value being negative (there are
just two -1 checks), so converting the field to u8 or unsigned int (and
using a sentinel different from -1) should both work, avoiding the need
for a cast.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/3] xen/x86: Avoid undefined behaviour by shifting into a sign bit

[Jan Beulich]

>>> On 05.08.16 at 15:50,  wrote:
> --- a/xen/arch/x86/cpu/common.c
> +++ b/xen/arch/x86/cpu/common.c
> @@ -476,7 +476,7 @@ void detect_extended_topology(struct cpuinfo_x86 *c)
>   sub_index++;
>   } while ( LEAFB_SUBTYPE(ecx) != INVALID_TYPE );
>  
> - core_select_mask = (~(-1 << core_plus_mask_width)) >> ht_mask_width;
> + core_select_mask = (~(-1u << core_plus_mask_width)) >> ht_mask_width;

-1u is kind of bogus; could I talk you into using ~0u instead?

With that
Reviewed-by: Jan Beulich 

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/3] xen/common: Avoid undefined behaviour by shifting into a sign bit

[George Dunlap]

On 05/08/16 15:04, Jan Beulich wrote:
 On 05.08.16 at 15:50,  wrote:
>> --- a/xen/common/domctl.c
>> +++ b/xen/common/domctl.c
>> @@ -188,7 +188,7 @@ void getdomaininfo(struct domain *d, struct 
>> xen_domctl_getdomaininfo *info)
>>  (d->controller_pause_count > 0  ? XEN_DOMINF_paused: 0) |
>>  (d->debugger_attached   ? XEN_DOMINF_debugged  : 0) |
>>  (d->is_xenstore ? XEN_DOMINF_xs_domain : 0) |
>> -d->shutdown_code << XEN_DOMINF_shutdownshift;
>> +(unsigned int)d->shutdown_code << XEN_DOMINF_shutdownshift;
> 
> Is adding a cast here really the most suitable fix? The only two places
> shutdown_code gets set (besides the -1 initialization) have their right
> side a u8. Nothing ever checks for the value being negative (there are
> just two -1 checks), so converting the field to u8 or unsigned int (and
> using a sentinel different from -1) should both work, avoiding the need
> for a cast.

This seems sensible if possible.

The other bits:

Reviewed-by: George Dunlap 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 3/3] x86/microcode: Avoid undefined behaviour from signed integer overflow

[Jan Beulich]

>>> On 05.08.16 at 15:50,  wrote:
> --- a/xen/arch/x86/microcode_intel.c
> +++ b/xen/arch/x86/microcode_intel.c
> @@ -143,7 +143,8 @@ static int microcode_sanity_check(void *mc)
>  struct extended_sigtable *ext_header = NULL;
>  struct extended_signature *ext_sig;
>  unsigned long total_size, data_size, ext_table_size;
> -int sum, orig_sum, ext_sigcount = 0, i;
> +uint32_t sum, orig_sum;
> +int ext_sigcount = 0, i;

Pretty clearly these other two want to be unsigned int? If you agree,
Reviewed-by: Jan Beulich .

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/3] xen: Have schedulers revise initial placement

[Dario Faggioli]

On Fri, 2016-08-05 at 07:24 -0600, Jan Beulich wrote:
> > > > On 01.08.16 at 14:32,  wrote:
> > On Mon, 2016-08-01 at 04:40 -0600, Jan Beulich wrote:
> > > > > > On 15.07.16 at 20:02,  wrote:
> > > > Signed-off-by: George Dunlap 
> > > Should this and patch 3 be backported?
> > > 
> > Yes, I think they're good backporting candidates.
> Well, they appear to work fine on 4.7, but putting them onto 4.5
> causes an early boot crash (BUG_ON( cpu != svc->vcpu->processor )
> in __runq_insert()). Pulling in e59321d154 ("credit: remove cpu
> argument to __runq_insert()") obviously makes that crash go
> away, just to, a little later, hit the similar one close to the top
> of
> csched_load_balance().
> 
Ah, I see.. Thanks for trying and letting us know.

> I'd really like to have those backported, but I have to ask one
> of you to identify which prereq-s are needed on 4.6 and 4.5
> (I'll revert them from 4.5 right away, but I'll wait for an osstest
> flight to confirm the same issue exists on 4.6). On 4.5 I'd then
> additionally depend on you to indicate whether sedf needs
> some additional adjustment.
> 
Ok. I can't, immediately.

But if Monday is fine, I'm happy to take care of this.

Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] live migration to qemu.git fails

[Olaf Hering]

On Fri, Aug 05, Anthony PERARD wrote:

> On Fri, Aug 05, 2016 at 03:37:32PM +0200, Olaf Hering wrote:

> If you compile older qemu out if xen tree, you would need to at least
> add:
> --extra-cflags="-DXC_WANT_COMPAT_EVTCHN_API=1 -DXC_WANT_COMPAT_GNTTAB_API=1 
> -DXC_WANT_COMPAT_MAP_FOREIGN_API=1"

Thanks, I will try that to fix compile errors.

> > And to my surprise live migration works in every
> > combination I have tried. Not sure what went wrong with earlier testing.
> 
> Good to hear.

After more testing even the staging-4.7 packages work again.
Not sure what change was made to hide the error.

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/3] xen: Have schedulers revise initial placement

[Jan Beulich]

>>> On 05.08.16 at 16:09,  wrote:
> On Fri, 2016-08-05 at 07:24 -0600, Jan Beulich wrote:
>> > > > On 01.08.16 at 14:32,  wrote:
>> > On Mon, 2016-08-01 at 04:40 -0600, Jan Beulich wrote:
>> > > > > > On 15.07.16 at 20:02,  wrote:
>> > > > Signed-off-by: George Dunlap 
>> > > Should this and patch 3 be backported?
>> > > 
>> > Yes, I think they're good backporting candidates.
>> Well, they appear to work fine on 4.7, but putting them onto 4.5
>> causes an early boot crash (BUG_ON( cpu != svc->vcpu->processor )
>> in __runq_insert()). Pulling in e59321d154 ("credit: remove cpu
>> argument to __runq_insert()") obviously makes that crash go
>> away, just to, a little later, hit the similar one close to the top
>> of
>> csched_load_balance().
>> 
> Ah, I see.. Thanks for trying and letting us know.
> 
>> I'd really like to have those backported, but I have to ask one
>> of you to identify which prereq-s are needed on 4.6 and 4.5
>> (I'll revert them from 4.5 right away, but I'll wait for an osstest
>> flight to confirm the same issue exists on 4.6). On 4.5 I'd then
>> additionally depend on you to indicate whether sedf needs
>> some additional adjustment.
>> 
> Ok. I can't, immediately.
> 
> But if Monday is fine, I'm happy to take care of this.

Oh, of course, any time next week would be fine. I'd like to have
this for 4.5.4, but in the worst case we'll ship that without it.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.6.1 crash with altp2m enabled bydefault

[Jan Beulich]

>>> On 05.08.16 at 14:51,  wrote:
> According to the xen dmesg
> 
> (XEN) RIP:e008:[] vmx_vmenter_helper+0x27e/0x30a
> (XEN) RFLAGS: 00010003   CONTEXT: hypervisor
> (XEN) rax: 8005003b   rbx: 8300e72fc000   rcx: 
> (XEN) rdx: 6c00   rsi: 830617fd7fc0   rdi: 8300e6fc
> (XEN) rbp: 830617fd7c40   rsp: 830617fd7c30   r8:  
> (XEN) r9:  830be8dc9310   r10:    r11: 3475e9cf85d0
> (XEN) r12: 0006   r13: 830c14ee1000   r14: 8300e6fc
> (XEN) r15: 830617fd   cr0: 8005003b   cr4: 26e0
> (XEN) cr3: 0001bd665000   cr2: 0451
> (XEN) ds:    es:    fs:    gs:    ss:    cs: e008
> 
> 0x82d0801fa0c3 :mov$0x6c00,%edx
> 0x82d0801fa0c8 :vmwrite %rax,%rdx
> 
> The vmwrite tries to write 0x8005003b   to 0x6c00.
> But the active VCPU has a 0-vmcs-pointer.

Which likely means altp2m manages to confuse some of VMX'es
VMCS management - vmx_vmenter_helper() being on the path
back to the guest, it should be impossible for the VMCS pointer to
be zero here. Can you perhaps identify the most recent vmread or
vmwrite which worked fine? There ought to be many on that path,
and the state corruption could then perhaps be narrowed to quite
small a range of code.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 1/3] livepach: Add .livepatch.hooks functions and test-case

[Jan Beulich]

>>> On 04.08.16 at 17:49,  wrote:
> In general, the hooks provide flexibility when having to deal with
> unforeseen cases, but their application should be rarely required (<
> 10%)."

But the greater flexibility of course comes with increased chances
of screwing things up. I'm therefore still not entirely convinced that
such XSAs wouldn't then better not be live patched.

> --- a/xen/arch/x86/test/xen_hello_world.c
> +++ b/xen/arch/x86/test/xen_hello_world.c
> @@ -4,14 +4,50 @@
>   */
>  
>  #include "config.h"
> +#include 
>  #include 
>  #include 
>  #include 
> +#include 
>  
>  #include 
>  
>  static char hello_world_patch_this_fnc[] = "xen_extra_version";
>  extern const char *xen_hello_world(void);
> +static unsigned int cnt;
> +
> +static void apply_hook(void)
> +{
> +printk(KERN_DEBUG "Hook executing.\n");
> +}
> +
> +static void revert_hook(void)
> +{
> +printk(KERN_DEBUG "Hook unloaded.\n");
> +}
> +
> +static void hi_func(void)
> +{
> +printk(KERN_DEBUG "%s: Hi! (called %u times)\n", __func__, ++cnt);
> +};
> +
> +/* If we are sorted we _MUST_ be the last .livepatch.hook section. */
> +static void Z_check_fnc(void)

And that Z_ prefix is supposed to guarantee that being last? Isn't
it that upper case letters sort before lower case ones?

> +{
> +printk(KERN_DEBUG "%s: Hi func called %u times\n", __func__, cnt);
> +BUG_ON(cnt == 0 || cnt > 2);
> +cnt = 0; /* Otherwise if you revert, apply, revert the value will be 4! 
> */

Isn't this an indication of a general weakness: Shouldn't a patch
module be allowed to assume it's being run in a clean state, i.e.
without any of its static data altered from their load time values?

> @@ -70,7 +71,11 @@ struct payload {
>  unsigned int nsyms;  /* Nr of entries in .strtab and 
> symbols. */
>  struct livepatch_build_id id;/* ELFNOTE_DESC(.note.gnu.build-id) 
> of the payload. */
>  struct livepatch_build_id dep;   /* 
> ELFNOTE_DESC(.livepatch.depends). */
> -char name[XEN_LIVEPATCH_NAME_SIZE]; /* Name of it. */
> +livepatch_loadcall_t **load_funcs;   /* The array of funcs to call after 
> */
> +livepatch_unloadcall_t **unload_funcs;/* load and unload of the payload. 
> */

These both seem like they want a const in the middle.

> @@ -515,6 +520,27 @@ static int prepare_payload(struct payload *payload,
>  }
>  }
>  
> +sec = livepatch_elf_sec_by_name(elf, ".livepatch.hooks.load");

Is the .hooks infix really needed?

> +if ( sec )
> +{
> +if ( !sec->sec->sh_size ||

What's wrong with a zero size section (holding no hooks)? We've
had that same case elsewhere in the original series ...

> @@ -999,6 +1025,17 @@ static int apply_payload(struct payload *data)
>  
>  arch_livepatch_quiesce();
>  
> +/*
> + * The hooks may call common code which expects spinlocks to be certain
> + * type, as such disable this temporarily.
> + */
> +spin_debug_disable();
> +for ( i = 0; i < data->n_load_funcs; i++ )
> +data->load_funcs[i]();
> +spin_debug_enable();

I have to admit that I can't really make sense of the comment, and
hence the code.

> --- /dev/null
> +++ b/xen/include/xen/livepatch_payload.h
> @@ -0,0 +1,49 @@
> +/*
> + * Copyright (C) 2016 Citrix Systems R&D Ltd.
> + */
> +
> +#ifndef __XEN_LIVEPATCH_PAYLOAD_H__
> +#define __XEN_LIVEPATCH_PAYLOAD_H__
> +
> +/*
> + * The following definitions are to be used in patches. They are taken
> + * from kpatch.
> + */
> +typedef void livepatch_loadcall_t(void);
> +typedef void livepatch_unloadcall_t(void);
> +
> +/*
> + * LIVEPATCH_LOAD_HOOK macro
> + *
> + * Declares a function pointer to be allocated in a new
> + * .livepatch.hook.load section.  This livepatch_load_data symbol is later
> + * stripped by create-diff-object so that it can be declared in multiple
> + * objects that are later linked together, avoiding global symbol
> + * collision.  Since multiple hooks can be registered, the
> + * .livepatch.hook.load section is a table of functions that will be
> + * executed in series by the livepatch infrastructure at patch load time.
> + */
> +#define LIVEPATCH_LOAD_HOOK(_fn) \
> +livepatch_loadcall_t *__attribute__((weak)) \
> +livepatch_load_data_##_fn __section(".livepatch.hooks.load") = _fn;

There's again a const desirable here, to render the section r/o.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 3/3] livepatch: Sync cache of build-id before using it first time.

[Jan Beulich]

>>> On 04.08.16 at 17:49,  wrote:
> --- a/xen/include/xen/livepatch.h
> +++ b/xen/include/xen/livepatch.h
> @@ -44,7 +44,7 @@ unsigned long livepatch_symbols_lookup_by_name(const char 
> *symname);
>  bool_t is_patch(const void *addr);
>  int xen_build_id_check(const Elf_Note *n, unsigned int n_sz,
> const void **p, unsigned int *len);
> -
> +void xen_build_init(void);
>  /* Arch hooks. */

Please don't ditch a blank line like this. But this is the wrong header
anyway, or else you'd have to make version.c include it (which would
again seem odd). And as I now realize that same aspect applies to
xen_build_id_check() too - this needs to move into xen/version.h.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 2/8] tools/console: introduce --start-notify-fd option for console client

[Ian Jackson]

Wei Liu writes ("[RFC PATCH 2/8] tools/console: introduce --start-notify-fd 
option for console client"):
> The console client will write 0x00 to that fd before entering console
> loop to indicate its readiness.

This could sensibly be in a comment in the code.

> + if (start_notify_fd != -1) {
> + const char msg[] = { 0x00 };
^ static

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 3/8] libxl: factor out libxl__console_tty_path

[Ian Jackson]

Wei Liu writes ("[RFC PATCH 3/8] libxl: factor out libxl__console_tty_path"):
> No user yet.

You mean no _other_ user!

With that change to the commit message,

Acked-by: Ian Jackson 


Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 1/8] tools/console: fix help string in client

[Ian Jackson]

Wei Liu writes ("[RFC PATCH 1/8] tools/console: fix help string in client"):
> There is no short '-t' option.

Acked-by: Ian Jackson 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 4/8] libxl: wait up to 5s in libxl_console_exec for xenconsoled

[Ian Jackson]

Wei Liu writes ("[RFC PATCH 4/8] libxl: wait up to 5s in libxl_console_exec for 
xenconsoled"):
> Wait until the tty node is available before exec'ing xenconsole.

You shouldn't poll.  We have a perfectly good xenstore watch
mechanism.

You could either:

(a) do something ad-hoc with poll(), the xenstore fd, etc.

(b) make libxl_console_exec by a weird special case which starts an AO
but always synchronously (ie, doesn't take an ao_how), and then use
the existing libxl__xswait and a callback.

Doing the latter would mean that you would already have done half (a
third?) of the work for a future libxl_console_get_fd function.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 5/8] libxl: libxl_{primary_, }console_exec now take notify_fd argument

[Ian Jackson]

Wei Liu writes ("[RFC PATCH 5/8] libxl: libxl_{primary_,}console_exec now take 
notify_fd argument"):
> The new argument will be passed down to xenconsole process, which then
> uses it to notify readiness.
...
>  int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num,
> -   libxl_console_type type)
> +   libxl_console_type type, int notify_fd)

Urgh.  I don't see a better way to do this though, given the existence
of libxl_console_exec.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 5/8] libxl: libxl_{primary_, }console_exec now take notify_fd argument

[Ian Jackson]

Ian Jackson writes ("Re: [RFC PATCH 5/8] libxl: libxl_{primary_,}console_exec 
now take notify_fd argument"):
> Wei Liu writes ("[RFC PATCH 5/8] libxl: libxl_{primary_,}console_exec now 
> take notify_fd argument"):
> > The new argument will be passed down to xenconsole process, which then
> > uses it to notify readiness.
> ...
> >  int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num,
> > -   libxl_console_type type)
> > +   libxl_console_type type, int notify_fd)
> 
> Urgh.  I don't see a better way to do this though, given the existence
> of libxl_console_exec.

By which I mean,

Acked-by: Ian Jackson 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 6/8] docs: document xenconsole startup protocol

[Ian Jackson]

Wei Liu writes ("[RFC PATCH 6/8] docs: document xenconsole startup protocol"):
> Signed-off-by: Wei Liu 

Oh!  Forget my comment on patch 02. about the commit message and/or
comment.  Maybe you want to squash this one into there, but either
way:

> +The xenconsole program supports a very simple protocol to notify parent about
> +its readiness. If xenconsole (the client) is exec'ed and has been given a fd
> +(normally the write end of a pipe) via --start-noitify-fd option, it will
  ^^^ not
> +write 0x00 to that fd after connecting to the guest but before entering the
> +event loop. Parent can read fromt the read end of the fd to know the status.
   ^

With the two typos fixed,

Acked-by: Ian Jackson 

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 7/8] xl: use xenconsole startup protocol

[Ian Jackson]

Wei Liu writes ("[RFC PATCH 7/8] xl: use xenconsole startup protocol"):
> If user asks xl to automatically connect to console when creating a
> guest, use the new startup protocol before trying to unpause domain so
> that we don't lose any console output.

Most of the logic here LGTM.

> @@ -2997,7 +2999,15 @@ start:
>  
>  libxl_asyncprogress_how autoconnect_console_how_buf;
>  if ( dom_info->console_autoconnect ) {
> +if (pipe(notify_pipe)) {

Use libxl_pipe.

> +if (autoconnect_console_how) {
> +char buf[1];
> +if (read(notify_pipe[0], buf, 1) != 1 && buf[0] != 0x00) {
> +fprintf(stderr,
> +"Failed to get notification from xenconsole, errno %d\n",
> +errno);

As you say in your 0/, poor error handling.  You need to print
the value of buf, if relevant, and handle EOF properly.

> +}
> +close(notify_pipe[0]);
> +close(notify_pipe[1]);
> +notify_pipe[0] = notify_pipe[1] = -1;
> +}

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 2/3] symbols: Generate an xen-sym.map

[Jan Beulich]

>>> On 04.08.16 at 17:49,  wrote:
> You could construct _most_ of the names of the functions
> by doing 'nm --defined' but unfortunatly you do not get the
>  prefix that is added on in Xen . For example:
> 
> $ cat xen-syms.symbols |grep do_domain_pause
> 0x82d080104920 t domain.c#do_domain_pause
> $ nm --defined xen-syms|grep do_domain_pause
> 82d080104920 t do_domain_pause
> 
> This is normally not an issue, but if one is doing livepatching and
> wants during build-time verify that the symbols the livepatch payloads
> will patch do correspond to the one the hypervisor has built - this helps a 
> lot.
> 
> Note that during runtime one can do:
> 82d080104920 t domain.c#do_domain_pause
> 
> But one may not want to build and verify a livepatch on the same host.

Well, I'm not particularly happy about getting yet another file
produced, but so be it.

> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -99,6 +99,9 @@ $(TARGET)-syms: prelink.o xen.lds 
> $(BASEDIR)/common/symbols-dummy.o
>   $(@D)/.$(@F).0.o -o $(@D)/.$(@F).1
>   $(NM) -pa --format=sysv $(@D)/.$(@F).1 \
>   | $(BASEDIR)/tools/symbols --sysv --sort >$(@D)/.$(@F).1.S
> + $(NM) -pa --format=sysv $(@D)/.$(@F).1 \
> + | $(BASEDIR)/tools/symbols --xensyms --sysv --sort \
> + >$(@D)/$(@F).map
>   $(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
>   $(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
>   $(@D)/.$(@F).1.o -o $@
> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -135,6 +135,10 @@ $(TARGET)-syms: prelink.o xen.lds 
> $(BASEDIR)/common/symbols-dummy.o
>   $(NM) -pa --format=sysv $(@D)/.$(@F).1 \
>   | $(BASEDIR)/tools/symbols $(all_symbols) --sysv --sort 
> --warn-dup \
>   >$(@D)/.$(@F).1.S
> + $(NM) -pa --format=sysv $(@D)/.$(@F).1 \
> + | $(BASEDIR)/tools/symbols --xensyms --sysv --sort --warn-dup \
> + >$(@D)/$(@F).map
> + $(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
>   $(MAKE) -f $(BASEDIR)/Rules.mk $(@D)/.$(@F).1.o
>   $(LD) $(LDFLAGS) -T xen.lds -N prelink.o $(build_id_linker) \
>   $(@D)/.$(@F).1.o -o $@

Why are you producing the maps from intermediate rather than
the final binaries?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 8/8] tools/console: remove 5s bodge in console client

[Ian Jackson]

Wei Liu writes ("[RFC PATCH 8/8] tools/console: remove 5s bodge in console 
client"):
> The bug described in the comment has been fixed in libxl. Since console
> client is a private binary and libxl is the only supported toolstack in
> upstream, remove the bodge to simplify code.

I don't think this is correct.

In particular, what about
   xl create /etc/xen/foo.cfg
   xl console foo
?

I think in this case xenconsole still needs to wait.

TBH I wonder if this means that
 [RFC PATCH 4/8] libxl: wait up to 5s in libxl_console_exec for xenconsoled
is actually wrong, despite my ack.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 8/8] tools/console: remove 5s bodge in console client

[Wei Liu]

On Fri, Aug 05, 2016 at 04:57:33PM +0100, Ian Jackson wrote:
> Wei Liu writes ("[RFC PATCH 8/8] tools/console: remove 5s bodge in console 
> client"):
> > The bug described in the comment has been fixed in libxl. Since console
> > client is a private binary and libxl is the only supported toolstack in
> > upstream, remove the bodge to simplify code.
> 
> I don't think this is correct.
> 
> In particular, what about
>xl create /etc/xen/foo.cfg
>xl console foo
> ?
> 
> I think in this case xenconsole still needs to wait.
> 

xl console will call libxl_console_exec which should wait for the tty
node to appear before exec'ing xenconsole.

> TBH I wonder if this means that
>  [RFC PATCH 4/8] libxl: wait up to 5s in libxl_console_exec for xenconsoled
> is actually wrong, despite my ack.
> 

You didn't ack that one. Maybe you mean something else?

Wei.

> Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] libxl: return any serial tty path in libxl_console_get_tty

[Ian Jackson]

Wei Liu writes ("Re: [PATCH v2] libxl: return any serial tty path in 
libxl_console_get_tty"):
> On Thu, Aug 04, 2016 at 09:07:45AM +0100, Wei Liu wrote:
> > Ian, I think this is a backport candidate.

Queued, thanks.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 8/8] tools/console: remove 5s bodge in console client

[Ian Jackson]

Wei Liu writes ("Re: [RFC PATCH 8/8] tools/console: remove 5s bodge in console 
client"):
> On Fri, Aug 05, 2016 at 04:57:33PM +0100, Ian Jackson wrote:
> > In particular, what about
> >xl create /etc/xen/foo.cfg
> >xl console foo
> > ?
> > 
> > I think in this case xenconsole still needs to wait.
> 
> xl console will call libxl_console_exec which should wait for the tty
> node to appear before exec'ing xenconsole.

Well, except that xenconsole is also on the path.  I (now) don't
understand why this functionality needs to be moved from xenconsole to
libxl.

> > TBH I wonder if this means that
> >  [RFC PATCH 4/8] libxl: wait up to 5s in libxl_console_exec for xenconsoled
> > is actually wrong, despite my ack.
> > 
> 
> You didn't ack that one. Maybe you mean something else?

I meant "despite my implicit approval of the principle".

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 8/8] tools/console: remove 5s bodge in console client

[Wei Liu]

On Fri, Aug 05, 2016 at 05:18:47PM +0100, Ian Jackson wrote:
> Wei Liu writes ("Re: [RFC PATCH 8/8] tools/console: remove 5s bodge in 
> console client"):
> > On Fri, Aug 05, 2016 at 04:57:33PM +0100, Ian Jackson wrote:
> > > In particular, what about
> > >xl create /etc/xen/foo.cfg
> > >xl console foo
> > > ?
> > > 
> > > I think in this case xenconsole still needs to wait.
> > 
> > xl console will call libxl_console_exec which should wait for the tty
> > node to appear before exec'ing xenconsole.
> 
> Well, except that xenconsole is also on the path.  I (now) don't
> understand why this functionality needs to be moved from xenconsole to
> libxl.
> 

There needs to be a entity that waits for the node to appear. I thought
libxl was in a better position.  That being said, I think let xenconsole
wait would also work.

I don't have a strong preference on how to proceed.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Device model operation hypercall (DMOP, re qemu depriv)

[Ian Jackson]

Jan Beulich writes ("Re: Device model operation hypercall (DMOP, re qemu 
depriv)"):
> On 04.08.16 at 13:21,  wrote:
> > What we cannot do is audit every HVMCTL, fix the class 2 problems, and
> > then declare HVMCTL to have the relevant security property, and
> > implement corresponding code in dom0's privcmd drivers which relies on
> > the security property.  This is because the dom0 privcmd driver
> > doesn't know whether the HVMCTLs it is allowing not-fully-trusted
> > userspace to make are actually trustworthy (with the specific
> > hypervisor version in question.)
> 
> I continue to not really understand this argumentation: Dom0's
> privcmd driver doesn't really matter here. If there's a bug in
> something qemu uses, this is a problem no matter whether that
> operation gets called though the to-be-added privcmd logic, or
> straight from a stubdom qemu. Both are less than fully privileged.
> What do I continue to be missing?

Let me try again.  Earlier I wrote:

  AFAICT there are two kinds of possible bug:

  1. An HVMCTL (or hvmop) that can have an adverse affect on the whole
  system.  Such bugs would already be security bugs, covered by our
  security support policy.  Such a bug would be a security bug whether
  the operation is an hvmop or an HVMCTL or a DMOP.

  2. An HVMCTL (or hvmop) that can have an adverse effect on the calling
  domain.  Such bugs are not currently security bugs.  But the of qemu
  depriv project requires them to be removed.  Such such a bug is a
  security bug if it is a DMOP[1] but not otherwise.

Bugs of class 1 are already security bugs.  They can already be
exploited by stub device models.

Bugs of class 2 are only security bugs if we allow unprivileged
callers within a privileged domain to use the corresponding hypercall.

That is, a bug of class 2 would allow the unprivileged qemu process in
dom0 to cause damage to other parts of dom0.

Bugs of class 2 are of no interest to an attacker who has control of a
stub device model, because all it allows them to do is damage the
device domain domain (which they already control).

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 8/8] tools/console: remove 5s bodge in console client

[Ian Jackson]

Wei Liu writes ("Re: [RFC PATCH 8/8] tools/console: remove 5s bodge in console 
client"):
> On Fri, Aug 05, 2016 at 05:18:47PM +0100, Ian Jackson wrote:
> > Well, except that xenconsole is also on the path.  I (now) don't
> > understand why this functionality needs to be moved from xenconsole to
> > libxl.
> 
> There needs to be a entity that waits for the node to appear. I thought
> libxl was in a better position.  That being said, I think let xenconsole
> wait would also work.

My point is that xenconsole seems to have the functionality already.
You are moving it to libxl.

The sole visible effect of this is to make xenconsole unreliable for
people who invoke it directly.

Or have I misunderstood ?

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 8/8] tools/console: remove 5s bodge in console client

[Wei Liu]

On Fri, Aug 05, 2016 at 05:32:30PM +0100, Ian Jackson wrote:
> Wei Liu writes ("Re: [RFC PATCH 8/8] tools/console: remove 5s bodge in 
> console client"):
> > On Fri, Aug 05, 2016 at 05:18:47PM +0100, Ian Jackson wrote:
> > > Well, except that xenconsole is also on the path.  I (now) don't
> > > understand why this functionality needs to be moved from xenconsole to
> > > libxl.
> > 
> > There needs to be a entity that waits for the node to appear. I thought
> > libxl was in a better position.  That being said, I think let xenconsole
> > wait would also work.
> 
> My point is that xenconsole seems to have the functionality already.
> You are moving it to libxl.
> 

Yes, that's what I did.

> The sole visible effect of this is to make xenconsole unreliable for
> people who invoke it directly.
> 

No, it should (at least by design) still be reliable since the libxl now
waits before exec xenconsole.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] gnttab: Add gntdev device mappings for FreeBSD

[Roger Pau Monné]

On Thu, Aug 04, 2016 at 06:23:51PM +0530, Akshay Jaggi wrote:
> Add grant table userspace device mappings for
> FreeBSD (enables support for qdisk backend
> on FreeBSD Dom0).
> 
> Signed-off-by: Akshay Jaggi 
> ---
>  tools/include/xen-sys/FreeBSD/gntdev.h | 118 
>  tools/libs/gnttab/Makefile |   2 +-
>  tools/libs/gnttab/freebsd.c| 335 
> +
>  3 files changed, 454 insertions(+), 1 deletion(-)
>  create mode 100644 tools/include/xen-sys/FreeBSD/gntdev.h
>  create mode 100644 tools/libs/gnttab/freebsd.c
> 
> diff --git a/tools/include/xen-sys/FreeBSD/gntdev.h 
> b/tools/include/xen-sys/FreeBSD/gntdev.h
> new file mode 100644
> index 000..1d09c5d
> --- /dev/null
> +++ b/tools/include/xen-sys/FreeBSD/gntdev.h
> @@ -0,0 +1,118 @@
> +/*-
> + * Copyright (c) 2016 Akshay Jaggi 
> + * All rights reserved.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + * 1. Redistributions of source code must retain the above copyright
> + *notice, this list of conditions and the following disclaimer.
> + * 2. Redistributions in binary form must reproduce the above copyright
> + *notice, this list of conditions and the following disclaimer in the
> + *documentation and/or other materials provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> + * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
> + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
> + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
> + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> + * SUCH DAMAGE.
> + *
> + * gntdev.h
> + * 
> + * Interface to /dev/xen/gntdev.
> + * 
> + */
> +
> +#ifndef __XEN_GNTDEV_H__
> +#define __XEN_GNTDEV_H__
> +
> +#include 
> +
> +#define IOCTL_GNTDEV_SET_UNMAP_NOTIFY
> \
> + _IOW('E', 0, struct ioctl_gntdev_unmap_notify)
> +struct ioctl_gntdev_unmap_notify {
> +/* IN parameters */
> +uint64_t index;
> +uint32_t action;
> +uint32_t event_channel_port;
> +};
> +
> +#define UNMAP_NOTIFY_CLEAR_BYTE 0x1
> +#define UNMAP_NOTIFY_SEND_EVENT 0x2
> +
> +/* Grant Creation IOCTLs  
> */
> +
> +#define IOCTL_GNTDEV_ALLOC_GREF  
> \
> + _IOWR('E', 1, struct ioctl_gntdev_alloc_gref)
> +struct ioctl_gntdev_alloc_gref {
> +/* IN parameters */
> +uint16_t domid;
> +uint16_t flags;
> +uint32_t count;
> +/* OUT parameters */
> +uint64_t index;
> +/* Variable OUT parameter */
> +uint32_t gref_ids[1];
> +};
> +
> +#define GNTDEV_ALLOC_FLAG_WRITABLE 1
> +
> +#define IOCTL_GNTDEV_DEALLOC_GREF\
> + _IOW('E', 2, struct ioctl_gntdev_dealloc_gref)
> +struct ioctl_gntdev_dealloc_gref {
> +/* IN parameters */
> +uint64_t index;
> +uint32_t count;
> +};
> +
> +/* Grant Accessing IOCTLs  
> ---*/
^ I would say "Mapping" here instead.

> +
> +struct ioctl_gntdev_grant_ref {
> +uint32_t domid;
> +uint32_t ref;
> +};
> +
> +#define IOCTL_GNTDEV_MAP_GRANT_REF   \
> + _IOWR('E', 3, struct ioctl_gntdev_map_grant_ref)
> +struct ioctl_gntdev_map_grant_ref {
> +/* IN parameters */
> +uint32_t count;
> +uint32_t pad0;
> +/* OUT parameters */
> +uint64_t index;
> +/* Variable IN parameter */
> +struct ioctl_gntdev_grant_ref refs[1];
> +};
> +
> +#define IOCTL_GNTDEV_UNMAP_GRANT_REF \
> + _IOW('E', 4, struct ioctl_gntdev_unmap_grant_ref)
> +struct ioctl_gntdev_unmap_grant_ref {
> +/* IN parameters */
> +uint64_t index;
> +uint32_t count;
> +};
> +
> +#define IOCTL_GNTDEV_GET_OFFSET_FOR_VADDR\
> + _IOWR('E', 5, struct ioctl_gntdev_get_offset_for_vaddr)
> +struct ioctl_gntdev_get_offset_for_vaddr {
> +/* IN parameters */
> +uint64_t vaddr;
> +/* OUT parameters */
> +uint64_t offset;
> +uint32_t count;
> +};
> +
> +#define IOCTL_GNTDEV_SET_MAX_GRANTS  \
> + _IOW('E', 6, struct ioctl_gntdev_set_max_grants)
> +struct ioctl_gntdev_set_max_grants {
> +/* IN parameters */
> +uint32_t count;

This seems usel

Re: [Xen-devel] [PATCH v2] domctl: relax getdomaininfo permissions

[Andrew Cooper]

On 05/08/16 14:54, Jan Beulich wrote:
 On 05.08.16 at 15:10,  wrote:
>> On 05/08/16 12:20, Jan Beulich wrote:
>>> I wonder what good the duplication of the returned domain ID does: I'm
>>> tempted to remove the one in the command-specific structure. Does
>>> anyone have insight into why it was done that way?
>> I wonder whether the first incarnation of this hypercall lacked a domid
>> field in the returned structure?  It seems like the kind of thing which
>> would be omitted, until the sysctl list version got introduced.
> Oh, good point - that makes clear why the field can't be dropped:
> That sysctl would break then.

Which domid were you referring to then?

The domid in the xen_domctl_getdomaininfo structure clearly needs to
stay, but the domctl "op->domain = op->u.getdomaininfo.domain;"
needn't.  OTOH, as we need to copy back the entire domctl structure
anyway, it doesn't hurt to keep it.

>
>>> --- a/xen/include/xsm/dummy.h
>>> +++ b/xen/include/xsm/dummy.h
>>> @@ -61,7 +61,12 @@ static always_inline int xsm_default_act
>>>  return 0;
>>>  case XSM_TARGET:
>>>  if ( src == target )
>>> +{
>>>  return 0;
>>> +case XSM_XS_PRIV:
>>> +if ( src->is_xenstore )
>>> +return 0;
>>> +}
>>>  /* fall through */
>>>  case XSM_DM_PRIV:
>>>  if ( target && src->target == target )
>>> @@ -71,10 +76,6 @@ static always_inline int xsm_default_act
>>>  if ( src->is_privileged )
>>>  return 0;
>>>  return -EPERM;
>>> -case XSM_XS_PRIV:
>>> -if ( src->is_xenstore || src->is_privileged )
>>> -return 0;
>>> -return -EPERM;
>>>  default:
>>>  LINKER_BUG_ON(1);
>>>  return -EPERM;
>> What is this change in relation to?  I can't see how it is related to
>> the XSM changes mentioned in the commit, as that is strictly for the use
>> of XSM_OTHER.
> I don't see any XSM changes mentioned in the description, there
> was only the XSM_OTHER related question outside the description.
> Anyway - the change above is what guarantees the XSM_XS_PRIV
> check, as invoked by xsm_domctl()'s XEN_DOMCTL_getdomaininfo
> case, to fall through into XSM_DM_PRIV - after all that's what the
> whole patch is about.

But the patch is about a qemu stubdom, which would be DM_PRIV, not XS_PRIV.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 8/8] tools/console: remove 5s bodge in console client

[Wei Liu]

On Fri, Aug 05, 2016 at 05:36:08PM +0100, Wei Liu wrote:
> On Fri, Aug 05, 2016 at 05:32:30PM +0100, Ian Jackson wrote:
> > Wei Liu writes ("Re: [RFC PATCH 8/8] tools/console: remove 5s bodge in 
> > console client"):
> > > On Fri, Aug 05, 2016 at 05:18:47PM +0100, Ian Jackson wrote:
> > > > Well, except that xenconsole is also on the path.  I (now) don't
> > > > understand why this functionality needs to be moved from xenconsole to
> > > > libxl.
> > > 
> > > There needs to be a entity that waits for the node to appear. I thought
> > > libxl was in a better position.  That being said, I think let xenconsole
> > > wait would also work.
> > 
> > My point is that xenconsole seems to have the functionality already.
> > You are moving it to libxl.
> > 
> 
> Yes, that's what I did.
> 
> > The sole visible effect of this is to make xenconsole unreliable for
> > people who invoke it directly.
> > 
> 
> No, it should (at least by design) still be reliable since the libxl now
> waits before exec xenconsole.

Now I have decided to drop patch 4 -- the less patch the better. :-)

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 05/18] mini-os: add nr_free_pages counter

[Juergen Gross]

Add a variable holding the number of available memory pages. This will
aid auto-ballooning later.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
---
 include/mm.h | 1 +
 mm.c | 6 ++
 2 files changed, 7 insertions(+)

diff --git a/include/mm.h b/include/mm.h
index a48f485..b97b43e 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -42,6 +42,7 @@
 #define STACK_SIZE_PAGE_ORDER __STACK_SIZE_PAGE_ORDER
 #define STACK_SIZE __STACK_SIZE
 
+extern unsigned long nr_free_pages;
 
 void init_mm(void);
 unsigned long alloc_pages(int order);
diff --git a/mm.c b/mm.c
index 0dd4862..263a356 100644
--- a/mm.c
+++ b/mm.c
@@ -53,6 +53,8 @@ static unsigned long *alloc_bitmap;
 #define allocated_in_map(_pn) \
 (alloc_bitmap[(_pn)/PAGES_PER_MAPWORD] & (1UL<<((_pn)&(PAGES_PER_MAPWORD-1
 
+unsigned long nr_free_pages;
+
 /*
  * Hint regarding bitwise arithmetic in map_{alloc,free}:
  *  -(1<= n. 
@@ -81,6 +83,8 @@ static void map_alloc(unsigned long first_page, unsigned long 
nr_pages)
 while ( ++curr_idx < end_idx ) alloc_bitmap[curr_idx] = ~0UL;
 alloc_bitmap[curr_idx] |= (1UL

[Xen-devel] [PATCH v2 14/18] mini-os: move p2m related macros to header file

[Juergen Gross]

In order to be able to use p2m related macros for ballooning move
their definitions to arch/x86/mm.h.

There is no need to define different macros regarding index masks and
number of entries for the different levels, as all levels share the
same entry format (a plain mfn). So reduce the number of macros
accordingly.

Add some macros to get the indices into p2m pages from a pfn and make
use of them in current p2m code.

Signed-off-by: Juergen Gross 
---
 arch/x86/mm.c | 31 +--
 include/x86/arch_mm.h | 21 +
 2 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index e10c2c5..a5c8959 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -609,40 +609,19 @@ static void clear_bootstrap(void)
 
 void arch_init_p2m(unsigned long max_pfn)
 {
-#ifdef __x86_64__
-#define L1_P2M_SHIFT9
-#define L2_P2M_SHIFT18
-#define L3_P2M_SHIFT27
-#else
-#define L1_P2M_SHIFT10
-#define L2_P2M_SHIFT20
-#define L3_P2M_SHIFT30
-#endif
-#define L1_P2M_ENTRIES  (1 << L1_P2M_SHIFT)
-#define L2_P2M_ENTRIES  (1 << (L2_P2M_SHIFT - L1_P2M_SHIFT))
-#define L3_P2M_ENTRIES  (1 << (L3_P2M_SHIFT - L2_P2M_SHIFT))
-#define L1_P2M_MASK (L1_P2M_ENTRIES - 1)
-#define L2_P2M_MASK (L2_P2M_ENTRIES - 1)
-#define L3_P2M_MASK (L3_P2M_ENTRIES - 1)
-
 unsigned long *l2_list = NULL, *l3_list;
 unsigned long pfn;
 
+p2m_chk_pfn(max_pfn - 1);
 l3_list = (unsigned long *)alloc_page(); 
-for ( pfn = 0; pfn < max_pfn; pfn += L1_P2M_ENTRIES )
+for ( pfn = 0; pfn < max_pfn; pfn += P2M_ENTRIES )
 {
-if ( !(pfn % (L1_P2M_ENTRIES * L2_P2M_ENTRIES)) )
+if ( !(pfn % (P2M_ENTRIES * P2M_ENTRIES)) )
 {
 l2_list = (unsigned long*)alloc_page();
-if ( (pfn >> L3_P2M_SHIFT) > 0 )
-{
-printk("Error: Too many pfns.\n");
-do_exit();
-}
-l3_list[(pfn >> L2_P2M_SHIFT)] = virt_to_mfn(l2_list);  
+l3_list[L3_P2M_IDX(pfn)] = virt_to_mfn(l2_list);
 }
-l2_list[(pfn >> L1_P2M_SHIFT) & L2_P2M_MASK] =
-virt_to_mfn(phys_to_machine_mapping + pfn);
+l2_list[L2_P2M_IDX(pfn)] = virt_to_mfn(phys_to_machine_mapping + pfn);
 }
 HYPERVISOR_shared_info->arch.pfn_to_mfn_frame_list_list = 
 virt_to_mfn(l3_list);
diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index d87fe55..7283f64 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -176,7 +176,28 @@ typedef unsigned long pgentry_t;
 #define IO_PROT_NOCACHE (L1_PROT | _PAGE_PCD)
 
 /* for P2M */
+#ifdef __x86_64__
+#define P2M_SHIFT   9
+#else
+#define P2M_SHIFT   10
+#endif
+#define P2M_ENTRIES (1UL << P2M_SHIFT)
+#define P2M_MASK(P2M_ENTRIES - 1)
+#define L1_P2M_SHIFTP2M_SHIFT
+#define L2_P2M_SHIFT(2 * P2M_SHIFT)
+#define L3_P2M_SHIFT(3 * P2M_SHIFT)
+#define L1_P2M_IDX(pfn) ((pfn) & P2M_MASK)
+#define L2_P2M_IDX(pfn) (((pfn) >> L1_P2M_SHIFT) & P2M_MASK)
+#define L3_P2M_IDX(pfn) (((pfn) >> L2_P2M_SHIFT) & P2M_MASK)
 #define INVALID_P2M_ENTRY (~0UL)
+static inline void p2m_chk_pfn(unsigned long pfn)
+{
+if ( (pfn >> L3_P2M_SHIFT) > 0 )
+{
+printk("Error: Too many pfns.\n");
+do_exit();
+}
+}
 
 #include "arch_limits.h"
 #define PAGE_SIZE   __PAGE_SIZE
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 01/18] mini-os: correct first free pfn

[Juergen Gross]

The first free pfn available for allocation is calculated by adding the
number of page table frames to the pfn of the first page table and
then the magic number 3 to account for start info page et al.

As the start info page, xenstore page and console page are allocated
_before_ the page tables leaving room for these pages behind the page
tables makes no sense.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
---
 arch/x86/mm.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index 51aa966..ae1036e 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -867,9 +867,8 @@ void arch_init_mm(unsigned long* start_pfn_p, unsigned 
long* max_pfn_p)
 printk("stack start: %p(VA)\n", stack);
 printk("   _end: %p(VA)\n", &_end);
 
-/* First page follows page table pages and 3 more pages (store page etc) */
-start_pfn = PFN_UP(to_phys(start_info.pt_base)) + 
-start_info.nr_pt_frames + 3;
+/* First page follows page table pages. */
+start_pfn = PFN_UP(to_phys(start_info.pt_base)) + start_info.nr_pt_frames;
 max_pfn = start_info.nr_pages;
 
 /* We need room for demand mapping and heap, clip available memory */
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 02/18] mini-os: remove unused alloc_contig_pages() function

[Juergen Gross]

alloc_contig_pages() is never used anywhere in mini-os. Remove it.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
---
 arch/x86/mm.c | 142 --
 include/mm.h  |   1 -
 2 files changed, 143 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index ae1036e..c59a5d3 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -652,148 +652,6 @@ int unmap_frames(unsigned long va, unsigned long 
num_frames)
 }
 
 /*
- * Allocate pages which are contiguous in machine memory.
- * Returns a VA to where they are mapped or 0 on failure.
- * 
- * addr_bits indicates if the region has restrictions on where it is
- * located. Typical values are 32 (if for example PCI devices can't access
- * 64bit memory) or 0 for no restrictions.
- *
- * Allocated pages can be freed using the page allocators free_pages() 
- * function.
- *
- * based on Linux function xen_create_contiguous_region()
- */
-#define MAX_CONTIG_ORDER 9 /* 2MB */
-unsigned long alloc_contig_pages(int order, unsigned int addr_bits)
-{
-unsigned long in_va, va;
-unsigned long in_frames[1UL << order], out_frames, mfn;
-multicall_entry_t call[1UL << order];
-unsigned int i, num_pages = 1UL << order;
-int ret, exch_success;
-
-/* pass in num_pages 'extends' of size 1 and
- * request 1 extend of size 'order */
-struct xen_memory_exchange exchange = {
-.in = {
-.nr_extents   = num_pages,
-.extent_order = 0,
-.domid= DOMID_SELF
-},
-.out = {
-.nr_extents   = 1,
-.extent_order = order,
-.address_bits = addr_bits,
-.domid= DOMID_SELF
-},
-.nr_exchanged = 0
-};
-
-if ( order > MAX_CONTIG_ORDER )
-{
-printk("alloc_contig_pages: order too large 0x%x > 0x%x\n",
-   order, MAX_CONTIG_ORDER);
-return 0;
-}
-
-/* Allocate some potentially discontiguous pages */
-in_va = alloc_pages(order);
-if ( !in_va )
-{
-printk("alloc_contig_pages: could not get enough pages (order=0x%x\n",
-   order);
-return 0;
-}
-
-/* set up arguments for exchange hyper call */
-set_xen_guest_handle(exchange.in.extent_start, in_frames);
-set_xen_guest_handle(exchange.out.extent_start, &out_frames);
-
-/* unmap current frames, keep a list of MFNs */
-for ( i = 0; i < num_pages; i++ )
-{
-int arg = 0;
-
-va = in_va + (PAGE_SIZE * i);
-in_frames[i] = virt_to_mfn(va);
-
-/* update P2M mapping */
-phys_to_machine_mapping[virt_to_pfn(va)] = INVALID_P2M_ENTRY;
-
-/* build multi call */
-call[i].op = __HYPERVISOR_update_va_mapping;
-call[i].args[arg++] = va;
-call[i].args[arg++] = 0;
-#ifdef __i386__
-call[i].args[arg++] = 0;
-#endif  
-call[i].args[arg++] = UVMF_INVLPG;
-}
-
-ret = HYPERVISOR_multicall(call, i);
-if ( ret )
-{
-printk("Odd, update_va_mapping hypercall failed with rc=%d.\n", ret);
-return 0;
-}
-
-/* try getting a contig range of MFNs */
-out_frames = virt_to_pfn(in_va); /* PFNs to populate */
-ret = HYPERVISOR_memory_op(XENMEM_exchange, &exchange);
-if ( ret ) {
-printk("mem exchanged order=0x%x failed with rc=%d, 
nr_exchanged=%lu\n",
-   order, ret, exchange.nr_exchanged);
-/* we still need to return the allocated pages above to the pool
- * ie. map them back into the 1:1 mapping etc. so we continue but 
- * in the end return the pages to the page allocator and return 0. */
-exch_success = 0;
-}
-else
-exch_success = 1;
-
-/* map frames into 1:1 and update p2m */
-for ( i = 0; i < num_pages; i++ )
-{
-int arg = 0;
-pte_t pte;
-
-va = in_va + (PAGE_SIZE * i);
-mfn = i < exchange.nr_exchanged ? (out_frames + i) : in_frames[i];
-pte = __pte(mfn << PAGE_SHIFT | L1_PROT);
-
-/* update P2M mapping */
-phys_to_machine_mapping[virt_to_pfn(va)] = mfn;
-
-/* build multi call */
-call[i].op = __HYPERVISOR_update_va_mapping;
-call[i].args[arg++] = va;
-#ifdef __x86_64__
-call[i].args[arg++] = (pgentry_t)pte.pte;
-#else
-call[i].args[arg++] = pte.pte_low;
-call[i].args[arg++] = pte.pte_high;
-#endif  
-call[i].args[arg++] = UVMF_INVLPG;
-}
-ret = HYPERVISOR_multicall(call, i);
-if ( ret )
-{
-printk("update_va_mapping hypercall no. 2 failed with rc=%d.\n", ret);
-return 0;
-}
-
-if ( !exch_success )
-{
-/* since the exchanged failed we just free the pages as well */
-free_pages((void *) in_va, order);
-return 0;
-}
-
-return in_va;
-}
-
-/*
  * Clear some of the bootstrap memory
  */
 static void clear_bootstrap(void)
diff --git a/include/mm.h b/include/mm.h
ind

[Xen-devel] [PATCH v2 09/18] mini-os: modify virtual memory layout for support of ballooning

[Juergen Gross]

In order to be able to support ballooning the virtual memory layout
of Mini-OS has to be modified: instead of a (nearly) consecutive
area used for physical memory mapping, on demand mappings, and heap
we need enough spare place for adding new memory.

So instead of dynamically place the different regions based on found
memory size locate them statically at fixed virtual addresses:

area   x86-64   x86-32

mapped physical memory    
kernel virtual mappings80 3f00
demand mappings  1000 4000
heap 2000 b000

This will enable Mini-OS to support up to 512GB of domain memory with
a 64 bit kernel and nearly 1GB with a 32 bit kernel.

For a 32 bit Mini-OS we have to avoid a conflict between heap and
m2p table which the hypervisor maps at f560. So the demand mapping
size is reduced by 256MB in order to keep the heap at about 1GB.

The kernel virtual mappings are a new area needed for being able to
grow the p2m list without having to relocate it in physical memory.

Modify the placement of the demand mappings and heap and adjust the
memory layout description.

Signed-off-by: Juergen Gross 
---
V2: avoid conflict with hypervisor mapped area on 32 bits
---
 arch/arm/mm.c |  2 +-
 arch/x86/mm.c | 44 +++-
 include/mm.h  |  2 +-
 include/x86/arch_mm.h | 35 ++-
 mm.c  |  2 +-
 5 files changed, 44 insertions(+), 41 deletions(-)

diff --git a/arch/arm/mm.c b/arch/arm/mm.c
index efecc51..f75888d 100644
--- a/arch/arm/mm.c
+++ b/arch/arm/mm.c
@@ -75,7 +75,7 @@ void arch_init_p2m(unsigned long max_pfn)
 {
 }
 
-void arch_init_demand_mapping_area(unsigned long cur_pfn)
+void arch_init_demand_mapping_area(void)
 {
 }
 
diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index c59a5d3..6aa4468 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -442,37 +442,21 @@ pgentry_t *need_pgt(unsigned long va)
  * Reserve an area of virtual address space for mappings and Heap
  */
 static unsigned long demand_map_area_start;
-#ifdef __x86_64__
-#define DEMAND_MAP_PAGES ((128ULL << 30) / PAGE_SIZE)
-#else
-#define DEMAND_MAP_PAGES ((2ULL << 30) / PAGE_SIZE)
-#endif
-
-#ifndef HAVE_LIBC
-#define HEAP_PAGES 0
-#else
+static unsigned long demand_map_area_end;
+#ifdef HAVE_LIBC
 unsigned long heap, brk, heap_mapped, heap_end;
-#ifdef __x86_64__
-#define HEAP_PAGES ((128ULL << 30) / PAGE_SIZE)
-#else
-#define HEAP_PAGES ((1ULL << 30) / PAGE_SIZE)
-#endif
 #endif
 
-void arch_init_demand_mapping_area(unsigned long cur_pfn)
+void arch_init_demand_mapping_area(void)
 {
-cur_pfn++;
-
-demand_map_area_start = (unsigned long) pfn_to_virt(cur_pfn);
-cur_pfn += DEMAND_MAP_PAGES;
-printk("Demand map pfns at %lx-%p.\n", 
-   demand_map_area_start, pfn_to_virt(cur_pfn));
+demand_map_area_start = VIRT_DEMAND_AREA;
+demand_map_area_end = demand_map_area_start + DEMAND_MAP_PAGES * PAGE_SIZE;
+printk("Demand map pfns at %lx-%lx.\n", demand_map_area_start,
+   demand_map_area_end);
 
 #ifdef HAVE_LIBC
-cur_pfn++;
-heap_mapped = brk = heap = (unsigned long) pfn_to_virt(cur_pfn);
-cur_pfn += HEAP_PAGES;
-heap_end = (unsigned long) pfn_to_virt(cur_pfn);
+heap_mapped = brk = heap = VIRT_HEAP_AREA;
+heap_end = heap_mapped + HEAP_PAGES * PAGE_SIZE;
 printk("Heap resides at %lx-%lx.\n", brk, heap_end);
 #endif
 }
@@ -729,14 +713,8 @@ void arch_init_mm(unsigned long* start_pfn_p, unsigned 
long* max_pfn_p)
 start_pfn = PFN_UP(to_phys(start_info.pt_base)) + start_info.nr_pt_frames;
 max_pfn = start_info.nr_pages;
 
-/* We need room for demand mapping and heap, clip available memory */
-#if defined(__i386__)
-{
-unsigned long virt_pfns = 1 + DEMAND_MAP_PAGES + 1 + HEAP_PAGES;
-if (max_pfn + virt_pfns >= 0x10)
-max_pfn = 0x10 - virt_pfns - 1;
-}
-#endif
+if ( max_pfn >= MAX_MEM_SIZE / PAGE_SIZE )
+max_pfn = MAX_MEM_SIZE / PAGE_SIZE - 1;
 
 printk("  start_pfn: %lx\n", start_pfn);
 printk("max_pfn: %lx\n", max_pfn);
diff --git a/include/mm.h b/include/mm.h
index b97b43e..a22dcd1 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -59,7 +59,7 @@ static __inline__ int get_order(unsigned long size)
 return order;
 }
 
-void arch_init_demand_mapping_area(unsigned long max_pfn);
+void arch_init_demand_mapping_area(void);
 void arch_init_mm(unsigned long* start_pfn_p, unsigned long* max_pfn_p);
 void arch_init_p2m(unsigned long max_pfn_p);
 
diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index f756dab..d87fe55 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -49,11 +49,13 @@
  *
  * Virtual address space usage:
  *
- * 1:1 mapping of physical memory starting at VA(0)
- * 

[Xen-devel] [PATCH v2 00/18] mini-os: support of auto-ballooning

[Juergen Gross]

Support ballooning Mini-OS automatically up in case of memory shortage.

Do some cleanups, a small correction and add some basic features to
lay groundwork for support of ballooning in Mini-OS (patches 1-14).

The main visible change is the virtual memory layout: to be able to
add memory to the running Mini-OS we need to have some spare areas
especially after the 1:1 mapping of physical memory.

Then add the ballooning functionality: the p2m map must be expanded,
the page allocator's bitmap must  be expanded and we must get new
memory from the hypervisor.

In case of a detected memory shortage the domain will balloon up until
either enough memory is available or the upper limit has been reached.

Ballooning has been tested with a xenstore stubdom.
Regression tests have been done with:
- pure mini-os
- ioemu stubdom
- pvgrub 64 bit

pvgrub 32 bit didn't work before applying the series, it just entered
the grub shell. With the series applied the behavior was exactly the
same. The grub shell however was working (I tried "help" and "reboot").

I tried to modify arm specific files in order not to break the
non-ballooning case, but I haven't tested it to either work or to
compile.

V1 of this series consisted of patches 1-9 only.

Changes in V2:
- added patches 10-18
- some coding style corrections
- patch 7: introduced balloon specific source files
- moved ballooning specific functions/definitions to ballon specific
  files
- patch 9: avoid conflict with hypervisor mapped area on 32 bits

Juergen Gross (18):
  mini-os: correct first free pfn
  mini-os: remove unused alloc_contig_pages() function
  mini-os: remove MM_DEBUG code
  mini-os: add description of x86 memory usage
  mini-os: add nr_free_pages counter
  mini-os: let memory allocation fail if no free page available
  mini-os: add ballooning config item
  mini-os: get maximum memory size from hypervisor
  mini-os: modify virtual memory layout for support of ballooning
  mini-os: remove unused mem_test() function
  mini-os: add checks for out of memory
  mini-os: don't allocate new pages for level 1 p2m tree
  mini-os: add function to map one frame
  mini-os: move p2m related macros to header file
  mini-os: remap p2m list in case of ballooning
  mini-os: map page allocator's bitmap to virtual kernel area for
ballooning
  mini-os: add support for ballooning up
  mini-os: balloon up in case of oom

 Makefile  |   3 +
 arch/arm/balloon.c|  39 +++
 arch/arm/mm.c |  10 +-
 arch/x86/balloon.c| 146 +++
 arch/x86/mm.c | 314 +++---
 balloon.c | 159 +
 include/arm/arch_mm.h |   2 +
 include/balloon.h |  59 ++
 include/mm.h  |  13 ++-
 include/x86/arch_mm.h |  70 +++
 mm.c  | 108 ++---
 11 files changed, 575 insertions(+), 348 deletions(-)
 create mode 100644 arch/arm/balloon.c
 create mode 100644 arch/x86/balloon.c
 create mode 100644 balloon.c
 create mode 100644 include/balloon.h

-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 06/18] mini-os: let memory allocation fail if no free page available

[Juergen Gross]

Instead of panicing when no page can be allocated try to fail the
memory allocation by returning NULL instead.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
---
V2: fixed minor style issue
---
 mm.c | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/mm.c b/mm.c
index 263a356..8cf3210 100644
--- a/mm.c
+++ b/mm.c
@@ -335,6 +335,13 @@ void *sbrk(ptrdiff_t increment)
 
 if (new_brk > heap_mapped) {
 unsigned long n = (new_brk - heap_mapped + PAGE_SIZE - 1) / PAGE_SIZE;
+
+if ( n > nr_free_pages )
+{
+printk("Memory exhausted: want %ld pages, but only %ld are left\n",
+   n, nr_free_pages);
+return NULL;
+}
 do_map_zero(heap_mapped, n);
 heap_mapped += n * PAGE_SIZE;
 }
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 13/18] mini-os: add function to map one frame

[Juergen Gross]

Add a function to map one physical frame to a specified virtual
address as read/write. This will be used later multiple times.

Signed-off-by: Juergen Gross 
---
 include/arm/arch_mm.h | 2 ++
 include/mm.h  | 1 +
 mm.c  | 5 +
 3 files changed, 8 insertions(+)

diff --git a/include/arm/arch_mm.h b/include/arm/arch_mm.h
index 085d4e5..f4685d8 100644
--- a/include/arm/arch_mm.h
+++ b/include/arm/arch_mm.h
@@ -14,6 +14,8 @@ extern uint32_t physical_address_offset;  /* Add this to 
a virtual address to get
 
 #define L1_PAGETABLE_SHIFT  12
 
+#define L1_PROT  0
+
 #define to_phys(x) (((paddr_t)(x)+physical_address_offset) & 
0x)
 #define to_virt(x) ((void *)(((x)-physical_address_offset) & 
0x))
 
diff --git a/include/mm.h b/include/mm.h
index 9244e26..6add683 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -72,6 +72,7 @@ int do_map_frames(unsigned long addr,
 const unsigned long *f, unsigned long n, unsigned long stride,
unsigned long increment, domid_t id, int *err, unsigned long prot);
 int unmap_frames(unsigned long va, unsigned long num_frames);
+int map_frame_rw(unsigned long addr, unsigned long mfn);
 #ifdef HAVE_LIBC
 extern unsigned long heap, brk, heap_mapped, heap_end;
 #endif
diff --git a/mm.c b/mm.c
index ff071ca..fd66115 100644
--- a/mm.c
+++ b/mm.c
@@ -319,6 +319,11 @@ int free_physical_pages(xen_pfn_t *mfns, int n)
 return HYPERVISOR_memory_op(XENMEM_decrease_reservation, &reservation);
 }
 
+int map_frame_rw(unsigned long addr, unsigned long mfn)
+{
+return do_map_frames(addr, &mfn, 1, 1, 1, DOMID_SELF, NULL, L1_PROT);
+}
+
 #ifdef HAVE_LIBC
 void *sbrk(ptrdiff_t increment)
 {
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 10/18] mini-os: remove unused mem_test() function

[Juergen Gross]

mem_test() isn't used anywhere and its value is rather questionable
with mini-os being in a mature state. Remove the function.

Signed-off-by: Juergen Gross 
---
 arch/x86/mm.c | 55 ---
 1 file changed, 55 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index 6aa4468..e2f026b 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -302,61 +302,6 @@ static void set_readonly(void *text, void *etext)
 }
 
 /*
- * A useful mem testing function. Write the address to every address in the
- * range provided and read back the value. If verbose, print page walk to
- * some VA
- * 
- * If we get MEM_TEST_MAX_ERRORS we might as well stop
- */
-#define MEM_TEST_MAX_ERRORS 10 
-int mem_test(unsigned long *start_va, unsigned long *end_va, int verbose)
-{
-unsigned long mask = 0x1;
-unsigned long *pointer;
-int error_count = 0;
- 
-/* write values and print page walks */
-if ( verbose && (((unsigned long)start_va) & 0xf) )
-{
-printk("MemTest Start: 0x%p\n", start_va);
-page_walk((unsigned long)start_va);
-}
-for ( pointer = start_va; pointer < end_va; pointer++ )
-{
-if ( verbose && !(((unsigned long)pointer) & 0xf) )
-{
-printk("Writing to %p\n", pointer);
-page_walk((unsigned long)pointer);
-}
-*pointer = (unsigned long)pointer & ~mask;
-}
-if ( verbose && (((unsigned long)end_va) & 0xf) )
-{
-printk("MemTest End: %p\n", end_va-1);
-page_walk((unsigned long)end_va-1);
-}
- 
-/* verify values */
-for ( pointer = start_va; pointer < end_va; pointer++ )
-{
-if ( ((unsigned long)pointer & ~mask) != *pointer )
-{
-printk("Read error at 0x%lx. Read: 0x%lx, should read 0x%lx\n",
-   (unsigned long)pointer, *pointer, 
-   ((unsigned long)pointer & ~mask));
-error_count++;
-if ( error_count >= MEM_TEST_MAX_ERRORS )
-{
-printk("mem_test: too many errors\n");
-return -1;
-}
-}
-}
-return 0;
-}
-
-
-/*
  * get the PTE for virtual address va if it exists. Otherwise NULL.
  */
 static pgentry_t *get_pgt(unsigned long va)
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 11/18] mini-os: add checks for out of memory

[Juergen Gross]

There are several core functions in Mini-OS not checking for failed
memory allocations. Add such checks.

Add do_map_frames() dummy function to arm architecture as it will be
needed in future for compilations to succeed.

Signed-off-by: Juergen Gross 
---
 arch/arm/mm.c |  8 
 arch/x86/mm.c | 26 +++---
 include/mm.h  |  2 +-
 3 files changed, 28 insertions(+), 8 deletions(-)

diff --git a/arch/arm/mm.c b/arch/arm/mm.c
index f75888d..fc8d4bc 100644
--- a/arch/arm/mm.c
+++ b/arch/arm/mm.c
@@ -1,6 +1,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -79,6 +80,13 @@ void arch_init_demand_mapping_area(void)
 {
 }
 
+int do_map_frames(unsigned long addr,
+const unsigned long *f, unsigned long n, unsigned long stride,
+unsigned long increment, domid_t id, int *err, unsigned long prot)
+{
+return -ENOSYS;
+}
+
 /* Get Xen's suggested physical page assignments for the grant table. */
 static paddr_t get_gnttab_base(void)
 {
diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index e2f026b..12f7fe4 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -34,6 +34,7 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
+#include 
 #include 
 #include 
 #include 
@@ -354,6 +355,8 @@ pgentry_t *need_pgt(unsigned long va)
 if ( !(tab[offset] & _PAGE_PRESENT) )
 {
 pt_pfn = virt_to_pfn(alloc_page());
+if ( !pt_pfn )
+return NULL;
 new_pt_frame(&pt_pfn, pt_mfn, offset, L3_FRAME);
 }
 ASSERT(tab[offset] & _PAGE_PRESENT);
@@ -364,6 +367,8 @@ pgentry_t *need_pgt(unsigned long va)
 if ( !(tab[offset] & _PAGE_PRESENT) ) 
 {
 pt_pfn = virt_to_pfn(alloc_page());
+if ( !pt_pfn )
+return NULL;
 new_pt_frame(&pt_pfn, pt_mfn, offset, L2_FRAME);
 }
 ASSERT(tab[offset] & _PAGE_PRESENT);
@@ -373,6 +378,8 @@ pgentry_t *need_pgt(unsigned long va)
 if ( !(tab[offset] & _PAGE_PRESENT) )
 {
 pt_pfn = virt_to_pfn(alloc_page());
+if ( !pt_pfn )
+return NULL;
 new_pt_frame(&pt_pfn, pt_mfn, offset, L1_FRAME);
 }
 ASSERT(tab[offset] & _PAGE_PRESENT);
@@ -445,10 +452,10 @@ unsigned long allocate_ondemand(unsigned long n, unsigned 
long alignment)
  * va. map f[i*stride]+i*increment for i in 0..n-1.
  */
 #define MAP_BATCH ((STACK_SIZE / 2) / sizeof(mmu_update_t))
-void do_map_frames(unsigned long va,
-   const unsigned long *mfns, unsigned long n, 
-   unsigned long stride, unsigned long incr, 
-   domid_t id, int *err, unsigned long prot)
+int do_map_frames(unsigned long va,
+  const unsigned long *mfns, unsigned long n,
+  unsigned long stride, unsigned long incr,
+  domid_t id, int *err, unsigned long prot)
 {
 pgentry_t *pgt = NULL;
 unsigned long done = 0;
@@ -458,7 +465,7 @@ void do_map_frames(unsigned long va,
 if ( !mfns ) 
 {
 printk("do_map_frames: no mfns supplied\n");
-return;
+return -EINVAL;
 }
 DEBUG("va=%p n=0x%lx, mfns[0]=0x%lx stride=0x%lx incr=0x%lx prot=0x%lx\n",
   va, n, mfns[0], stride, incr, prot);
@@ -484,7 +491,9 @@ void do_map_frames(unsigned long va,
 {
 if ( !pgt || !(va & L1_MASK) )
 pgt = need_pgt(va);
-
+if ( !pgt )
+return -ENOMEM;
+
 mmu_updates[i].ptr = virt_to_mach(pgt) | MMU_NORMAL_PT_UPDATE;
 mmu_updates[i].val = ((pgentry_t)(mfns[(done + i) * stride] +
   (done + i) * incr)
@@ -505,6 +514,8 @@ void do_map_frames(unsigned long va,
 }
 done += todo;
 }
+
+return 0;
 }
 
 /*
@@ -521,7 +532,8 @@ void *map_frames_ex(const unsigned long *mfns, unsigned 
long n,
 if ( !va )
 return NULL;
 
-do_map_frames(va, mfns, n, stride, incr, id, err, prot);
+if ( do_map_frames(va, mfns, n, stride, incr, id, err, prot) )
+return NULL;
 
 return (void *)va;
 }
diff --git a/include/mm.h b/include/mm.h
index a22dcd1..9244e26 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -68,7 +68,7 @@ unsigned long allocate_ondemand(unsigned long n, unsigned 
long alignment);
 void *map_frames_ex(const unsigned long *f, unsigned long n, unsigned long 
stride,
unsigned long increment, unsigned long alignment, domid_t id,
int *err, unsigned long prot);
-void do_map_frames(unsigned long addr,
+int do_map_frames(unsigned long addr,
 const unsigned long *f, unsigned long n, unsigned long stride,
unsigned long increment, domid_t id, int *err, unsigned long prot);
 int unmap_frames(unsigned long va, unsigned long num_frames);
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 08/18] mini-os: get maximum memory size from hypervisor

[Juergen Gross]

Add support for obtaining the maximum memory size from the hypervisor.
This will make it possible to support ballooning.

Signed-off-by: Juergen Gross 
---
V2: Moved new stuff to balloon.c
---
 balloon.c | 22 ++
 include/balloon.h |  6 ++
 mm.c  |  3 ++-
 3 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/balloon.c b/balloon.c
index 9cabde0..4c18c5c 100644
--- a/balloon.c
+++ b/balloon.c
@@ -21,4 +21,26 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
+#include 
 #include 
+#include 
+#include 
+#include 
+
+unsigned long nr_max_pages;
+
+void get_max_pages(void)
+{
+long ret;
+domid_t domid = DOMID_SELF;
+
+ret = HYPERVISOR_memory_op(XENMEM_maximum_reservation, &domid);
+if ( ret < 0 )
+{
+printk("Could not get maximum pfn\n");
+return;
+}
+
+nr_max_pages = ret;
+printk("Maximum memory size: %ld pages\n", nr_max_pages);
+}
diff --git a/include/balloon.h b/include/balloon.h
index 399fff4..e7219f8 100644
--- a/include/balloon.h
+++ b/include/balloon.h
@@ -26,7 +26,13 @@
 
 #ifdef CONFIG_BALLOON
 
+extern unsigned long nr_max_pages;
+
+void get_max_pages(void);
+
 #else /* CONFIG_BALLOON */
 
+static inline void get_max_pages(void) { }
+
 #endif /* CONFIG_BALLOON */
 #endif /* _BALLOON_H_ */
diff --git a/mm.c b/mm.c
index 8cf3210..25ee3da 100644
--- a/mm.c
+++ b/mm.c
@@ -38,6 +38,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -353,7 +354,6 @@ void *sbrk(ptrdiff_t increment)
 #endif
 
 
-
 void init_mm(void)
 {
 
@@ -361,6 +361,7 @@ void init_mm(void)
 
 printk("MM: Init\n");
 
+get_max_pages();
 arch_init_mm(&start_pfn, &max_pfn);
 /*
  * now we can initialise the page allocator
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 12/18] mini-os: don't allocate new pages for level 1 p2m tree

[Juergen Gross]

When constructing the 3 level p2m tree there is no need to allocate
new pages for the level 1 containing the p2m info for all pages. The
pages from the linear p2m list constructed by the hypervisor can be
used for that purpose.

Signed-off-by: Juergen Gross 
---
 arch/x86/mm.c | 14 --
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index 12f7fe4..e10c2c5 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -625,11 +625,11 @@ void arch_init_p2m(unsigned long max_pfn)
 #define L2_P2M_MASK (L2_P2M_ENTRIES - 1)
 #define L3_P2M_MASK (L3_P2M_ENTRIES - 1)
 
-unsigned long *l1_list = NULL, *l2_list = NULL, *l3_list;
+unsigned long *l2_list = NULL, *l3_list;
 unsigned long pfn;
 
 l3_list = (unsigned long *)alloc_page(); 
-for ( pfn=0; pfn> L2_P2M_SHIFT)] = virt_to_mfn(l2_list);  
 }
-if ( !(pfn % (L1_P2M_ENTRIES)) )
-{
-l1_list = (unsigned long*)alloc_page();
-l2_list[(pfn >> L1_P2M_SHIFT) & L2_P2M_MASK] = 
-virt_to_mfn(l1_list); 
-}
-
-l1_list[pfn & L1_P2M_MASK] = pfn_to_mfn(pfn); 
+l2_list[(pfn >> L1_P2M_SHIFT) & L2_P2M_MASK] =
+virt_to_mfn(phys_to_machine_mapping + pfn);
 }
 HYPERVISOR_shared_info->arch.pfn_to_mfn_frame_list_list = 
 virt_to_mfn(l3_list);
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 17/18] mini-os: add support for ballooning up

[Juergen Gross]

Add support for ballooning the domain up by a specified amount of
pages. Following steps are performed:

- extending the p2m map
- extending the page allocator's bitmap
- getting new memory pages from the hypervisor
- adding the memory at the current end of guest memory

Signed-off-by: Juergen Gross 
---
 arch/arm/balloon.c |  9 ++
 arch/x86/balloon.c | 94 ++
 balloon.c  | 64 +
 include/balloon.h  |  5 +++
 mm.c   |  4 +++
 5 files changed, 176 insertions(+)

diff --git a/arch/arm/balloon.c b/arch/arm/balloon.c
index a76db3b..fbb8007 100644
--- a/arch/arm/balloon.c
+++ b/arch/arm/balloon.c
@@ -27,4 +27,13 @@
 
 unsigned long virt_kernel_area_end;   /* TODO: find a virtual area */
 
+int arch_expand_p2m(unsigned long max_pfn)
+{
+return 0;
+}
+
+void arch_pfn_add(unsigned long pfn, unsigned long mfn)
+{
+}
+
 #endif
diff --git a/arch/x86/balloon.c b/arch/x86/balloon.c
index db37e8f..3d7692f 100644
--- a/arch/x86/balloon.c
+++ b/arch/x86/balloon.c
@@ -23,6 +23,7 @@
 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -30,9 +31,36 @@
 
 unsigned long virt_kernel_area_end = VIRT_KERNEL_AREA;
 
+static void p2m_invalidate(unsigned long *list, unsigned long start_idx)
+{
+unsigned long idx;
+
+for ( idx = start_idx; idx < P2M_ENTRIES; idx++ )
+list[idx] = INVALID_P2M_ENTRY;
+}
+
+static inline unsigned long *p2m_l3list(void)
+{
+return 
mfn_to_virt(HYPERVISOR_shared_info->arch.pfn_to_mfn_frame_list_list);
+}
+
+static inline unsigned long *p2m_to_virt(unsigned long p2m)
+{
+return ( p2m == INVALID_P2M_ENTRY ) ? NULL : mfn_to_virt(p2m);
+}
+
 void arch_remap_p2m(unsigned long max_pfn)
 {
 unsigned long pfn;
+unsigned long *l3_list, *l2_list, *l1_list;
+
+l3_list = p2m_l3list();
+l2_list = p2m_to_virt(l3_list[L3_P2M_IDX(max_pfn - 1)]);
+l1_list = p2m_to_virt(l2_list[L2_P2M_IDX(max_pfn - 1)]);
+
+p2m_invalidate(l3_list, L3_P2M_IDX(max_pfn - 1) + 1);
+p2m_invalidate(l2_list, L2_P2M_IDX(max_pfn - 1) + 1);
+p2m_invalidate(l1_list, L1_P2M_IDX(max_pfn - 1) + 1);
 
 if ( p2m_pages(nr_max_pages) <= p2m_pages(max_pfn) )
 return;
@@ -49,4 +77,70 @@ void arch_remap_p2m(unsigned long max_pfn)
 virt_kernel_area_end += PAGE_SIZE * p2m_pages(nr_max_pages);
 }
 
+int arch_expand_p2m(unsigned long max_pfn)
+{
+unsigned long pfn;
+unsigned long *l1_list, *l2_list, *l3_list;
+
+p2m_chk_pfn(max_pfn - 1);
+l3_list = p2m_l3list();
+
+for ( pfn = (HYPERVISOR_shared_info->arch.max_pfn + P2M_MASK) & ~P2M_MASK;
+  pfn < max_pfn; pfn += P2M_ENTRIES )
+{
+l2_list = p2m_to_virt(l3_list[L3_P2M_IDX(pfn)]);
+if ( !l2_list )
+{
+l2_list = (unsigned long*)alloc_page();
+if ( !l2_list )
+return -ENOMEM;
+p2m_invalidate(l2_list, 0);
+l3_list[L3_P2M_IDX(pfn)] = virt_to_mfn(l2_list);
+}
+l1_list = p2m_to_virt(l2_list[L2_P2M_IDX(pfn)]);
+if ( !l1_list )
+{
+l1_list = (unsigned long*)alloc_page();
+if ( !l1_list )
+return -ENOMEM;
+p2m_invalidate(l1_list, 0);
+l2_list[L2_P2M_IDX(pfn)] = virt_to_mfn(l1_list);
+
+if ( map_frame_rw((unsigned long)(phys_to_machine_mapping + pfn),
+  l2_list[L2_P2M_IDX(pfn)]) )
+return -ENOMEM;
+}
+}
+
+HYPERVISOR_shared_info->arch.max_pfn = max_pfn;
+
+/* Make sure the new last page can be mapped. */
+if ( !need_pgt((unsigned long)pfn_to_virt(max_pfn - 1)) )
+return -ENOMEM;
+
+return 0;
+}
+
+void arch_pfn_add(unsigned long pfn, unsigned long mfn)
+{
+mmu_update_t mmu_updates[1];
+pgentry_t *pgt;
+int rc;
+
+phys_to_machine_mapping[pfn] = mfn;
+
+pgt = need_pgt((unsigned long)pfn_to_virt(pfn));
+ASSERT(pgt);
+mmu_updates[0].ptr = virt_to_mach(pgt) | MMU_NORMAL_PT_UPDATE;
+mmu_updates[0].val = (pgentry_t)(mfn << PAGE_SHIFT) |
+ _PAGE_PRESENT | _PAGE_RW;
+rc = HYPERVISOR_mmu_update(mmu_updates, 1, NULL, DOMID_SELF);
+if ( rc < 0 )
+{
+printk("ERROR: build_pagetable(): PTE could not be updated\n");
+printk("   mmu_update failed with rc=%d\n", rc);
+do_exit();
+}
+}
+
 #endif
diff --git a/balloon.c b/balloon.c
index 75b87c8..30e571c 100644
--- a/balloon.c
+++ b/balloon.c
@@ -23,11 +23,13 @@
 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
 
 unsigned long nr_max_pages;
+unsigned long nr_mem_pages;
 
 void get_max_pages(void)
 {
@@ -61,3 +63,65 @@ void alloc_bitmap_remap(void)
 alloc_bitmap = (unsigned long *)virt_kernel_area_end;
 virt_kernel_area_end += round_pgup((nr_max_pages + 1) >> (PAGE_SHIFT + 3));
 }
+
+#define N_BALLOON_FRAMES 64
+static unsigned long balloon_frames[N_BALLOON_FRAMES];
+
+int balloo

[Xen-devel] [PATCH v2 16/18] mini-os: map page allocator's bitmap to virtual kernel area for ballooning

[Juergen Gross]

In case of CONFIG_BALLOON the page allocator's bitmap needs some space
to be able to grow. Remap it to kernel virtual area if the preallocated
area isn't large enough.

Signed-off-by: Juergen Gross 
---
 balloon.c | 17 +
 include/balloon.h |  2 ++
 include/mm.h  |  6 ++
 mm.c  | 19 ++-
 4 files changed, 35 insertions(+), 9 deletions(-)

diff --git a/balloon.c b/balloon.c
index 4c18c5c..75b87c8 100644
--- a/balloon.c
+++ b/balloon.c
@@ -44,3 +44,20 @@ void get_max_pages(void)
 nr_max_pages = ret;
 printk("Maximum memory size: %ld pages\n", nr_max_pages);
 }
+
+void alloc_bitmap_remap(void)
+{
+unsigned long i;
+
+if ( alloc_bitmap_size >= ((nr_max_pages + 1) >> (PAGE_SHIFT + 3)) )
+return;
+
+for ( i = 0; i < alloc_bitmap_size; i += PAGE_SIZE )
+{
+map_frame_rw(virt_kernel_area_end + i,
+ virt_to_mfn((unsigned long)(alloc_bitmap) + i));
+}
+
+alloc_bitmap = (unsigned long *)virt_kernel_area_end;
+virt_kernel_area_end += round_pgup((nr_max_pages + 1) >> (PAGE_SHIFT + 3));
+}
diff --git a/include/balloon.h b/include/balloon.h
index b8d9335..0e2340b 100644
--- a/include/balloon.h
+++ b/include/balloon.h
@@ -31,11 +31,13 @@ extern unsigned long virt_kernel_area_end;
 
 void get_max_pages(void);
 void arch_remap_p2m(unsigned long max_pfn);
+void alloc_bitmap_remap(void);
 
 #else /* CONFIG_BALLOON */
 
 static inline void get_max_pages(void) { }
 static inline void arch_remap_p2m(unsigned long max_pfn) { }
+static inline void alloc_bitmap_remap(void) { }
 
 #endif /* CONFIG_BALLOON */
 #endif /* _BALLOON_H_ */
diff --git a/include/mm.h b/include/mm.h
index 6add683..73c59c4 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -42,8 +42,14 @@
 #define STACK_SIZE_PAGE_ORDER __STACK_SIZE_PAGE_ORDER
 #define STACK_SIZE __STACK_SIZE
 
+#define round_pgdown(_p)  ((_p) & PAGE_MASK)
+#define round_pgup(_p)(((_p) + (PAGE_SIZE - 1)) & PAGE_MASK)
+
 extern unsigned long nr_free_pages;
 
+extern unsigned long *alloc_bitmap;
+extern unsigned long alloc_bitmap_size;
+
 void init_mm(void);
 unsigned long alloc_pages(int order);
 #define alloc_page()alloc_pages(0)
diff --git a/mm.c b/mm.c
index fd66115..13bb3e5 100644
--- a/mm.c
+++ b/mm.c
@@ -48,7 +48,9 @@
  *  One bit per page of memory. Bit set => page is allocated.
  */
 
-static unsigned long *alloc_bitmap;
+unsigned long *alloc_bitmap;
+unsigned long alloc_bitmap_size;
+
 #define PAGES_PER_MAPWORD (sizeof(unsigned long) * 8)
 
 #define allocated_in_map(_pn) \
@@ -137,9 +139,6 @@ static chunk_head_t *free_head[FREELIST_SIZE];
 static chunk_head_t  free_tail[FREELIST_SIZE];
 #define FREELIST_EMPTY(_l) ((_l)->next == NULL)
 
-#define round_pgdown(_p)  ((_p)&PAGE_MASK)
-#define round_pgup(_p)(((_p)+(PAGE_SIZE-1))&PAGE_MASK)
-
 /*
  * Initialise allocator, placing addresses [@min,@max] in free pool.
  * @min and @max are PHYSICAL addresses.
@@ -147,7 +146,7 @@ static chunk_head_t  free_tail[FREELIST_SIZE];
 static void init_page_allocator(unsigned long min, unsigned long max)
 {
 int i;
-unsigned long range, bitmap_size;
+unsigned long range;
 chunk_head_t *ch;
 chunk_tail_t *ct;
 for ( i = 0; i < FREELIST_SIZE; i++ )
@@ -161,14 +160,14 @@ static void init_page_allocator(unsigned long min, 
unsigned long max)
 max = round_pgdown(max);
 
 /* Allocate space for the allocation bitmap. */
-bitmap_size  = (max+1) >> (PAGE_SHIFT+3);
-bitmap_size  = round_pgup(bitmap_size);
+alloc_bitmap_size  = (max + 1) >> (PAGE_SHIFT + 3);
+alloc_bitmap_size  = round_pgup(alloc_bitmap_size);
 alloc_bitmap = (unsigned long *)to_virt(min);
-min += bitmap_size;
+min += alloc_bitmap_size;
 range= max - min;
 
 /* All allocated by default. */
-memset(alloc_bitmap, ~0, bitmap_size);
+memset(alloc_bitmap, ~0, alloc_bitmap_size);
 /* Free up the memory we've been given to play with. */
 map_free(PHYS_PFN(min), range>>PAGE_SHIFT);
 
@@ -198,6 +197,8 @@ static void init_page_allocator(unsigned long min, unsigned 
long max)
 free_head[i]= ch;
 ct->level   = i;
 }
+
+alloc_bitmap_remap();
 }
 
 
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 07/18] mini-os: add ballooning config item

[Juergen Gross]

Add CONFIG_BALLOON defaulting to 'n' as a config item to Mini-OS.

Add balloon.c, balloon.h and arch/*/balloon.c for future use.

Signed-off-by: Juergen Gross 
---
V2: Added dummy sources and header
---
 Makefile   |  3 +++
 arch/arm/balloon.c | 28 
 arch/x86/balloon.c | 28 
 balloon.c  | 24 
 include/balloon.h  | 32 
 5 files changed, 115 insertions(+)
 create mode 100644 arch/arm/balloon.c
 create mode 100644 arch/x86/balloon.c
 create mode 100644 balloon.c
 create mode 100644 include/balloon.h

diff --git a/Makefile b/Makefile
index 2e4bdba..f5b7011 100644
--- a/Makefile
+++ b/Makefile
@@ -33,6 +33,7 @@ CONFIG_CONSFRONT ?= y
 CONFIG_XENBUS ?= y
 CONFIG_XC ?=y
 CONFIG_LWIP ?= $(lwip)
+CONFIG_BALLOON ?= n
 
 # Export config items as compiler directives
 flags-$(CONFIG_START_NETWORK) += -DCONFIG_START_NETWORK
@@ -48,6 +49,7 @@ flags-$(CONFIG_KBDFRONT) += -DCONFIG_KBDFRONT
 flags-$(CONFIG_FBFRONT) += -DCONFIG_FBFRONT
 flags-$(CONFIG_CONSFRONT) += -DCONFIG_CONSFRONT
 flags-$(CONFIG_XENBUS) += -DCONFIG_XENBUS
+flags-$(CONFIG_BALLOON) += -DCONFIG_BALLOON
 
 DEF_CFLAGS += $(flags-y)
 
@@ -96,6 +98,7 @@ src-$(CONFIG_NETFRONT) += netfront.c
 src-$(CONFIG_PCIFRONT) += pcifront.c
 src-y += sched.c
 src-$(CONFIG_TEST) += test.c
+src-$(CONFIG_BALLOON) += balloon.c
 
 src-y += lib/ctype.c
 src-y += lib/math.c
diff --git a/arch/arm/balloon.c b/arch/arm/balloon.c
new file mode 100644
index 000..dc6270d
--- /dev/null
+++ b/arch/arm/balloon.c
@@ -0,0 +1,28 @@
+/* -*-  Mode:C; c-basic-offset:4; tab-width:4 -*-
+ *
+ * (C) 2016 - Juergen Gross, SUSE Linux GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ * 
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ * 
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include 
+
+#ifdef CONFIG_BALLOON
+
+#endif
diff --git a/arch/x86/balloon.c b/arch/x86/balloon.c
new file mode 100644
index 000..dc6270d
--- /dev/null
+++ b/arch/x86/balloon.c
@@ -0,0 +1,28 @@
+/* -*-  Mode:C; c-basic-offset:4; tab-width:4 -*-
+ *
+ * (C) 2016 - Juergen Gross, SUSE Linux GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ * 
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ * 
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include 
+
+#ifdef CONFIG_BALLOON
+
+#endif
diff --git a/balloon.c b/balloon.c
new file mode 100644
index 000..9cabde0
--- /dev/null
+++ b/balloon.c
@@ -0,0 +1,24 @@
+/* -*-  Mode:C; c-basic-offset:4; tab-width:4 -*-
+ *
+ * (C) 2016 - Juergen Gross, SUSE Linux GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ * 
+ * The above copyright notice and this per

[Xen-devel] [PATCH v2 03/18] mini-os: remove MM_DEBUG code

[Juergen Gross]

mm.c contains unused code inside #ifdef MM_DEBUG areas. Its usability
is rather questionable and some parts are even wrong (e.g.
print_chunks() called with nr_pages > 1000 will clobber an arbitrary
stack content with a 0 byte).

Remove this code.

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
---
 mm.c | 60 
 1 file changed, 60 deletions(-)

diff --git a/mm.c b/mm.c
index 31aaf83..0dd4862 100644
--- a/mm.c
+++ b/mm.c
@@ -42,13 +42,6 @@
 #include 
 #include 
 
-#ifdef MM_DEBUG
-#define DEBUG(_f, _a...) \
-printk("MINI_OS(file=mm.c, line=%d) " _f "\n", __LINE__, ## _a)
-#else
-#define DEBUG(_f, _a...)((void)0)
-#endif
-
 /*
  * ALLOCATION BITMAP
  *  One bit per page of memory. Bit set => page is allocated.
@@ -140,59 +133,6 @@ static chunk_head_t  free_tail[FREELIST_SIZE];
 #define round_pgdown(_p)  ((_p)&PAGE_MASK)
 #define round_pgup(_p)(((_p)+(PAGE_SIZE-1))&PAGE_MASK)
 
-#ifdef MM_DEBUG
-/*
- * Prints allocation[0/1] for @nr_pages, starting at @start
- * address (virtual).
- */
-USED static void print_allocation(void *start, int nr_pages)
-{
-unsigned long pfn_start = virt_to_pfn(start);
-int count;
-for(count = 0; count < nr_pages; count++)
-if(allocated_in_map(pfn_start + count)) printk("1");
-else printk("0");
-
-printk("\n");
-}
-
-/*
- * Prints chunks (making them with letters) for @nr_pages starting
- * at @start (virtual).
- */
-USED static void print_chunks(void *start, int nr_pages)
-{
-char chunks[1001], current='A';
-int order, count;
-chunk_head_t *head;
-unsigned long pfn_start = virt_to_pfn(start);
-   
-memset(chunks, (int)'_', 1000);
-if(nr_pages > 1000) 
-{
-DEBUG("Can only pring 1000 pages. Increase buffer size.");
-}
-
-for(order=0; order < FREELIST_SIZE; order++)
-{
-head = free_head[order];
-while(!FREELIST_EMPTY(head))
-{
-for(count = 0; count < 1UL<< head->level; count++)
-{
-if(count + virt_to_pfn(head) - pfn_start < 1000)
-chunks[count + virt_to_pfn(head) - pfn_start] = current;
-}
-head = head->next;
-current++;
-}
-}
-chunks[nr_pages] = '\0';
-printk("%s\n", chunks);
-}
-#endif
-
-
 /*
  * Initialise allocator, placing addresses [@min,@max] in free pool.
  * @min and @max are PHYSICAL addresses.
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] acpi: Re-license ACPI builder files from GPLv2 to LGPLv2.1

[Keir Fraser]

Re: [Xen-devel] [PATCH] acpi: Re-license ACPI builder files from.eml

Subject:
Re: [Xen-devel] [PATCH] acpi: Re-license ACPI builder files from GPLv2
to LGPLv2.1
From:
"Tian, Kevin" mailto:kevin.t...@intel.com>>
Date:
Thu, 4 Aug 2016 06:52:34 +

To:
Boris Ostrovsky mailto:boris.ostrov...@oracle.com>>, Simon Horman mailto:ho...@verge.net.au>>, Keir Fraser mailto:k...@xen.org>>, Jan Beulich mailto:jbeul...@suse.com>>
CC:
Lars Kurth mailto:lars.ku...@citrix.com>>,
Stefan Berger mailto:stef...@us.ibm.com>>, Ian
Jackson mailto:ian.jack...@eu.citrix.com>>,
Kouya Shimura mailto:ko...@jp.fujitsu.com>>,
"xen-de...@lists.xenproject.org "
mailto:xen-de...@lists.xenproject.org>>, Daniel Kiper
mailto:dki...@net-space.pl>>



>  From: Boris Ostrovsky
>  Sent: Monday, August 01, 2016 9:57 PM
>  On 07/18/2016 10:01 AM, Boris Ostrovsky wrote:

>  >  ACPI builder is currently distributed under GPLv2 license.
>  >
>  >  We plan to make the builder available to components other
>  >  than the hvmloader (which is also GPLv2). Some of these
>  >  components (such as libxl) may be distributed under LGPL-2.1
>  >  so that they can be used by non-GPLv2 callers.  But this
>  >  will not be possible if we incorporate the ACPI builder in
>  >  those other components.
>  >
>  >  To avoid this problem we are relicensing sources in ACPI
>  >  bulder directory to the Lesser GNU Public License (LGPL)
>  >  version 2.1
>  >
>  >  Signed-off-by: Boris Ostrovsky
>  >  CC: Kouya Shimura
>  >  CC: Daniel Kiper
>  >  CC: Stefan Berger
>  >  CC: Simon Horman
>  >  CC: Keir Fraser
>  >  CC: Ian Jackson
>  >  CC: Lars Kurth


Acked-by: Kevin Tian  for all Intel related code.


Acked-by: Keir Fraser 



Thanks
Kevin
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 15/18] mini-os: remap p2m list in case of ballooning

[Juergen Gross]

In case of enabled ballooning we must be prepared for a growing p2m
list. If the maximum memory size of the domain can't be covered by the
actual p2m list remap it to the kernel virtual mapping area and leave
enough space at the end.

Signed-off-by: Juergen Gross 
---
 arch/arm/balloon.c|  2 ++
 arch/x86/balloon.c| 24 
 arch/x86/mm.c |  3 +++
 include/balloon.h |  3 +++
 include/x86/arch_mm.h |  4 
 5 files changed, 36 insertions(+)

diff --git a/arch/arm/balloon.c b/arch/arm/balloon.c
index dc6270d..a76db3b 100644
--- a/arch/arm/balloon.c
+++ b/arch/arm/balloon.c
@@ -25,4 +25,6 @@
 
 #ifdef CONFIG_BALLOON
 
+unsigned long virt_kernel_area_end;   /* TODO: find a virtual area */
+
 #endif
diff --git a/arch/x86/balloon.c b/arch/x86/balloon.c
index dc6270d..db37e8f 100644
--- a/arch/x86/balloon.c
+++ b/arch/x86/balloon.c
@@ -21,8 +21,32 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
+#include 
 #include 
+#include 
+#include 
 
 #ifdef CONFIG_BALLOON
 
+unsigned long virt_kernel_area_end = VIRT_KERNEL_AREA;
+
+void arch_remap_p2m(unsigned long max_pfn)
+{
+unsigned long pfn;
+
+if ( p2m_pages(nr_max_pages) <= p2m_pages(max_pfn) )
+return;
+
+for ( pfn = 0; pfn < max_pfn; pfn += P2M_ENTRIES )
+{
+map_frame_rw(virt_kernel_area_end + PAGE_SIZE * (pfn / P2M_ENTRIES),
+ virt_to_mfn(phys_to_machine_mapping + pfn));
+}
+
+phys_to_machine_mapping = (unsigned long *)virt_kernel_area_end;
+printk("remapped p2m list to %p\n", phys_to_machine_mapping);
+
+virt_kernel_area_end += PAGE_SIZE * p2m_pages(nr_max_pages);
+}
+
 #endif
diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index a5c8959..8fa3b4c 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -37,6 +37,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -626,6 +627,8 @@ void arch_init_p2m(unsigned long max_pfn)
 HYPERVISOR_shared_info->arch.pfn_to_mfn_frame_list_list = 
 virt_to_mfn(l3_list);
 HYPERVISOR_shared_info->arch.max_pfn = max_pfn;
+
+arch_remap_p2m(max_pfn);
 }
 
 void arch_init_mm(unsigned long* start_pfn_p, unsigned long* max_pfn_p)
diff --git a/include/balloon.h b/include/balloon.h
index e7219f8..b8d9335 100644
--- a/include/balloon.h
+++ b/include/balloon.h
@@ -27,12 +27,15 @@
 #ifdef CONFIG_BALLOON
 
 extern unsigned long nr_max_pages;
+extern unsigned long virt_kernel_area_end;
 
 void get_max_pages(void);
+void arch_remap_p2m(unsigned long max_pfn);
 
 #else /* CONFIG_BALLOON */
 
 static inline void get_max_pages(void) { }
+static inline void arch_remap_p2m(unsigned long max_pfn) { }
 
 #endif /* CONFIG_BALLOON */
 #endif /* _BALLOON_H_ */
diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index 7283f64..e5d9c57 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -198,6 +198,10 @@ static inline void p2m_chk_pfn(unsigned long pfn)
 do_exit();
 }
 }
+static inline unsigned long p2m_pages(unsigned long pages)
+{
+return (pages + P2M_ENTRIES - 1) >> L1_P2M_SHIFT;
+}
 
 #include "arch_limits.h"
 #define PAGE_SIZE   __PAGE_SIZE
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 18/18] mini-os: balloon up in case of oom

[Juergen Gross]

If a memory shortage is detected balloon up.

Be careful to always leave some pages free as ballooning up might need
some memory, too:

- new p2m frames
- page tables for addressing new p2m frame
- new frame for page allocation bitmap
- page table for addressing new page allocation bitmap frame
- page tables for addressing new 1:1 mapped frames

For the moment we only balloon up synchronously when memory shortage
is detected in allocation routines with irqs on.

Signed-off-by: Juergen Gross 
---
 balloon.c | 32 
 include/balloon.h | 11 +++
 mm.c  |  4 +++-
 3 files changed, 46 insertions(+), 1 deletion(-)

diff --git a/balloon.c b/balloon.c
index 30e571c..30b338c 100644
--- a/balloon.c
+++ b/balloon.c
@@ -125,3 +125,35 @@ int balloon_up(unsigned long n_pages)
 
 return rc;
 }
+
+static int in_balloon;
+
+int chk_free_pages(unsigned long needed)
+{
+unsigned long n_pages;
+
+/* No need for ballooning if plenty of space available. */
+if ( needed + BALLOON_EMERGENCY_PAGES <= nr_free_pages )
+return 0;
+
+/* If we are already ballooning up just hope for the best. */
+if ( in_balloon )
+return 0;
+
+/* Interrupts disabled can't be handled right now. */
+if ( irqs_disabled() )
+return 0;
+
+in_balloon = 1;
+
+while ( needed + BALLOON_EMERGENCY_PAGES > nr_free_pages )
+{
+n_pages = needed + BALLOON_EMERGENCY_PAGES - nr_free_pages;
+if ( !balloon_up(n_pages) )
+break;
+}
+
+in_balloon = 0;
+
+return needed > nr_free_pages;
+}
diff --git a/include/balloon.h b/include/balloon.h
index 8bf77e5..c08b3f8 100644
--- a/include/balloon.h
+++ b/include/balloon.h
@@ -26,6 +26,12 @@
 
 #ifdef CONFIG_BALLOON
 
+/*
+ * Always keep some pages free for allocations while ballooning or
+ * interrupts disabled.
+ */
+#define BALLOON_EMERGENCY_PAGES   64
+
 extern unsigned long nr_max_pages;
 extern unsigned long virt_kernel_area_end;
 extern unsigned long nr_mem_pages;
@@ -37,12 +43,17 @@ void arch_remap_p2m(unsigned long max_pfn);
 void alloc_bitmap_remap(void);
 int arch_expand_p2m(unsigned long max_pfn);
 void arch_pfn_add(unsigned long pfn, unsigned long mfn);
+int chk_free_pages(unsigned long needed);
 
 #else /* CONFIG_BALLOON */
 
 static inline void get_max_pages(void) { }
 static inline void arch_remap_p2m(unsigned long max_pfn) { }
 static inline void alloc_bitmap_remap(void) { }
+static inline int chk_free_pages(unsigned long needed)
+{
+return needed > nr_free_pages;
+}
 
 #endif /* CONFIG_BALLOON */
 #endif /* _BALLOON_H_ */
diff --git a/mm.c b/mm.c
index 092c5f9..2b57f46 100644
--- a/mm.c
+++ b/mm.c
@@ -209,6 +209,8 @@ unsigned long alloc_pages(int order)
 chunk_head_t *alloc_ch, *spare_ch;
 chunk_tail_t*spare_ct;
 
+if ( chk_free_pages(1UL << order) )
+goto no_memory;
 
 /* Find smallest order which can satisfy the request. */
 for ( i = order; i < FREELIST_SIZE; i++ ) {
@@ -343,7 +345,7 @@ void *sbrk(ptrdiff_t increment)
 if (new_brk > heap_mapped) {
 unsigned long n = (new_brk - heap_mapped + PAGE_SIZE - 1) / PAGE_SIZE;
 
-if ( n > nr_free_pages )
+if ( chk_free_pages(n) )
 {
 printk("Memory exhausted: want %ld pages, but only %ld are left\n",
n, nr_free_pages);
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 04/18] mini-os: add description of x86 memory usage

[Juergen Gross]

Add a brief description how the physical and virtual address usage
looks like on x86 to include/x86/arch_mm.h

Signed-off-by: Juergen Gross 
Reviewed-by: Wei Liu 
---
 include/x86/arch_mm.h | 20 
 1 file changed, 20 insertions(+)

diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index 58f29fc..f756dab 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -36,6 +36,26 @@
 #endif
 #endif
 
+/*
+ * Physical address space usage:
+ *
+ * 0..._edata: kernel text/data
+ * *stack: kernel stack (thread 0)
+ * hypervisor allocated data: p2m_list, start_info page, xenstore page,
+ *console page, initial page tables
+ * bitmap of allocated pages
+ * pages controlled by the page allocator
+ *
+ *
+ * Virtual address space usage:
+ *
+ * 1:1 mapping of physical memory starting at VA(0)
+ * 1 unallocated page
+ * demand map area (32 bits: 2 GB, 64 bits: 128 GB) for virtual allocations
+ * 1 unallocated page
+ * with libc: heap area (32 bits: 1 GB, 64 bits: 128 GB)
+ */
+
 #define L1_FRAME1
 #define L2_FRAME2
 #define L3_FRAME3
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 4/6] libxl: libxl_{primary_, }console_exec now take notify_fd argument

[Wei Liu]

The new argument will be passed down to xenconsole process, which then
uses it to notify readiness.

Signed-off-by: Wei Liu 
---
 tools/libxl/libxl.c | 15 +++
 tools/libxl/libxl.h | 43 ---
 2 files changed, 51 insertions(+), 7 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 9a7104c..da79dc1 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -1777,12 +1777,13 @@ out:
 }
 
 int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num,
-   libxl_console_type type)
+   libxl_console_type type, int notify_fd)
 {
 GC_INIT(ctx);
 char *p = GCSPRINTF("%s/xenconsole", libxl__private_bindir_path());
 char *domid_s = GCSPRINTF("%d", domid);
 char *cons_num_s = GCSPRINTF("%d", cons_num);
+char *notify_fd_s;
 char *cons_type_s;
 
 switch (type) {
@@ -1796,7 +1797,13 @@ int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, 
int cons_num,
 goto out;
 }
 
-execl(p, p, domid_s, "--num", cons_num_s, "--type", cons_type_s, (void 
*)NULL);
+if (notify_fd != -1) {
+notify_fd_s = GCSPRINTF("%d", notify_fd);
+execl(p, p, domid_s, "--num", cons_num_s, "--type", cons_type_s,
+  "--start-notify-fd", notify_fd_s, (void *)NULL);
+} else
+execl(p, p, domid_s, "--num", cons_num_s, "--type", cons_type_s,
+  (void *)NULL);
 
 out:
 GC_FREE;
@@ -1868,7 +1875,7 @@ out:
 return rc;
 }
 
-int libxl_primary_console_exec(libxl_ctx *ctx, uint32_t domid_vm)
+int libxl_primary_console_exec(libxl_ctx *ctx, uint32_t domid_vm, int 
notify_fd)
 {
 uint32_t domid;
 int cons_num;
@@ -1877,7 +1884,7 @@ int libxl_primary_console_exec(libxl_ctx *ctx, uint32_t 
domid_vm)
 
 rc = libxl__primary_console_find(ctx, domid_vm, &domid, &cons_num, &type);
 if ( rc ) return rc;
-return libxl_console_exec(ctx, domid, cons_num, type);
+return libxl_console_exec(ctx, domid, cons_num, type, notify_fd);
 }
 
 int libxl_primary_console_get_tty(libxl_ctx *ctx, uint32_t domid_vm,
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 9b59e6e..030aad5 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -67,6 +67,14 @@
  * the same $(XEN_VERSION) (e.g. throughout a major release).
  */
 
+/* LIBXL_HAVE_CONSOLE_NOTIFY_FD
+ *
+ * If this is defined, libxl_console_exec and
+ * libxl_primary_console_exe take a notify_fd parameter. That
+ * parameter will be used to notify the caller that the console is connected.
+ */
+#define LIBXL_HAVE_CONSOLE_NOTIFY_FD 1
+
 /* LIBXL_HAVE_CONST_COPY_AND_LENGTH_FUNCTIONS
  *
  * If this is defined, the copy functions have constified src parameter and the
@@ -1438,15 +1446,44 @@ int libxl_wait_for_memory_target(libxl_ctx *ctx, 
uint32_t domid, int wait_secs);
 #endif
 
 int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass);
-int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num, 
libxl_console_type type);
+
+/*
+ * If notify_fd is not -1, xenconsole will write 0x00 to it to nofity
+ * the caller that it has connected to the guest console.
+ */
+int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num,
+   libxl_console_type type, int notify_fd);
 /* libxl_primary_console_exec finds the domid and console number
  * corresponding to the primary console of the given vm, then calls
  * libxl_console_exec with the right arguments (domid might be different
  * if the guest is using stubdoms).
  * This function can be called after creating the device model, in
  * case of HVM guests, and before libxl_run_bootloader in case of PV
- * guests using pygrub. */
-int libxl_primary_console_exec(libxl_ctx *ctx, uint32_t domid_vm);
+ * guests using pygrub.
+ * If notify_fd is not -1, xenconsole will write 0x00 to it to nofity
+ * the caller that it has connected to the guest console.
+ */
+int libxl_primary_console_exec(libxl_ctx *ctx, uint32_t domid_vm,
+   int notify_fd);
+
+#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION < 0x040800
+
+static inline int libxl_console_exec_0x040700(libxl_ctx *ctx,
+  uint32_t domid, int cons_num,
+  libxl_console_type type)
+{
+return libxl_console_exec(ctx, domid, cons_num, type, -1);
+}
+#define libxl_console_exec libxl_console_exec_0x040700
+
+static inline libxl_primary_console_exec_0x040700(libxl_ctx *ctx,
+  uint32_t domid_vm)
+{
+return libxl_primary_console_exec(ctx, domid_vm, -1);
+}
+#define libxl_primary_console_exec libxl_primary_console_exec_0x040700
+
+#endif
 
 /* libxl_console_get_tty retrieves the specified domain's console tty path
  * and stores it in path. Caller is responsible for freeing the memory.
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@

[Xen-devel] [PATCH v2 5/6] docs: document xenconsole startup protocol

[Wei Liu]

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
 docs/misc/console.txt | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/docs/misc/console.txt b/docs/misc/console.txt
index ed7b795..16da805 100644
--- a/docs/misc/console.txt
+++ b/docs/misc/console.txt
@@ -124,3 +124,9 @@ can only have one pv console with xenstored as backend (the 
stubdom
 could provide pv console backends to the hvm guest but then it would
 need another pv console connection for each console backend to export
 the pty to dom0).
+
+The xenconsole program supports a very simple protocol to notify parent about
+its readiness. If xenconsole (the client) is exec'ed and has been given a fd
+(normally the write end of a pipe) via --start-notify-fd option, it will
+write 0x00 to that fd after connecting to the guest but before entering the
+event loop. Parent can read from the read end of the fd to know the status.
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 0/6] Fix console synchronisation issues

[Wei Liu]

Version 2 of this series. Now it properly handles read and write errors. I also
dropped two patches to not move the wait bodge from xenconsole to libxl.

Wei Liu (6):
  tools/console: fix help string in client
  tools/console: introduce --start-notify-fd option for console client
  libxl: factor out libxl__console_tty_path
  libxl: libxl_{primary_,}console_exec now take notify_fd argument
  docs: document xenconsole startup protocol
  xl: use xenconsole startup protocol

 docs/misc/console.txt   |  6 
 tools/console/client/main.c | 24 ++-
 tools/libxl/libxl.c | 72 ++---
 tools/libxl/libxl.h | 43 +--
 tools/libxl/xl_cmdimpl.c| 43 +--
 5 files changed, 157 insertions(+), 31 deletions(-)

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 6/6] xl: use xenconsole startup protocol

[Wei Liu]

If user asks xl to automatically connect to console when creating a
guest, use the new startup protocol before trying to unpause domain so
that we don't lose any console output.

Signed-off-by: Wei Liu 
---
v2: properly handle read(2) errors
---
 tools/libxl/xl_cmdimpl.c | 43 ---
 1 file changed, 40 insertions(+), 3 deletions(-)

diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 180fc8d..92002dc 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -2728,6 +2728,7 @@ static void autoconnect_console(libxl_ctx *ctx_ignored,
 libxl_event *ev, void *priv)
 {
 uint32_t bldomid = ev->domid;
+int notify_fd = *(int*)priv; /* write end of the notification pipe */
 
 libxl_event_free(ctx, ev);
 
@@ -2740,7 +2741,7 @@ static void autoconnect_console(libxl_ctx *ctx_ignored,
 postfork();
 
 sleep(1);
-libxl_primary_console_exec(ctx, bldomid);
+libxl_primary_console_exec(ctx, bldomid, notify_fd);
 /* Do not return. xl continued in child process */
 perror("xl: unable to exec console client");
 _exit(1);
@@ -2810,6 +2811,7 @@ static int create_domain(struct domain_create *dom_info)
 int restore_fd_to_close = -1;
 int send_back_fd = -1;
 const libxl_asyncprogress_how *autoconnect_console_how;
+int notify_pipe[2] = { -1, -1 };
 struct save_file_header hdr;
 uint32_t domid_soft_reset = INVALID_DOMID;
 
@@ -2997,7 +2999,15 @@ start:
 
 libxl_asyncprogress_how autoconnect_console_how_buf;
 if ( dom_info->console_autoconnect ) {
+if (libxl_pipe(ctx, notify_pipe)) {
+fprintf(stderr,
+"Failed to create console notification pipe, errno %d\n",
+errno);
+ret = ERROR_FAIL;
+goto error_out;
+}
 autoconnect_console_how_buf.callback = autoconnect_console;
+autoconnect_console_how_buf.for_callback = ¬ify_pipe[1];
 autoconnect_console_how = &autoconnect_console_how_buf;
 }else{
 autoconnect_console_how = 0;
@@ -3047,6 +3057,33 @@ start:
 restore_fd_to_close = -1;
 }
 
+if (autoconnect_console_how) {
+char buf[1];
+int r;
+
+/* Try to get notification from xenconsole. Just move on if
+ * error occurs -- it's only minor annoyance if console
+ * doesn't show up.
+ */
+do {
+r = read(notify_pipe[0], buf, 1);
+} while (r == -1 && errno == EINTR);
+
+if (r == -1)
+fprintf(stderr,
+"Failed to get notification from xenconsole: %s\n",
+strerror(errno));
+else if (r == 0)
+fprintf(stderr, "Got EOF from xenconsole notification fd\n");
+else if (r == 1 && buf[0] != 0x00)
+fprintf(stderr, "Got unexpected response from xenconsole: %x\n",
+buf[0]);
+
+close(notify_pipe[0]);
+close(notify_pipe[1]);
+notify_pipe[0] = notify_pipe[1] = -1;
+}
+
 if (!paused)
 libxl_domain_unpause(ctx, domid);
 
@@ -3754,9 +3791,9 @@ int main_console(int argc, char **argv)
 
 domid = find_domain(argv[optind]);
 if (!type)
-libxl_primary_console_exec(ctx, domid);
+libxl_primary_console_exec(ctx, domid, -1);
 else
-libxl_console_exec(ctx, domid, num, type);
+libxl_console_exec(ctx, domid, num, type, -1);
 fprintf(stderr, "Unable to attach console\n");
 return EXIT_FAILURE;
 }
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 2/6] tools/console: introduce --start-notify-fd option for console client

[Wei Liu]

The console client will write 0x00 to that fd before entering console
loop to indicate its readiness.

Signed-off-by: Wei Liu 
---
v2: properly handle write(2) errors
---
 tools/console/client/main.c | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/tools/console/client/main.c b/tools/console/client/main.c
index be39700..99f 100644
--- a/tools/console/client/main.c
+++ b/tools/console/client/main.c
@@ -77,6 +77,7 @@ static void usage(const char *program) {
   "  -h, --help   display this help and exit\n"
   "  -n, --num N  use console number N\n"
   "  --type TYPE  console type. must be 'pv' or 'serial'\n"
+  "  --start-notify-fd N file descriptor used to notify parent\n"
   , program);
 }
 
@@ -327,10 +328,12 @@ int main(int argc, char **argv)
int ch;
unsigned int num = 0;
int opt_ind=0;
+   int start_notify_fd = -1;
struct option lopt[] = {
{ "type", 1, 0, 't' },
{ "num", 1, 0, 'n' },
{ "help",0, 0, 'h' },
+   { "start-notify-fd", 1, 0, 's' },
{ 0 },
 
};
@@ -364,6 +367,9 @@ int main(int argc, char **argv)
exit(EINVAL);
}
break;
+   case 's':
+   start_notify_fd = atoi(optarg);
+   break;
default:
fprintf(stderr, "Invalid argument\n");
fprintf(stderr, "Try `%s --help' for more 
information.\n", 
@@ -462,6 +468,22 @@ int main(int argc, char **argv)
init_term(STDIN_FILENO, &stdin_old_attr);
atexit(restore_term_stdin); /* if this fails, oh dear */
}
+
+   if (start_notify_fd != -1) {
+   /* Write 0x00 to notify parent about client's readiness */
+   static const char msg[] = { 0x00 };
+   int r;
+
+   do {
+   r = write(start_notify_fd, msg, 1);
+   } while ((r == -1 && errno == EINTR) || r == 0);
+
+   if (r == -1)
+   err(errno, "Could not notify parent with fd %d",
+   start_notify_fd);
+   close(start_notify_fd);
+   }
+
console_loop(spty, xs, path, interactive);
 
free(path);
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 1/6] tools/console: fix help string in client

[Wei Liu]

There is no short '-t' option.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
 tools/console/client/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/console/client/main.c b/tools/console/client/main.c
index f660e10..be39700 100644
--- a/tools/console/client/main.c
+++ b/tools/console/client/main.c
@@ -76,7 +76,7 @@ static void usage(const char *program) {
   "\n"
   "  -h, --help   display this help and exit\n"
   "  -n, --num N  use console number N\n"
-  "  -t, --type TYPE  console type. must be 'pv' or 'serial'\n"
+  "  --type TYPE  console type. must be 'pv' or 'serial'\n"
   , program);
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 3/6] libxl: factor out libxl__console_tty_path

[Wei Liu]

No other user yet.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
 tools/libxl/libxl.c | 57 ++---
 1 file changed, 37 insertions(+), 20 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 7bd3e8c..9a7104c 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -1742,6 +1742,40 @@ static void domain_destroy_domid_cb(libxl__egc *egc,
 dis->callback(egc, dis, rc);
 }
 
+static int libxl__console_tty_path(libxl__gc *gc, uint32_t domid, int cons_num,
+   libxl_console_type type, char **tty_path)
+{
+int rc;
+char *dom_path;
+
+dom_path = libxl__xs_get_dompath(gc, domid);
+if (!dom_path) {
+rc = ERROR_FAIL;
+goto out;
+}
+
+switch (type) {
+case LIBXL_CONSOLE_TYPE_SERIAL:
+*tty_path = GCSPRINTF("%s/serial/%d/tty", dom_path, cons_num);
+rc = 0;
+break;
+case LIBXL_CONSOLE_TYPE_PV:
+if (cons_num == 0)
+*tty_path = GCSPRINTF("%s/console/tty", dom_path);
+else
+*tty_path = GCSPRINTF("%s/device/console/%d/tty", dom_path,
+  cons_num);
+rc = 0;
+break;
+default:
+rc = ERROR_INVAL;
+goto out;
+}
+
+out:
+return rc;
+}
+
 int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num,
libxl_console_type type)
 {
@@ -1773,30 +1807,13 @@ int libxl_console_get_tty(libxl_ctx *ctx, uint32_t 
domid, int cons_num,
   libxl_console_type type, char **path)
 {
 GC_INIT(ctx);
-char *dom_path;
 char *tty_path;
 char *tty;
 int rc;
 
-dom_path = libxl__xs_get_dompath(gc, domid);
-if (!dom_path) {
-rc = ERROR_FAIL;
-goto out;
-}
-
-switch (type) {
-case LIBXL_CONSOLE_TYPE_SERIAL:
-tty_path = GCSPRINTF("%s/serial/0/tty", dom_path);
-break;
-case LIBXL_CONSOLE_TYPE_PV:
-if (cons_num == 0)
-tty_path = GCSPRINTF("%s/console/tty", dom_path);
-else
-tty_path = GCSPRINTF("%s/device/console/%d/tty", dom_path,
-cons_num);
-break;
-default:
-rc = ERROR_INVAL;
+rc = libxl__console_tty_path(gc, domid, cons_num, type, &tty_path);
+if (rc) {
+LOG(ERROR, "Failed to get tty path for domain %d\n", domid);
 goto out;
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] monitor access to pages with a specific p2m_type_t

[Tamas K Lengyel]

On Fri, Aug 5, 2016 at 5:35 AM, sepanta s  wrote:

>
>
> On Tue, Aug 2, 2016 at 8:23 PM, Tamas K Lengyel 
> wrote:
>
>>
>>
>> On Tue, Aug 2, 2016 at 12:19 AM, sepanta s  wrote:
>>
>>>
>>>
>>> On Sat, Jul 23, 2016 at 3:49 PM, sepanta s  wrote:
>>>

>> Hi,
>> Is there any sample code which I can undestand how to capture the
>> events on the gfns which have p2m_ram_shared enabled ?
>> I couldn't find any ... .
>> I would be grateful if any help , as there is not any documents
>> through net to use :(
>>
>>
> Should I just set the ring_page as the pages which are shared and mark
>> them read-only, then capture the write events?
>>
>
> Not sure what ring_page you are talking about, but if you mark the
> pages read-only with mem_access you will get notifications for events that
> lead to unsharing with p2m_ram_shared type pages as well.
>

 There was a function in mem-sharing.c which is intended to announce the
 failed unshared pages. It is "mem_sharing_notify_enomem" .
 I added "mem_sharing_notify_unshare" as a new function and call it in
 also XEN_DOMCTL_VM_EVENT_OP_UNSHARING and "HVM_PARAM_USHARING_RING_PFN".
 I also added the required codes in /xen/common/vm_event.c and
 /tools/libxc/xc_vm_event so as
 I have added a new event for the unsharing actions of a page.
 Then, I wrote a sample code line xen-access and create a ring for the
 pages of a domain and listen to unshared events of it.

>
>> BTW, I added a function called mem_sharing_notify_unshare to
>> mem_sharing.c and added it to __mem_sharing_unshare_page at this part:
>>
>> *if ( p2m_change_type_one(d, gfn, p2m_ram_shared, p2m_ram_rw) )*
>> *{*
>> *gdprintk(XENLOG_ERR, "Could not change p2m type d %hu gfn %lx.\n", *
>> *d->domain_id, gfn);*
>> *BUG();*
>> *}else {*
>>
>>
>> * mem_sharing_notify_unshare(d,gfn.0);*
>> *}*
>>
>>
> IMHO this duplicates a lot of what mem_access does already, I don't
> think there is a need for a separate notification on another ring.
>
>
 You are right, xen-access should work but I couldn't change its code
 and couldn't get the mem-access events.
  I just added the above function to be sure that unsharing a page
 happens and works fine. Because I couldn't get the access requests on
 shared-pages of a vm.
 In xen-access, Instead of setting all the pages' default access to rx ,
 I just call xc_set_mem_access for the pages with p2m_ram_shared and assign
 rx as the default access but there is no requests on this ring.

>

 So by having a vm event channel listening to unsharing event, I can see
>> the notification in xen-access . To do so, I
>> have used vm_event_enable which uses HVM_PARAM_SHARING_RING_PFN .
>> But, when I used memshrtool to share all the pages of two vms - my
>> vm1 and its clone vm2 .
>> About 900 MB of the ram is shared but there are a lot of unshared
>> events happening.
>>
>
> Yes, I would say that's expected.
>

>
>> When I do the sharing one more time, there is not any pages unshared
>> as the total number of shared pages stay the same.
>>
>
> Well, if you let the domain run for a while after sharing, then you do
> the sharing like that again you are likely going to crash the VM.
>
>
>> Seems no unsharing is done as the number of shared pages are the same.
>> Does any page fault triggers when a write operation is done on a
>> shared page among two vms ?
>>
>
> I would guess the the VM crashed and that's why you don't see any more
> unsharing at that point.
>
 Yes it was crashed as I checked it.
 The scenario of sharing is I use:
 I pause the origin VM and then run memshrtool on origin VM and clone
 VM. After sharing all the pages between these two VMs,Clone VM seems to be
 inaccessible. The clone seems to work as the attached photo shows, its cpu
 time increases and it exceeds the dom0 cpu time but when I use gvncviewer
 to see the GUI of the Clone VM, the mouse or keyboard don't work. (origin
 VM is ubunut-64-1 and clone VM is ubuntu-64-clone-1). Is there anything I
 have missed in sharing between two VMs?

 [image: Inline image 2]

>>>
>>> Can someone help please ? still have problem with the machine .
>>> is it because sharing is not based on content?
>>>
>>>
>> Well, the emulated device contexts probably get all messed up when you
>> just clone over the memory of the domain and that's why they don't work.
>> The only way I got this to work is by doing a save/restore of the origin
>> domain before doing the memory sharing.
>>
>> Tamas
>>
> I do so , as I use the following commands to create the clone :
> xl save -c origin-domid /tmp/clone
> xl restore -p -e  /etc/xen/clone.config  /tmp/clone
> Also I keep t

Re: [Xen-devel] monitor access to pages with a specific p2m_type_t

[sepanta s]

On Fri, Aug 5, 2016 at 10:45 PM, Tamas K Lengyel 
wrote:

>
>
> On Fri, Aug 5, 2016 at 5:35 AM, sepanta s  wrote:
>
>>
>>
>> On Tue, Aug 2, 2016 at 8:23 PM, Tamas K Lengyel 
>> wrote:
>>
>>>
>>>
>>> On Tue, Aug 2, 2016 at 12:19 AM, sepanta s  wrote:
>>>


 On Sat, Jul 23, 2016 at 3:49 PM, sepanta s 
 wrote:

>
>>> Hi,
>>> Is there any sample code which I can undestand how to capture the
>>> events on the gfns which have p2m_ram_shared enabled ?
>>> I couldn't find any ... .
>>> I would be grateful if any help , as there is not any documents
>>> through net to use :(
>>>
>>>
>> Should I just set the ring_page as the pages which are shared and
>>> mark them read-only, then capture the write events?
>>>
>>
>> Not sure what ring_page you are talking about, but if you mark the
>> pages read-only with mem_access you will get notifications for events 
>> that
>> lead to unsharing with p2m_ram_shared type pages as well.
>>
>
> There was a function in mem-sharing.c which is intended to announce
> the failed unshared pages. It is "mem_sharing_notify_enomem" .
> I added "mem_sharing_notify_unshare" as a new function and call it in
> also XEN_DOMCTL_VM_EVENT_OP_UNSHARING and "HVM_PARAM_USHARING_RING_PFN".
> I also added the required codes in /xen/common/vm_event.c and
> /tools/libxc/xc_vm_event so as
> I have added a new event for the unsharing actions of a page.
> Then, I wrote a sample code line xen-access and create a ring for the
> pages of a domain and listen to unshared events of it.
>
>>
>>> BTW, I added a function called mem_sharing_notify_unshare to
>>> mem_sharing.c and added it to __mem_sharing_unshare_page at this part:
>>>
>>> *if ( p2m_change_type_one(d, gfn, p2m_ram_shared, p2m_ram_rw) )*
>>> *{*
>>> *gdprintk(XENLOG_ERR, "Could not change p2m type d %hu gfn %lx.\n", *
>>> *d->domain_id, gfn);*
>>> *BUG();*
>>> *}else {*
>>>
>>>
>>> * mem_sharing_notify_unshare(d,gfn.0);*
>>> *}*
>>>
>>>
>> IMHO this duplicates a lot of what mem_access does already, I don't
>> think there is a need for a separate notification on another ring.
>>
>>
> You are right, xen-access should work but I couldn't change its code
> and couldn't get the mem-access events.
>  I just added the above function to be sure that unsharing a page
> happens and works fine. Because I couldn't get the access requests on
> shared-pages of a vm.
> In xen-access, Instead of setting all the pages' default access to rx
> , I just call xc_set_mem_access for the pages with p2m_ram_shared and
> assign rx as the default access but there is no requests on this ring.
>
>>
>
> So by having a vm event channel listening to unsharing event, I can
>>> see the notification in xen-access . To do so, I
>>> have used vm_event_enable which uses HVM_PARAM_SHARING_RING_PFN .
>>> But, when I used memshrtool to share all the pages of two vms - my
>>> vm1 and its clone vm2 .
>>> About 900 MB of the ram is shared but there are a lot of unshared
>>> events happening.
>>>
>>
>> Yes, I would say that's expected.
>>
>
>>
>>> When I do the sharing one more time, there is not any pages unshared
>>> as the total number of shared pages stay the same.
>>>
>>
>> Well, if you let the domain run for a while after sharing, then you
>> do the sharing like that again you are likely going to crash the VM.
>>
>>
>>> Seems no unsharing is done as the number of shared pages are the
>>> same.
>>> Does any page fault triggers when a write operation is done on a
>>> shared page among two vms ?
>>>
>>
>> I would guess the the VM crashed and that's why you don't see any
>> more unsharing at that point.
>>
> Yes it was crashed as I checked it.
> The scenario of sharing is I use:
> I pause the origin VM and then run memshrtool on origin VM and clone
> VM. After sharing all the pages between these two VMs,Clone VM seems to be
> inaccessible. The clone seems to work as the attached photo shows, its cpu
> time increases and it exceeds the dom0 cpu time but when I use gvncviewer
> to see the GUI of the Clone VM, the mouse or keyboard don't work. (origin
> VM is ubunut-64-1 and clone VM is ubuntu-64-clone-1). Is there anything I
> have missed in sharing between two VMs?
>
> [image: Inline image 2]
>

 Can someone help please ? still have problem with the machine .
 is it because sharing is not based on content?


>>> Well, the emulated device contexts probably get all messed up when you
>>> just clone over the memory of the domain and that's why they don't work.
>>> The only way I got this to work is by doing a save/restore of the origin
>>> domain before doing

[Xen-devel] [xen-unstable-smoke test] 99970: tolerable all pass - PUSHED

[osstest service owner]

flight 99970 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99970/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  7fb0a87d97201f9c3639f85615eacd93110dc1c5
baseline version:
 xen  35dbf099ac18924d40533c9d1b9bfbf1ecb818c9

Last test of basis99958  2016-08-05 10:01:24 Z0 days
Testing same since99970  2016-08-05 17:03:22 Z0 days1 attempts


People who touched revisions under test:
  Jan Beulich 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=7fb0a87d97201f9c3639f85615eacd93110dc1c5
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
7fb0a87d97201f9c3639f85615eacd93110dc1c5
+ branch=xen-unstable-smoke
+ revision=7fb0a87d97201f9c3639f85615eacd93110dc1c5
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-4.7-testing
+ '[' x7fb0a87d97201f9c3639f85615eacd93110dc1c5 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src 
'[fetch=try]'
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local 'options=[fetch=try]'
 getconfig GitCacheProxy
 perl -e '
use Osstest;
readglobalconfig();
print $c{"GitCacheProxy"} or die $!;
'
+++ local cache=git://cache:9419/
+++ '[' xgit://cache:9419/ '!=' x ']'
+++ echo 
'git://cache:9419/https://github.

Re: [Xen-devel] [PATCH v2 3/3] livepatch: Sync cache of build-id before using it first time.

[Konrad Rzeszutek Wilk]

On Fri, Aug 05, 2016 at 09:40:43AM -0600, Jan Beulich wrote:
> >>> On 04.08.16 at 17:49,  wrote:
> > --- a/xen/include/xen/livepatch.h
> > +++ b/xen/include/xen/livepatch.h
> > @@ -44,7 +44,7 @@ unsigned long livepatch_symbols_lookup_by_name(const char 
> > *symname);
> >  bool_t is_patch(const void *addr);
> >  int xen_build_id_check(const Elf_Note *n, unsigned int n_sz,
> > const void **p, unsigned int *len);
> > -
> > +void xen_build_init(void);
> >  /* Arch hooks. */
> 
> Please don't ditch a blank line like this. But this is the wrong header
> anyway, or else you'd have to make version.c include it (which would
> again seem odd). And as I now realize that same aspect applies to
> xen_build_id_check() too - this needs to move into xen/version.h.

Back in April (feels like eons ago) with
"xsplice: Stacking build-id dependency checking."

I have this comment (on v7 of it) - 
https://lists.xen.org/archives/html/xen-devel/2016-04/msg01492.html:
" Moved xen_build_id_check definition to xsplice.h from version.h
(and dropping the #include's in version.h)"

on that patch.

Digging in the archive, the reason is outlined in:
https://lists.xen.org/archives/html/xen-devel/2016-04/msg00407.html
where:

 @@ -17,4 +17,7 @@ const char *xen_deny(void);
>  #include 
>  int xen_build_id(const void **p, unsigned int *len);
>  
> +#include 
> +int xen_build_id_check(const Elf_Note *n, const void **p, unsigned 
int *len);

The #include is misplaced again, and I'm rather hesitant to see
version.h gain this dependency. Couldn't this go into xen/elf.h?

And I somehow made it go in version.h not elf.h (I believe that was due
to the Elf_Note being a macro - that is dependent on ELF_SIZE - which
was not set in elf.h - and would cause compilation issues).

Anyhow if I move those two in version.h:

In file included from console.c:13:0:
/home/konrad/ssd/konrad/xtt-x86_64/bootstrap/xen.git/xen/include/xen/version.h:19:30:
 error: unknown type name ‘Elf_Note’
 int xen_build_id_check(const Elf_Note *n, unsigned int n_sz,
  ^~~~
make[4]: *** [console.o] Error 1
make[3]: *** [char/built_in.o] Error 2
make[3]: *** Waiting for unfinished jobs
In file included from kernel.c:10:0:
/home/konrad/ssd/konrad/xtt-x86_64/bootstrap/xen.git/xen/include/xen/version.h:19:30:
 error: unknown type name ‘Elf_Note’
 int xen_build_id_check(const Elf_Note *n, unsigned int n_sz,
  ^~~~
make[3]: *** [kernel.o] Error 1
make[2]: *** 
[/home/konrad/ssd/konrad/xtt-x86_64/bootstrap/xen.git/xen/common/built_in.o] Err


I belive I have to include the #include  to deal with 
Elf_Note macro.

P.S.
This is the patch (copy-n-paste):

diff --git a/xen/include/xen/livepatch.h b/xen/include/xen/livepatch.h
index cfc9601..c297056 100644
--- a/xen/include/xen/livepatch.h
+++ b/xen/include/xen/livepatch.h
@@ -42,9 +42,6 @@ int livepatch_op(struct xen_sysctl_livepatch_op *);
 void check_for_livepatch_work(void);
 unsigned long livepatch_symbols_lookup_by_name(const char *symname);
 bool_t is_patch(const void *addr);
-int xen_build_id_check(const Elf_Note *n, unsigned int n_sz,
-   const void **p, unsigned int *len);
-void xen_build_init(void);
 /* Arch hooks. */
 int arch_livepatch_verify_elf(const struct livepatch_elf *elf);
 int arch_livepatch_perform_rel(struct livepatch_elf *elf,
diff --git a/xen/include/xen/version.h b/xen/include/xen/version.h
index 400160f..19a3aaa 100644
--- a/xen/include/xen/version.h
+++ b/xen/include/xen/version.h
@@ -15,4 +15,10 @@ const char *xen_banner(void);
 const char *xen_deny(void);
 int xen_build_id(const void **p, unsigned int *len);
 
+#ifdef BUILD_ID
+int xen_build_id_check(const Elf_Note *n, unsigned int n_sz,
+   const void **p, unsigned int *len);
+void xen_build_init(void);
+#endif
+
 #endif /* __XEN_VERSION_H__ */


> 
> Jan
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 1/3] livepach: Add .livepatch.hooks functions and test-case

[Konrad Rzeszutek Wilk]

On Fri, Aug 05, 2016 at 09:35:49AM -0600, Jan Beulich wrote:
> >>> On 04.08.16 at 17:49,  wrote:
> > In general, the hooks provide flexibility when having to deal with
> > unforeseen cases, but their application should be rarely required (<
> > 10%)."
> 
> But the greater flexibility of course comes with increased chances
> of screwing things up. I'm therefore still not entirely convinced that
> such XSAs wouldn't then better not be live patched.
> 
> > --- a/xen/arch/x86/test/xen_hello_world.c
> > +++ b/xen/arch/x86/test/xen_hello_world.c
> > @@ -4,14 +4,50 @@
> >   */
> >  
> >  #include "config.h"
> > +#include 
> >  #include 
> >  #include 
> >  #include 
> > +#include 
> >  
> >  #include 
> >  
> >  static char hello_world_patch_this_fnc[] = "xen_extra_version";
> >  extern const char *xen_hello_world(void);
> > +static unsigned int cnt;
> > +
> > +static void apply_hook(void)
> > +{
> > +printk(KERN_DEBUG "Hook executing.\n");
> > +}
> > +
> > +static void revert_hook(void)
> > +{
> > +printk(KERN_DEBUG "Hook unloaded.\n");
> > +}
> > +
> > +static void hi_func(void)
> > +{
> > +printk(KERN_DEBUG "%s: Hi! (called %u times)\n", __func__, ++cnt);
> > +};
> > +
> > +/* If we are sorted we _MUST_ be the last .livepatch.hook section. */
> > +static void Z_check_fnc(void)
> 
> And that Z_ prefix is supposed to guarantee that being last? Isn't
> it that upper case letters sort before lower case ones?

Yes. And the code is actually flexible enough not to be the last one.

B/c:
> 
> > +{
> > +printk(KERN_DEBUG "%s: Hi func called %u times\n", __func__, cnt);
> > +BUG_ON(cnt == 0 || cnt > 2);
> > +cnt = 0; /* Otherwise if you revert, apply, revert the value will be 
> > 4! */
> 
> Isn't this an indication of a general weakness: Shouldn't a patch
> module be allowed to assume it's being run in a clean state, i.e.
> without any of its static data altered from their load time values?

Of this bug. (which I obviously need to fix).
> 
> > @@ -70,7 +71,11 @@ struct payload {
> >  unsigned int nsyms;  /* Nr of entries in .strtab and 
> > symbols. */
> >  struct livepatch_build_id id;/* 
> > ELFNOTE_DESC(.note.gnu.build-id) of the payload. */
> >  struct livepatch_build_id dep;   /* 
> > ELFNOTE_DESC(.livepatch.depends). */
> > -char name[XEN_LIVEPATCH_NAME_SIZE]; /* Name of it. */
> > +livepatch_loadcall_t **load_funcs;   /* The array of funcs to call 
> > after */
> > +livepatch_unloadcall_t **unload_funcs;/* load and unload of the 
> > payload. */
> 
> These both seem like they want a const in the middle.

I did that initially and found out that the calling ->load_funcs[i] resulted
in _no_ code being called.

I did not dig in the assembler output to figure out what happend, let me
do that now.


> 
> > @@ -515,6 +520,27 @@ static int prepare_payload(struct payload *payload,
> >  }
> >  }
> >  
> > +sec = livepatch_elf_sec_by_name(elf, ".livepatch.hooks.load");
> 
> Is the .hooks infix really needed?
> 
> > +if ( sec )
> > +{
> > +if ( !sec->sec->sh_size ||
> 
> What's wrong with a zero size section (holding no hooks)? We've
> had that same case elsewhere in the original series ...

That would be a bit odd (having zero size).

But yes there is nothing wrong with it.  I will fix it up.

> 
> > @@ -999,6 +1025,17 @@ static int apply_payload(struct payload *data)
> >  
> >  arch_livepatch_quiesce();
> >  
> > +/*
> > + * The hooks may call common code which expects spinlocks to be certain
> > + * type, as such disable this temporarily.
> > + */
> > +spin_debug_disable();
> > +for ( i = 0; i < data->n_load_funcs; i++ )
> > +data->load_funcs[i]();
> > +spin_debug_enable();
> 
> I have to admit that I can't really make sense of the comment, and
> hence the code.

Perhaps:

Since we are running with IRQs disabled and the hooks may call common
code - which expects the spinlocks to run with IRQs enabled - we temporarly
disable the spin locks "IRQ affinity state"

?
> 
> > --- /dev/null
> > +++ b/xen/include/xen/livepatch_payload.h
> > @@ -0,0 +1,49 @@
> > +/*
> > + * Copyright (C) 2016 Citrix Systems R&D Ltd.
> > + */
> > +
> > +#ifndef __XEN_LIVEPATCH_PAYLOAD_H__
> > +#define __XEN_LIVEPATCH_PAYLOAD_H__
> > +
> > +/*
> > + * The following definitions are to be used in patches. They are taken
> > + * from kpatch.
> > + */
> > +typedef void livepatch_loadcall_t(void);
> > +typedef void livepatch_unloadcall_t(void);
> > +
> > +/*
> > + * LIVEPATCH_LOAD_HOOK macro
> > + *
> > + * Declares a function pointer to be allocated in a new
> > + * .livepatch.hook.load section.  This livepatch_load_data symbol is later
> > + * stripped by create-diff-object so that it can be declared in multiple
> > + * objects that are later linked together, avoiding global symbol
> > + * collision.  Since multiple hooks can be registered, the
> > + * .livepatch.hook.load section is a table of functions th

[Xen-devel] [xen-4.7-testing test] 99961: regressions - FAIL

[osstest service owner]

flight 99961 xen-4.7-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/99961/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-arndale   7 host-ping-check-xen   fail REGR. vs. 99754

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop  fail like 99754
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop  fail like 99754

Tests which did not succeed, but are not blocking:
 test-amd64-i386-rumpuserxen-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-rumpuserxen-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 build-i386-rumpuserxen6 xen-buildfail   never pass
 build-amd64-rumpuserxen   6 xen-buildfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt-raw 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  f2160ba6e60e990060de96f2fc9be645f51f5995
baseline version:
 xen  899495b60a6e55fc2afa69d4616cb08af212de12

Last test of basis99754  2016-07-28 14:03:00 Z8 days
Testing same since99961  2016-08-05 12:21:29 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Jan Beulich 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-prev pass
 build-i386-prev  pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pas

[Xen-devel] [PATCH v4 00/19] x86: multiboot2 protocol support

[Daniel Kiper]

Hi,

I am sending fourth version of multiboot2 protocol support for
legacy BIOS and EFI platforms. This patch series release contains
fixes for all known issues.

The final goal is xen.efi binary file which could be loaded by EFI
loader, multiboot (v1) protocol (only on legacy BIOS platforms) and
multiboot2 protocol. This way we will have:
  - smaller Xen code base,
  - one code base for xen.gz and xen.efi,
  - one build method for xen.gz and xen.efi;
xen.efi will be extracted from xen(-syms)
file using objcopy or special custom tool,
  - xen.efi build will not so strongly depend
on a given GCC and binutils version.

Here is short list of changes:
  - new patches: 01, 02, 04, 12, 13,
  - changed patches: 03, 05, 06, 08, 09, 10, 11, 14, 15, 16, 17, 18, 19.

This patch series was build with following tools:
  - gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)
and binutils 2.17-3+etch1,
  - gcc version 4.4.7 20120313 (Red Hat 4.4.7-11) (GCC)
and binutils 2.23.2-2.el6,
  - gcc version 4.7.2 (Debian 4.7.2-5)
and binutils 2.22-8,
  - gcc version 5.1.1 20150618 (Red Hat 5.1.1-4) (GCC)
and binutils 2.25-9.fc22.

I hope that features provided by this patch series will be included
in Xen 4.8 release in one way or another.

If you are not interested in this patch series at all please
drop me a line and I will remove you from distribution list.

Daniel

PS FYI, I will be on vacation in August 6-15 and 20-28.
   It will be nice if you review my patches during that time.

 .gitignore|5 +-
 xen/arch/x86/Makefile |8 +-
 xen/arch/x86/Rules.mk |4 +
 xen/arch/x86/boot/Makefile|   10 +-
 xen/arch/x86/boot/build32.lds |   53 
 xen/arch/x86/boot/build32.mk  |   12 +-
 xen/arch/x86/boot/cmdline.S   |  367 
-
 xen/arch/x86/boot/cmdline.c   |  376 
++
 xen/arch/x86/boot/edd.S   |3 -
 xen/arch/x86/boot/head.S  |  568 
+-
 xen/arch/x86/boot/reloc.c |  246 
 xen/arch/x86/boot/trampoline.S|   22 +++-
 xen/arch/x86/boot/video.S |6 -
 xen/arch/x86/boot/wakeup.S|4 +-
 xen/arch/x86/boot/x86_64.S|   51 +++-
 xen/arch/x86/dmi_scan.c   |4 +-
 xen/arch/x86/domain_page.c|2 +-
 xen/arch/x86/efi/Makefile |   11 +-
 xen/arch/x86/efi/efi-boot.h   |  108 ++--
 xen/arch/x86/efi/stub.c   |   30 -
 xen/arch/x86/mpparse.c|4 +-
 xen/arch/x86/setup.c  |   48 +++
 xen/arch/x86/shutdown.c   |5 +-
 xen/arch/x86/time.c   |2 +-
 xen/arch/x86/x86_64/asm-offsets.c |   12 ++
 xen/arch/x86/xen.lds.S|   14 +-
 xen/common/efi/boot.c |   31 -
 xen/common/efi/runtime.c  |   20 +--
 xen/common/lib.c  |   10 +-
 xen/drivers/acpi/osl.c|2 +-
 xen/include/asm-x86/config.h  |1 +
 xen/include/asm-x86/page.h|2 +-
 xen/include/xen/efi.h |   14 +-
 xen/include/xen/multiboot2.h  |  182 ++
 34 files changed, 1619 insertions(+), 618 deletions(-)

Daniel Kiper (19):
  x86: allow EFI reboot method neither on EFI platforms...
  x86/boot: remove multiboot1_header_end from symbol table
  x86/boot: create *.lnk files with linker script
  x86/boot/reloc: reduce assembly usage as much as possible
  x86/boot: call reloc() using stdcall calling convention
  x86/boot/reloc: create generic alloc and copy functions
  x86/boot: use %ecx instead of %eax
  x86/boot/reloc: Rename some variables and rearrange code a bit
  x86: add multiboot2 protocol support
  efi: move efi struct initialization to xen/common/lib.c
  efi: create efi_enabled()
  efi: introduce EFI_RS to ease control on runtime services usage
  efi: EFI_RS bit in efi.flags must be controlled by efi=[no-]rs command 
line argument
  efi: build xen.gz with EFI code
  x86/efi: create new early memory allocator
  x86: add multiboot2 protocol support for EFI platforms
  x86/boot: implement early command line parser in C
  x86: make Xen early boot code relocatable
  x86: add multiboot2 protocol support for relocatable images


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4 02/19] x86/boot: remove multiboot1_header_end from symbol table

[Daniel Kiper]

Its visibility is not needed and just pollute symbol table.

Suggested-by: Jan Beulich 
Signed-off-by: Daniel Kiper 
---
 xen/arch/x86/boot/head.S |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index 85770e8..e34351c 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -32,7 +32,7 @@ multiboot1_header_start:   /*** MULTIBOOT1 HEADER /
 .long   MULTIBOOT_HEADER_FLAGS
 /* Checksum: must be the negated sum of the first two fields. */
 .long   -(MULTIBOOT_HEADER_MAGIC + MULTIBOOT_HEADER_FLAGS)
-multiboot1_header_end:
+.Lmultiboot1_header_end:
 
 .section .init.rodata, "a", @progbits
 .align 4
-- 
1.7.10.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel