Re: [Xen-devel] [PATCH v4 2/9] x86: introduce a new set of APIs to manage Xen page tables

2019-12-04 Thread Xia, Hongyan
> There's no need for the map/unmap functions to have a _new > suffix, is there? I thought this was weird at first also, but what I find really useful is that we can just change all call sites to the new API in small steps without breaking. Otherwise we have to merge a huge batch of changes (aroun

Re: [Xen-devel] [PATCH v4 3/9] x86/mm: introduce l{1, 2}t local variables to map_pages_to_xen

2019-12-04 Thread Xia, Hongyan
>> @@ -5272,6 +5279,7 @@ int map_pages_to_xen( >> ((1u << PAGETABLE_ORDER) - 1)) == 0)) ) >> { >> unsigned long base_mfn; >> +l1_pgentry_t *l1t; > > const, as this looks to be used for lookup only? I cannot do this for now since va

Re: [Xen-devel] [PATCH v4 0/9] Add alternative API for Xen PTEs

2019-12-05 Thread Xia, Hongyan
I have addressed the comments that I can find in the archive. Some big debates (like _new or not _new, whether to modularise map_pages_to_xen) have not been touched. Various acked-by and reviewed-by added. Hongyan On Thu, 2019-12-05 at 10:14 +0100, Jan Beulich wrote: > On 04.12.2019 18:10, Hongya

Re: [Xen-devel] [PATCH v4 6/9] x86/mm: add an end_of_loop label in map_pages_to_xen

2019-12-05 Thread Xia, Hongyan
>On 02.10.2019 19:16, Hongyan Xia wrote: >> We will soon need to clean up mappings whenever the out most loop is >> ended. Add a new label and turn relevant continue's into goto's. > >I think already when this still was RFC I did indicate that I'm not >happy about the introduction of these labels (

Re: [Xen-devel] [PATCH v4 0/9] Add alternative API for Xen PTEs

2019-12-05 Thread Xia, Hongyan
are stuck with the old API for now. I can review relevant functions and const qualify other applicable ones. Hongyan On Thu, 2019-12-05 at 10:51 +0100, Jan Beulich wrote: > On 05.12.2019 10:41, Xia, Hongyan wrote: > > I have addressed the comments that I can find in the archive. > &g

Re: [Xen-devel] [PATCH v4 6/9] x86/mm: add an end_of_loop label in map_pages_to_xen

2019-12-05 Thread Xia, Hongyan
:02, Durrant, Paul wrote: > > > -Original Message- > > > From: Xen-devel On > > > Behalf Of Jan > > > Beulich > > > Sent: 05 December 2019 10:26 > > > To: Xia, Hongyan > > > Cc: andrew.coop...@citrix.com; xen-devel@lists.xenproject.

Re: [Xen-devel] [PATCH v4 6/9] x86/mm: add an end_of_loop label in map_pages_to_xen

2019-12-06 Thread Xia, Hongyan
t; Beulich > > > Sent: 05 December 2019 10:26 > > > To: Xia, Hongyan > > > Cc: andrew.coop...@citrix.com; xen-devel@lists.xenproject.org; > > > w...@xen.org; > > > roger@citrix.com > > > Subject: Re: [Xen-devel] [PATCH v4 6/9] x86/mm: a

Re: [Xen-devel] [PATCH 1/2] x86/mm: factor out the code for shattering an l3 PTE

2019-12-07 Thread Xia, Hongyan
Hi Andrew, On Fri, 2019-12-06 at 17:50 +, Andrew Cooper wrote: > On 06/12/2019 15:53, Hongyan Xia wrote: > > map_pages_to_xen and modify_xen_mappings are performing almost > > exactly > > the same operations when shattering an l3 PTE, the only difference > > being whether we want to flush. > >

Re: [Xen-devel] [PATCH 1/2] x86/mm: factor out the code for shattering an l3 PTE

2019-12-10 Thread Xia, Hongyan
On Mon, 2019-12-09 at 14:22 +0100, Jan Beulich wrote: > On 07.12.2019 19:20, Xia, Hongyan wrote: > > On Fri, 2019-12-06 at 17:50 +, Andrew Cooper wrote: > > > On 06/12/2019 15:53, Hongyan Xia wrote: > > > > +/* Shatter an l3 entry and populate l2. If virt is pass

Re: [Xen-devel] [PATCH v2 1/2] x86/mm: factor out the code for shattering an l3 PTE

2019-12-11 Thread Xia, Hongyan
On Tue, 2019-12-10 at 16:20 +0100, Jan Beulich wrote: > > ol2e = l2e_from_intpte( >l2e_get_intpte(ol2e) + (PAGE_SIZE << > PAGETABLE_ORDER)); > > Of course, as mentioned before, I'm not overly happy to see type > safety lost in case like this one, where it's not needed

Re: [Xen-devel] [PATCH v2 1/2] x86/mm: factor out the code for shattering an l3 PTE

2019-12-11 Thread Xia, Hongyan
On Tue, 2019-12-10 at 16:20 +0100, Jan Beulich wrote: > On 09.12.2019 12:48, Hongyan Xia wrote: > > --- a/xen/arch/x86/mm.c > > +++ b/xen/arch/x86/mm.c > > @@ -5151,6 +5151,51 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long > > v) > > flush_area_local((const void *)v, f) :

Re: [Xen-devel] [PATCH v2 1/2] x86/mm: factor out the code for shattering an l3 PTE

2019-12-11 Thread Xia, Hongyan
On Wed, 2019-12-11 at 11:10 +, Julien Grall wrote: > Hi Hongyan, > ... > > While this involves more instructions, how often do we expect the > code > to be called? > > Cheers, > I don't expect this to be called very often in the current Xen. Although with direct map removal, a lot of the m

Re: [Xen-devel] [PATCH v3 1/2] x86/mm: factor out the code for shattering an l3 PTE

2019-12-11 Thread Xia, Hongyan
On Wed, 2019-12-11 at 16:32 +0100, Jan Beulich wrote: > On 11.12.2019 11:58, Hongyan Xia wrote: > > @@ -5578,27 +5597,8 @@ int modify_xen_mappings(unsigned long s, > > unsigned long e, unsigned int nf) > > } > > > > /* PAGE1GB: shatter the superpage and fall through. */

Re: [Xen-devel] [PATCH v3 1/2] x86/mm: factor out the code for shattering an l3 PTE

2019-12-11 Thread Xia, Hongyan
On Wed, 2019-12-11 at 16:29 +0100, Jan Beulich wrote: > > +} > > + > > +if ( locking ) > > +spin_lock(&map_pgdir_lock); > > +if ( (l3e_get_flags(*pl3e) & _PAGE_PRESENT) && > > + (l3e_get_flags(*pl3e) & _PAGE_PSE) ) > > +{ > > +l3e_write_atomic(pl3e, > > +

Re: [Xen-devel] [PATCH v4 1/2] x86/mm: factor out the code for shattering an l3 PTE

2019-12-13 Thread Xia, Hongyan
Hi, On Fri, 2019-12-13 at 14:19 +, Andrew Cooper wrote: > On 12/12/2019 12:46, Hongyan Xia wrote: > > diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c > > index 7d4dd80a85..8def4fb8d8 100644 > > --- a/xen/arch/x86/mm.c > > +++ b/xen/arch/x86/mm.c > > @@ -5151,6 +5151,52 @@ l1_pgentry_t *virt

Re: [Xen-devel] [PATCH v4 1/2] x86/mm: factor out the code for shattering an l3 PTE

2019-12-13 Thread Xia, Hongyan
On Fri, 2019-12-13 at 14:58 +, Andrew Cooper wrote: > On 13/12/2019 14:36, Jan Beulich wrote: > > On 13.12.2019 15:19, Andrew Cooper wrote: > > > On 12/12/2019 12:46, Hongyan Xia wrote: > > > > diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c > > > > index 7d4dd80a85..8def4fb8d8 100644 > > >

Re: [Xen-devel] [[PATCH for-4.13]] xen/arm: mm: Allow generic xen page-tables helpers to be called early

2019-10-11 Thread Xia, Hongyan
Not commenting on the patch, but I had exactly the same problem when removing the direct map in x86. map_domain_page has to be usable without the direct map and even before alloc_boot_pages can be used (so that I can map the bootmem_regions_list, although I have made it statically allocated now).

Re: [Xen-devel] [[PATCH for-4.13]] xen/arm: mm: Allow generic xen page-tables helpers to be called early

2019-10-16 Thread Xia, Hongyan
yan, > > On 11/10/2019 10:53, Xia, Hongyan wrote: > > Not commenting on the patch, but I had exactly the same problem > > when > > removing the direct map in x86. map_domain_page has to be usable > > without the direct map and even before alloc_boot_pages can be us

Re: [Xen-devel] [PATCH v4 3/3] xen/efi: use directmap to access runtime services table

2019-10-24 Thread Xia, Hongyan
It is certainly nice to have less users of the direct map. My non-EFI builds already work without the direct map now but once I start testing EFI, it is nice to have one less thing to worry about. How important and performance-critical is this? If we really want to avoid switching the page table,

Re: Memory ordering question in the shutdown deferral code

2020-09-21 Thread Xia, Hongyan
ge Dunlap > > ; Durrant, Paul > > > > Cc: Xia, Hongyan ; > > xen-devel@lists.xenproject.org > > Subject: RE: [EXTERNAL] Memory ordering question in the shutdown > > deferral code > > > > CAUTION: This email originated from outside of the organization.

Re: [Xen-devel] [PATCH v2] x86/domain_page: implement pure per-vCPU mapping infrastructure

2020-02-21 Thread Xia, Hongyan
On Fri, 2020-02-21 at 11:50 +, Wei Liu wrote: > On Thu, Feb 06, 2020 at 06:58:23PM +, Hongyan Xia wrote: > > ... > > > > Benchmarks > > (baseline uses direct map instead of the mapcache in > > map_domain_page, > > old is the existing mapcache and new is after this patch): > > > > Geekbenc

Re: [Xen-devel] [PATCH v2] x86/domain_page: implement pure per-vCPU mapping infrastructure

2020-02-21 Thread Xia, Hongyan
On Fri, 2020-02-21 at 13:02 +, Andrew Cooper wrote: > On 21/02/2020 12:52, Xia, Hongyan wrote: > > On Fri, 2020-02-21 at 11:50 +, Wei Liu wrote: > > > Given that: > > > > > > 1. mapcache_domain is now a structure with only one member. > > &

Re: [Xen-devel] [PATCH v2] x86/domain_page: implement pure per-vCPU mapping infrastructure

2020-02-21 Thread Xia, Hongyan
On Fri, 2020-02-21 at 14:31 +0100, Jan Beulich wrote: > On 21.02.2020 13:52, Xia, Hongyan wrote: > > On Fri, 2020-02-21 at 11:50 +, Wei Liu wrote: > > > On Thu, Feb 06, 2020 at 06:58:23PM +, Hongyan Xia wrote: > > > > +if ( hashmfn

Re: [Xen-devel] [PATCH v2] x86/mm: switch to new APIs in arch_init_memory

2020-02-27 Thread Xia, Hongyan
On Thu, 2020-02-27 at 11:51 +, Julien Grall wrote: > Hi Hongyan, > > On 27/02/2020 10:27, Hongyan Xia wrote: > > ... > > diff --git a/xen/include/xen/domain_page.h > > b/xen/include/xen/domain_page.h > > index 32669a3339..bfc3bf6aeb 100644 > > --- a/xen/include/xen/domain_page.h > > +++ b/xen/

Re: [Xen-devel] [PATCH v3 1/6] domain: introduce alloc/free_shared_info() helpers...

2020-03-05 Thread Xia, Hongyan
ng to say that free_xenheap_page() can deal with NULL so we do not have to return on NULL. But then I found that the final patch needs to unmap it which cannot deal with NULL anyway, so I no longer have a strong opinion to do clean-ups here. Other than that, Reviewed-by: Hongyan Xia Hongyan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3 1/6] domain: introduce alloc/free_shared_info() helpers...

2020-03-05 Thread Xia, Hongyan
On Thu, 2020-03-05 at 13:23 +, Hongyan Xia wrote: > On Thu, 2020-03-05 at 12:44 +, pdurr...@amzn.com wrote: > > ... > > Other than that, > Reviewed-by: Hongyan Xia Sorry, I meant hongy...@amazon.com ___ Xen-devel mailing list Xen-devel@lists.xe

Re: [Xen-devel] [PATCH v5 1/7] x86: move some xen mm function declarations

2020-01-07 Thread Xia, Hongyan
On Tue, 2020-01-07 at 14:09 +0100, Jan Beulich wrote: > ... > > Looks like I simply forgot every time I went through my list of > pending (for the various stages of processing) patches. I guess > patches 3 and 4 are also independent of patch 2 and hence could > go in as well. If so, looks like pa

Re: [Xen-devel] Xen 4.14 and future work

2020-01-09 Thread Xia, Hongyan
On Mon, 2019-12-02 at 19:51 +, Andrew Cooper wrote: > ... > > Other areas in need of work is the boot time directmap at 0 (which > hides > NULL pointer deferences during boot), and the correct handling of > %dr6 > for all kinds of guests. > Sorry for the late reply to this thread. Talking ab

Re: [Xen-devel] [PATCH v5 2/7] x86: introduce a new set of APIs to manage Xen page tables

2020-01-27 Thread Xia, Hongyan
On Thu, 2020-01-16 at 13:04 +0100, Jan Beulich wrote: > ... > > > +void free_xen_pagetable(void *v) > > +{ > > +mfn_t mfn = v ? virt_to_mfn(v) : INVALID_MFN; > > + > > +if ( system_state != SYS_STATE_early_boot ) > > +free_xen_pagetable_new(mfn); > > The condition is (partly) redu

Re: [Xen-devel] [PATCH v5 2/7] x86: introduce a new set of APIs to manage Xen page tables

2020-01-27 Thread Xia, Hongyan
On Mon, 2020-01-27 at 15:43 +0100, Jan Beulich wrote: > On 27.01.2020 15:33, Xia, Hongyan wrote: > > ... > > > > Reflecting back to your comment in v3 about whether the new Xen > > page > > table mapping APIs (map/unmap_xen_pagetable) are really necessary, >

Re: [Xen-devel] [PATCH 6/8] x86/setup: move vm_init() before end_boot_allocator()

2020-02-03 Thread Xia, Hongyan
On Sat, 2020-02-01 at 00:33 +, David Woodhouse wrote: > From: David Woodhouse > > We would like to be able to use vmap() to map the live update data, > and > we need to do a first pass of the live update data before we prime > the > heap because we need to know which pages need to be preserve

Re: [Xen-devel] [PATCH] x86/domain_page: implement pure per-vCPU mapping infrastructure

2020-02-04 Thread Xia, Hongyan
On Tue, 2020-02-04 at 14:00 +, Wei Liu wrote: > On Mon, Feb 03, 2020 at 06:36:53PM +, Hongyan Xia wrote: > [...] > > diff --git a/xen/arch/x86/domain_page.c > > b/xen/arch/x86/domain_page.c > > index dd32712d2f..52971d2ecc 100644 > > --- a/xen/arch/x86/domain_page.c > > +++ b/xen/arch/x86/d

Re: [Xen-devel] [PATCH] x86/domain_page: implement pure per-vCPU mapping infrastructure

2020-02-04 Thread Xia, Hongyan
On Tue, 2020-02-04 at 14:17 +, Andrew Cooper wrote: > On 03/02/2020 18:36, Hongyan Xia wrote: > > Rewrite the mapcache to be purely per-vCPU instead of partly per- > > vCPU > > and partly per-domain. > > > > This patch is needed to address performance issues when we start > > relying > > on th

Re: [Xen-devel] [PATCH] x86/domain_page: implement pure per-vCPU mapping infrastructure

2020-02-04 Thread Xia, Hongyan
On Tue, 2020-02-04 at 12:08 +, Wei Liu wrote: > Thanks, I welcome effort to make Xen more scalable. :-) > > On Mon, Feb 03, 2020 at 06:36:53PM +, Hongyan Xia wrote: > > Rewrite the mapcache to be purely per-vCPU instead of partly per- > > vCPU > > and partly per-domain. > > > > This patc

Re: [Xen-devel] [PATCH v6] x86: introduce a new set of APIs to manage Xen page tables

2020-02-05 Thread Xia, Hongyan
Ping. On Tue, 2020-01-28 at 13:50 +, Hongyan Xia wrote: > From: Wei Liu > > We are going to switch to using domheap page for page tables. > A new set of APIs is introduced to allocate and free pages of page > tables based on mfn instead of the xenheap direct map address. The > allocation and

Re: [Xen-devel] [PATCH 2/2] xen/mm: Introduce PG_state_uninitialised

2020-02-07 Thread Xia, Hongyan
On Fri, 2020-02-07 at 15:57 +, David Woodhouse wrote: > From: David Woodhouse > > ... > > Finally, make free_xenheap_pages() and free_domheap_pages() call > through to init_heap_pages() instead of directly to free_heap_pages() > in the case where pages are being free which have never passed

Re: [Xen-devel] [PATCH 2/2] xen/mm: Introduce PG_state_uninitialised

2020-02-07 Thread Xia, Hongyan
On Fri, 2020-02-07 at 16:32 +, David Woodhouse wrote: > > ... > > Ideally not because init_heap_pages() then calls free_heap_pages() > and the "recursion" is best avoided. Kind of depends on where we change its pg to initialised. If we do that in free_heap_pages this does recurse, but if it

Re: [PATCH RFC 1/2] docs/design: Add a design document for Live Update

2021-05-07 Thread Xia, Hongyan
On Fri, 2021-05-07 at 14:15 +0200, Jan Beulich wrote: > On 07.05.2021 13:44, Julien Grall wrote: [...] > > > > It is a known convenient place. It may be difficult to find a > > similar > > spot on host that have been long-running. > > I'm not convinced: If it was placed in the kexec area at a 2M

Re: [PATCH RFCv2 12/15] xen/arm: add Persistent Map (PMAP) infrastructure

2021-04-26 Thread Xia, Hongyan
On Sun, 2021-04-25 at 21:13 +0100, Julien Grall wrote: > From: Wei Liu > > The basic idea is like Persistent Kernel Map (PKMAP) in Linux. We > pre-populate all the relevant page tables before the system is fully > set up. > > We will need it on Arm in order to rework the arm64 version of > xenhe