Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-24 Thread Kai Huang
On 03/24/2015 04:14 PM, Jan Beulich wrote: On 24.03.15 at 09:06, wrote: On 03/24/2015 03:53 PM, Jan Beulich wrote: The command line option parsing is (and should be) independent of the chosen default anyway, i.e. overrides in either direction should be possible. While the parse_ept_param

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-24 Thread Jan Beulich
>>> On 24.03.15 at 09:06, wrote: > On 03/24/2015 03:53 PM, Jan Beulich wrote: >> The command line >> option parsing is (and should be) independent of the chosen >> default anyway, i.e. overrides in either direction should be >> possible. > While the parse_ept_param function does support "ept=pml

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-24 Thread Kai Huang
On 03/24/2015 03:53 PM, Jan Beulich wrote: On 24.03.15 at 07:42, wrote: On 02/17/2015 06:19 PM, Jan Beulich wrote: On 12.02.15 at 03:39, wrote: On 02/11/2015 07:52 PM, Andrew Cooper wrote: On 11/02/15 08:28, Kai Huang wrote: Design == - PML feature is used globally A new Xen boot p

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-24 Thread Jan Beulich
>>> On 24.03.15 at 07:42, wrote: > > On 02/17/2015 06:19 PM, Jan Beulich wrote: > On 12.02.15 at 03:39, wrote: >>> On 02/11/2015 07:52 PM, Andrew Cooper wrote: On 11/02/15 08:28, Kai Huang wrote: > Design > == > > - PML feature is used globally > > A new Xen

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-23 Thread Kai Huang
On 02/17/2015 06:19 PM, Jan Beulich wrote: On 12.02.15 at 03:39, wrote: On 02/11/2015 07:52 PM, Andrew Cooper wrote: On 11/02/15 08:28, Kai Huang wrote: Design == - PML feature is used globally A new Xen boot parameter, say 'opt_enable_pml', will be introduced to control PML feature d

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-13 Thread Kai Huang
On Thu, Mar 12, 2015 at 7:19 PM, Andrew Cooper wrote: > On 12/03/15 07:36, Kai Huang wrote: >> > It might also be nice to be able to enable or disable this feature > with a sysctl call; but that's just a nice-to-have. This feature should either be used or not. It is impractical to >>

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-12 Thread Andrew Cooper
On 12/03/15 07:36, Kai Huang wrote: > It might also be nice to be able to enable or disable this feature with a sysctl call; but that's just a nice-to-have. >>> This feature should either be used or not. It is impractical to >>> enable/disable at runtime. >>> >>> However, it absolutely w

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-12 Thread Kai Huang
On 03/11/2015 11:53 PM, George Dunlap wrote: On Wed, Mar 11, 2015 at 11:11 AM, Andrew Cooper wrote: On 11/03/15 10:59, George Dunlap wrote: On Tue, Feb 17, 2015 at 10:19 AM, Jan Beulich wrote: On 12.02.15 at 03:39, wrote: On 02/11/2015 07:52 PM, Andrew Cooper wrote: On 11/02/15 08:28, K

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-11 Thread George Dunlap
On Wed, Mar 11, 2015 at 11:11 AM, Andrew Cooper wrote: > On 11/03/15 10:59, George Dunlap wrote: >> On Tue, Feb 17, 2015 at 10:19 AM, Jan Beulich wrote: >> On 12.02.15 at 03:39, wrote: On 02/11/2015 07:52 PM, Andrew Cooper wrote: > On 11/02/15 08:28, Kai Huang wrote: >> Design >

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-11 Thread Andrew Cooper
On 11/03/15 10:59, George Dunlap wrote: > On Tue, Feb 17, 2015 at 10:19 AM, Jan Beulich wrote: > On 12.02.15 at 03:39, wrote: >>> On 02/11/2015 07:52 PM, Andrew Cooper wrote: On 11/02/15 08:28, Kai Huang wrote: > Design > == > > - PML feature is used globally > >>

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-11 Thread George Dunlap
On Tue, Feb 17, 2015 at 10:19 AM, Jan Beulich wrote: On 12.02.15 at 03:39, wrote: >> On 02/11/2015 07:52 PM, Andrew Cooper wrote: >>> On 11/02/15 08:28, Kai Huang wrote: Design == - PML feature is used globally A new Xen boot parameter, say 'opt_enable_pml',

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-03-01 Thread Kai Huang
On 02/17/2015 06:23 PM, Jan Beulich wrote: On 12.02.15 at 03:49, wrote: On 02/11/2015 09:06 PM, Jan Beulich wrote: On 11.02.15 at 09:28, wrote: - PML buffer flush There are two places we need to flush PML buffer. The first place is PML buffer full VMEXIT handler (apparently), and the secon

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-17 Thread Tim Deegan
At 10:19 + on 17 Feb (1424164775), Jan Beulich wrote: > >>> On 12.02.15 at 03:39, wrote: > > On 02/11/2015 07:52 PM, Andrew Cooper wrote: > >> On 11/02/15 08:28, Kai Huang wrote: > >>> Currently, PML will be used as long as there's guest memory in dirty > >>> logging mode, no matter globally o

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-17 Thread Jan Beulich
>>> On 13.02.15 at 03:28, wrote: >> From: Tim Deegan [mailto:t...@xen.org] >> Sent: Thursday, February 12, 2015 8:42 PM >> >> At 07:08 + on 12 Feb (1423721283), Tian, Kevin wrote: >> > for general log dirty, ept_invalidate_emt is required because there is >> > access permission change (dirti

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-17 Thread Jan Beulich
>>> On 16.02.15 at 12:44, wrote: > On 14/02/15 03:01, Kai Huang wrote: >> To me looks currently tracking vram (HAP) shatters all superpages, >> instead only superpages in vram range would be. Am I misunderstanding >> here? > > You are completely correct. > > Having just re-reviewed the HAP code,

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-17 Thread Jan Beulich
>>> On 12.02.15 at 03:49, wrote: > On 02/11/2015 09:06 PM, Jan Beulich wrote: > On 11.02.15 at 09:28, wrote: >>> - PML buffer flush >>> >>> There are two places we need to flush PML buffer. The first place is PML >>> buffer full VMEXIT handler (apparently), and the second place is in >>> pagi

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-17 Thread Jan Beulich
>>> On 12.02.15 at 03:39, wrote: > On 02/11/2015 07:52 PM, Andrew Cooper wrote: >> On 11/02/15 08:28, Kai Huang wrote: >>> Design >>> == >>> >>> - PML feature is used globally >>> >>> A new Xen boot parameter, say 'opt_enable_pml', will be introduced to >>> control PML feature detection, and P

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-16 Thread Tim Deegan
Hi, At 22:01 +0800 on 16 Feb (1424120474), Kai Huang wrote: > On Fri, Feb 13, 2015 at 10:50 AM, Kai Huang wrote: > > > > On 02/12/2015 08:34 PM, Tim Deegan wrote: > >> > >> Hi, > >> > >> Thanks for posting this design! > >> > >> At 16:28 +0800 on 11 Feb (1423668493), Kai Huang wrote: > >>> > >>>

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-16 Thread Kai Huang
On Mon, Feb 16, 2015 at 7:44 PM, Andrew Cooper wrote: > On 14/02/15 03:01, Kai Huang wrote: This will only function correctly if superpage shattering is used. As soon as a superpage D bit transitions from 0 to 1, the gfn is logged and the guest can make further updated in the s

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-16 Thread Kai Huang
On Fri, Feb 13, 2015 at 10:50 AM, Kai Huang wrote: > > On 02/12/2015 08:34 PM, Tim Deegan wrote: >> >> Hi, >> >> Thanks for posting this design! >> >> At 16:28 +0800 on 11 Feb (1423668493), Kai Huang wrote: >>> >>> Design >>> == >>> >>> - PML feature is used globally >>> >>> A new Xen boot par

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-16 Thread Andrew Cooper
On 14/02/15 03:01, Kai Huang wrote: >>> This will only function correctly if superpage shattering is used. >>> >>> As soon as a superpage D bit transitions from 0 to 1, the gfn is logged >>> and the guest can make further updated in the same frame without further >>> log entries being recorded. The

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-13 Thread Kai Huang
On Fri, Feb 13, 2015 at 11:52 PM, Kai Huang wrote: > On Fri, Feb 13, 2015 at 11:28 PM, Andrew Cooper > wrote: >> On 13/02/15 14:32, Kai Huang wrote: >>> On Fri, Feb 13, 2015 at 6:57 PM, Andrew Cooper >>> wrote: On 13/02/15 02:11, Kai Huang wrote: On 02/12/2015 10:10 PM, Andre

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-13 Thread Kai Huang
On Fri, Feb 13, 2015 at 11:28 PM, Andrew Cooper wrote: > On 13/02/15 14:32, Kai Huang wrote: >> On Fri, Feb 13, 2015 at 6:57 PM, Andrew Cooper >> wrote: >>> On 13/02/15 02:11, Kai Huang wrote: >>> >>> >>> On 02/12/2015 10:10 PM, Andrew Cooper wrote: >>> >>> On 12/02/15 06:54, Tian, Kevin wrote: >

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-13 Thread Andrew Cooper
On 13/02/15 14:32, Kai Huang wrote: > On Fri, Feb 13, 2015 at 6:57 PM, Andrew Cooper > wrote: >> On 13/02/15 02:11, Kai Huang wrote: >> >> >> On 02/12/2015 10:10 PM, Andrew Cooper wrote: >> >> On 12/02/15 06:54, Tian, Kevin wrote: >> >> which presumably >> means that the PML buffer flush needs to

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-13 Thread Kai Huang
On Fri, Feb 13, 2015 at 6:57 PM, Andrew Cooper wrote: > On 13/02/15 02:11, Kai Huang wrote: > > > On 02/12/2015 10:10 PM, Andrew Cooper wrote: > > On 12/02/15 06:54, Tian, Kevin wrote: > > which presumably > means that the PML buffer flush needs to be aware of which gfns are > mapped by superpages

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-13 Thread Andrew Cooper
On 13/02/15 02:11, Kai Huang wrote: > > On 02/12/2015 10:10 PM, Andrew Cooper wrote: >> On 12/02/15 06:54, Tian, Kevin wrote: > which presumably > means that the PML buffer flush needs to be aware of which gfns are > mapped by superpages to be able to correctly set a block of bits in th

[Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-13 Thread Bing
​>On 12/02/15 06:54, Tian, Kevin wrote: >> which presumably means that the PML buffer flush needs to be aware of which gfns are mapped by superpages to be able to correctly set a block of bits in the logdirty bitmap. >>> Unfortunately PML itself can't tell us if the logged

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-12 Thread Kai Huang
On 02/12/2015 08:34 PM, Tim Deegan wrote: Hi, Thanks for posting this design! At 16:28 +0800 on 11 Feb (1423668493), Kai Huang wrote: Design == - PML feature is used globally A new Xen boot parameter, say 'opt_enable_pml', will be introduced to control PML feature detection, and PML fe

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-12 Thread Tian, Kevin
> From: Tim Deegan [mailto:t...@xen.org] > Sent: Thursday, February 12, 2015 8:42 PM > > At 07:08 + on 12 Feb (1423721283), Tian, Kevin wrote: > > for general log dirty, ept_invalidate_emt is required because there is > > access permission change (dirtied page becomes rw after 1st fault, > > s

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-12 Thread Kai Huang
On 02/12/2015 08:42 PM, Tim Deegan wrote: At 07:08 + on 12 Feb (1423721283), Tian, Kevin wrote: for general log dirty, ept_invalidate_emt is required because there is access permission change (dirtied page becomes rw after 1st fault, so need to change them back to ro again for the new dirty

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-12 Thread Kai Huang
On 02/12/2015 10:10 PM, Andrew Cooper wrote: On 12/02/15 06:54, Tian, Kevin wrote: which presumably means that the PML buffer flush needs to be aware of which gfns are mapped by superpages to be able to correctly set a block of bits in the logdirty bitmap. Unfortunately PML itself can't tell

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-12 Thread Andrew Cooper
On 12/02/15 06:54, Tian, Kevin wrote: > >>> which presumably >>> means that the PML buffer flush needs to be aware of which gfns are >>> mapped by superpages to be able to correctly set a block of bits in the >>> logdirty bitmap. >>> >> Unfortunately PML itself can't tell us if the logged GPA comes

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-12 Thread Tim Deegan
At 07:08 + on 12 Feb (1423721283), Tian, Kevin wrote: > for general log dirty, ept_invalidate_emt is required because there is > access permission change (dirtied page becomes rw after 1st fault, > so need to change them back to ro again for the new dirty tracking > round). But for PML, there'

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-12 Thread Tim Deegan
Hi, Thanks for posting this design! At 16:28 +0800 on 11 Feb (1423668493), Kai Huang wrote: > Design > == > > - PML feature is used globally > > A new Xen boot parameter, say 'opt_enable_pml', will be introduced to control > PML feature detection, and PML feature will only be detected if

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Kai Huang
On 02/12/2015 03:08 PM, Tian, Kevin wrote: From: Kai Huang [mailto:kai.hu...@linux.intel.com] Sent: Thursday, February 12, 2015 2:46 PM On 02/12/2015 02:25 PM, Tian, Kevin wrote: From: Kai Huang [mailto:kai.hu...@linux.intel.com] Sent: Thursday, February 12, 2015 10:35 AM On 02/11/2015 09:13

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Kai Huang
On 02/12/2015 03:09 PM, Tian, Kevin wrote: From: Kai Huang [mailto:kai.hu...@linux.intel.com] Sent: Thursday, February 12, 2015 2:57 PM On 02/12/2015 02:54 PM, Tian, Kevin wrote: From: Kai Huang [mailto:kai.hu...@linux.intel.com] Sent: Thursday, February 12, 2015 10:39 AM PML needs to be enab

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Kai Huang
On 02/12/2015 03:02 PM, Tian, Kevin wrote: From: Kai Huang [mailto:kai.hu...@linux.intel.com] Sent: Thursday, February 12, 2015 10:50 AM - PML buffer flush There are two places we need to flush PML buffer. The first place is PML buffer full VMEXIT handler (apparently), and the second place is

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Tian, Kevin
> From: Kai Huang [mailto:kai.hu...@linux.intel.com] > Sent: Thursday, February 12, 2015 2:57 PM > > On 02/12/2015 02:54 PM, Tian, Kevin wrote: > >> From: Kai Huang [mailto:kai.hu...@linux.intel.com] > >> Sent: Thursday, February 12, 2015 10:39 AM > PML needs to be enabled (allocate PML buffe

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Tian, Kevin
> From: Kai Huang [mailto:kai.hu...@linux.intel.com] > Sent: Thursday, February 12, 2015 2:46 PM > > On 02/12/2015 02:25 PM, Tian, Kevin wrote: > >> From: Kai Huang [mailto:kai.hu...@linux.intel.com] > >> Sent: Thursday, February 12, 2015 10:35 AM > >> > >> On 02/11/2015 09:13 PM, Jan Beulich wrot

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Kai Huang
On 02/12/2015 02:54 PM, Tian, Kevin wrote: From: Kai Huang [mailto:kai.hu...@linux.intel.com] Sent: Thursday, February 12, 2015 10:39 AM PML needs to be enabled (allocate PML buffer, initialize PML index, PML base address, turn PML on VMCS, etc) for all vcpus of the domain, as PML buffer and PM

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Tian, Kevin
> From: Kai Huang [mailto:kai.hu...@linux.intel.com] > Sent: Thursday, February 12, 2015 10:50 AM > > >> - PML buffer flush > >> > >> There are two places we need to flush PML buffer. The first place is PML > >> buffer full VMEXIT handler (apparently), and the second place is in > >> paging_log_di

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Tian, Kevin
> From: Kai Huang [mailto:kai.hu...@linux.intel.com] > Sent: Thursday, February 12, 2015 10:39 AM > > > >> PML needs to be enabled (allocate PML buffer, initialize PML index, > >> PML base address, turn PML on VMCS, etc) for all vcpus of the domain, > >> as PML buffer and PML index are per-vcpu, bu

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Kai Huang
On 02/12/2015 02:25 PM, Tian, Kevin wrote: From: Kai Huang [mailto:kai.hu...@linux.intel.com] Sent: Thursday, February 12, 2015 10:35 AM On 02/11/2015 09:13 PM, Jan Beulich wrote: On 11.02.15 at 12:52, wrote: On 11/02/15 08:28, Kai Huang wrote: With PML, we don't have to use write protectio

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Tian, Kevin
> From: Kai Huang [mailto:kai.hu...@linux.intel.com] > Sent: Thursday, February 12, 2015 10:35 AM > > On 02/11/2015 09:13 PM, Jan Beulich wrote: > On 11.02.15 at 12:52, wrote: > >> On 11/02/15 08:28, Kai Huang wrote: > >>> With PML, we don't have to use write protection but just clear D-bit

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Kai Huang
On 02/12/2015 10:49 AM, Kai Huang wrote: On 02/11/2015 09:06 PM, Jan Beulich wrote: On 11.02.15 at 09:28, wrote: - PML enable/disable for particular Domain PML needs to be enabled (allocate PML buffer, initialize PML index, PML base address, turn PML on VMCS, etc) for all vcpus of the doma

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Kai Huang
On 02/11/2015 09:06 PM, Jan Beulich wrote: On 11.02.15 at 09:28, wrote: - PML enable/disable for particular Domain PML needs to be enabled (allocate PML buffer, initialize PML index, PML base address, turn PML on VMCS, etc) for all vcpus of the domain, as PML buffer and PML index are per-vcpu

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Kai Huang
On 02/11/2015 07:52 PM, Andrew Cooper wrote: On 11/02/15 08:28, Kai Huang wrote: Hi all, PML (Page Modification Logging) is a new feature on Intel's Boardwell server platfrom targeted to reduce overhead of dirty logging mechanism. Below is the design for Xen. Would you help to review and give

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Kai Huang
On 02/11/2015 09:13 PM, Jan Beulich wrote: On 11.02.15 at 12:52, wrote: On 11/02/15 08:28, Kai Huang wrote: With PML, we don't have to use write protection but just clear D-bit of EPT entry of guest memory to do dirty logging, with an additional PML buffer full VMEXIT for 512 dirty GPAs. Theo

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Jan Beulich
>>> On 11.02.15 at 17:33, wrote: > On 11/02/15 13:13, Jan Beulich wrote: > On 11.02.15 at 12:52, wrote: >>> On 11/02/15 08:28, Kai Huang wrote: We handle above two cases by flushing PML buffer at the beginning of all VMEXITs. This solves the first case above, and it also solves the

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Andrew Cooper
On 11/02/15 13:13, Jan Beulich wrote: On 11.02.15 at 12:52, wrote: >> On 11/02/15 08:28, Kai Huang wrote: >>> With PML, we don't have to use write protection but just clear D-bit >>> of EPT entry of guest memory to do dirty logging, with an additional >>> PML buffer full VMEXIT for 512 dirty

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Jan Beulich
>>> On 11.02.15 at 12:52, wrote: > On 11/02/15 08:28, Kai Huang wrote: >> With PML, we don't have to use write protection but just clear D-bit >> of EPT entry of guest memory to do dirty logging, with an additional >> PML buffer full VMEXIT for 512 dirty GPAs. Theoretically, this can >> reduce hyp

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Jan Beulich
>>> On 11.02.15 at 09:28, wrote: > - PML enable/disable for particular Domain > > PML needs to be enabled (allocate PML buffer, initialize PML index, PML base > address, turn PML on VMCS, etc) for all vcpus of the domain, as PML buffer > and PML index are per-vcpu, but EPT table may be shared b

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Andrew Cooper
On 11/02/15 08:28, Kai Huang wrote: > Hi all, > > PML (Page Modification Logging) is a new feature on Intel's Boardwell > server platfrom targeted to reduce overhead of dirty logging > mechanism. Below is the design for Xen. Would you help to review and > give comments? Thankyou for this design.

[Xen-devel] PML (Page Modification Logging) design for Xen

2015-02-11 Thread Kai Huang
Hi all, PML (Page Modification Logging) is a new feature on Intel's Boardwell server platfrom targeted to reduce overhead of dirty logging mechanism. Below is the design for Xen. Would you help to review and give comments? Background == Currently, dirty logging is done via write prote