Re: [RFC PATCH v2 2/2] KVM: x86: Not wr-protect huge page with init_all_set dirty log

On Tue, Apr 20, 2021 at 12:49 AM Keqian Zhu wrote: > > Hi Ben, > > On 2021/4/20 3:20, Ben Gardon wrote: > > On Fri, Apr 16, 2021 at 1:25 AM Keqian Zhu wrote: > >> > >> Currently during start dirty logging, if we're with init-all-set, > >>

Re: [PATCH 09/15] KVM: selftests: Move per-VM GPA into perf_test_args

On Thu, Feb 11, 2021 at 7:58 AM Sean Christopherson wrote: > > On Thu, Feb 11, 2021, Paolo Bonzini wrote: > > On 11/02/21 02:56, Sean Christopherson wrote: > > > > > + pta->gpa = (vm_get_max_gfn(vm) - guest_num_pages) * > > > > > pta->guest_page_size; > > > > > + pta->gpa &= ~(pta->ho

Re: [PATCH] locking/arch: Move qrwlock.h include after qspinlock.h

ment to verify that. > > Cheers, > Longman > Thanks Longman and Guenter for developing and testing this fix! I don't have the environment to test this either, but the patch looks good to me. Reviewed-by: Ben Gardon

Re: [PATCH 02/15] KVM: selftests: Expose align() helpers to tests

; to ensure gpa alignment for hugepages. > > No functional change intended. > > Signed-off-by: Sean Christopherson Reviewed-by: Ben Gardon > --- > tools/testing/selftests/kvm/include/kvm_util.h | 15 +++ > tools/testing/selftests/kvm/lib/kvm_util.c | 1

Re: [PATCH 01/15] KVM: selftests: Explicitly state indicies for vm_guest_mode_params array

On Wed, Feb 10, 2021 at 3:06 PM Sean Christopherson wrote: > > Explicitly state the indices when populating vm_guest_mode_params to > make it marginally easier to visualize what's going on. > > No functional change intended. > > Signed-off-by: Sean Christopherson

Re: [PATCH 03/15] KVM: selftests: Align HVA for HugeTLB-backed memslots

On Wed, Feb 10, 2021 at 3:06 PM Sean Christopherson wrote: > > Align the HVA for HugeTLB memslots, not just THP memslots. Add an > assert so any future backing types are forced to assess whether or not > they need to be aligned. > > Cc: Ben Gardon > Cc: Yanan Wang >

Re: [PATCH 05/15] KVM: selftests: Require GPA to be aligned when backed by hugepages

> hugepages when dirty logging is activated is otherwise not exercised. > > Add a comment explaining that guest_page_size is for non-huge pages to > try and avoid confusion about what it actually tracks. > > Cc: Ben Gardon > Cc: Yanan Wang > Cc: Andrew Jones > Cc:

Re: [PATCH 06/15] KVM: selftests: Use shorthand local var to access struct perf_tests_args

On Wed, Feb 10, 2021 at 3:06 PM Sean Christopherson wrote: > > Use 'pta' as a local pointer to the global perf_tests_args in order to > shorten line lengths and make the code borderline readable. > > No functional change intended. > > Signed-off-by: Sean Christoph

Re: [PATCH 09/15] KVM: selftests: Move per-VM GPA into perf_test_args

> > No functional change intended. > > Signed-off-by: Sean Christopherson Reviewed-by: Ben Gardon > --- > .../selftests/kvm/include/perf_test_util.h| 8 +- > .../selftests/kvm/lib/perf_test_util.c| 28 --- > .../kvm/memslot_modificatio

Re: [PATCH 07/15] KVM: selftests: Capture per-vCPU GPA in perf_test_vcpu_args

On Wed, Feb 10, 2021 at 3:06 PM Sean Christopherson wrote: > > Capture the per-vCPU GPA in perf_test_vcpu_args so that tests can get > the GPA without having to calculate the GPA on their own. > > No functional change intended. > > Signed-off-by: Sean Christopherson Re

Re: [PATCH 08/15] KVM: selftests: Use perf util's per-vCPU GPA/pages in demand paging test

t clear > that the test's calculations are guaranteed to yield the same value as > perf_util, e.g. if guest_percpu_mem_size != vcpu_args->pages. > > Signed-off-by: Sean Christopherson Reviewed-by: Ben Gardon > --- > .../selftests/kvm/demand_paging_test.c| 20 +--

Re: [PATCH 10/15] KVM: selftests: Remove perf_test_args.host_page_size

, > e.g. that the logic is unchanged if hugepages are in play. > > No functional change intended. > > Signed-off-by: Sean Christopherson Reviewed-by: Ben Gardon > --- > tools/testing/selftests/kvm/demand_paging_test.c | 8 > tools/testing/selfte

Re: [PATCH 11/15] KVM: selftests: Create VM with adjusted number of guest pages for perf tests

rson Reviewed-by: Ben Gardon > --- > tools/testing/selftests/kvm/lib/perf_test_util.c | 9 ++--- > 1 file changed, 6 insertions(+), 3 deletions(-) > > diff --git a/tools/testing/selftests/kvm/lib/perf_test_util.c > b/tools/testing/selftests/kvm/lib/perf_test_util.c >

[PATCH v2 00/28] Allow parallel MMU operations with TDP MMU

the TDP MMU is disabled - Merged RCU commits - Extended additional MMU operations to operate in parallel - Ammended dirty log perf test to cover newly parallelized code paths - Misc refactorings (see changelogs for individual commits) - Big thanks to Sean and Paolo for their thorough review of

[PATCH v2 02/28] KVM: x86/mmu: Add comment on __tdp_mmu_set_spte

-by: Paolo Bonzini Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 16 1 file changed, 16 insertions(+) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index e9f9ff81a38e..3d8cca238eba 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu

[PATCH v2 03/28] KVM: x86/mmu: Add lockdep when setting a TDP MMU SPTE

Add lockdep to __tdp_mmu_set_spte to ensure that SPTEs are only modified under the MMU lock. No functional change intended. Reviewed-by: Peter Feiner Reviewed-by: Sean Christopherson Acked-by: Paolo Bonzini Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 2 ++ 1 file changed, 2

[PATCH v2 01/28] KVM: x86/mmu: change TDP MMU yield function returns to match cond_resched

semantics as cond_resched: true if the thread yielded, false if it did not. If the function yielded in the _flush_ version, then the TLBs will have been flushed. Reviewed-by: Peter Feiner Acked-by: Paolo Bonzini Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 39

[PATCH v2 04/28] KVM: x86/mmu: Don't redundantly clear TDP MMU pt memory

The KVM MMU caches already guarantee that shadow page table memory will be zeroed, so there is no reason to re-zero the page in the TDP MMU page fault handler. No functional change intended. Reviewed-by: Peter Feiner Reviewed-by: Sean Christopherson Acked-by: Paolo Bonzini Signed-off-by: Ben

[PATCH v2 06/28] locking/rwlocks: Add contention detection for rwlocks

Acked-by: Paolo Bonzini Signed-off-by: Ben Gardon --- include/asm-generic/qrwlock.h | 24 ++-- include/linux/rwlock.h| 7 +++ 2 files changed, 25 insertions(+), 6 deletions(-) diff --git a/include/asm-generic/qrwlock.h b/include/asm-generic/qrwlock.h index

[PATCH v2 07/28] sched: Add needbreak for rwlocks

Acked-by: Waiman Long Acked-by: Paolo Bonzini Signed-off-by: Ben Gardon --- include/linux/sched.h | 17 + 1 file changed, 17 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index 6e3a5eeec509..5d1378e5a040 100644 --- a/include/linux/sched.h +++ b/include

[PATCH v2 14/28] KVM: x86/mmu: Yield in TDU MMU iter even if no SPTES changed

yielding only happens after an SPTE is modified. Fix this issue by moving the yield to the beginning of the loop. Fixes: a6a0b05da9f3 ("kvm: x86/mmu: Support dirty logging for the TDP MMU") Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- v1 -> v2 - Split patch into three arc

[PATCH v2 12/28] KVM: x86/mmu: Rename goal_gfn to next_last_level_gfn

The goal_gfn field in tdp_iter can be misleading as it implies that it is the iterator's final goal. It is really a taget for the lowest gfn mapped by the leaf level SPTE the iterator will traverse towards. Change the field's name to be more precise. Signed-off-by: Ben Gardon --- ar

[PATCH v2 05/28] KVM: x86/mmu: Factor out handling of removed page tables

Signed-off-by: Ben Gardon --- v1 -> v2 - Replaced "disconnected" with "removed" updated derivative comments and code arch/x86/kvm/mmu/tdp_mmu.c | 71 ++ 1 file changed, 42 insertions(+), 29 deletions(-) diff --git a/arch/x86/kvm/mmu/t

[PATCH v2 15/28] KVM: x86/mmu: Skip no-op changes in TDP MMU functions

Skip setting SPTEs if no change is expected. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- v1 -> v2 - Merged no-op checks into exiting old_spte check arch/x86/kvm/mmu/tdp_mmu.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.

[PATCH v2 19/28] KVM: x86/mmu: Factor out functions to add/remove TDP MMU pages

Move the work of adding and removing TDP MMU pages to/from "secondary" data structures to helper functions. These functions will be built on in future commits to enable MMU operations to proceed (mostly) in parallel. No functional change expected. Signed-off-by: Ben Gardon --- ar

[PATCH v2 16/28] KVM: x86/mmu: Clear dirtied pages mask bit before early break

d-by: Sean Christopherson Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index aeb05f626b55..a75e92164a8b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x

[PATCH v2 22/28] KVM: x86/mmu: Mark SPTEs in disconnected pages as removed

already acquired a pointer to that memory and installs a mapping in an already cleared entry. This can lead to memory leaks and accounting errors. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 36 ++-- 1 file changed, 30

[PATCH v2 24/28] KVM: x86/mmu: Allow zap gfn range to operate under the mmu read lock

To reduce lock contention and interference with page fault handlers, allow the TDP MMU function to zap a GFN range to operate under the MMU read lock. Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c | 13 ++- arch/x86/kvm/mmu/mmu_internal.h | 6 +- arch/x86/kvm/mmu/tdp_mmu.c

[PATCH v2 26/28] KVM: x86/mmu: Allow enabling / disabling dirty logging under MMU read lock

To reduce lock contention and interference with page fault handlers, allow the TDP MMU functions which enable and disable dirty logging to operate under the MMU read lock. Extend dirty logging enable disable functions read lock-ness Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c

[PATCH v2 23/28] KVM: x86/mmu: Allow parallel page faults for the TDP MMU

Make the last few changes necessary to enable the TDP MMU to handle page faults in parallel while holding the mmu_lock in read mode. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a

[PATCH v2 25/28] KVM: x86/mmu: Allow zapping collapsible SPTEs to use MMU read lock

To speed the process of disabling dirty logging, change the TDP MMU function which zaps collapsible SPTEs to run under the MMU read lock. Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c | 5 ++--- arch/x86/kvm/mmu/tdp_mmu.c | 22 +++--- 2 files changed, 17 insertions

[PATCH v2 27/28] KVM: selftests: Add backing src parameter to dirty_log_perf_test

Add a parameter to control the backing memory type for dirty_log_perf_test so that the test can be run with hugepages. To: linux-kselft...@vger.kernel.org CC: Peter Xu CC: Andrew Jones CC: Thomas Huth Signed-off-by: Ben Gardon --- .../selftests/kvm/demand_paging_test.c| 3

[PATCH v2 28/28] KVM: selftests: Disable dirty logging with vCPUs running

: Andrew Jones CC: Thomas Huth Signed-off-by: Ben Gardon --- tools/testing/selftests/kvm/dirty_log_perf_test.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/kvm/dirty_log_perf_test.c b/tools/testing/selftests/kvm/dirty_log_perf_test.c

[PATCH v2 21/28] KVM: x86/mmu: Flush TLBs after zap in TDP MMU PF handler

are flushed properly before other threads are allowed to install any present value for the SPTE. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- v1 -> v2 - Renamed "FROZEN_SPTE" to "REMOVED_SPTE" and updated derivative comments and code arch/x86

[PATCH v2 20/28] KVM: x86/mmu: Use atomic ops to set SPTEs in TDP MMU map

To prepare for handling page faults in parallel, change the TDP MMU page fault handler to use atomic operations to set SPTEs so that changes are not lost if multiple threads attempt to modify the same SPTE. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- v1 -> v2 - Rename "ato

[PATCH v2 18/28] KVM: x86/mmu: Use an rwlock for the x86 MMU

Add a read / write lock to be used in place of the MMU spinlock on x86. The rwlock will enable the TDP MMU to handle page faults, and other operations in parallel in future commits. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- v1 -> v2 - Removed MMU lock wrappers - Complet

[PATCH v2 17/28] KVM: x86/mmu: Protect TDP MMU page table memory with RCU

MIO in the TDP MMU") Reviewed-by: Peter Feiner Suggested-by: Sean Christopherson Signed-off-by: Ben Gardon --- v1 -> v2 - Moved RCU read unlock before the TLB flush - Merged the RCU commits from v1 into a single commit - Changed the way accesses to page table memory are annotated wit

[PATCH v2 13/28] KVM: x86/mmu: Ensure forward progress when yielding in TDP MMU iter

quot;) Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- v1 -> v2 - Moved forward progress check into tdp_mmu_iter_cond_resched - Folded tdp_iter_refresh_walk into tdp_mmu_iter_cond_resched - Split patch into three and renamed all arch/x86/kvm/mmu/tdp_iter.c | 18 +-

[PATCH v2 11/28] KVM: x86/mmu: Merge flush and non-flush tdp_mmu_iter_cond_resched

The flushing and non-flushing variants of tdp_mmu_iter_cond_resched have almost identical implementations. Merge the two functions and add a flush parameter. Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 42 -- 1 file changed, 13 insertions

[PATCH v2 10/28] KVM: x86/mmu: Fix TDP MMU zap collapsible SPTEs

Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index c3075fb568eb..e3066d08c1dc 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c

[PATCH v2 09/28] KVM: x86/mmu: Fix braces in kvm_recover_nx_lpages

No functional change intended. Fixes: 29cf0f5007a2 ("kvm: x86/mmu: NX largepage recovery for TDP MMU") Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 6d

[PATCH v2 08/28] sched: Add cond_resched_rwlock

Signed-off-by: Ben Gardon --- include/linux/sched.h | 12 kernel/sched/core.c | 40 2 files changed, 52 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index 5d1378e5a040..3052d16da3cf 100644 --- a/include/linux

Re: [PATCH v2 23/28] KVM: x86/mmu: Allow parallel page faults for the TDP MMU

On Wed, Feb 3, 2021 at 4:40 AM Paolo Bonzini wrote: > > On 02/02/21 19:57, Ben Gardon wrote: > > > > - write_lock(&vcpu->kvm->mmu_lock); > > + > > + if (is_tdp_mmu_root(vcpu->kvm, vcpu->arch.mmu->root_hpa)) > > +

Re: [PATCH v2 24/28] KVM: x86/mmu: Allow zap gfn range to operate under the mmu read lock

On Wed, Feb 3, 2021 at 3:26 AM Paolo Bonzini wrote: > > On 02/02/21 19:57, Ben Gardon wrote: > > +#ifdef CONFIG_LOCKDEP > > + if (shared) > > + lockdep_assert_held_read(&kvm->mmu_lock); > > + else > > + lockdep_ass

Re: [PATCH v2 25/28] KVM: x86/mmu: Allow zapping collapsible SPTEs to use MMU read lock

On Wed, Feb 3, 2021 at 3:34 AM Paolo Bonzini wrote: > > On 02/02/21 19:57, Ben Gardon wrote: > > @@ -1485,7 +1489,9 @@ void kvm_tdp_mmu_zap_collapsible_sptes(struct kvm > > *kvm, > > struct kvm_mmu_page *root; > > int root_as_id; > > > >

[PATCH] KVM: VMX: Optimize flushing the PML buffer

vmx_flush_pml_buffer repeatedly calls kvm_vcpu_mark_page_dirty, which SRCU-derefrences kvm->memslots. In order to give the compiler more freedom to optimize the function, SRCU-dereference the pointer kvm->memslots only once. Reviewed-by: Makarand Sonare Signed-off-by: Ben Gardon --- Tes

Re: [PATCH] KVM: VMX: Optimize flushing the PML buffer

On Thu, Feb 4, 2021 at 2:51 PM Peter Xu wrote: > > Hi, Ben, > > On Thu, Feb 04, 2021 at 02:19:59PM -0800, Ben Gardon wrote: > > The average time for each run demonstrated a strange bimodal distribution, > > with clusters around 2 seconds and 2.5 seconds. This may have b

Re: [RFC PATCH 1/2] KVM: selftests: Add a macro to get string of vm_mem_backing_src_type

On Mon, Feb 8, 2021 at 1:08 AM Yanan Wang wrote: > > Add a macro to get string of the backing source memory type, so that > application can add choices for source types in the help() function, > and users can specify which type to use for testing. Coincidentally, I sent out a change last week to

Re: [RFC PATCH 2/2] KVM: selftests: Add a test for kvm page table code

On Mon, Feb 8, 2021 at 1:08 AM Yanan Wang wrote: > > This test serves as a performance tester and a bug reproducer for > kvm page table code (GPA->HPA mappings), so it gives guidance for > people trying to make some improvement for kvm. > > The function guest_code() is designed to cover conditions

Re: [RFC PATCH 1/2] KVM: selftests: Add a macro to get string of vm_mem_backing_src_type

On Tue, Feb 9, 2021 at 3:21 AM wangyanan (Y) wrote: > > > On 2021/2/9 2:13, Ben Gardon wrote: > > On Mon, Feb 8, 2021 at 1:08 AM Yanan Wang wrote: > >> Add a macro to get string of the backing source memory type, so that > >> application can add choices for so

Re: [RFC PATCH 2/2] KVM: selftests: Add a test for kvm page table code

On Mon, Feb 8, 2021 at 11:22 PM wangyanan (Y) wrote: > > Hi Ben, > > On 2021/2/9 4:29, Ben Gardon wrote: > > On Mon, Feb 8, 2021 at 1:08 AM Yanan Wang wrote: > >> This test serves as a performance tester and a bug reproducer for > >> kvm page table code (GPA-

Re: [RFC PATCH 2/2] KVM: selftests: Add a test for kvm page table code

On Tue, Feb 9, 2021 at 1:43 AM wangyanan (Y) wrote: > > > On 2021/2/9 4:29, Ben Gardon wrote: > > On Mon, Feb 8, 2021 at 1:08 AM Yanan Wang wrote: > >> This test serves as a performance tester and a bug reproducer for > >> kvm page table code (GPA->HPA

Re: [RFC PATCH v2 4/7] KVM: selftests: Add a helper to get system configured THP page size

e can also > read file /sys/kernel/mm/transparent_hugepage/hpage_pmd_size to get THP > hugepage size. > > Signed-off-by: Yanan Wang Reviewed-by: Ben Gardon > --- > .../testing/selftests/kvm/include/test_util.h | 2 ++ > tools/testing/selftests/kvm/lib/test_util.c | 36 +

Re: [RFC PATCH v2 5/7] KVM: selftests: List all hugetlb src types specified with page sizes

On Wed, Feb 24, 2021 at 10:03 PM Yanan Wang wrote: > > With VM_MEM_SRC_ANONYMOUS_HUGETLB, we currently can only use system > default hugetlb pages to back the testing guest memory. In order to > add flexibility, now list all the known hugetlb backing src types with > different page sizes, so that

Re: [RFC PATCH v2 6/7] KVM: selftests: Adapt vm_userspace_mem_region_add to new helpers

On Wed, Feb 24, 2021 at 10:03 PM Yanan Wang wrote: > > With VM_MEM_SRC_ANONYMOUS_THP specified in vm_userspace_mem_region_add(), > we have to get the transparent hugepage size for HVA alignment. With the > new helpers, we can use get_backing_src_pagesz() to check whether THP is > configured and th

Re: [RFC PATCH v2 0/7] Some improvement and a new test for kvm page table

On Wed, Feb 24, 2021 at 9:59 PM Yanan Wang wrote: > > Hi, > This v2 series can mainly include two parts. > Based on kvm queue branch: > https://git.kernel.org/pub/scm/virt/kvm/kvm.git/log/?h=queue > Links of v1: > https://lore.kernel.org/lkml/20210208090841.333724-1-wangyana...@huawei.com/ > > I

Re: [PATCH v2 0/2] KVM: x86/mmu: Zap orphaned kids for nested TDP MMU

his doesn't provide the advertised performance benefits. > > Ben Gardon (1): > KVM: x86/MMU: Recursively zap nested TDP SPs when zapping last/only > parent > > Sean Christopherson (1): > KVM: x86/mmu: Move flush logic from mmu_page_zap_pte() to > FN

Re: [PATCH 1/1] kvm: mmu: zap pages when zapping only parent

On Tue, Aug 4, 2020 at 2:14 PM Sean Christopherson wrote: > > On Mon, Jul 27, 2020 at 01:33:24PM -0700, Ben Gardon wrote: > > When the KVM MMU zaps a page, it will recursively zap the unsynced child > > pages, but not the synced ones. This can create problems over time whe

Re: [RFC 0/9] KVM:x86/mmu:Introduce parallel memory virtualization to boost performance

On Wed, Aug 5, 2020 at 9:53 AM Yulei Zhang wrote: > > From: Yulei Zhang > > Currently in KVM memory virtulization we relay on mmu_lock to synchronize > the memory mapping update, which make vCPUs work in serialize mode and > slow down the execution, especially after migration to do substantial >

[PATCH 00/24] Allow parallel page faults with TDP MMU

+ no TDP MMU works DBX + TDP MMU works Cross-compilation was also checked for PowerPC and ARM64. This series can be viewed in Gerrit at: https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux/+/7172 Ben Gardon (24): locking/rwlocks: Add contention detection for rwlocks sched:

[PATCH 02/24] sched: Add needbreak for rwlocks

Acked-by: Waiman Long Acked-by: Paolo Bonzini Signed-off-by: Ben Gardon --- include/linux/sched.h | 17 + 1 file changed, 17 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index 6e3a5eeec509..5d1378e5a040 100644 --- a/include/linux/sched.h +++ b/include

[PATCH 05/24] kvm: x86/mmu: Fix yielding in TDP MMU

only yielding if the loop has made forward progress since the last yield. Fixes: a6a0b05da9f3 ("kvm: x86/mmu: Support dirty logging for the TDP MMU") Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 83 +++--- 1 f

[PATCH 07/24] kvm: x86/mmu: Add comment on __tdp_mmu_set_spte

Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 16 1 file changed, 16 insertions(+) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 2650fa9fe066..b033da8243fc 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -357,6

[PATCH 09/24] kvm: x86/mmu: Don't redundantly clear TDP MMU pt memory

The KVM MMU caches already guarantee that shadow page table memory will be zeroed, so there is no reason to re-zero the page in the TDP MMU page fault handler. No functional change intended. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 1 - 1 file

[PATCH 04/24] kvm: x86/mmu: change TDP MMU yield function returns to match cond_resched

semantics as cond_resched: true if the thread yielded, false if it did not. If the function yielded in the _flush_ version, then the TLBs will have been flushed. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 38 +- 1 file

[PATCH 08/24] kvm: x86/mmu: Add lockdep when setting a TDP MMU SPTE

Add lockdep to __tdp_mmu_set_spte to ensure that SPTEs are only modified under the MMU lock. This lockdep will be updated in future commits to reflect and validate changes to the TDP MMU's synchronization strategy. No functional change intended. Reviewed-by: Peter Feiner Signed-off-by

[PATCH 14/24] kvm: mmu: Wrap mmu_lock lock / unlock in a function

other archs. No functional change intended. Signed-off-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/arm64/kvm/mmu.c | 36 ++--- arch/mips/kvm/mips.c | 8 +-- arch/mips/kvm/mmu.c| 14 ++--- arch/powerpc/kvm

[PATCH 17/24] kvm: mmu: Move mmu_lock to struct kvm_arch

Move the mmu_lock to struct kvm_arch so that it can be replaced with a rwlock on x86 without affecting the performance of other archs. No functional change intended. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- Documentation/virt/kvm/locking.rst | 2 +- arch/arm64/include/asm

[PATCH 18/24] kvm: x86/mmu: Use an rwlock for the x86 TDP MMU

lock is still used as locking and unlocking a spin lock is slightly faster. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/include/asm/kvm_host.h | 8 ++- arch/x86/kvm/mmu/mmu.c | 89 + arch/x86/kvm/mmu/mmu_internal.h | 9 arch

[PATCH 10/24] kvm: x86/mmu: Factor out handle disconnected pt

Factor out the code to handle a disconnected subtree of the TDP paging structure from the code to handle the change to an individual SPTE. Future commits will build on this to allow asynchronous page freeing. No functional change intended. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon

[PATCH 22/24] kvm: x86/mmu: Flush TLBs after zap in TDP MMU PF handler

are flushed properly before other threads are allowed to install any present value for the SPTE. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/spte.h| 16 +- arch/x86/kvm/mmu/tdp_mmu.c | 62 -- 2 files changed, 68

[PATCH 01/24] locking/rwlocks: Add contention detection for rwlocks

Acked-by: Paolo Bonzini Signed-off-by: Ben Gardon --- include/asm-generic/qrwlock.h | 24 ++-- include/linux/rwlock.h| 7 +++ 2 files changed, 25 insertions(+), 6 deletions(-) diff --git a/include/asm-generic/qrwlock.h b/include/asm-generic/qrwlock.h index

[PATCH 19/24] kvm: x86/mmu: Protect tdp_mmu_pages with a lock

pages will cause some serialization, but only on non-leaf page table entries, so the lock is not expected to be very contended. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/include/asm/kvm_host.h | 15 arch/x86/kvm/mmu/tdp_mmu.c | 67

[PATCH 13/24] kvm: x86/mmu: Only free tdp_mmu pages after a grace period

By waiting until an RCU grace period has elapsed to free TDP MMU PT memory, the system can ensure that no kernel threads access the memory after it has been freed. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/mmu_internal.h | 3 +++ arch/x86/kvm/mmu/tdp_mmu.c

[PATCH 15/24] kvm: mmu: Wrap mmu_lock cond_resched and needbreak

Feiner Signed-off-by: Ben Gardon --- arch/arm64/kvm/mmu.c | 2 +- arch/x86/kvm/mmu/mmu.c | 16 arch/x86/kvm/mmu/tdp_mmu.c | 8 include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c| 10 ++ 5 files changed, 25 insertions(+), 13 deletions

[PATCH 24/24] kvm: x86/mmu: Allow parallel page faults for the TDP MMU

Make the last few changes necessary to enable the TDP MMU to handle page faults in parallel while holding the mmu_lock in read mode. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a

[PATCH 21/24] kvm: x86/mmu: Use atomic ops to set SPTEs in TDP MMU map

To prepare for handling page faults in parallel, change the TDP MMU page fault handler to use atomic operations to set SPTEs so that changes are not lost if multiple threads attempt to modify the same SPTE. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c

[PATCH 16/24] kvm: mmu: Wrap mmu_lock assertions

kvm_arch so that it can be replaced with an rwlock for x86. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/arm64/kvm/mmu.c | 2 +- arch/powerpc/include/asm/kvm_book3s_64.h | 7 +++ arch/powerpc/kvm/book3s_hv_nested.c | 3 +-- arch/x86/kvm/mmu

[PATCH 12/24] kvm: x86/kvm: RCU dereference tdp mmu page table links

In order to protect TDP MMU PT memory with RCU, ensure that page table links are properly rcu_derefenced. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_iter.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/tdp_iter.c b

[PATCH 23/24] kvm: x86/mmu: Freeze SPTEs in disconnected pages

already acquired a pointer to that memory and installs a mapping in an already cleared entry. This can lead to memory leaks and accounting errors. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 35 +-- 1 file changed, 29

[PATCH 20/24] kvm: x86/mmu: Add atomic option for setting SPTEs

In order to allow multiple TDP MMU operations to proceed in parallel, there must be an option to modify SPTEs atomically so that changes are not lost. Add that option to __tdp_mmu_set_spte and __handle_changed_spte. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu

[PATCH 11/24] kvm: x86/mmu: Put TDP MMU PT walks in RCU read-critical section

. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 53 -- 1 file changed, 51 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index e8f35cd46b4c..662907d374b3 100644 --- a/arch/x86

[PATCH 03/24] sched: Add cond_resched_rwlock

Signed-off-by: Ben Gardon --- include/linux/sched.h | 12 kernel/sched/core.c | 40 2 files changed, 52 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index 5d1378e5a040..3052d16da3cf 100644 --- a/include/linux

[PATCH 06/24] kvm: x86/mmu: Skip no-op changes in TDP MMU functions

Skip setting SPTEs if no change is expected. Reviewed-by: Peter Feiner Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 1987da0da66e..2650fa9fe066 100644 --- a/arch

[PATCH 2/6] KVM: selftests: Avoid flooding debug log while populating memory

-by: Ben Gardon --- tools/testing/selftests/kvm/dirty_log_perf_test.c | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/kvm/dirty_log_perf_test.c b/tools/testing/selftests/kvm/dirty_log_perf_test.c index 16efe6589b43..15a9c45bdb5f 100644 --- a

[PATCH 5/6] KVM: selftests: Add option to overlap vCPU memory access

kernel. Reviewed-by: Jacob Xu Reviewed-by: Makarand Sonare Signed-off-by: Ben Gardon --- .../selftests/kvm/demand_paging_test.c| 32 +++ .../selftests/kvm/dirty_log_perf_test.c | 14 ++-- .../selftests/kvm/include/perf_test_util.h| 4 ++- .../selftests/kvm

[PATCH 1/6] KVM: selftests: Rename timespec_diff_now to timespec_elapsed

In response to some earlier comments from Peter Xu, rename timespec_diff_now to the much more sensible timespec_elapsed. No functional change intended. Reviewed-by: Jacob Xu Reviewed-by: Makarand Sonare Signed-off-by: Ben Gardon --- tools/testing/selftests/kvm/demand_paging_test.c | 8

[PATCH 6/6] KVM: selftests: Add memslot modification stress test

. Adding and removing a memslot also has the effect of tearing down the entire paging structure, which leads to more page faults and pressure on the page fault handling path than a one-and-done memory population test. Reviewed-by: Jacob Xu Signed-off-by: Ben Gardon --- tools/testing/selftests/kvm

[PATCH 4/6] KVM: selftests: Fix population stage in dirty_log_perf_test

Currently the population stage in the dirty_log_perf_test does nothing as the per-vCPU iteration counters are not initialized and the loop does not wait for each vCPU. Remedy those errors. Reviewed-by: Jacob Xu Reviewed-by: Makarand Sonare Signed-off-by: Ben Gardon --- tools/testing

[PATCH 3/6] KVM: selftests: Convert iterations to int in dirty_log_perf_test

In order to add an iteration -1 to indicate that the memory population phase has not yet completed, convert the interations counters to ints. No functional change intended. Reviewed-by: Jacob Xu Signed-off-by: Ben Gardon --- .../selftests/kvm/dirty_log_perf_test.c | 26

[PATCH 0/6] KVM: selftests: Perf test cleanups and memslot modification test

ource.com/c/linux/kernel/git/torvalds/linux/+/7216 Ben Gardon (6): KVM: selftests: Rename timespec_diff_now to timespec_elapsed KVM: selftests: Avoid flooding debug log while populating memory KVM: selftests: Convert iterations to int in dirty_log_perf_test KVM: selftests: Fix populati

Re: [PATCH v2 3/3] KVM: x86/mmu: Don't allow TDP MMU to yield when recovering NX pages

break its loop > before getting a chance to yield. > > But that is all very, very subtle, and will break at the slightest sneeze, > e.g. zapping while holding mmu_lock for read would break as the TDP MMU > wouldn't be guaranteed to see the present shadow page, and thus could step

Re: [PATCH v2 2/3] KVM: x86/mmu: Ensure TLBs are flushed for TDP MMU during NX zapping

to do the > flush. If pages are zapped from the TDP MMU but not the legacy MMU, then > no flush will occur. > > Fixes: 29cf0f5007a2 ("kvm: x86/mmu: NX largepage recovery for TDP MMU") > Cc: sta...@vger.kernel.org > Cc: Ben Gardon > Signed-off-by: Sean Christopherson

Re: [PATCH 1/2] KVM: x86/mmu: Ensure TLBs are flushed when yielding during GFN range zap

25b3 ("KVM: x86/mmu: Yield in TDU MMU iter even if no SPTES > changed") > Cc: sta...@vger.kernel.org > Cc: Ben Gardon > Signed-off-by: Sean Christopherson Reviewed-By: Ben Gardon > --- > arch/x86/kvm/mmu/tdp_mmu.c | 23 --- > 1 file changed,

Re: [PATCH 2/2] KVM: x86/mmu: Ensure TLBs are flushed when yielding during NX zapping

ond, tell the TDP MMU a flush is pending if the list of zapped pages > from legacy MMUs is not empty, i.e. the legacy MMU needs a flush. This > fixes the case where the TDP MMU yields, but it iteslf does not require a > flush. > > Fixes: 29cf0f5007a2 ("kvm: x86/mmu: NX lar

Re: [PATCH 2/2] KVM: x86/mmu: Ensure TLBs are flushed when yielding during NX zapping

On Mon, Mar 22, 2021 at 5:15 PM Sean Christopherson wrote: > > On Mon, Mar 22, 2021, Ben Gardon wrote: > > On Fri, Mar 19, 2021 at 4:20 PM Sean Christopherson > > wrote: > > > @@ -5960,19 +5963,21 @@ static void kvm_recove

Re: [PATCH 2/2] KVM: x86/mmu: Ensure TLBs are flushed when yielding during NX zapping

On Tue, Mar 23, 2021 at 11:58 AM Sean Christopherson wrote: > > On Tue, Mar 23, 2021, Ben Gardon wrote: > > On Mon, Mar 22, 2021 at 5:15 PM Sean Christopherson > > wrote: > > > > > > On Mon, Mar 22, 2021, Ben Gardon wrote: > > > > It could be fi

Re: [PATCH 00/18] KVM: Consolidate and optimize MMU notifiers

On Thu, Mar 25, 2021 at 7:20 PM Sean Christopherson wrote: > > The end goal of this series is to optimize the MMU notifiers to take > mmu_lock if and only if the notification is relevant to KVM, i.e. the hva > range overlaps a memslot. Large VMs (hundreds of vCPUs) are very > sensitive to mmu_lo

Re: [PATCH 03/15] KVM: x86/mmu: Ensure MMU pages are available when allocating roots

a > bug where KVM would fail to free valid PAE roots if a one of the later > roots failed to allocate. > > Note, KVM still leaks the PAE roots if the lm_root allocation fails. > This will be addressed in a future commit. > > Cc: Ben Gardon > Signed-off-by: Sean Christopher

Re: [PATCH 02/15] KVM: x86/mmu: Alloc page for PDPTEs when shadowing 32-bit NPT with 64-bit

tdp > enabled") > Fixes: b6b80c78af83 ("KVM: x86/mmu: Allocate PAE root array when using SVM's > 32-bit NPT") > Cc: sta...@vger.kernel.org > Signed-off-by: Sean Christopherson Reviewed-by: Ben Gardon > --- > arch/x86/kvm/mmu/mmu.c | 44 +

  1   2   3   4   >