Re: [PATCH v1 04/10] vfio/type1: Prepare is_invalid_reserved_pfn() for PG_reserved changes

2019-11-08 Thread David Hildenbrand
On 08.11.19 19:29, Dan Williams wrote: On Fri, Nov 8, 2019 at 2:22 AM David Hildenbrand wrote: On 08.11.19 08:14, David Hildenbrand wrote: On 08.11.19 06:09, Dan Williams wrote: On Thu, Nov 7, 2019 at 2:07 PM David Hildenbrand wrote: On 07.11.19 19:22, David Hildenbrand wrote: Am

Re: [PATCH v1 04/10] vfio/type1: Prepare is_invalid_reserved_pfn() for PG_reserved changes

2019-11-08 Thread David Hildenbrand
On 08.11.19 08:14, David Hildenbrand wrote: On 08.11.19 06:09, Dan Williams wrote: On Thu, Nov 7, 2019 at 2:07 PM David Hildenbrand wrote: On 07.11.19 19:22, David Hildenbrand wrote: Am 07.11.2019 um 16:40 schrieb Dan Williams : On Thu, Oct 24, 2019 at 5:12 AM David Hildenbrand wrote

Re: [PATCH v1 04/10] vfio/type1: Prepare is_invalid_reserved_pfn() for PG_reserved changes

2019-11-07 Thread David Hildenbrand
On 08.11.19 06:09, Dan Williams wrote: On Thu, Nov 7, 2019 at 2:07 PM David Hildenbrand wrote: On 07.11.19 19:22, David Hildenbrand wrote: Am 07.11.2019 um 16:40 schrieb Dan Williams : On Thu, Oct 24, 2019 at 5:12 AM David Hildenbrand wrote: Right now, ZONE_DEVICE memory is always

Re: [PATCH v1 04/10] vfio/type1: Prepare is_invalid_reserved_pfn() for PG_reserved changes

2019-11-07 Thread David Hildenbrand
On 07.11.19 19:22, David Hildenbrand wrote: Am 07.11.2019 um 16:40 schrieb Dan Williams : On Thu, Oct 24, 2019 at 5:12 AM David Hildenbrand wrote: Right now, ZONE_DEVICE memory is always set PG_reserved. We want to change that. KVM has this weird use case that you can map anything from

Re: [PATCH v1 04/10] vfio/type1: Prepare is_invalid_reserved_pfn() for PG_reserved changes

2019-11-07 Thread David Hildenbrand
> Am 07.11.2019 um 16:40 schrieb Dan Williams : > > On Thu, Oct 24, 2019 at 5:12 AM David Hildenbrand wrote: >> >> Right now, ZONE_DEVICE memory is always set PG_reserved. We want to >> change that. >> >> KVM has this weird use case that you can map a

Re: [PATCH v1 03/10] KVM: Prepare kvm_is_reserved_pfn() for PG_reserved changes

2019-11-05 Thread David Hildenbrand
, 2019 at 03:02:40PM -0800, Dan Williams wrote: On Tue, Nov 5, 2019 at 12:31 PM David Hildenbrand wrote: The scarier code (for me) is transparent_hugepage_adjust() and kvm_mmu_zap_collapsible_spte(), as I don't at all understand the interaction between THP and _PAGE_DEVMAP. The x86 KVM MMU co

Re: [PATCH v1 03/10] KVM: Prepare kvm_is_reserved_pfn() for PG_reserved changes

2019-11-05 Thread David Hildenbrand
I think I know what's going wrong: Pages that are pinned via gfn_to_pfn() and friends take a references, however are often released via kvm_release_pfn_clean()/kvm_release_pfn_dirty()/kvm_release_page_clean()... E.g., in arch/x86/kvm/x86.c:reexecute_instruction() ... pfn = gfn_to_pfn(vcpu->kvm

Re: [PATCH v1 02/10] KVM: x86/mmu: Prepare kvm_is_mmio_pfn() for PG_reserved changes

2019-11-05 Thread David Hildenbrand
On 05.11.19 02:37, Dan Williams wrote: On Thu, Oct 24, 2019 at 5:10 AM David Hildenbrand wrote: Right now, ZONE_DEVICE memory is always set PG_reserved. We want to change that. KVM has this weird use case that you can map anything from /dev/mem into the guest. pfn_valid() is not a reliable

Re: [PATCH v1 09/10] mm/memory_hotplug: Don't mark pages PG_reserved when initializing the memmap

2019-11-05 Thread David Hildenbrand
On 04.11.19 23:44, Boris Ostrovsky wrote: On 10/24/19 8:09 AM, David Hildenbrand wrote: diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c index 4f2e78a5e4db..af69f057913a 100644 --- a/drivers/xen/balloon.c +++ b/drivers/xen/balloon.c @@ -374,6 +374,13 @@ static void xen_online_page

Re: [PATCH v1 03/10] KVM: Prepare kvm_is_reserved_pfn() for PG_reserved changes

2019-11-05 Thread David Hildenbrand
On 05.11.19 10:49, David Hildenbrand wrote: On 05.11.19 10:17, David Hildenbrand wrote: On 05.11.19 05:38, Dan Williams wrote: On Thu, Oct 24, 2019 at 5:11 AM David Hildenbrand wrote: Right now, ZONE_DEVICE memory is always set PG_reserved. We want to change that. KVM has this weird use

Re: [PATCH v1 03/10] KVM: Prepare kvm_is_reserved_pfn() for PG_reserved changes

2019-11-05 Thread David Hildenbrand
On 05.11.19 10:17, David Hildenbrand wrote: On 05.11.19 05:38, Dan Williams wrote: On Thu, Oct 24, 2019 at 5:11 AM David Hildenbrand wrote: Right now, ZONE_DEVICE memory is always set PG_reserved. We want to change that. KVM has this weird use case that you can map anything from /dev/mem

Re: [PATCH v1 01/10] mm/memory_hotplug: Don't allow to online/offline memory blocks with holes

2019-11-05 Thread David Hildenbrand
On 05.11.19 02:30, Dan Williams wrote: On Thu, Oct 24, 2019 at 5:10 AM David Hildenbrand wrote: Our onlining/offlining code is unnecessarily complicated. Only memory blocks added during boot can have holes (a range that is not IORESOURCE_SYSTEM_RAM). Hotplugged memory never has holes (e.g

Re: [PATCH v1 03/10] KVM: Prepare kvm_is_reserved_pfn() for PG_reserved changes

2019-11-05 Thread David Hildenbrand
On 05.11.19 05:38, Dan Williams wrote: On Thu, Oct 24, 2019 at 5:11 AM David Hildenbrand wrote: Right now, ZONE_DEVICE memory is always set PG_reserved. We want to change that. KVM has this weird use case that you can map anything from /dev/mem into the guest. pfn_valid() is not a reliable

Re: [PATCH v1 00/10] mm: Don't mark hotplugged pages PG_reserved (including ZONE_DEVICE)

2019-11-01 Thread David Hildenbrand
On 24.10.19 14:09, David Hildenbrand wrote: This is the result of a recent discussion with Michal ([1], [2]). Right now we set all pages PG_reserved when initializing hotplugged memmaps. This includes ZONE_DEVICE memory. In case of system memory, PG_reserved is cleared again when onlining the

Re: [PATCH RFC v1 00/12] mm: Don't mark hotplugged pages PG_reserved (including ZONE_DEVICE)

2019-10-24 Thread David Hildenbrand
On 23.10.19 09:26, David Hildenbrand wrote: On 22.10.19 23:54, Dan Williams wrote: Hi David, Thanks for tackling this! Thanks for having a look :) [...] I am probably a little bit too careful (but I don't want to break things). In most places (besides KVM and vfio that are nuts)

[PATCH v1 08/10] x86/mm: Prepare __ioremap_check_ram() for PG_reserved changes

2019-10-24 Thread David Hildenbrand
: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Signed-off-by: David Hildenbrand --- arch/x86/mm/ioremap.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index a39dcdb5ae34..db6913b48edf 100644

[PATCH v1 07/10] powerpc/mm: Prepare maybe_pte_to_page() for PG_reserved changes

2019-10-24 Thread David Hildenbrand
Leroy Cc: "Aneesh Kumar K.V" Cc: Allison Randal Cc: Nicholas Piggin Cc: Thomas Gleixner Signed-off-by: David Hildenbrand --- arch/powerpc/mm/pgtable.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtab

[PATCH v1 10/10] mm/usercopy.c: Update comment in check_page_span() regarding ZONE_DEVICE

2019-10-24 Thread David Hildenbrand
Cc: Thomas Gleixner Signed-off-by: David Hildenbrand --- mm/usercopy.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/usercopy.c b/mm/usercopy.c index 660717a1ea5c..80f254024c97 100644 --- a/mm/usercopy.c +++ b/mm/usercopy.c @@ -199,9 +199,9 @@ static inline void check_

[PATCH v1 09/10] mm/memory_hotplug: Don't mark pages PG_reserved when initializing the memmap

2019-10-24 Thread David Hildenbrand
Weiner Cc: Anthony Yznaga Cc: Michal Hocko Cc: Oscar Salvador Cc: Dan Williams Cc: Mel Gorman Cc: Mike Rapoport Cc: Anshuman Khandual Cc: Matt Sickler Cc: Kees Cook Suggested-by: Michal Hocko Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c| 6 ++ drivers/xen/balloon.

[PATCH v1 06/10] powerpc/64s: Prepare hash_page_do_lazy_icache() for PG_reserved changes

2019-10-24 Thread David Hildenbrand
: "Aneesh Kumar K.V" Cc: Christophe Leroy Cc: Nicholas Piggin Cc: Andrew Morton Cc: Mike Rapoport Cc: YueHaibing Signed-off-by: David Hildenbrand --- arch/powerpc/mm/book3s64/hash_utils.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/m

[PATCH v1 04/10] vfio/type1: Prepare is_invalid_reserved_pfn() for PG_reserved changes

2019-10-24 Thread David Hildenbrand
have an initialized memmap (and don't have ZONE_DEVICE memory). Rewrite is_invalid_reserved_pfn() similar to kvm_is_reserved_pfn() to make sure the function produces the same result once we stop setting ZONE_DEVICE pages PG_reserved. Cc: Alex Williamson Cc: Cornelia Huck Signed-off-by:

[PATCH v1 05/10] powerpc/book3s: Prepare kvmppc_book3s_instantiate_page() for PG_reserved changes

2019-10-24 Thread David Hildenbrand
t Cc: Michael Ellerman Signed-off-by: David Hildenbrand --- arch/powerpc/kvm/book3s_64_mmu_radix.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 2d415c36a61d..05397c0561fc 100644

[PATCH v1 03/10] KVM: Prepare kvm_is_reserved_pfn() for PG_reserved changes

2019-10-24 Thread David Hildenbrand
arimAllah Ahmed Signed-off-by: David Hildenbrand --- virt/kvm/kvm_main.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index e9eb666eb6e8..9d18cc67d124 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -151,9 +151,1

[PATCH v1 00/10] mm: Don't mark hotplugged pages PG_reserved (including ZONE_DEVICE)

2019-10-24 Thread David Hildenbrand
: Dan Williams Cc: kvm-...@vger.kernel.org Cc: linuxppc-...@lists.ozlabs.org Cc: k...@vger.kernel.org Cc: linux-hyp...@vger.kernel.org Cc: de...@driverdev.osuosl.org Cc: xen-de...@lists.xenproject.org Cc: x...@kernel.org Cc: Alexander Duyck David Hildenbrand (10): mm/memory_hotplug: Don't allow to o

[PATCH v1 02/10] KVM: x86/mmu: Prepare kvm_is_mmio_pfn() for PG_reserved changes

2019-10-24 Thread David Hildenbrand
Wanpeng Li Cc: Jim Mattson Cc: Joerg Roedel Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: KarimAllah Ahmed Cc: Michal Hocko Cc: Dan Williams Signed-off-by: David Hildenbrand --- arch/x86/kvm/mmu.c | 29 + 1 file chan

[PATCH v1 01/10] mm/memory_hotplug: Don't allow to online/offline memory blocks with holes

2019-10-24 Thread David Hildenbrand
: Michal Hocko Cc: Oscar Salvador Cc: Pavel Tatashin Cc: Dan Williams Cc: Anshuman Khandual Signed-off-by: David Hildenbrand --- mm/memory_hotplug.c | 26 -- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index

Re: [PATCH RFC v1 01/12] mm/memory_hotplug: Don't allow to online/offline memory blocks with holes

2019-10-24 Thread David Hildenbrand
On 24.10.19 05:53, Anshuman Khandual wrote: On 10/22/2019 10:42 PM, David Hildenbrand wrote: Our onlining/offlining code is unnecessarily complicated. Only memory blocks added during boot can have holes. Hotplugged memory never has holes. That memory is already online. Why hot plugged memory

Re: [PATCH RFC v1 00/12] mm: Don't mark hotplugged pages PG_reserved (including ZONE_DEVICE)

2019-10-23 Thread David Hildenbrand
On 23.10.19 21:39, Dan Williams wrote: > On Wed, Oct 23, 2019 at 10:28 AM David Hildenbrand wrote: >> >>>> I dislike this for three reasons >>>> >>>> a) It does not protect against any races, really, it does not improve >>>&g

Re: [PATCH RFC v1 00/12] mm: Don't mark hotplugged pages PG_reserved (including ZONE_DEVICE)

2019-10-23 Thread David Hildenbrand
>> I dislike this for three reasons >> >> a) It does not protect against any races, really, it does not improve things. >> b) We do have the exact same problem with pfn_to_online_page(). As long as we >>don't hold the memory hotplug lock, memory can get offlined and remove >> any time. Racy. >

Re: [PATCH RFC v1 02/12] mm/usercopy.c: Prepare check_page_span() for PG_reserved changes

2019-10-23 Thread David Hildenbrand
On 23.10.19 18:25, Kees Cook wrote: > On Wed, Oct 23, 2019 at 10:20:14AM +0200, David Hildenbrand wrote: >> On 22.10.19 19:12, David Hildenbrand wrote: >>> Right now, ZONE_DEVICE memory is always set PG_reserved. We want to >>> change that. >>> >>> L

Re: [PATCH RFC v1 02/12] mm/usercopy.c: Prepare check_page_span() for PG_reserved changes

2019-10-23 Thread David Hildenbrand
On 22.10.19 19:12, David Hildenbrand wrote: Right now, ZONE_DEVICE memory is always set PG_reserved. We want to change that. Let's make sure that the logic in the function won't change. Once we no longer set these pages to reserved, we can rework this function to perform separate

Re: [PATCH RFC v1 06/12] staging/gasket: Prepare gasket_release_page() for PG_reserved changes

2019-10-23 Thread David Hildenbrand
On 22.10.19 19:12, David Hildenbrand wrote: Right now, ZONE_DEVICE memory is always set PG_reserved. We want to change that. The pages are obtained via get_user_pages_fast(). I assume, these could be ZONE_DEVICE pages. Let's just exclude them as well explicitly. Cc: Rob Springer Cc:

Re: [PATCH RFC v1 00/12] mm: Don't mark hotplugged pages PG_reserved (including ZONE_DEVICE)

2019-10-23 Thread David Hildenbrand
On 22.10.19 23:54, Dan Williams wrote: > Hi David, > > Thanks for tackling this! Thanks for having a look :) [...] >> I am probably a little bit too careful (but I don't want to break things). >> In most places (besides KVM and vfio that are nuts), the >> pfn_to_online_page() check could most

Re: [PATCH RFC v1 07/12] staging: kpc2000: Prepare transfer_complete_cb() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
On 22.10.19 19:55, Matt Sickler wrote: Right now, ZONE_DEVICE memory is always set PG_reserved. We want to change that. The pages are obtained via get_user_pages_fast(). I assume, these could be ZONE_DEVICE pages. Let's just exclude them as well explicitly. I'm not sure what ZONE_DEVICE pages

[PATCH RFC v1 10/12] powerpc/mm: Prepare maybe_pte_to_page() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: Christophe Leroy Cc: "Aneesh Kumar K.V" Cc: Allison Randal Cc: Nicholas Piggin Cc: Thomas Gleixner Signed-off-by: David Hildenbrand --- arch/powerpc/mm/pgtable.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletion

[PATCH RFC v1 09/12] powerpc/64s: Prepare hash_page_do_lazy_icache() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: "Aneesh Kumar K.V" Cc: Christophe Leroy Cc: Nicholas Piggin Cc: Andrew Morton Cc: Mike Rapoport Cc: YueHaibing Signed-off-by: David Hildenbrand --- arch/powerpc/mm/book3s64/hash_utils.c | 10 ++ 1 file changed, 6

[PATCH RFC v1 11/12] x86/mm: Prepare __ioremap_check_ram() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Signed-off-by: David Hildenbrand --- arch/x86/mm/ioremap.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/x86/mm/ioremap.c

[PATCH RFC v1 01/12] mm/memory_hotplug: Don't allow to online/offline memory blocks with holes

2019-10-22 Thread David Hildenbrand
-by: David Hildenbrand --- mm/memory_hotplug.c | 26 -- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 561371ead39a..7210f4375279 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1447,10 +1447,19

[PATCH RFC v1 06/12] staging/gasket: Prepare gasket_release_page() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
rtman Signed-off-by: David Hildenbrand --- drivers/staging/gasket/gasket_page_table.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/gasket/gasket_page_table.c b/drivers/staging/gasket/gasket_page_table.c index f6d715787da8..d43fed58bf65 100644 --- a/drivers/st

[PATCH RFC v1 08/12] powerpc/book3s: Prepare kvmppc_book3s_instantiate_page() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
-by: David Hildenbrand --- arch/powerpc/kvm/book3s_64_mmu_radix.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 2d415c36a61d..05397c0561fc 100644 --- a/arch/powerpc/kvm

[PATCH RFC v1 12/12] mm/memory_hotplug: Don't mark pages PG_reserved when initializing the memmap

2019-10-22 Thread David Hildenbrand
n Khandual Suggested-by: Michal Hocko Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c| 6 ++ drivers/xen/balloon.c | 7 +++ include/linux/page-flags.h | 8 +--- mm/memory_hotplug.c| 17 +++-- mm/page_alloc.c| 11 --

[PATCH RFC v1 07/12] staging: kpc2000: Prepare transfer_complete_cb() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
" Cc: Dan Carpenter Cc: Nishka Dasgupta Cc: Madhumitha Prabakaran Cc: Fabio Estevam Cc: Matt Sickler Cc: Jeremy Sowden Signed-off-by: David Hildenbrand --- drivers/staging/kpc2000/kpc_dma/fileops.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/staging/

[PATCH RFC v1 05/12] vfio/type1: Prepare is_invalid_reserved_pfn() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
accessed). Cc: Alex Williamson Cc: Cornelia Huck Signed-off-by: David Hildenbrand --- drivers/vfio/vfio_iommu_type1.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 2ada8e6cdb88

[PATCH RFC v1 04/12] KVM: Prepare kvm_is_reserved_pfn() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
accessed). Cc: Paolo Bonzini Cc: "Radim Krčmář" Cc: Michal Hocko Cc: Dan Williams Cc: KarimAllah Ahmed Signed-off-by: David Hildenbrand --- virt/kvm/kvm_main.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_ma

[PATCH RFC v1 03/12] KVM: x86/mmu: Prepare kvm_is_mmio_pfn() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
Cc: "H. Peter Anvin" Cc: KarimAllah Ahmed Cc: Michal Hocko Cc: Dan Williams Signed-off-by: David Hildenbrand --- arch/x86/kvm/mmu.c | 30 ++ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index 2

[PATCH RFC v1 02/12] mm/usercopy.c: Prepare check_page_span() for PG_reserved changes

2019-10-22 Thread David Hildenbrand
ecks). Cc: Kees Cook Cc: Andrew Morton Cc: Kate Stewart Cc: Allison Randal Cc: "Isaac J. Manjarres" Cc: Qian Cai Cc: Thomas Gleixner Signed-off-by: David Hildenbrand --- mm/usercopy.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/usercopy.c b

[PATCH RFC v1 00/12] mm: Don't mark hotplugged pages PG_reserved (including ZONE_DEVICE)

2019-10-22 Thread David Hildenbrand
tps://lkml.org/lkml/2019/10/21/736 [2] https://lkml.org/lkml/2019/10/21/1034 Cc: Michal Hocko Cc: Dan Williams Cc: kvm-...@vger.kernel.org Cc: linuxppc-...@lists.ozlabs.org Cc: k...@vger.kernel.org Cc: linux-hyp...@vger.kernel.org Cc: de...@driverdev.osuosl.org Cc: xen-de...@lists.xenpr

Re: [PATCH RFCv2 0/4] mm/memory_hotplug: Introduce memory block types

2019-03-27 Thread David Hildenbrand
On 20.12.18 14:08, Michal Hocko wrote: > On Thu 20-12-18 13:58:16, David Hildenbrand wrote: >> On 30.11.18 18:59, David Hildenbrand wrote: >>> This is the second approach, introducing more meaningful memory block >>> types and not changing online behavior in the kernel

Re: [PATCH v2 3/8] kexec: export PG_offline to VMCOREINFO

2019-03-11 Thread David Hildenbrand
On 11.03.19 10:04, Dave Young wrote: > Hi David, > On 11/22/18 at 11:06am, David Hildenbrand wrote: >> Right now, pages inflated as part of a balloon driver will be dumped >> by dump tools like makedumpfile. While XEN is able to check in the >> crash kernel whether a certa

Re: [PATCH v2] makedumpfile: exclude pages that are logically offline

2019-03-07 Thread David Hildenbrand
e kernel by >> dumping, when reading memory of a partially onlined memory segment >> (for memory added by the Hyper-V balloon driver). >> >> Therefore, don't read and dump pages that are marked as being logically >> offline. >> >> Signed-off-by: David Hilde

Re: [PATCH] hv_balloon: avoid touching uninitialized struct page during tail onlining

2019-01-08 Thread David Hildenbrand
On 07.01.19 14:44, Vitaly Kuznetsov wrote: > David Hildenbrand writes: > >> On 04.01.19 15:19, Vitaly Kuznetsov wrote: >>> Hyper-V memory hotplug protocol has 2M granularity and in Linux x86 we use >>> 128M. To deal with it we implement partial section onlining

Re: [PATCH] hv_balloon: avoid touching uninitialized struct page during tail onlining

2019-01-07 Thread David Hildenbrand
On 04.01.19 15:19, Vitaly Kuznetsov wrote: > Hyper-V memory hotplug protocol has 2M granularity and in Linux x86 we use > 128M. To deal with it we implement partial section onlining by registering > custom page onlining callback (hv_online_page()). Later, when more memory > arrives we try to online

Re: [PATCH RFCv2 0/4] mm/memory_hotplug: Introduce memory block types

2018-12-20 Thread David Hildenbrand
On 20.12.18 14:08, Michal Hocko wrote: > On Thu 20-12-18 13:58:16, David Hildenbrand wrote: >> On 30.11.18 18:59, David Hildenbrand wrote: >>> This is the second approach, introducing more meaningful memory block >>> types and not changing online behavior in the kernel

Re: [PATCH RFCv2 0/4] mm/memory_hotplug: Introduce memory block types

2018-12-20 Thread David Hildenbrand
On 30.11.18 18:59, David Hildenbrand wrote: > This is the second approach, introducing more meaningful memory block > types and not changing online behavior in the kernel. It is based on > latest linux-next. > > As we found out during dicussion, user space should always handl

Re: [PATCH RFCv2 3/4] mm/memory_hotplug: Introduce and use more memory types

2018-12-04 Thread David Hildenbrand
On 04.12.18 10:44, Michal Suchánek wrote: > On Fri, 30 Nov 2018 18:59:21 +0100 > David Hildenbrand wrote: > >> Let's introduce new types for different kinds of memory blocks and use >> them in existing code. As I don't see an easy way to split this up, >>

Re: [PATCH RFCv2 2/4] mm/memory_hotplug: Replace "bool want_memblock" by "int type"

2018-12-03 Thread David Hildenbrand
On 01.12.18 02:50, Wei Yang wrote: > On Fri, Nov 30, 2018 at 06:59:20PM +0100, David Hildenbrand wrote: >> Let's pass a memory block type instead. Pass "MEMORY_BLOCK_NONE" for device >> memory and for now "MEMORY_BLOCK_UNSPECIFIED" for anything else. No &g

Re: [PATCH RFCv2 1/4] mm/memory_hotplug: Introduce memory block types

2018-12-03 Thread David Hildenbrand
On 01.12.18 02:25, Wei Yang wrote: > On Fri, Nov 30, 2018 at 06:59:19PM +0100, David Hildenbrand wrote: >> Memory onlining should always be handled by user space, because only user >> space knows which use cases it wants to satisfy. E.g. memory might be >> onlined to the MO

[PATCH RFCv2 3/4] mm/memory_hotplug: Introduce and use more memory types

2018-11-30 Thread David Hildenbrand
Neuling Cc: Nathan Fontenot Cc: YueHaibing Cc: Vasily Gorbik Cc: Ingo Molnar Cc: Stephen Rothwell Cc: "mike.tra...@hpe.com" Cc: Oscar Salvador Cc: Joonsoo Kim Cc: Mathieu Malaterre Cc: Michal Hocko Cc: Arun KS Cc: Andrew Banman Cc: Dave Hansen Cc: Michal Suchánek Cc: V

[PATCH RFCv2 4/4] mm/memory_hotplug: Drop MEMORY_TYPE_UNSPECIFIED

2018-11-30 Thread David Hildenbrand
Cc: Michal Hocko Cc: Michal Suchánek Cc: Vitaly Kuznetsov Cc: Dan Williams Cc: Pavel Tatashin Signed-off-by: David Hildenbrand --- drivers/base/memory.c | 3 --- include/linux/memory.h | 5 - 2 files changed, 8 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.

[PATCH RFCv2 1/4] mm/memory_hotplug: Introduce memory block types

2018-11-30 Thread David Hildenbrand
hwidefsky Cc: Heiko Carstens Signed-off-by: David Hildenbrand --- drivers/base/memory.c | 38 +++--- include/linux/memory.h | 27 +++ 2 files changed, 62 insertions(+), 3 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memo

[PATCH RFCv2 2/4] mm/memory_hotplug: Replace "bool want_memblock" by "int type"

2018-11-30 Thread David Hildenbrand
c: Rob Herring Cc: Pavel Tatashin Cc: "mike.tra...@hpe.com" Cc: Joonsoo Kim Cc: Wei Yang Cc: Logan Gunthorpe Cc: "Jérôme Glisse" Cc: "Jan H. Schönherr" Cc: Dave Jiang Cc: Matthew Wilcox Cc: Mathieu Malaterre Signed-off-by: David Hildenbrand --- arch/ia6

[PATCH RFCv2 0/4] mm/memory_hotplug: Introduce memory block types

2018-11-30 Thread David Hildenbrand
e.g. online_type) we are using in that context. - The "removable" property should never have been named like that. It should have been "offlinable". Can we still rename that? E.g. boot memory is sometimes marked as removable ... David Hildenbrand (4): mm/memory_ho

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-11-27 Thread David Hildenbrand
On 27.11.18 17:32, Michal Suchánek wrote: > On Mon, 26 Nov 2018 16:59:14 +0100 > David Hildenbrand wrote: > >> On 26.11.18 15:20, Michal Suchánek wrote: >>> On Mon, 26 Nov 2018 14:33:29 +0100 >>> David Hildenbrand wrote: >>> >>>>

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-11-26 Thread David Hildenbrand
On 26.11.18 15:20, Michal Suchánek wrote: > On Mon, 26 Nov 2018 14:33:29 +0100 > David Hildenbrand wrote: > >> On 26.11.18 13:30, David Hildenbrand wrote: >>> On 23.11.18 19:06, Michal Suchánek wrote: > >>>> >>>> If we are going to fake the

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-11-26 Thread David Hildenbrand
On 26.11.18 13:30, David Hildenbrand wrote: > On 23.11.18 19:06, Michal Suchánek wrote: >> On Fri, 23 Nov 2018 12:13:58 +0100 >> David Hildenbrand wrote: >> >>> On 28.09.18 17:03, David Hildenbrand wrote: >>>> How to/when to online hotplugged memor

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-11-26 Thread David Hildenbrand
On 23.11.18 19:06, Michal Suchánek wrote: > On Fri, 23 Nov 2018 12:13:58 +0100 > David Hildenbrand wrote: > >> On 28.09.18 17:03, David Hildenbrand wrote: >>> How to/when to online hotplugged memory is hard to manage for >>> distributions because differe

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-11-23 Thread David Hildenbrand
On 28.09.18 17:03, David Hildenbrand wrote: > How to/when to online hotplugged memory is hard to manage for > distributions because different memory types are to be treated differently. > Right now, we need complicated udev rules that e.g. check if we are > running on s390x, on a phy

[PATCH v2] makedumpfile: exclude pages that are logically offline

2018-11-22 Thread David Hildenbrand
balloon driver). Therefore, don't read and dump pages that are marked as being logically offline. Signed-off-by: David Hildenbrand --- v1 -> v2: - Fix PAGE_BUDDY_MAPCOUNT_VALUE vs. PAGE_OFFLINE_MAPCOUNT_VALUE makedumpfile.c | 34 ++ makedumpfile.h | 1 +

[PATCH v2 6/8] vmw_balloon: mark inflated pages PG_offline

2018-11-22 Thread David Hildenbrand
ael S. Tsirkin" Acked-by: Nadav Amit Signed-off-by: David Hildenbrand --- drivers/misc/vmw_balloon.c | 32 1 file changed, 32 insertions(+) diff --git a/drivers/misc/vmw_balloon.c b/drivers/misc/vmw_balloon.c index e6126a4b95d3..877611b5659b 100644 --- a/dr

[PATCH v2 7/8] PM / Hibernate: use pfn_to_online_page()

2018-11-22 Thread David Hildenbrand
ot; Suggested-by: Michal Hocko Acked-by: Michal Hocko Acked-by: Pavel Machek Acked-by: Rafael J. Wysocki Signed-off-by: David Hildenbrand --- kernel/power/snapshot.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/kernel/power/snapshot.c b/kernel/power/snap

[PATCH v2 4/8] xen/balloon: mark inflated pages PG_offline

2018-11-22 Thread David Hildenbrand
Mark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped. Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Andrew Morton Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Signed-off

[PATCH v2 8/8] PM / Hibernate: exclude all PageOffline() pages

2018-11-22 Thread David Hildenbrand
t;Rafael J. Wysocki" Cc: Pavel Machek Cc: Len Brown Cc: Andrew Morton Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Acked-by: Pavel Machek Acked-by: Rafael J. Wysocki Signed-off-by: David Hildenbrand --- kernel/power/snapshot.c | 9 +++-- 1 file changed, 7 i

[PATCH v2 5/8] hv_balloon: mark inflated pages PG_offline

2018-11-22 Thread David Hildenbrand
"Michael S. Tsirkin" Acked-by: Pankaj gupta Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c index 211f3fe3a038..47719862e57f 100

[PATCH v2 3/8] kexec: export PG_offline to VMCOREINFO

2018-11-22 Thread David Hildenbrand
doval Cc: Arnd Bergmann Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Cc: Lianbo Jiang Cc: Borislav Petkov Cc: Kazuhito Hagio Acked-by: Michael S. Tsirkin Acked-by: Dave Young Signed-off-by: David Hildenbrand --- kernel/crash_core.c | 2 ++ 1 file changed, 2 in

[PATCH v2 1/8] mm: balloon: update comment about isolation/migration/compaction

2018-11-22 Thread David Hildenbrand
on Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Acked-by: Michael S. Tsirkin Signed-off-by: David Hildenbrand --- include/linux/balloon_compaction.h | 26 +- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/include/linux/balloo

[PATCH v2 0/8] mm/kdump: allow to exclude pages that are logically offline

2018-11-22 Thread David Hildenbrand
a macro - "vmw_balloon: mark inflated pages PG_offline" -- Use helper function + adapt comments - "PM / Hibernate: exclude all PageOffline() pages" -- Perform the check separate from swsusp checks. - Added RBs/ACKs David Hildenbrand (8): mm: balloon: update comment about i

[PATCH v2 2/8] mm: convert PG_balloon to PG_offline

2018-11-22 Thread David Hildenbrand
Miles Chen Cc: David Rientjes Cc: Konstantin Khlebnikov Cc: Kazuhito Hagio Acked-by: Konstantin Khlebnikov Acked-by: Michael S. Tsirkin Acked-by: Pankaj gupta Signed-off-by: David Hildenbrand --- Documentation/admin-guide/mm/pagemap.rst | 9 ++--- fs/proc/page.c

Re: [PATCH v1] makedumpfile: exclude pages that are logically offline

2018-11-21 Thread David Hildenbrand
by we can crash the kernel by >> dumping, when reading memory of a partially onlined memory segment >> (for memory added by the Hyper-V balloon driver). >> >> Therefore, don't read and dump pages that are marked as being logically

Re: [PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages

2018-11-21 Thread David Hildenbrand
On 21.11.18 12:35, William Kucharski wrote: > If you are adding PageOffline(page) to the condition list of the already > existing if in > saveable_highmem_page(), why explicitly add it as a separate statement in > saveable_page()? > > It would seem more consistent to make the second check: > >

Re: [PATCH v1 6/8] vmw_balloon: mark inflated pages PG_offline

2018-11-21 Thread David Hildenbrand
On 21.11.18 04:22, Nadav Amit wrote: > Thanks for this patch! > >> On Nov 19, 2018, at 2:16 AM, David Hildenbrand wrote: >> >> Mark inflated and never onlined pages PG_offline, to tell the world that >> the content is stale and should not be dumped. >> >>

Re: [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO

2018-11-21 Thread David Hildenbrand
On 21.11.18 07:04, Baoquan He wrote: > On 11/19/18 at 11:16am, David Hildenbrand wrote: >> diff --git a/kernel/crash_core.c b/kernel/crash_core.c >> index 933cb3e45b98..093c9f917ed0 100644 >> --- a/kernel/crash_core.c >> +++ b/kernel/crash_core.c >> @@ -

Re: [PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline

2018-11-20 Thread David Hildenbrand
> Cc: Stephen Hemminger >> Cc: Kairui Song >> Cc: Vitaly Kuznetsov >> Cc: Andrew Morton >> Cc: Matthew Wilcox >> Cc: Michal Hocko >> Cc: "Michael S. Tsirkin" >> Signed-off-by: David Hildenbrand >> --- >> drivers/hv/hv_ba

Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline

2018-11-19 Thread David Hildenbrand
> > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > index 50ce1bddaf56..f91da3d0a67e 100644 > --- a/include/linux/page-flags.h > +++ b/include/linux/page-flags.h > @@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap) > #define PAGE_TYPE_BASE 0xf000 > /* Reserve

Re: [PATCH v1 4/8] xen/balloon: mark inflated pages PG_offline

2018-11-19 Thread David Hildenbrand
On 19.11.18 13:22, Juergen Gross wrote: > On 19/11/2018 11:16, David Hildenbrand wrote: >> Mark inflated and never onlined pages PG_offline, to tell the world that >> the content is stale and should not be dumped. >> >> Cc: Boris Ostrovsky >> Cc: Juergen Gross

[PATCH v1] makedumpfile: exclude pages that are logically offline

2018-11-19 Thread David Hildenbrand
balloon driver). Therefore, don't read and dump pages that are marked as being logically offline. Signed-off-by: David Hildenbrand --- makedumpfile.c | 34 ++ makedumpfile.h | 1 + 2 files changed, 31 insertions(+), 4 deletions(-) diff --git a/makedumpfil

[PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages

2018-11-19 Thread David Hildenbrand
; Acked-by: Pavel Machek Signed-off-by: David Hildenbrand --- kernel/power/snapshot.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index 87e6dd57819f..8d7b4d458842 100644 --- a/kernel/power/snapshot.c +++ b/kernel/power/snap

[PATCH v1 7/8] PM / Hibernate: use pfn_to_online_page()

2018-11-19 Thread David Hildenbrand
ot; Suggested-by: Michal Hocko Signed-off-by: David Hildenbrand --- kernel/power/snapshot.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index 640b2034edd6..87e6dd57819f 100644 --- a/kernel/power/snapshot.c +++

[PATCH v1 6/8] vmw_balloon: mark inflated pages PG_offline

2018-11-19 Thread David Hildenbrand
ael S. Tsirkin" Signed-off-by: David Hildenbrand --- drivers/misc/vmw_balloon.c | 32 1 file changed, 32 insertions(+) diff --git a/drivers/misc/vmw_balloon.c b/drivers/misc/vmw_balloon.c index e6126a4b95d3..8cc8bd9a4e32 100644 --- a/drivers/misc/vmw_ballo

[PATCH v1 4/8] xen/balloon: mark inflated pages PG_offline

2018-11-19 Thread David Hildenbrand
Mark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped. Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Andrew Morton Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Signed-off

[PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO

2018-11-19 Thread David Hildenbrand
mping them. Cc: Andrew Morton Cc: Dave Young Cc: "Kirill A. Shutemov" Cc: Baoquan He Cc: Omar Sandoval Cc: Arnd Bergmann Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Cc: Lianbo Jiang Cc: Borislav Petkov Cc: Kazuhito Hagio Signed-off-by: David

[PATCH v1 1/8] mm: balloon: update comment about isolation/migration/compaction

2018-11-19 Thread David Hildenbrand
on Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Signed-off-by: David Hildenbrand --- include/linux/balloon_compaction.h | 26 +- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/include/linux/balloon_compaction.h b/include/linux/b

[PATCH v1 2/8] mm: convert PG_balloon to PG_offline

2018-11-19 Thread David Hildenbrand
Miles Chen Cc: David Rientjes Cc: Konstantin Khlebnikov Cc: Kazuhito Hagio Signed-off-by: David Hildenbrand --- Documentation/admin-guide/mm/pagemap.rst | 9 ++--- fs/proc/page.c | 4 ++-- include/linux/balloon_compaction.h | 8 include/linux/p

[PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline

2018-11-19 Thread David Hildenbrand
"Michael S. Tsirkin" Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c index 211f3fe3a038..47719862e57f 100644 --- a/drivers/hv/hv_balloo

[PATCH v1 0/8] mm/kdump: allow to exclude pages that are logically offline

2018-11-19 Thread David Hildenbrand
ated pages PG_offline" - "mm: convert PG_balloon to PG_offline" -- After discussions, also rename the UAPI bit name (KPF_BALLOON -> KPF_OFFLINE) David Hildenbrand (8): mm: balloon: update comment about isolation/migration/compaction mm: convert PG_balloon to PG_offline kexec:

Re: [PATCH RFC 0/6] mm/kdump: allow to exclude pages that are logically offline

2018-11-16 Thread David Hildenbrand
On 14.11.18 22:16, David Hildenbrand wrote: > Right now, pages inflated as part of a balloon driver will be dumped > by dump tools like makedumpfile. While XEN is able to check in the > crash kernel whether a certain pfn is actuall backed by memory in the > hypervisor (see xen_oldme

Re: [PATCH RFC 6/6] PM / Hibernate: exclude all PageOffline() pages

2018-11-15 Thread David Hildenbrand
On 15.11.18 13:23, Michal Hocko wrote: > On Wed 14-11-18 22:17:04, David Hildenbrand wrote: > [...] >> diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c >> index b0308a2c6000..01db1d13481a 100644 >> --- a/kernel/power/snapshot.c >> +++ b/kernel/power/sn

Re: [PATCH RFC 3/6] kexec: export PG_offline to VMCOREINFO

2018-11-15 Thread David Hildenbrand
On 15.11.18 12:52, Borislav Petkov wrote: > On Thu, Nov 15, 2018 at 12:20:40PM +0100, David Hildenbrand wrote: >> Sorry to say, but that is the current practice without which >> makedumpfile would not be able to work at all. (exclude user pages, >> exclude page cache, exclud

Re: [PATCH RFC 3/6] kexec: export PG_offline to VMCOREINFO

2018-11-15 Thread David Hildenbrand
On 15.11.18 12:10, Borislav Petkov wrote: > On Thu, Nov 15, 2018 at 02:19:23PM +0800, Dave Young wrote: >> It would be good to copy some background info from cover letter to the >> patch description so that we can get better understanding why this is >> needed now. >> >> BTW, Lianbo is working on a

Re: [PATCH RFC 3/6] kexec: export PG_offline to VMCOREINFO

2018-11-15 Thread David Hildenbrand
On 15.11.18 07:19, Dave Young wrote: > Hi David, > > On 11/14/18 at 10:17pm, David Hildenbrand wrote: >> Let's export PG_offline via PAGE_OFFLINE_MAPCOUNT_VALUE, so >> makedumpfile can directly skip pages that are logically offline and the >> content therefore stal

Re: [PATCH RFC 2/6] mm: convert PG_balloon to PG_offline

2018-11-15 Thread David Hildenbrand
On 15.11.18 03:07, Mike Rapoport wrote: > On Wed, Nov 14, 2018 at 11:49:15PM +0100, David Hildenbrand wrote: >> On 14.11.18 23:23, Matthew Wilcox wrote: >>> On Wed, Nov 14, 2018 at 10:17:00PM +0100, David Hildenbrand wrote: >>>> Rename PG_balloon to PG_offline. This

  1   2   >