Re: [PATCH RFCv2 3/6] mm/memory_hotplug: fix online/offline_pages called w.o. mem_hotplug_lock

2018-09-17 Thread David Hildenbrand
Am 03.09.18 um 02:36 schrieb Rashmica: > Hi David, > > > On 21/08/18 20:44, David Hildenbrand wrote: > >> There seem to be some problems as result of 30467e0b3be ("mm, hotplug: >> fix concurrent memory hot-add deadlock"), which tried to fix a possible >&g

[PATCH v1 1/6] mm/memory_hotplug: make remove_memory() take the device_hotplug_lock

2018-09-18 Thread David Hildenbrand
car Salvador Cc: YASUAKI ISHIMATSU Cc: Mathieu Malaterre Reviewed-by: Pavel Tatashin Signed-off-by: David Hildenbrand --- arch/powerpc/platforms/powernv/memtrace.c | 2 -- arch/powerpc/platforms/pseries/hotplug-memory.c | 6 +++--- drivers/acpi/acpi_memhotplug.c

[PATCH v1 2/6] mm/memory_hotplug: make add_memory() take the device_hotplug_lock

2018-09-18 Thread David Hildenbrand
: Boris Ostrovsky Cc: Juergen Gross Cc: Nathan Fontenot Cc: John Allen Cc: Andrew Morton Cc: Michal Hocko Cc: Dan Williams Cc: Joonsoo Kim Cc: Vlastimil Babka Cc: Oscar Salvador Cc: Mathieu Malaterre Cc: Pavel Tatashin Cc: YASUAKI ISHIMATSU Reviewed-by: Pavel Tatashin Signed-off-by: David H

[PATCH v1 0/6] mm: online/offline_pages called w.o. mem_hotplug_lock

2018-09-18 Thread David Hildenbrand
Added rb's RFC -> RFCv2: - Don't export device_hotplug_lock, provide proper remove_memory/add_memory wrappers. - Split up the patches a bit. - Try to improve powernv memtrace locking - Add some documentation for locking that matches my knowledge David Hildenbrand (6): mm/memory_

[PATCH v1 3/6] mm/memory_hotplug: fix online/offline_pages called w.o. mem_hotplug_lock

2018-09-18 Thread David Hildenbrand
n Reviewed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- drivers/base/memory.c | 13 + mm/memory_hotplug.c | 28 2 files changed, 21 insertions(+), 20 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 40cac122e

[PATCH v1 6/6] memory-hotplug.txt: Add some details about locking internals

2018-09-18 Thread David Hildenbrand
ff-by: David Hildenbrand --- Documentation/memory-hotplug.txt | 39 +++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt index 7f49ebf3ddb2..03aaad7d7373 100644 --- a/Documentation/m

[PATCH v1 5/6] powerpc/powernv: hold device_hotplug_lock in memtrace_offline_pages()

2018-09-18 Thread David Hildenbrand
ashin Signed-off-by: David Hildenbrand --- arch/powerpc/platforms/powernv/memtrace.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/platforms/powernv/memtrace.c b/arch/powerpc/platforms/powernv/memtrace.c index ef7181d4fe68..473e59842ec5 100644 --- a

[PATCH v1 4/6] powerpc/powernv: hold device_hotplug_lock when calling device_online()

2018-09-18 Thread David Hildenbrand
device_online() should be called with device_hotplug_lock() held. Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: Rashmica Gupta Cc: Balbir Singh Cc: Michael Neuling Reviewed-by: Pavel Tatashin Signed-off-by: David Hildenbrand --- arch/powerpc/platforms/powernv

Re: [PATCH v1 0/6] mm: online/offline_pages called w.o. mem_hotplug_lock

2018-09-19 Thread David Hildenbrand
Am 19.09.18 um 03:22 schrieb Balbir Singh: > On Tue, Sep 18, 2018 at 01:48:16PM +0200, David Hildenbrand wrote: >> Reading through the code and studying how mem_hotplug_lock is to be used, >> I noticed that there are two places where we can end up calling >> device_on

[PATCH v2 1/6] mm/memory_hotplug: make remove_memory() take the device_hotplug_lock

2018-09-25 Thread David Hildenbrand
ed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- arch/powerpc/platforms/powernv/memtrace.c | 2 +- arch/powerpc/platforms/pseries/hotplug-memory.c | 6 +++--- drivers/acpi/acpi_memhotplug.c | 2 +- include/linux/memory_hotplug.h | 3 ++- mm/m

[PATCH v2 0/6] mm: online/offline_pages called w.o. mem_hotplug_lock

2018-09-25 Thread David Hildenbrand
e_memory/add_memory wrappers. - Split up the patches a bit. - Try to improve powernv memtrace locking - Add some documentation for locking that matches my knowledge David Hildenbrand (6): mm/memory_hotplug: make remove_memory() take the device_hotplug_lock mm/memory_hotplug: make add_memory() ta

[PATCH v2 2/6] mm/memory_hotplug: make add_memory() take the device_hotplug_lock

2018-09-25 Thread David Hildenbrand
Rafael J. Wysocki Reviewed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- .../platforms/pseries/hotplug-memory.c| 2 +- drivers/acpi/acpi_memhotplug.c| 2 +- drivers/base/memory.c | 9 ++-- drivers/xen/balloon.c

[PATCH v2 6/6] memory-hotplug.txt: Add some details about locking internals

2018-09-25 Thread David Hildenbrand
ed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- Documentation/memory-hotplug.txt | 42 +++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt index 7f49ebf3ddb2..ce4faa5530fa 100644

[PATCH v2 5/6] powerpc/powernv: hold device_hotplug_lock when calling memtrace_offline_pages()

2018-09-25 Thread David Hildenbrand
ashin Reviewed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- arch/powerpc/platforms/powernv/memtrace.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/platforms/powernv/memtrace.c b/arch/powerpc/platforms/powernv/memtrace.c index fdd48f1a39f7..d84d09c

[PATCH v2 3/6] mm/memory_hotplug: fix online/offline_pages called w.o. mem_hotplug_lock

2018-09-25 Thread David Hildenbrand
n Reviewed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- drivers/base/memory.c | 13 + mm/memory_hotplug.c | 28 2 files changed, 21 insertions(+), 20 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 40cac122e

[PATCH v2 4/6] powerpc/powernv: hold device_hotplug_lock when calling device_online()

2018-09-25 Thread David Hildenbrand
device_online() should be called with device_hotplug_lock() held. Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: Rashmica Gupta Cc: Balbir Singh Cc: Michael Neuling Reviewed-by: Pavel Tatashin Reviewed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- arch

Re: [PATCH v2 5/6] powerpc/powernv: hold device_hotplug_lock when calling memtrace_offline_pages()

2018-09-26 Thread David Hildenbrand
On 25/09/2018 14:15, Balbir Singh wrote: > On Tue, Sep 25, 2018 at 11:14:56AM +0200, David Hildenbrand wrote: >> Let's perform all checking + offlining + removing under >> device_hotplug_lock, so nobody can mess with these devices via >> sysfs concurrently. >> >

[PATCH v3 3/6] mm/memory_hotplug: fix online/offline_pages called w.o. mem_hotplug_lock

2018-09-27 Thread David Hildenbrand
n Reviewed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- drivers/base/memory.c | 13 + mm/memory_hotplug.c | 28 2 files changed, 21 insertions(+), 20 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 40cac122e

[PATCH v3 2/6] mm/memory_hotplug: make add_memory() take the device_hotplug_lock

2018-09-27 Thread David Hildenbrand
Rafael J. Wysocki Reviewed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- .../platforms/pseries/hotplug-memory.c| 2 +- drivers/acpi/acpi_memhotplug.c| 2 +- drivers/base/memory.c | 9 ++-- drivers/xen/balloon.c

[PATCH v3 1/6] mm/memory_hotplug: make remove_memory() take the device_hotplug_lock

2018-09-27 Thread David Hildenbrand
ed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- arch/powerpc/platforms/powernv/memtrace.c | 2 +- arch/powerpc/platforms/pseries/hotplug-memory.c | 6 +++--- drivers/acpi/acpi_memhotplug.c | 2 +- include/linux/memory_hotplug.h | 3 ++- mm/m

[PATCH v3 0/6] mm: online/offline_pages called w.o. mem_hotplug_lock

2018-09-27 Thread David Hildenbrand
remove_memory() in patch #1 - Minor patch description fixes. - Added rb's RFC -> RFCv2: - Don't export device_hotplug_lock, provide proper remove_memory/add_memory wrappers. - Split up the patches a bit. - Try to improve powernv memtrace locking - Add some documentation for locking that

[PATCH v3 5/6] powerpc/powernv: hold device_hotplug_lock when calling memtrace_offline_pages()

2018-09-27 Thread David Hildenbrand
ashin Reviewed-by: Rashmica Gupta Acked-by: Balbir Singh Signed-off-by: David Hildenbrand --- arch/powerpc/platforms/powernv/memtrace.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/platforms/powernv/memtrace.c b/arch/powerpc/platforms/powernv/memtrace.c

[PATCH v3 4/6] powerpc/powernv: hold device_hotplug_lock when calling device_online()

2018-09-27 Thread David Hildenbrand
device_online() should be called with device_hotplug_lock() held. Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: Rashmica Gupta Cc: Balbir Singh Cc: Michael Neuling Reviewed-by: Pavel Tatashin Reviewed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- arch

[PATCH v3 6/6] memory-hotplug.txt: Add some details about locking internals

2018-09-27 Thread David Hildenbrand
ed-by: Rashmica Gupta Signed-off-by: David Hildenbrand --- Documentation/memory-hotplug.txt | 42 +++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt index 7f49ebf3ddb2..ce4faa5530fa 100644

[PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-09-28 Thread David Hildenbrand
Kroah-Hartman Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Boris Ostrovsky Cc: Juergen Gross Cc: "Jérôme Glisse" Cc: Andrew Morton Cc: Mike Rapoport Cc: Dan Williams Cc: Stephen Rothwell Cc: Michal Hocko Cc: "Kirill A. Shutemov" Cc: David

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-01 Thread David Hildenbrand
On 28/09/2018 19:02, Dave Hansen wrote: > It's really nice if these kinds of things are broken up. First, replace > the old want_memblock parameter, then add the parameter to the > __add_page() calls. Definitely, once we agree that is is not nuts, I will split it up for the next version :) > >>

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-01 Thread David Hildenbrand
On 01/10/2018 10:40, Michal Hocko wrote: > On Fri 28-09-18 17:03:57, David Hildenbrand wrote: > [...] > > I haven't read the patch itself but I just wanted to note one thing > about this part > >> For paravirtualized devices it is relevant that memory is onlined as

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-02 Thread David Hildenbrand
On 02/10/2018 15:47, Michal Hocko wrote: > On Mon 01-10-18 11:34:25, David Hildenbrand wrote: >> On 01/10/2018 10:40, Michal Hocko wrote: >>> On Fri 28-09-18 17:03:57, David Hildenbrand wrote: >>> [...] >>> >>> I haven't read the patch itself but I

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-03 Thread David Hildenbrand
On 03/10/2018 15:54, Michal Hocko wrote: > On Tue 02-10-18 17:25:19, David Hildenbrand wrote: >> On 02/10/2018 15:47, Michal Hocko wrote: > [...] >>> Zone imbalance is an inherent problem of the highmem zone. It is >>> essentially the highmem zone we all loved so much

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-03 Thread David Hildenbrand
On 03/10/2018 16:24, Michal Hocko wrote: > On Wed 03-10-18 15:52:24, Vitaly Kuznetsov wrote: > [...] >>> As David said some of the memory cannot be onlined without further steps >>> (e.g. when it is standby as David called it) and then I fail to see how >>> eBPF help in any way. >> >> and also, we

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-03 Thread David Hildenbrand
On 03/10/2018 16:34, Vitaly Kuznetsov wrote: > Dave Hansen writes: > >> On 10/03/2018 06:52 AM, Vitaly Kuznetsov wrote: >>> It is more than just memmaps (e.g. forking udev process doing memory >>> onlining also needs memory) but yes, the main idea is to make the >>> onlining synchronous with hotp

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-04 Thread David Hildenbrand
On 04/10/2018 08:28, Michal Hocko wrote: > On Wed 03-10-18 19:00:29, David Hildenbrand wrote: > [...] >> Let me rephrase: You state that user space has to make the decision and >> that user should be able to set/reconfigure rules. That is perfectly fine. >> >> But

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-04 Thread David Hildenbrand
On 01/10/2018 18:24, Dave Hansen wrote: >> How should a policy in user space look like when new memory gets added >> - on s390x? Not onlining paravirtualized memory is very wrong. > > Because we're going to balloon it away in a moment anyway? No, rether somebody wanted this VM to have more memory

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-04 Thread David Hildenbrand
On 03/10/2018 16:24, Michal Hocko wrote: > On Wed 03-10-18 15:52:24, Vitaly Kuznetsov wrote: > [...] >>> As David said some of the memory cannot be onlined without further steps >>> (e.g. when it is standby as David called it) and then I fail to see how >>> eBPF help in any way. >> >> and also, we

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-04 Thread David Hildenbrand
On 04/10/2018 08:19, Michal Hocko wrote: > On Wed 03-10-18 19:14:05, David Hildenbrand wrote: >> On 03/10/2018 16:34, Vitaly Kuznetsov wrote: >>> Dave Hansen writes: >>> >>>> On 10/03/2018 06:52 AM, Vitaly Kuznetsov wrote: >>>>> It is more tha

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-04 Thread David Hildenbrand
On 04/10/2018 17:28, Michal Suchánek wrote: > On Thu, 4 Oct 2018 10:13:48 +0200 > David Hildenbrand wrote: > > ok, so what is the problem here? > > Handling the hotplug in userspace through udev may be suboptimal and > kernel handling might be faster but that's or

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-05 Thread David Hildenbrand
On 04/10/2018 19:50, Michal Suchánek wrote: > On Thu, 4 Oct 2018 17:45:13 +0200 > David Hildenbrand wrote: > >> On 04/10/2018 17:28, Michal Suchánek wrote: > >>> >>> The state of the art is to determine what to do with hotplugged >>> memory in usersp

[PATCH RFC 0/6] mm/kdump: allow to exclude pages that are logically offline

2018-11-14 Thread David Hildenbrand
and a makedumpfile change will be required to skip dumping these pages). [1] https://lkml.org/lkml/2018/7/20/566 David Hildenbrand (6): mm: balloon: update comment about isolation/migration/compaction mm: convert PG_balloon to PG_offline kexec: export PG_offline to VMCOREINFO xen/balloon:

[PATCH RFC 1/6] mm: balloon: update comment about isolation/migration/compaction

2018-11-14 Thread David Hildenbrand
on Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Signed-off-by: David Hildenbrand --- include/linux/balloon_compaction.h | 26 +- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/include/linux/balloon_compaction.h b/include/linux/b

[PATCH RFC 3/6] kexec: export PG_offline to VMCOREINFO

2018-11-14 Thread David Hildenbrand
thew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Signed-off-by: David Hildenbrand --- kernel/crash_core.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/crash_core.c b/kernel/crash_core.c index 933cb3e45b98..093c9f917ed0 100644 --- a/kernel/crash_core.c +++ b/kern

[PATCH RFC 2/6] mm: convert PG_balloon to PG_offline

2018-11-14 Thread David Hildenbrand
t;Kirill A. Shutemov" Cc: Stephen Rothwell Cc: Matthew Wilcox Cc: "Michael S. Tsirkin" Cc: Michal Hocko Cc: Pavel Tatashin Cc: Alexander Duyck Cc: Naoya Horiguchi Cc: Miles Chen Cc: David Rientjes Signed-off-by: David Hildenbrand --- Documentation/admin-guide/mm/pagemap.rst |

[PATCH RFC 4/6] xen/balloon: mark inflated pages PG_offline

2018-11-14 Thread David Hildenbrand
Mark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped. Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Andrew Morton Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Signed-off

[PATCH RFC 6/6] PM / Hibernate: exclude all PageOffline() pages

2018-11-14 Thread David Hildenbrand
The content of pages that are marked PG_offline is not of interest (e.g. inflated by a balloon driver), let's skip these pages. Cc: "Rafael J. Wysocki" Cc: Pavel Machek Cc: Len Brown Cc: Andrew Morton Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin&qu

[PATCH RFC 5/6] hv_balloon: mark inflated pages PG_offline

2018-11-14 Thread David Hildenbrand
"Michael S. Tsirkin" Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c index 5728dc470eeb..778b6f879d1c 100644 --- a/drivers/hv/hv_balloo

Re: [PATCH RFC 2/6] mm: convert PG_balloon to PG_offline

2018-11-14 Thread David Hildenbrand
On 14.11.18 23:23, Matthew Wilcox wrote: > On Wed, Nov 14, 2018 at 10:17:00PM +0100, David Hildenbrand wrote: >> Rename PG_balloon to PG_offline. This is an indicator that the page is >> logically offline, the content stale and that it should not be touched >> (e.g. a hyp

Re: [PATCH RFC 0/6] mm/kdump: allow to exclude pages that are logically offline

2018-11-14 Thread David Hildenbrand
On 14.11.18 23:57, Nadav Amit wrote: > From: David Hildenbrand > Sent: November 14, 2018 at 9:16:58 PM GMT >> Subject: [PATCH RFC 0/6] mm/kdump: allow to exclude pages that are logically >> offline >> >> >> Right now, pages inflated as part of a balloon driver

Re: [PATCH RFC 2/6] mm: convert PG_balloon to PG_offline

2018-11-15 Thread David Hildenbrand
On 15.11.18 03:07, Mike Rapoport wrote: > On Wed, Nov 14, 2018 at 11:49:15PM +0100, David Hildenbrand wrote: >> On 14.11.18 23:23, Matthew Wilcox wrote: >>> On Wed, Nov 14, 2018 at 10:17:00PM +0100, David Hildenbrand wrote: >>>> Rename PG_balloon to PG_offline. This

Re: [PATCH RFC 3/6] kexec: export PG_offline to VMCOREINFO

2018-11-15 Thread David Hildenbrand
On 15.11.18 07:19, Dave Young wrote: > Hi David, > > On 11/14/18 at 10:17pm, David Hildenbrand wrote: >> Let's export PG_offline via PAGE_OFFLINE_MAPCOUNT_VALUE, so >> makedumpfile can directly skip pages that are logically offline and the >> content therefore stal

Re: [PATCH RFC 3/6] kexec: export PG_offline to VMCOREINFO

2018-11-15 Thread David Hildenbrand
On 15.11.18 12:10, Borislav Petkov wrote: > On Thu, Nov 15, 2018 at 02:19:23PM +0800, Dave Young wrote: >> It would be good to copy some background info from cover letter to the >> patch description so that we can get better understanding why this is >> needed now. >> >> BTW, Lianbo is working on a

Re: [PATCH RFC 3/6] kexec: export PG_offline to VMCOREINFO

2018-11-15 Thread David Hildenbrand
On 15.11.18 12:52, Borislav Petkov wrote: > On Thu, Nov 15, 2018 at 12:20:40PM +0100, David Hildenbrand wrote: >> Sorry to say, but that is the current practice without which >> makedumpfile would not be able to work at all. (exclude user pages, >> exclude page cache, exclud

Re: [PATCH RFC 6/6] PM / Hibernate: exclude all PageOffline() pages

2018-11-15 Thread David Hildenbrand
On 15.11.18 13:23, Michal Hocko wrote: > On Wed 14-11-18 22:17:04, David Hildenbrand wrote: > [...] >> diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c >> index b0308a2c6000..01db1d13481a 100644 >> --- a/kernel/power/snapshot.c >> +++ b/kernel/power/sn

Re: [PATCH RFC 0/6] mm/kdump: allow to exclude pages that are logically offline

2018-11-16 Thread David Hildenbrand
On 14.11.18 22:16, David Hildenbrand wrote: > Right now, pages inflated as part of a balloon driver will be dumped > by dump tools like makedumpfile. While XEN is able to check in the > crash kernel whether a certain pfn is actuall backed by memory in the > hypervisor (see xen_oldme

[PATCH v1 0/8] mm/kdump: allow to exclude pages that are logically offline

2018-11-19 Thread David Hildenbrand
ated pages PG_offline" - "mm: convert PG_balloon to PG_offline" -- After discussions, also rename the UAPI bit name (KPF_BALLOON -> KPF_OFFLINE) David Hildenbrand (8): mm: balloon: update comment about isolation/migration/compaction mm: convert PG_balloon to PG_offline kexec:

[PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO

2018-11-19 Thread David Hildenbrand
mping them. Cc: Andrew Morton Cc: Dave Young Cc: "Kirill A. Shutemov" Cc: Baoquan He Cc: Omar Sandoval Cc: Arnd Bergmann Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Cc: Lianbo Jiang Cc: Borislav Petkov Cc: Kazuhito Hagio Signed-off-by: David

[PATCH v1 1/8] mm: balloon: update comment about isolation/migration/compaction

2018-11-19 Thread David Hildenbrand
on Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Signed-off-by: David Hildenbrand --- include/linux/balloon_compaction.h | 26 +- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/include/linux/balloon_compaction.h b/include/linux/b

[PATCH v1 2/8] mm: convert PG_balloon to PG_offline

2018-11-19 Thread David Hildenbrand
Miles Chen Cc: David Rientjes Cc: Konstantin Khlebnikov Cc: Kazuhito Hagio Signed-off-by: David Hildenbrand --- Documentation/admin-guide/mm/pagemap.rst | 9 ++--- fs/proc/page.c | 4 ++-- include/linux/balloon_compaction.h | 8 include/linux/p

[PATCH v1 4/8] xen/balloon: mark inflated pages PG_offline

2018-11-19 Thread David Hildenbrand
Mark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped. Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Andrew Morton Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Signed-off

[PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline

2018-11-19 Thread David Hildenbrand
"Michael S. Tsirkin" Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c index 211f3fe3a038..47719862e57f 100644 --- a/drivers/hv/hv_balloo

[PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages

2018-11-19 Thread David Hildenbrand
; Acked-by: Pavel Machek Signed-off-by: David Hildenbrand --- kernel/power/snapshot.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index 87e6dd57819f..8d7b4d458842 100644 --- a/kernel/power/snapshot.c +++ b/kernel/power/snap

[PATCH v1 7/8] PM / Hibernate: use pfn_to_online_page()

2018-11-19 Thread David Hildenbrand
ot; Suggested-by: Michal Hocko Signed-off-by: David Hildenbrand --- kernel/power/snapshot.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index 640b2034edd6..87e6dd57819f 100644 --- a/kernel/power/snapshot.c +++

[PATCH v1 6/8] vmw_balloon: mark inflated pages PG_offline

2018-11-19 Thread David Hildenbrand
ael S. Tsirkin" Signed-off-by: David Hildenbrand --- drivers/misc/vmw_balloon.c | 32 1 file changed, 32 insertions(+) diff --git a/drivers/misc/vmw_balloon.c b/drivers/misc/vmw_balloon.c index e6126a4b95d3..8cc8bd9a4e32 100644 --- a/drivers/misc/vmw_ballo

[PATCH v1] makedumpfile: exclude pages that are logically offline

2018-11-19 Thread David Hildenbrand
balloon driver). Therefore, don't read and dump pages that are marked as being logically offline. Signed-off-by: David Hildenbrand --- makedumpfile.c | 34 ++ makedumpfile.h | 1 + 2 files changed, 31 insertions(+), 4 deletions(-) diff --git a/makedumpfil

Re: [PATCH v1 4/8] xen/balloon: mark inflated pages PG_offline

2018-11-19 Thread David Hildenbrand
On 19.11.18 13:22, Juergen Gross wrote: > On 19/11/2018 11:16, David Hildenbrand wrote: >> Mark inflated and never onlined pages PG_offline, to tell the world that >> the content is stale and should not be dumped. >> >> Cc: Boris Ostrovsky >> Cc: Juergen Gross

Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline

2018-11-19 Thread David Hildenbrand
> > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > index 50ce1bddaf56..f91da3d0a67e 100644 > --- a/include/linux/page-flags.h > +++ b/include/linux/page-flags.h > @@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap) > #define PAGE_TYPE_BASE 0xf000 > /* Reserve

Re: [PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline

2018-11-20 Thread David Hildenbrand
> Cc: Stephen Hemminger >> Cc: Kairui Song >> Cc: Vitaly Kuznetsov >> Cc: Andrew Morton >> Cc: Matthew Wilcox >> Cc: Michal Hocko >> Cc: "Michael S. Tsirkin" >> Signed-off-by: David Hildenbrand >> --- >> drivers/hv/hv_ba

Re: [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO

2018-11-21 Thread David Hildenbrand
On 21.11.18 07:04, Baoquan He wrote: > On 11/19/18 at 11:16am, David Hildenbrand wrote: >> diff --git a/kernel/crash_core.c b/kernel/crash_core.c >> index 933cb3e45b98..093c9f917ed0 100644 >> --- a/kernel/crash_core.c >> +++ b/kernel/crash_core.c >> @@ -

Re: [PATCH v1 6/8] vmw_balloon: mark inflated pages PG_offline

2018-11-21 Thread David Hildenbrand
On 21.11.18 04:22, Nadav Amit wrote: > Thanks for this patch! > >> On Nov 19, 2018, at 2:16 AM, David Hildenbrand wrote: >> >> Mark inflated and never onlined pages PG_offline, to tell the world that >> the content is stale and should not be dumped. >> >>

Re: [PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages

2018-11-21 Thread David Hildenbrand
On 21.11.18 12:35, William Kucharski wrote: > If you are adding PageOffline(page) to the condition list of the already > existing if in > saveable_highmem_page(), why explicitly add it as a separate statement in > saveable_page()? > > It would seem more consistent to make the second check: > >

Re: [PATCH v1] makedumpfile: exclude pages that are logically offline

2018-11-21 Thread David Hildenbrand
by we can crash the kernel by >> dumping, when reading memory of a partially onlined memory segment >> (for memory added by the Hyper-V balloon driver). >> >> Therefore, don't read and dump pages that are marked as being logically

[PATCH v2 2/8] mm: convert PG_balloon to PG_offline

2018-11-22 Thread David Hildenbrand
Miles Chen Cc: David Rientjes Cc: Konstantin Khlebnikov Cc: Kazuhito Hagio Acked-by: Konstantin Khlebnikov Acked-by: Michael S. Tsirkin Acked-by: Pankaj gupta Signed-off-by: David Hildenbrand --- Documentation/admin-guide/mm/pagemap.rst | 9 ++--- fs/proc/page.c

[PATCH v2 0/8] mm/kdump: allow to exclude pages that are logically offline

2018-11-22 Thread David Hildenbrand
a macro - "vmw_balloon: mark inflated pages PG_offline" -- Use helper function + adapt comments - "PM / Hibernate: exclude all PageOffline() pages" -- Perform the check separate from swsusp checks. - Added RBs/ACKs David Hildenbrand (8): mm: balloon: update comment about i

[PATCH v2 1/8] mm: balloon: update comment about isolation/migration/compaction

2018-11-22 Thread David Hildenbrand
on Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Acked-by: Michael S. Tsirkin Signed-off-by: David Hildenbrand --- include/linux/balloon_compaction.h | 26 +- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/include/linux/balloo

[PATCH v2 3/8] kexec: export PG_offline to VMCOREINFO

2018-11-22 Thread David Hildenbrand
doval Cc: Arnd Bergmann Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Cc: Lianbo Jiang Cc: Borislav Petkov Cc: Kazuhito Hagio Acked-by: Michael S. Tsirkin Acked-by: Dave Young Signed-off-by: David Hildenbrand --- kernel/crash_core.c | 2 ++ 1 file changed, 2 in

[PATCH v2 8/8] PM / Hibernate: exclude all PageOffline() pages

2018-11-22 Thread David Hildenbrand
t;Rafael J. Wysocki" Cc: Pavel Machek Cc: Len Brown Cc: Andrew Morton Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Acked-by: Pavel Machek Acked-by: Rafael J. Wysocki Signed-off-by: David Hildenbrand --- kernel/power/snapshot.c | 9 +++-- 1 file changed, 7 i

[PATCH v2 5/8] hv_balloon: mark inflated pages PG_offline

2018-11-22 Thread David Hildenbrand
"Michael S. Tsirkin" Acked-by: Pankaj gupta Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c index 211f3fe3a038..47719862e57f 100

[PATCH v2 6/8] vmw_balloon: mark inflated pages PG_offline

2018-11-22 Thread David Hildenbrand
ael S. Tsirkin" Acked-by: Nadav Amit Signed-off-by: David Hildenbrand --- drivers/misc/vmw_balloon.c | 32 1 file changed, 32 insertions(+) diff --git a/drivers/misc/vmw_balloon.c b/drivers/misc/vmw_balloon.c index e6126a4b95d3..877611b5659b 100644 --- a/dr

[PATCH v2 7/8] PM / Hibernate: use pfn_to_online_page()

2018-11-22 Thread David Hildenbrand
ot; Suggested-by: Michal Hocko Acked-by: Michal Hocko Acked-by: Pavel Machek Acked-by: Rafael J. Wysocki Signed-off-by: David Hildenbrand --- kernel/power/snapshot.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/kernel/power/snapshot.c b/kernel/power/snap

[PATCH v2 4/8] xen/balloon: mark inflated pages PG_offline

2018-11-22 Thread David Hildenbrand
Mark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped. Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Andrew Morton Cc: Matthew Wilcox Cc: Michal Hocko Cc: "Michael S. Tsirkin" Signed-off

[PATCH v2] makedumpfile: exclude pages that are logically offline

2018-11-22 Thread David Hildenbrand
balloon driver). Therefore, don't read and dump pages that are marked as being logically offline. Signed-off-by: David Hildenbrand --- v1 -> v2: - Fix PAGE_BUDDY_MAPCOUNT_VALUE vs. PAGE_OFFLINE_MAPCOUNT_VALUE makedumpfile.c | 34 ++ makedumpfile.h | 1 +

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-11-23 Thread David Hildenbrand
On 28.09.18 17:03, David Hildenbrand wrote: > How to/when to online hotplugged memory is hard to manage for > distributions because different memory types are to be treated differently. > Right now, we need complicated udev rules that e.g. check if we are > running on s390x, on a phy

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-11-26 Thread David Hildenbrand
On 23.11.18 19:06, Michal Suchánek wrote: > On Fri, 23 Nov 2018 12:13:58 +0100 > David Hildenbrand wrote: > >> On 28.09.18 17:03, David Hildenbrand wrote: >>> How to/when to online hotplugged memory is hard to manage for >>> distributions because differe

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-11-26 Thread David Hildenbrand
On 26.11.18 13:30, David Hildenbrand wrote: > On 23.11.18 19:06, Michal Suchánek wrote: >> On Fri, 23 Nov 2018 12:13:58 +0100 >> David Hildenbrand wrote: >> >>> On 28.09.18 17:03, David Hildenbrand wrote: >>>> How to/when to online hotplugged memor

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-11-26 Thread David Hildenbrand
On 26.11.18 15:20, Michal Suchánek wrote: > On Mon, 26 Nov 2018 14:33:29 +0100 > David Hildenbrand wrote: > >> On 26.11.18 13:30, David Hildenbrand wrote: >>> On 23.11.18 19:06, Michal Suchánek wrote: > >>>> >>>> If we are going to fake the

Re: [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-11-27 Thread David Hildenbrand
On 27.11.18 17:32, Michal Suchánek wrote: > On Mon, 26 Nov 2018 16:59:14 +0100 > David Hildenbrand wrote: > >> On 26.11.18 15:20, Michal Suchánek wrote: >>> On Mon, 26 Nov 2018 14:33:29 +0100 >>> David Hildenbrand wrote: >>> >>>>

[PATCH RFCv2 0/4] mm/memory_hotplug: Introduce memory block types

2018-11-30 Thread David Hildenbrand
e.g. online_type) we are using in that context. - The "removable" property should never have been named like that. It should have been "offlinable". Can we still rename that? E.g. boot memory is sometimes marked as removable ... David Hildenbrand (4): mm/memory_ho

[PATCH RFCv2 1/4] mm/memory_hotplug: Introduce memory block types

2018-11-30 Thread David Hildenbrand
hwidefsky Cc: Heiko Carstens Signed-off-by: David Hildenbrand --- drivers/base/memory.c | 38 +++--- include/linux/memory.h | 27 +++ 2 files changed, 62 insertions(+), 3 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memo

[PATCH RFCv2 2/4] mm/memory_hotplug: Replace "bool want_memblock" by "int type"

2018-11-30 Thread David Hildenbrand
c: Rob Herring Cc: Pavel Tatashin Cc: "mike.tra...@hpe.com" Cc: Joonsoo Kim Cc: Wei Yang Cc: Logan Gunthorpe Cc: "Jérôme Glisse" Cc: "Jan H. Schönherr" Cc: Dave Jiang Cc: Matthew Wilcox Cc: Mathieu Malaterre Signed-off-by: David Hildenbrand --- arch/ia6

[PATCH RFCv2 4/4] mm/memory_hotplug: Drop MEMORY_TYPE_UNSPECIFIED

2018-11-30 Thread David Hildenbrand
Cc: Michal Hocko Cc: Michal Suchánek Cc: Vitaly Kuznetsov Cc: Dan Williams Cc: Pavel Tatashin Signed-off-by: David Hildenbrand --- drivers/base/memory.c | 3 --- include/linux/memory.h | 5 - 2 files changed, 8 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.

[PATCH RFCv2 3/4] mm/memory_hotplug: Introduce and use more memory types

2018-11-30 Thread David Hildenbrand
Neuling Cc: Nathan Fontenot Cc: YueHaibing Cc: Vasily Gorbik Cc: Ingo Molnar Cc: Stephen Rothwell Cc: "mike.tra...@hpe.com" Cc: Oscar Salvador Cc: Joonsoo Kim Cc: Mathieu Malaterre Cc: Michal Hocko Cc: Arun KS Cc: Andrew Banman Cc: Dave Hansen Cc: Michal Suchánek Cc: V

Re: [PATCH RFCv2 1/4] mm/memory_hotplug: Introduce memory block types

2018-12-03 Thread David Hildenbrand
On 01.12.18 02:25, Wei Yang wrote: > On Fri, Nov 30, 2018 at 06:59:19PM +0100, David Hildenbrand wrote: >> Memory onlining should always be handled by user space, because only user >> space knows which use cases it wants to satisfy. E.g. memory might be >> onlined to the MO

Re: [PATCH RFCv2 2/4] mm/memory_hotplug: Replace "bool want_memblock" by "int type"

2018-12-03 Thread David Hildenbrand
On 01.12.18 02:50, Wei Yang wrote: > On Fri, Nov 30, 2018 at 06:59:20PM +0100, David Hildenbrand wrote: >> Let's pass a memory block type instead. Pass "MEMORY_BLOCK_NONE" for device >> memory and for now "MEMORY_BLOCK_UNSPECIFIED" for anything else. No &g

Re: [PATCH RFCv2 3/4] mm/memory_hotplug: Introduce and use more memory types

2018-12-04 Thread David Hildenbrand
On 04.12.18 10:44, Michal Suchánek wrote: > On Fri, 30 Nov 2018 18:59:21 +0100 > David Hildenbrand wrote: > >> Let's introduce new types for different kinds of memory blocks and use >> them in existing code. As I don't see an easy way to split this up, >>

Re: [PATCH RFCv2 0/4] mm/memory_hotplug: Introduce memory block types

2018-12-20 Thread David Hildenbrand
On 30.11.18 18:59, David Hildenbrand wrote: > This is the second approach, introducing more meaningful memory block > types and not changing online behavior in the kernel. It is based on > latest linux-next. > > As we found out during dicussion, user space should always handl

Re: [PATCH RFCv2 0/4] mm/memory_hotplug: Introduce memory block types

2018-12-20 Thread David Hildenbrand
On 20.12.18 14:08, Michal Hocko wrote: > On Thu 20-12-18 13:58:16, David Hildenbrand wrote: >> On 30.11.18 18:59, David Hildenbrand wrote: >>> This is the second approach, introducing more meaningful memory block >>> types and not changing online behavior in the kernel

Re: [PATCH] hv_balloon: avoid touching uninitialized struct page during tail onlining

2019-01-07 Thread David Hildenbrand
On 04.01.19 15:19, Vitaly Kuznetsov wrote: > Hyper-V memory hotplug protocol has 2M granularity and in Linux x86 we use > 128M. To deal with it we implement partial section onlining by registering > custom page onlining callback (hv_online_page()). Later, when more memory > arrives we try to online

Re: [PATCH] hv_balloon: avoid touching uninitialized struct page during tail onlining

2019-01-08 Thread David Hildenbrand
On 07.01.19 14:44, Vitaly Kuznetsov wrote: > David Hildenbrand writes: > >> On 04.01.19 15:19, Vitaly Kuznetsov wrote: >>> Hyper-V memory hotplug protocol has 2M granularity and in Linux x86 we use >>> 128M. To deal with it we implement partial section onlining

Re: [PATCH v2] makedumpfile: exclude pages that are logically offline

2019-03-07 Thread David Hildenbrand
e kernel by >> dumping, when reading memory of a partially onlined memory segment >> (for memory added by the Hyper-V balloon driver). >> >> Therefore, don't read and dump pages that are marked as being logically >> offline. >> >> Signed-off-by: David Hilde

Re: [PATCH v2 3/8] kexec: export PG_offline to VMCOREINFO

2019-03-11 Thread David Hildenbrand
On 11.03.19 10:04, Dave Young wrote: > Hi David, > On 11/22/18 at 11:06am, David Hildenbrand wrote: >> Right now, pages inflated as part of a balloon driver will be dumped >> by dump tools like makedumpfile. While XEN is able to check in the >> crash kernel whether a certa

Re: [PATCH RFCv2 0/4] mm/memory_hotplug: Introduce memory block types

2019-03-27 Thread David Hildenbrand
On 20.12.18 14:08, Michal Hocko wrote: > On Thu 20-12-18 13:58:16, David Hildenbrand wrote: >> On 30.11.18 18:59, David Hildenbrand wrote: >>> This is the second approach, introducing more meaningful memory block >>> types and not changing online behavior in the kernel

[PATCH RFC v1 00/12] mm: Don't mark hotplugged pages PG_reserved (including ZONE_DEVICE)

2019-10-22 Thread David Hildenbrand
tps://lkml.org/lkml/2019/10/21/736 [2] https://lkml.org/lkml/2019/10/21/1034 Cc: Michal Hocko Cc: Dan Williams Cc: kvm-...@vger.kernel.org Cc: linuxppc-...@lists.ozlabs.org Cc: k...@vger.kernel.org Cc: linux-hyp...@vger.kernel.org Cc: de...@driverdev.osuosl.org Cc: xen-de...@lists.xenpr

  1   2   >