Re: [PATCH v2 4/6] mm: replace vma->vm_flags indirect modification in ksm_madvise

2023-01-25 Thread Michal Hocko
On Wed 25-01-23 08:57:48, Suren Baghdasaryan wrote: > On Wed, Jan 25, 2023 at 1:38 AM 'Michal Hocko' via kernel-team > wrote: > > > > On Wed 25-01-23 00:38:49, Suren Baghdasaryan wrote: > > > Replace indirect modifications to vma->vm_flags with calls t

Re: [PATCH v2 6/6] mm: export dump_mm()

2023-01-25 Thread Michal Hocko
gt; > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko > --- > mm/debug.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/mm/debug.c b/mm/debug.c > index 9d3d893dc7f4..96d594e16292 100644 > --- a/mm/debug.c > +++ b/mm/debug.c > @@ -215,6

Re: [PATCH v2 5/6] mm: introduce mod_vm_flags_nolock and use it in untrack_pfn

2023-01-25 Thread Michal Hocko
e isolated before we downgraded mmap_lock. > + */ > + unmap_region(mm, &mt_detach, vma, prev, next, start, end, !downgrade); > /* Statistics and freeing VMAs */ > mas_set(&mas_detach, start); > remove_mt(mm, &mas_detach); > @@ -2704,7 +2708,7

Re: [PATCH v2 4/6] mm: replace vma->vm_flags indirect modification in ksm_madvise

2023-01-25 Thread Michal Hocko
cation attempts. Those BUG_ONs scream to much IMHO. KSM is an MM internal code so I gueess we should be willing to trust it. > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko -- Michal Hocko SUSE Labs

Re: [PATCH v2 3/6] mm: replace vma->vm_flags direct modifications with modifier calls

2023-01-25 Thread Michal Hocko
sors which would also prevent any future direct setting of those flags in uncontrolled way as well. Anyway Acked-by: Michal Hocko -- Michal Hocko SUSE Labs

Re: [PATCH v2 1/6] mm: introduce vma->vm_flags modifier functions

2023-01-25 Thread Michal Hocko
; operations. Introduce modifier functions for vm_flags to be used whenever > flags are updated. This way we can better check and control correct > locking behavior during these updates. > > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko > --- >

Re: [PATCH v2 2/6] mm: replace VM_LOCKED_CLEAR_MASK with VM_LOCKED_MASK

2023-01-25 Thread Michal Hocko
On Wed 25-01-23 00:38:47, Suren Baghdasaryan wrote: > To simplify the usage of VM_LOCKED_CLEAR_MASK in clear_vm_flags(), > replace it with VM_LOCKED_MASK bitmask and convert all users. > > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko > --- > include/linux/mm.h

Re: [RFC PATCH 00/30] Code tagging framework and applications

2022-09-08 Thread Michal Hocko
On Thu 08-09-22 03:29:50, Kent Overstreet wrote: > On Thu, Sep 08, 2022 at 09:12:45AM +0200, Michal Hocko wrote: > > Then you have probably missed a huge part of my emails. Please > > re-read. If those arguments are not clear, feel free to ask for > > clarification. Reducing t

Re: [RFC PATCH 00/30] Code tagging framework and applications

2022-09-08 Thread Michal Hocko
On Thu 08-09-22 02:35:48, Kent Overstreet wrote: > On Wed, Sep 07, 2022 at 09:45:18AM -0400, Steven Rostedt wrote: > > On Wed, 7 Sep 2022 09:04:28 -0400 > > Kent Overstreet wrote: > > > > > On Wed, Sep 07, 2022 at 01:00:09PM +0200, Michal Hocko wrote: >

Re: [RFC PATCH 00/30] Code tagging framework and applications

2022-09-07 Thread Michal Hocko
sense here. I know how to use my time better. -- Michal Hocko SUSE Labs

Re: [RFC PATCH 00/30] Code tagging framework and applications

2022-09-06 Thread Michal Hocko
On Mon 05-09-22 11:03:35, Suren Baghdasaryan wrote: > On Mon, Sep 5, 2022 at 1:12 AM Michal Hocko wrote: > > > > On Sun 04-09-22 18:32:58, Suren Baghdasaryan wrote: > > > On Thu, Sep 1, 2022 at 12:15 PM Michal Hocko wrote: > > [...] > > > > Yes, tr

Re: [RFC PATCH 00/30] Code tagging framework and applications

2022-09-06 Thread Michal Hocko
On Mon 05-09-22 19:46:49, Kent Overstreet wrote: > On Mon, Sep 05, 2022 at 10:49:38AM +0200, Michal Hocko wrote: > > This is really my main concern about this whole work. Not only it adds a > > considerable maintenance burden to the core MM because > > [citation needed] I t

Re: [RFC PATCH 00/30] Code tagging framework and applications

2022-09-05 Thread Michal Hocko
we really want to add all the tagging source code level changes to each and every one of them. This is really my main concern about this whole work. Not only it adds a considerable maintenance burden to the core MM because it adds on top of our existing allocator layers complexity but it would need to spread beyond MM to be useful because it is usually outside of MM where leaks happen. -- Michal Hocko SUSE Labs

Re: [RFC PATCH 00/30] Code tagging framework and applications

2022-09-05 Thread Michal Hocko
On Sun 04-09-22 18:32:58, Suren Baghdasaryan wrote: > On Thu, Sep 1, 2022 at 12:15 PM Michal Hocko wrote: [...] > > Yes, tracking back the call trace would be really needed. The question > > is whether this is really prohibitively expensive. How much overhead are > > we talki

Re: [RFC PATCH 00/30] Code tagging framework and applications

2022-09-01 Thread Michal Hocko
On Thu 01-09-22 08:33:19, Suren Baghdasaryan wrote: > On Thu, Sep 1, 2022 at 12:18 AM Michal Hocko wrote: [...] > > So I find Peter's question completely appropriate while your response to > > that not so much! Maybe ftrace is not the right tool for the intented > >

Re: [RFC PATCH 00/30] Code tagging framework and applications

2022-09-01 Thread Michal Hocko
On Wed 31-08-22 15:01:54, Kent Overstreet wrote: > On Wed, Aug 31, 2022 at 12:47:32PM +0200, Michal Hocko wrote: > > On Wed 31-08-22 11:19:48, Mel Gorman wrote: > > > Whatever asking for an explanation as to why equivalent functionality > > > cannot not be created from

Re: [RFC PATCH 00/30] Code tagging framework and applications

2022-08-31 Thread Michal Hocko
; is tricky and would need combining multiple different elements and that > is development effort but not impossible. > > Whatever asking for an explanation as to why equivalent functionality > cannot not be created from ftrace/kprobe/eBPF/whatever is reasonable. Fully agreed and this is especially true for a change this size 77 files changed, 3406 insertions(+), 703 deletions(-) -- Michal Hocko SUSE Labs

Re: [PATCH] mm, page_alloc: fix build_zonerefs_node()

2022-04-07 Thread Michal Hocko
On Thu 07-04-22 14:12:38, David Hildenbrand wrote: > On 07.04.22 14:04, Michal Hocko wrote: > > On Thu 07-04-22 13:58:44, David Hildenbrand wrote: > > [...] > >>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c > >>> index 3589febc6d31..130a2feceddc 100644 >

Re: [PATCH v2] mm, page_alloc: fix build_zonerefs_node()

2022-04-07 Thread Michal Hocko
efb74 ("mm, vmscan: only allocate and reclaim from zones with > pages managed by the buddy allocator") > Reported-by: Marek Marczykowski-Górecki > Signed-off-by: Juergen Gross > Acked-by: Michal Hocko > --- > V2: > - updated commit message (Michal Hocko) >

Re: [PATCH] mm, page_alloc: fix build_zonerefs_node()

2022-04-07 Thread Michal Hocko
that we are making that distinction has led to some bugs and I suspect it makes the code more complex without a very good reason. > I'd vote for going with the simple fix first, which should be good > enough AFAIKT. yes, see the other reply -- Michal Hocko SUSE Labs

Re: [PATCH] mm, page_alloc: fix build_zonerefs_node()

2022-04-07 Thread Michal Hocko
On Thu 07-04-22 13:40:25, Michal Hocko wrote: [...] > Now to your patch. I suspect this is not sufficient for the full hotplug > situation. Consider a new NUMA node to be hotadded. hotadd_new_pgdat > will call build_all_zonelists but the zone is not populated yet at that > moment

Re: [PATCH] mm, page_alloc: fix build_zonerefs_node()

2022-04-07 Thread Michal Hocko
On Thu 07-04-22 13:17:19, Juergen Gross wrote: > On 07.04.22 13:07, Michal Hocko wrote: > > On Thu 07-04-22 12:45:41, Juergen Gross wrote: > > > On 07.04.22 12:34, Michal Hocko wrote: > > > > Ccing Mel > > > > > > > > On Thu 07-04-22

Re: [PATCH] mm, page_alloc: fix build_zonerefs_node()

2022-04-07 Thread Michal Hocko
On Thu 07-04-22 12:45:41, Juergen Gross wrote: > On 07.04.22 12:34, Michal Hocko wrote: > > Ccing Mel > > > > On Thu 07-04-22 11:32:21, Juergen Gross wrote: > > > Since commit 9d3be21bf9c0 ("mm, page_alloc: simplify zonelist > > > initialization&quo

Re: [PATCH] mm, page_alloc: fix build_zonerefs_node()

2022-04-07 Thread Michal Hocko
type--; > zone = pgdat->node_zones + zone_type; > - if (managed_zone(zone)) { > + if (populated_zone(zone)) { > zoneref_set_zone(zone, &zonerefs[nr_zones++]); > check_highest_zone(zone_type); > } > -- > 2.34.1 -- Michal Hocko SUSE Labs

Re: blocking vs. non-blocking mmu notifiers

2022-03-24 Thread Michal Hocko
On Wed 23-03-22 14:04:04, Jason Gunthorpe wrote: > On Wed, Mar 23, 2022 at 05:49:43PM +0100, Michal Hocko wrote: > > > The bug here is that prior to commit a81461b0546c ("xen/gntdev: update > > > to new mmu_notifier semantic") wired the mn_invl_range_start

Re: blocking vs. non-blocking mmu notifiers

2022-03-23 Thread Michal Hocko
On Wed 23-03-22 13:31:46, Jason Gunthorpe wrote: > On Wed, Mar 23, 2022 at 10:45:30AM +0100, Michal Hocko wrote: > > [Let me add more people to the CC list - I am not really sure who is the > > most familiar with all the tricks that mmu notifiers might do] > > > &g

Re: blocking vs. non-blocking mmu notifiers

2022-03-23 Thread Michal Hocko
syscall_64 at 81004954 > #24 [c9004222ff50] entry_SYSCALL_64_after_hwframe at 818000b6 > > It was found that the notifier of the Xen gntdev driver was using a > mutex resulting in the deadlock. > > Michal Hocko suggested that backporting above m

Re: [PATCH v2 3/5] mm/page_alloc: move pages to tail in move_to_free_list()

2020-10-06 Thread Michal Hocko
eviewed-by: Oscar Salvador > Acked-by: Pankaj Gupta > Reviewed-by: Wei Yang > Cc: Andrew Morton > Cc: Alexander Duyck > Cc: Mel Gorman > Cc: Michal Hocko > Cc: Dave Hansen > Cc: Vlastimil Babka > Cc: Wei Yang > Cc: Oscar Salvador > Cc: Mike Rapopo

Re: [PATCH v1 3/5] mm/page_alloc: always move pages to the tail of the freelist in unset_migratetype_isolate()

2020-10-04 Thread Michal Hocko
On Fri 02-10-20 17:20:09, David Hildenbrand wrote: > On 02.10.20 15:24, Michal Hocko wrote: > > On Mon 28-09-20 20:21:08, David Hildenbrand wrote: > >> Page isolation doesn't actually touch the pages, it simply isolates > >> pageblocks and moves all free pages

Re: [PATCH v1 5/5] mm/memory_hotplug: update comment regarding zone shuffling

2020-10-02 Thread Michal Hocko
c: Alexander Duyck > Cc: Mel Gorman > Cc: Michal Hocko > Cc: Dave Hansen > Cc: Vlastimil Babka > Cc: Wei Yang > Cc: Oscar Salvador > Cc: Mike Rapoport > Signed-off-by: David Hildenbrand Acked-by: Michal Hocko > --- > mm/memory_hotplug.c | 11 --- >

Re: [PATCH v1 4/5] mm/page_alloc: place pages to tail in __free_pages_core()

2020-10-02 Thread Michal Hocko
briefly. I do not expect this to make a huge difference but who knows. It makes some sense to add pages in the order they show up in the physical address ordering. > Reviewed-by: Vlastimil Babka > Reviewed-by: Oscar Salvador > Cc: Andrew Morton > Cc: Alexander Duyck > Cc: Mel Gorman

Re: [PATCH v1 3/5] mm/page_alloc: always move pages to the tail of the freelist in unset_migratetype_isolate()

2020-10-02 Thread Michal Hocko
by: Oscar Salvador > Cc: Andrew Morton > Cc: Alexander Duyck > Cc: Mel Gorman > Cc: Michal Hocko > Cc: Dave Hansen > Cc: Vlastimil Babka > Cc: Wei Yang > Cc: Oscar Salvador > Cc: Mike Rapoport > Cc: Scott Cheloha > Cc: Michael Ellerman > Sign

Re: [PATCH v1 2/5] mm/page_alloc: place pages to tail in __putback_isolated_page()

2020-10-02 Thread Michal Hocko
whole > zone when undoing isolation of larger ranges, and after > free_contig_range(). > > Reviewed-by: Alexander Duyck > Reviewed-by: Oscar Salvador > Cc: Andrew Morton > Cc: Alexander Duyck > Cc: Mel Gorman > Cc: Michal Hocko > Cc: Dave Hansen > Cc: Vlas

Re: [PATCH v1 1/5] mm/page_alloc: convert "report" flag of __free_one_page() to a proper flag

2020-10-02 Thread Michal Hocko
ld > be good enough for internal purposes. > > Reviewed-by: Alexander Duyck > Reviewed-by: Vlastimil Babka > Reviewed-by: Oscar Salvador > Cc: Andrew Morton > Cc: Alexander Duyck > Cc: Mel Gorman > Cc: Michal Hocko > Cc: Dave Hansen > Cc: Vlastimil Bab

Re: [PATCH 3/3] memory: introduce an option to force onlining of hotplug memory

2020-07-24 Thread Michal Hocko
map+identity mapping+iomem resource. I think reserving such a > region during boot as suggested is the easiest approach, but I am > *absolutely* not an expert on all these XEN-specific things :) I am late to the discussion but FTR I completely agree. -- Michal Hocko SUSE Labs

Re: [Xen-devel] [PATCH 0/3] Remove __online_page_set_limits()

2019-09-09 Thread Michal Hocko
- > include/linux/memory_hotplug.h | 1 - > mm/memory_hotplug.c| 5 - > 4 files changed, 8 deletions(-) To the whole series Acked-by: Michal Hocko Thanks! -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xen

Re: [Xen-devel] [PATCH 00/34] put_user_pages(): miscellaneous call sites

2019-08-07 Thread Michal Hocko
On Wed 07-08-19 10:37:26, Jan Kara wrote: > On Fri 02-08-19 12:14:09, John Hubbard wrote: > > On 8/2/19 7:52 AM, Jan Kara wrote: > > > On Fri 02-08-19 07:24:43, Matthew Wilcox wrote: > > > > On Fri, Aug 02, 2019 at 02:41:46PM +0200, Jan Kara wrote: > > > >

Re: [Xen-devel] [PATCH 00/34] put_user_pages(): miscellaneous call sites

2019-08-02 Thread Michal Hocko
e changes that would break the balance though. -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] mm: fix regression with deferred struct page init

2019-06-28 Thread Michal Hocko
On Fri 28-06-19 19:38:13, Juergen Gross wrote: > On 28.06.19 17:17, Michal Hocko wrote: > > On Thu 20-06-19 18:08:21, Juergen Gross wrote: > > > Commit 0e56acae4b4dd4a9 ("mm: initialize MAX_ORDER_NR_PAGES at a time > > > instead of doing larger sections&q

Re: [Xen-devel] [PATCH] mm: fix regression with deferred struct page init

2019-06-28 Thread Michal Hocko
t_page_from_freelist() is > assuming forward progress can be made by doing some more struct page > initialization. The patch looks correct. The code is subtle but the comment helps. > Cc: Alexander Duyck > Fixes: 0e56acae4b4dd4a9 ("mm: initialize MAX_ORDER_NR_PAGES at a time instead >

Re: [Xen-devel] [PATCH RFCv2 0/4] mm/memory_hotplug: Introduce memory block types

2018-12-20 Thread Michal Hocko
w, I am doing it just like > > the other types (e.g. online_type) we are using in that context. > > - The "removable" property should never have been named like that. It > > should have been "offlinable". Can we still rename that? E.g. boot memory

Re: [Xen-devel] [PATCH v1] mm/memory_hotplug: drop "online" parameter from add_memory_resource()

2018-11-23 Thread Michal Hocko
On Fri 23-11-18 13:58:16, David Hildenbrand wrote: > On 23.11.18 13:54, Michal Hocko wrote: > > On Fri 23-11-18 13:37:40, David Hildenbrand wrote: > >> User space should always be in charge of how to online memory and > >> if memory should be onlined automatically in

Re: [Xen-devel] [PATCH v1] mm/memory_hotplug: drop "online" parameter from add_memory_resource()

2018-11-23 Thread Michal Hocko
be unaware of the memhp_auto_online leading to a different behavior. > Cc: Boris Ostrovsky > Cc: Juergen Gross > Cc: Stefano Stabellini > Cc: Andrew Morton > Cc: Dan Williams > Cc: Oscar Salvador > Cc: Pavel Tatashin > Cc: Michal Hocko > Cc: David Hildenbrand >

Re: [Xen-devel] [PATCH v1 7/8] PM / Hibernate: use pfn_to_online_page()

2018-11-19 Thread Michal Hocko
Andrew Morton > Cc: Matthew Wilcox > Cc: Michal Hocko > Cc: "Michael S. Tsirkin" > Suggested-by: Michal Hocko > Signed-off-by: David Hildenbrand I have only a very vague understanding of this specific code but I do not really see any real reason for checking offl

Re: [Xen-devel] [PATCH RFC 6/6] PM / Hibernate: exclude all PageOffline() pages

2018-11-15 Thread Michal Hocko
ly do not want to touch offline pfn ranges in general. A separate patch for that of course. Thanks! -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH RFC 2/6] mm: convert PG_balloon to PG_offline

2018-11-15 Thread Michal Hocko
hen virtio-balloon pages were always marked > with PG_balloon. So the documentation is actually wrong ("balloon page" > vs. "balloon compaction page"). > > I have no idea who actually used that information. I suspect this was > just some debugg

Re: [Xen-devel] [PATCH RFC 3/6] kexec: export PG_offline to VMCOREINFO

2018-11-15 Thread Michal Hocko
page state. I am not really sure what is the concern here exactly. Kdump is so closly tight to the specific kernel version that the api exported specifically for its purpose cannot be seriously considered an ABI. Kdump has to adopt all the time. -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] mm: convert totalram_pages, totalhigh_pages and managed_pages to atomic.

2018-10-22 Thread Michal Hocko
s hard to review manually. -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v5 1/2] memory_hotplug: Free pages as higher order

2018-10-19 Thread Michal Hocko
On Thu 18-10-18 19:18:25, Andrew Morton wrote: [...] > So this patch needs more work, yes? Yes, I've talked to Arun (he is offline until next week) offlist and he will play with this some more. -- Michal Hocko SUSE Labs ___ Xen-devel mailing

Re: [Xen-devel] [PATCH v5 1/2] memory_hotplug: Free pages as higher order

2018-10-11 Thread Michal Hocko
> > https://lore.kernel.org/patchwork/patch/989445/ > > Looks like deferred_free_range() has the same problem calling > __free_pages_core() to adjust zone->managed_pages. deferred initialization has one thread per node AFAIR so we cannot race on managed_pages updates. Wel

Re: [Xen-devel] [PATCH v5 1/2] memory_hotplug: Free pages as higher order

2018-10-11 Thread Michal Hocko
On Thu 11-10-18 07:59:32, Arun KS wrote: > On 2018-10-10 23:03, Michal Hocko wrote: > > On Wed 10-10-18 22:26:41, Arun KS wrote: > > > On 2018-10-10 21:00, Vlastimil Babka wrote: > > > > On 10/5/18 10:10 AM, Arun KS wrote: > > > > > When free pa

Re: [Xen-devel] [PATCH v5 1/2] memory_hotplug: Free pages as higher order

2018-10-10 Thread Michal Hocko
_lock. Are we safe without the lock? If yes, there > > should perhaps be a comment explaining why. > > Looks unsafe without managed_page_count_lock. Why does it matter actually? We cannot online/offline memory in parallel. This is not the case for the boot where we initialize memory in par

Re: [Xen-devel] [PATCH v5 1/2] memory_hotplug: Free pages as higher order

2018-10-09 Thread Michal Hocko
On Tue 09-10-18 15:24:13, Arun KS wrote: > On 2018-10-09 14:59, Michal Hocko wrote: > > On Fri 05-10-18 13:40:05, Arun KS wrote: > > > When free pages are done with higher order, time spend on > > > coalescing pages by buddy allocator can be reduced. With > >

Re: [Xen-devel] [PATCH v5 1/2] memory_hotplug: Free pages as higher order

2018-10-09 Thread Michal Hocko
; improving the hot add latency by 60%. Modify external > providers of online callback to align with the change. Acked-by: Michal Hocko Thanks for your patience with all the resubmission. -- Michal Hocko SUSE Labs ___ Xen-devel mail

Re: [Xen-devel] [PATCH v5 2/2] mm/page_alloc: remove software prefetching in __free_pages_core

2018-10-09 Thread Michal Hocko
+, p++) { > - prefetchw(p + 1); > + for (loop = 0; loop < nr_pages ; loop++, p++) { > __ClearPageReserved(p); > set_page_count(p, 0); > } > - __ClearPageReserved(p); > - set_page_count(p, 0); > > page_zone(page)-&

Re: [Xen-devel] [PATCH v4] memory_hotplug: Free pages as higher order

2018-10-04 Thread Michal Hocko
__ClearPageReserved(p); > set_page_count(p, 0); > } > - __ClearPageReserved(p); > - set_page_count(p, 0); > > page_zone(page)->managed_pages += nr_pages; > set_page_refcounted(page); I think this is wort a separate patch

Re: [Xen-devel] [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-03 Thread Michal Hocko
g to enable them anyway (e.g. RAS would require to have movable_node kernel parameters, ballooning a kernel module etc.). Really, one udev script to rule them all will simply never work. -- Michal Hocko SUSE Labs ___ Xen-devel mailing li

Re: [Xen-devel] [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-03 Thread Michal Hocko
ree that allocating that from the zone normal and off node is not great. Especially the second part which is noticeable for whole node hotplug. I have a feeling that arguing about fork not able to proceed or OOMing for the memory hotplug is a bit of a stretch and a sign a of misconfiguration. -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-03 Thread Michal Hocko
d of days here trying to come up with > an onlining solution which would work for everyone and eBPF would move > this decision to distro level. The point is that there is _no_ general onlining solution. This is basically policy which belongs to the userspa

Re: [Xen-devel] [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-03 Thread Michal Hocko
On Tue 02-10-18 17:25:19, David Hildenbrand wrote: > On 02/10/2018 15:47, Michal Hocko wrote: [...] > > Zone imbalance is an inherent problem of the highmem zone. It is > > essentially the highmem zone we all loved so much back in 32b days. > > Yes the movable zone doesn&

Re: [Xen-devel] [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-03 Thread Michal Hocko
On Wed 03-10-18 15:38:04, Vitaly Kuznetsov wrote: > David Hildenbrand writes: > > > On 02/10/2018 15:47, Michal Hocko wrote: > ... > >> > >> Why do you need a generic hotplug rule in the first place? Why don't you > >> simply provide different

Re: [Xen-devel] [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-02 Thread Michal Hocko
On Mon 01-10-18 11:34:25, David Hildenbrand wrote: > On 01/10/2018 10:40, Michal Hocko wrote: > > On Fri 28-09-18 17:03:57, David Hildenbrand wrote: > > [...] > > > > I haven't read the patch itself but I just wanted to note one thing > > about this part &g

Re: [Xen-devel] [PATCH RFC] mm/memory_hotplug: Introduce memory block types

2018-10-01 Thread Michal Hocko
purpose. You said that distributions have hard time to distinguish different types of onlinining policies but isn't this something that is inherently usecase specific? -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org

Re: [Xen-devel] [PATCH v2] memory_hotplug: Free pages as higher order

2018-09-25 Thread Michal Hocko
On Tue 25-09-18 11:59:09, Vlastimil Babka wrote: [...] > This seems like almost complete copy of __free_pages_boot_core(), could > you do some code reuse instead? I think Michal Hocko also suggested that. Yes, please try to reuse as much code as possible -- Michal Hocko SUS

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
about? From f7ac75277d526dccd011f343818dc6af627af2af Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Fri, 24 Aug 2018 15:32:24 +0200 Subject: [PATCH] mm, mmu_notifier: be explicit about range invalition non-blocking mode If invalidate_range_start is called for !blocking mode then all callbacks have to guarantee

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 23:52:25, Tetsuo Handa wrote: > On 2018/08/24 22:32, Michal Hocko wrote: > > On Fri 24-08-18 22:02:23, Tetsuo Handa wrote: > >> I worry that (currently > >> out-of-tree) users of this API are involving work / recursion. > > > > I do not giv

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 15:44:03, Christian König wrote: > Am 24.08.2018 um 15:40 schrieb Michal Hocko: > > On Fri 24-08-18 15:28:33, Christian König wrote: > > > Am 24.08.2018 um 15:24 schrieb Michal Hocko: > > > > On Fri 24-08-18 15:10:08, Christian König wrote: > >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 15:28:33, Christian König wrote: > Am 24.08.2018 um 15:24 schrieb Michal Hocko: > > On Fri 24-08-18 15:10:08, Christian König wrote: > > > Am 24.08.2018 um 15:01 schrieb Michal Hocko: > > > > On Fri 24-08-18 14:52:26, Christian König wrote: > >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 22:02:23, Tetsuo Handa wrote: > On 2018/08/24 20:36, Michal Hocko wrote: > >> That is, this API seems to be currently used by only out-of-tree users. > >> Since > >> we can't check that nobody has memory allocation dependency, I think that >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 15:10:08, Christian König wrote: > Am 24.08.2018 um 15:01 schrieb Michal Hocko: > > On Fri 24-08-18 14:52:26, Christian König wrote: > > > Am 24.08.2018 um 14:33 schrieb Michal Hocko: > > [...] > > > > Thiking about it some more, I can im

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 14:52:26, Christian König wrote: > Am 24.08.2018 um 14:33 schrieb Michal Hocko: [...] > > Thiking about it some more, I can imagine that a notifier callback which > > performs an allocation might trigger a memory reclaim and that in turn > > might trigger a n

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 14:18:44, Christian König wrote: > Am 24.08.2018 um 14:03 schrieb Michal Hocko: > > On Fri 24-08-18 13:57:52, Christian König wrote: > > > Am 24.08.2018 um 13:52 schrieb Michal Hocko: > > > > On Fri 24-08-18 13:43:16, Christian König wrote: > >

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 13:57:52, Christian König wrote: > Am 24.08.2018 um 13:52 schrieb Michal Hocko: > > On Fri 24-08-18 13:43:16, Christian König wrote: [...] > > > That won't work like this there might be multiple > > > invalidate_range_start()/invalidate_ra

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 13:43:16, Christian König wrote: > Am 24.08.2018 um 13:32 schrieb Michal Hocko: > > On Fri 24-08-18 19:54:19, Tetsuo Handa wrote: > > > Two more worries for this patch. > > > > > > > > > > > > > --- a/drivers/gpu/drm

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
pagetables(mirror, > action, > > That is, this API seems to be currently used by only out-of-tree users. Since > we can't check that nobody has memory allocation dependency, I think that > hmm_invalidate_range_start() should return -EAGAIN if blockable == false for > now

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
int amdgpu_mn_read_lock(struct amdgpu_mn *amn, bool blockable) */ static void amdgpu_mn_read_unlock(struct amdgpu_mn *amn) { - if (atomic_dec_return(&amn->recursion) == 0) - up_read_non_owner(&amn->lock); + up_read(&amn->lock); } /** -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/gntdev: fix up blockable calls to mn_invl_range_start

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 07:03:28, Juergen Gross wrote: > On 23/08/18 21:09, Michal Hocko wrote: > > On Thu 23-08-18 10:06:53, Boris Ostrovsky wrote: > >> On 08/23/2018 09:51 AM, Michal Hocko wrote: > >>> On Thu 23-08-18 22:44:07, Tetsuo Handa wrote: > >>>

Re: [Xen-devel] [PATCH] xen/gntdev: fix up blockable calls to mn_invl_range_start

2018-08-23 Thread Michal Hocko
On Thu 23-08-18 10:06:53, Boris Ostrovsky wrote: > On 08/23/2018 09:51 AM, Michal Hocko wrote: > > On Thu 23-08-18 22:44:07, Tetsuo Handa wrote: > >> On 2018/08/23 21:07, Michal Hocko wrote: > >>> diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c > >

Re: [Xen-devel] [PATCH] xen/gntdev: fix up blockable calls to mn_invl_range_start

2018-08-23 Thread Michal Hocko
On Thu 23-08-18 22:44:07, Tetsuo Handa wrote: > On 2018/08/23 21:07, Michal Hocko wrote: > > diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c > > index 57390c7666e5..e7d8bb1bee2a 100644 > > --- a/drivers/xen/gntdev.c > > +++ b/drivers/xen/gntdev.c > >

[Xen-devel] [PATCH] xen/gntdev: fix up blockable calls to mn_invl_range_start

2018-08-23 Thread Michal Hocko
From: Michal Hocko 93065ac753e4 ("mm, oom: distinguish blockable mode for mmu notifiers") has introduced blockable parameter to all mmu_notifiers and the notifier has to back off when called in !blockable case and it could block down the road. The above commit implemente

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-24 Thread Michal Hocko
unction a bit to have unified function exit paths. Suggested-by: Andrew Morton Signed-off-by: Michal Hocko " -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-24 Thread Michal Hocko
oom_reaper: reaped process %d (%s), now anon-rss:%lukB, > file-rss:%lukB, shmem-rss:%lukB\n", > task_pid_nr(tsk), tsk->comm, -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-24 Thread Michal Hocko
K(get_mm_counter(mm, MM_SHMEMPAGES))); +out_finish: + trace_finish_task_reaping(tsk->pid); +out_unlock: up_read(&mm->mmap_sem); - trace_finish_task_reaping(tsk->pid); - return true; + return ret; } #define MAX_OOM_REAP_RETRIES 10 -- Mic

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Fri 20-07-18 16:01:25, Andrew Morton wrote: > On Tue, 17 Jul 2018 10:12:01 +0200 Michal Hocko wrote: > > > > Any suggestions regarding how the driver developers can test this code > > > path? I don't think we presently have a way to fake an oom-killing > &

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Mon 23-07-18 09:11:54, Michal Hocko wrote: > On Mon 23-07-18 09:03:06, Michal Hocko wrote: > > On Fri 20-07-18 17:09:02, Andrew Morton wrote: > > [...] > > > Please take a look? > > > > Are you OK to have these in a separate patch? > > Btw. I will

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Mon 23-07-18 09:03:06, Michal Hocko wrote: > On Fri 20-07-18 17:09:02, Andrew Morton wrote: > [...] > > Please take a look? > > Are you OK to have these in a separate patch? Btw. I will rebase this patch once oom stuff in linux-next settles. At least oom_lock removal fr

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-23 Thread Michal Hocko
On Fri 20-07-18 17:09:02, Andrew Morton wrote: [...] > Please take a look? Are you OK to have these in a separate patch? -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listi

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-19 Thread Michal Hocko
Does anybody see any reasons why this should get into mmotm tree? I do not want to rush this in but if general feeling is to push it for the upcoming merge window then I will not object. -- Michal Hocko SUSE Labs ___ Xen-devel mailing list Xen-devel

Re: [Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-17 Thread Michal Hocko
On Mon 16-07-18 16:12:49, Andrew Morton wrote: > On Mon, 16 Jul 2018 13:50:58 +0200 Michal Hocko wrote: > > > From: Michal Hocko > > > > There are several blockable mmu notifiers which might sleep in > > mmu_notifier_invalidate_range_start and that is a problem

[Xen-devel] [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-16 Thread Michal Hocko
From: Michal Hocko There are several blockable mmu notifiers which might sleep in mmu_notifier_invalidate_range_start and that is a problem for the oom_reaper because it needs to guarantee a forward progress so it cannot depend on any sleepable locks. Currently we simply back off and mark an

Re: [Xen-devel] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-11 Thread Michal Hocko
On Wed 11-07-18 13:14:47, Leon Romanovsky wrote: > On Wed, Jul 11, 2018 at 11:03:53AM +0200, Michal Hocko wrote: > > On Tue 10-07-18 19:20:20, Leon Romanovsky wrote: > > > On Tue, Jul 10, 2018 at 04:14:10PM +0200, Michal Hocko wrote: > > > > On Tue 10-07-18 1

Re: [Xen-devel] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-11 Thread Michal Hocko
On Tue 10-07-18 19:20:20, Leon Romanovsky wrote: > On Tue, Jul 10, 2018 at 04:14:10PM +0200, Michal Hocko wrote: > > On Tue 10-07-18 16:40:40, Leon Romanovsky wrote: > > > On Mon, Jul 09, 2018 at 02:29:08PM +0200, Michal Hocko wrote: > > > > On Wed 27-06-1

Re: [Xen-devel] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-10 Thread Michal Hocko
On Tue 10-07-18 16:40:40, Leon Romanovsky wrote: > On Mon, Jul 09, 2018 at 02:29:08PM +0200, Michal Hocko wrote: > > On Wed 27-06-18 09:44:21, Michal Hocko wrote: > > > This is the v2 of RFC based on the feedback I've received so far. The > > > code even compiles

Re: [Xen-devel] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-09 Thread Michal Hocko
On Wed 27-06-18 09:44:21, Michal Hocko wrote: > This is the v2 of RFC based on the feedback I've received so far. The > code even compiles as a bonus ;) I haven't runtime tested it yet, mostly > because I have no idea how. > > Any further feedback is highly apprecia

Re: [Xen-devel] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-02 Thread Michal Hocko
de would simply back of without releasing any memory. The patch should help to reclaim some memory. > But do you know a way to let the OOM killer kill a specific process? Yes, you can set its oom_score_adj to 1000 which means always select that task.

Re: [Xen-devel] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-02 Thread Michal Hocko
On Mon 02-07-18 14:24:29, Christian König wrote: > Am 02.07.2018 um 14:20 schrieb Michal Hocko: > > On Mon 02-07-18 14:13:42, Christian König wrote: > > > Am 02.07.2018 um 13:54 schrieb Michal Hocko: > > > > On Mon 02-07-18 11:14:58, Christian König wrote: > >

Re: [Xen-devel] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-02 Thread Michal Hocko
On Mon 02-07-18 14:13:42, Christian König wrote: > Am 02.07.2018 um 13:54 schrieb Michal Hocko: > > On Mon 02-07-18 11:14:58, Christian König wrote: > > > Am 27.06.2018 um 09:44 schrieb Michal Hocko: > > > > This is the v2 of RFC based on the feedback I've rec

Re: [Xen-devel] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-07-02 Thread Michal Hocko
On Mon 02-07-18 11:14:58, Christian König wrote: > Am 27.06.2018 um 09:44 schrieb Michal Hocko: > > This is the v2 of RFC based on the feedback I've received so far. The > > code even compiles as a bonus ;) I haven't runtime tested it yet, mostly > > because I

Re: [Xen-devel] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-06-27 Thread Michal Hocko
7 00:00:00 2001 From: Michal Hocko Date: Wed, 20 Jun 2018 15:03:20 +0200 Subject: [PATCH] mm, oom: distinguish blockable mode for mmu notifiers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit There are several blockable mmu notifiers which migh

Re: [Xen-devel] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-06-25 Thread Michal Hocko
On Mon 25-06-18 10:01:03, Michal Hocko wrote: > On Fri 22-06-18 16:09:06, Felix Kuehling wrote: > > On 2018-06-22 11:24 AM, Michal Hocko wrote: > > > On Fri 22-06-18 17:13:02, Christian König wrote: > > >> Hi Michal, > > >> > > >> [Adding Fel

  1   2   >