Re: Pinning ZONE_MOVABLE pages

2020-11-24 Thread David Hildenbrand
On 24.11.20 09:43, Michal Hocko wrote: > On Mon 23-11-20 11:31:59, Pavel Tatashin wrote: > [...] >> Also, we still need to take care of the fault scenario. > > Forgot to reply to this part. I believe you mean this to be fault at gup > time, right? Then the easiest way forward would be to either ad

Re: Pinning ZONE_MOVABLE pages

2020-11-24 Thread Michal Hocko
On Mon 23-11-20 11:31:59, Pavel Tatashin wrote: [...] > Also, we still need to take care of the fault scenario. Forgot to reply to this part. I believe you mean this to be fault at gup time, right? Then the easiest way forward would be to either add yet another scoped flag or (maybe) better to gen

Re: Pinning ZONE_MOVABLE pages

2020-11-24 Thread Michal Hocko
On Mon 23-11-20 11:31:59, Pavel Tatashin wrote: > > Makes sense, as this means no userspace change. > > > > > 2. Add an internal move_pages_zone() similar to move_pages() syscall > > > but instead of migrating to a different NUMA node, migrate pages from > > > ZONE_MOVABLE to another zone. > > > Ca

Re: Pinning ZONE_MOVABLE pages

2020-11-24 Thread Michal Hocko
On Mon 23-11-20 11:06:21, Pavel Tatashin wrote: > On Mon, Nov 23, 2020 at 4:01 AM Michal Hocko wrote: > > > > On Fri 20-11-20 15:27:46, Pavel Tatashin wrote: > > > Recently, I encountered a hang that is happening during memory hot > > > remove operation. It turns out that the hang is caused by pin

Re: Pinning ZONE_MOVABLE pages

2020-11-23 Thread John Hubbard
On 11/20/20 12:27 PM, Pavel Tatashin wrote: Recently, I encountered a hang that is happening during memory hot remove operation. It turns out that the hang is caused by pinned user pages in ZONE_MOVABLE. Kernel expects that all pages in ZONE_MOVABLE can be migrated, but this is not the case if a

Re: Pinning ZONE_MOVABLE pages

2020-11-23 Thread Jason Gunthorpe
On Mon, Nov 23, 2020 at 12:54:16PM -0500, Pavel Tatashin wrote: > > I agree with the other emails, ZONE_MOVABLE needs to be reconciled > > with FOLL_LONGTERM - most likely by preventing ZONE_MOVABLE pages from > > being returned. This will need migration like CMA does and the point > > about faulti

Re: Pinning ZONE_MOVABLE pages

2020-11-23 Thread Pavel Tatashin
On Mon, Nov 23, 2020 at 12:15 PM Jason Gunthorpe wrote: > > On Mon, Nov 23, 2020 at 11:06:21AM -0500, Pavel Tatashin wrote: > > > What I mean here is allowing users to guarantee that the page's PA is > > going to stay the same. Sort of a stronger mlock. Mlock only > > guarantees that the page is n

Re: Pinning ZONE_MOVABLE pages

2020-11-23 Thread Jason Gunthorpe
On Mon, Nov 23, 2020 at 11:06:21AM -0500, Pavel Tatashin wrote: > What I mean here is allowing users to guarantee that the page's PA is > going to stay the same. Sort of a stronger mlock. Mlock only > guarantees that the page is not swapped, but something like You've just described get/pin_user_p

Re: Pinning ZONE_MOVABLE pages

2020-11-23 Thread Pavel Tatashin
> Makes sense, as this means no userspace change. > > > 2. Add an internal move_pages_zone() similar to move_pages() syscall > > but instead of migrating to a different NUMA node, migrate pages from > > ZONE_MOVABLE to another zone. > > Call move_pages_zone() on demand prior to pinning pages from >

Re: Pinning ZONE_MOVABLE pages

2020-11-23 Thread Pavel Tatashin
On Mon, Nov 23, 2020 at 4:01 AM Michal Hocko wrote: > > On Fri 20-11-20 15:27:46, Pavel Tatashin wrote: > > Recently, I encountered a hang that is happening during memory hot > > remove operation. It turns out that the hang is caused by pinned user > > pages in ZONE_MOVABLE. > > > > Kernel expects

Re: Pinning ZONE_MOVABLE pages

2020-11-23 Thread Pavel Tatashin
> > I've spoken with Stephen Hemminger, and he said that DPDK is moving in > > the direction of using transparent huge pages instead of HugeTLBs, > > which means that we need to allow at least anonymous, and anonymous > > transparent huge pages to come from non-movable zones on demand. > > > > I'd

Re: Pinning ZONE_MOVABLE pages

2020-11-23 Thread Vlastimil Babka
+CC John Hubbard On 11/20/20 9:27 PM, Pavel Tatashin wrote: Recently, I encountered a hang that is happening during memory hot remove operation. It turns out that the hang is caused by pinned user pages in ZONE_MOVABLE. Kernel expects that all pages in ZONE_MOVABLE can be migrated, but this is

Re: Pinning ZONE_MOVABLE pages

2020-11-23 Thread Michal Hocko
On Fri 20-11-20 15:27:46, Pavel Tatashin wrote: > Recently, I encountered a hang that is happening during memory hot > remove operation. It turns out that the hang is caused by pinned user > pages in ZONE_MOVABLE. > > Kernel expects that all pages in ZONE_MOVABLE can be migrated, but > this is not

Re: Pinning ZONE_MOVABLE pages

2020-11-22 Thread David Rientjes
On Fri, 20 Nov 2020, Pavel Tatashin wrote: > Recently, I encountered a hang that is happening during memory hot > remove operation. It turns out that the hang is caused by pinned user > pages in ZONE_MOVABLE. > > Kernel expects that all pages in ZONE_MOVABLE can be migrated, but > this is not the

Re: Pinning ZONE_MOVABLE pages

2020-11-20 Thread David Hildenbrand
> Am 20.11.2020 um 22:58 schrieb Pavel Tatashin : > > On Fri, Nov 20, 2020 at 3:59 PM David Hildenbrand wrote: >> >> Am 20.11.2020 um 21:28 schrieb Pavel Tatashin : >>> >>> Recently, I encountered a hang that is happening during memory hot >>> remove operation. It turns out that the h

Re: Pinning ZONE_MOVABLE pages

2020-11-20 Thread Pavel Tatashin
On Fri, Nov 20, 2020 at 3:59 PM David Hildenbrand wrote: > > > > Am 20.11.2020 um 21:28 schrieb Pavel Tatashin : > > > > Recently, I encountered a hang that is happening during memory hot > > remove operation. It turns out that the hang is caused by pinned user > > pages in ZONE_MOVABLE. > > > >

Re: Pinning ZONE_MOVABLE pages

2020-11-20 Thread Pavel Tatashin
On Fri, Nov 20, 2020 at 4:34 PM David Hildenbrand wrote: > > > > Am 20.11.2020 um 22:17 schrieb Matthew Wilcox : > > > > On Fri, Nov 20, 2020 at 09:59:24PM +0100, David Hildenbrand wrote: > >> > Am 20.11.2020 um 21:28 schrieb Pavel Tatashin > : > >>> > >>> Recently, I encountered a ha

Re: Pinning ZONE_MOVABLE pages

2020-11-20 Thread David Hildenbrand
> Am 20.11.2020 um 22:17 schrieb Matthew Wilcox : > > On Fri, Nov 20, 2020 at 09:59:24PM +0100, David Hildenbrand wrote: >> Am 20.11.2020 um 21:28 schrieb Pavel Tatashin : >>> >>> Recently, I encountered a hang that is happening during memory hot >>> remove operation. It turns out that

Re: Pinning ZONE_MOVABLE pages

2020-11-20 Thread Matthew Wilcox
On Fri, Nov 20, 2020 at 09:59:24PM +0100, David Hildenbrand wrote: > > > Am 20.11.2020 um 21:28 schrieb Pavel Tatashin : > > > > Recently, I encountered a hang that is happening during memory hot > > remove operation. It turns out that the hang is caused by pinned user > > pages in ZONE_MOVABLE.

Re: Pinning ZONE_MOVABLE pages

2020-11-20 Thread David Hildenbrand
> Am 20.11.2020 um 21:28 schrieb Pavel Tatashin : > > Recently, I encountered a hang that is happening during memory hot > remove operation. It turns out that the hang is caused by pinned user > pages in ZONE_MOVABLE. > > Kernel expects that all pages in ZONE_MOVABLE can be migrated, but > thi

Pinning ZONE_MOVABLE pages

2020-11-20 Thread Pavel Tatashin
Recently, I encountered a hang that is happening during memory hot remove operation. It turns out that the hang is caused by pinned user pages in ZONE_MOVABLE. Kernel expects that all pages in ZONE_MOVABLE can be migrated, but this is not the case if a user applications such as through dpdk librar