[PATCH v9 5/5] mm/page_alloc: reuse tail struct pages for compound devmaps

2022-04-20 Thread Joao Martins
s a lot smaller than the total amount of struct pages being mapped. The altmap path is left alone since it does not support memory savings based on compound pages devmap. Signed-off-by: Joao Martins Reviewed-by: Muchun Song --- mm/page_alloc.c | 17 - 1 file changed, 16 inserti

[PATCH v9 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps

2022-04-20 Thread Joao Martins
to the already in use vmemmap_populate(). It is worth noting that altmap for devmap mappings was there to relieve the pressure of inordinate amounts of memmap space to map terabytes of pmem. With compound pages the motivation for altmaps for pmem gets reduced. Signed-off-by: Joao Martins Review

[PATCH v9 3/5] mm/hugetlb_vmemmap: move comment block to Documentation/vm

2022-04-20 Thread Joao Martins
In preparation for device-dax for using hugetlbfs compound page tail deduplication technique, move the comment block explanation into a common place in Documentation/vm. Cc: Muchun Song Cc: Mike Kravetz Suggested-by: Dan Williams Signed-off-by: Joao Martins Reviewed-by: Muchun Song Reviewed

[PATCH v9 2/5] mm/sparse-vmemmap: refactor core of vmemmap_populate_basepages() to helper

2022-04-20 Thread Joao Martins
* as return value, rather than an hardcoded errno of 0 or -ENOMEM. Signed-off-by: Joao Martins Reviewed-by: Muchun Song --- mm/sparse-vmemmap.c | 53 ++--- 1 file changed, 36 insertions(+), 17 deletions(-) diff --git a/mm/sparse-vmemmap.c b/mm/sparse

[PATCH v9 0/5] sparse-vmemmap: memory savings for compound devmaps (device-dax)

2022-04-20 Thread Joao Martins
tps://lore.kernel.org/linux-mm/20220210193345.23628-1-joao.m.mart...@oracle.com/ [6] https://lore.kernel.org/linux-mm/20210827145819.16471-15-joao.m.mart...@oracle.com/ [7] https://lore.kernel.org/linux-mm/20220223194807.12070-1-joao.m.mart...@oracle.com/ [8] https://lore.kernel.org/linux-mm/

[PATCH v9 1/5] mm/sparse-vmemmap: add a pgmap argument to section activation

2022-04-20 Thread Joao Martins
/ Signed-off-by: Joao Martins Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- include/linux/memory_hotplug.h | 5 - include/linux/mm.h | 3 ++- mm/memory_hotplug.c| 3 ++- mm/sparse-vmemmap.c| 3 ++- mm/sparse.c| 26

Re: [PATCH v8 3/5] mm/hugetlb_vmemmap: move comment block to Documentation/vm

2022-03-10 Thread Joao Martins
On 3/10/22 10:32, Mike Rapoport wrote: > Hi, > > On Mon, Mar 07, 2022 at 12:24:55PM +0000, Joao Martins wrote: >> In preparation for device-dax for using hugetlbfs compound page tail >> deduplication technique, move the comment block explanation into a >> commo

[PATCH v8 1/5] mm/sparse-vmemmap: add a pgmap argument to section activation

2022-03-07 Thread Joao Martins
/ Signed-off-by: Joao Martins Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- include/linux/memory_hotplug.h | 5 - include/linux/mm.h | 3 ++- mm/memory_hotplug.c| 3 ++- mm/sparse-vmemmap.c| 3 ++- mm/sparse.c| 26

[PATCH v8 5/5] mm/page_alloc: reuse tail struct pages for compound devmaps

2022-03-07 Thread Joao Martins
s a lot smaller than the total amount of struct pages being mapped. The altmap path is left alone since it does not support memory savings based on compound pages devmap. Signed-off-by: Joao Martins Reviewed-by: Muchun Song --- mm/page_alloc.c | 17 - 1 file changed, 16 inserti

[PATCH v8 3/5] mm/hugetlb_vmemmap: move comment block to Documentation/vm

2022-03-07 Thread Joao Martins
In preparation for device-dax for using hugetlbfs compound page tail deduplication technique, move the comment block explanation into a common place in Documentation/vm. Cc: Muchun Song Cc: Mike Kravetz Suggested-by: Dan Williams Signed-off-by: Joao Martins Reviewed-by: Muchun Song Reviewed

[PATCH v8 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps

2022-03-07 Thread Joao Martins
to the already in use vmemmap_populate(). It is worth noting that altmap for devmap mappings was there to relieve the pressure of inordinate amounts of memmap space to map terabytes of pmem. With compound pages the motivation for altmaps for pmem gets reduced. Signed-off-by: Joao Martins Review

[PATCH v8 0/5] sparse-vmemmap: memory savings for compound devmaps (device-dax)

2022-03-07 Thread Joao Martins
57-1-joao.m.mart...@oracle.com/ [4] https://lore.kernel.org/linux-mm/20210827145819.16471-1-joao.m.mart...@oracle.com/ [5] https://lore.kernel.org/linux-mm/20220210193345.23628-1-joao.m.mart...@oracle.com/ [6] https://lore.kernel.org/linux-mm/20210827145819.16471-15-joao.m.mart...@oracle.com/ [

[PATCH v8 2/5] mm/sparse-vmemmap: refactor core of vmemmap_populate_basepages() to helper

2022-03-07 Thread Joao Martins
* as return value, rather than an hardcoded errno of 0 or -ENOMEM. Signed-off-by: Joao Martins Reviewed-by: Muchun Song --- mm/sparse-vmemmap.c | 53 ++--- 1 file changed, 36 insertions(+), 17 deletions(-) diff --git a/mm/sparse-vmemmap.c b/mm/sparse

Re: [PATCH v7 3/5] mm/hugetlb_vmemmap: move comment block to Documentation/vm

2022-03-05 Thread Joao Martins
On 3/4/22 15:21, Jonathan Corbet wrote: > Joao Martins writes: > >> In preparation for device-dax for using hugetlbfs compound page tail >> deduplication technique, move the comment block explanation into a >> common place in Documentation/vm. >> >>

Re: [PATCH v7 5/5] mm/page_alloc: reuse tail struct pages for compound devmaps

2022-03-04 Thread Joao Martins
On 3/4/22 03:27, Muchun Song wrote: > On Fri, Mar 4, 2022 at 5:33 AM Joao Martins wrote: >> >> Currently memmap_init_zone_device() ends up initializing 32768 pages >> when it only needs to initialize 128 given tail page reuse. That >> number is worse with 1GB compound p

Re: [PATCH v7 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps

2022-03-04 Thread Joao Martins
On 3/4/22 03:09, Muchun Song wrote: > On Fri, Mar 4, 2022 at 5:33 AM Joao Martins wrote: >> A compound devmap is a dev_pagemap with @vmemmap_shift > 0 and it >> means that pages are mapped at a given huge page alignment and utilize >> uses compound pages as opposed to o

[PATCH v7 3/5] mm/hugetlb_vmemmap: move comment block to Documentation/vm

2022-03-03 Thread Joao Martins
In preparation for device-dax for using hugetlbfs compound page tail deduplication technique, move the comment block explanation into a common place in Documentation/vm. Cc: Muchun Song Cc: Mike Kravetz Suggested-by: Dan Williams Signed-off-by: Joao Martins Reviewed-by: Muchun Song Reviewed

[PATCH v7 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps

2022-03-03 Thread Joao Martins
to the already in use vmemmap_populate(). It is worth noting that altmap for devmap mappings was there to relieve the pressure of inordinate amounts of memmap space to map terabytes of pmem. With compound pages the motivation for altmaps for pmem gets reduced. Signed-off-by: Joao Martins ---

[PATCH v7 5/5] mm/page_alloc: reuse tail struct pages for compound devmaps

2022-03-03 Thread Joao Martins
s a lot smaller than the total amount of struct pages being mapped. The altmap path is left alone since it does not support memory savings based on compound pages devmap. Signed-off-by: Joao Martins --- mm/page_alloc.c | 17 - 1 file changed, 16 insertions(+), 1 deletion(-) d

[PATCH v7 2/5] mm/sparse-vmemmap: refactor core of vmemmap_populate_basepages() to helper

2022-03-03 Thread Joao Martins
* as return value, rather than an hardcoded errno of 0 or -ENOMEM. Signed-off-by: Joao Martins Reviewed-by: Muchun Song --- mm/sparse-vmemmap.c | 53 ++--- 1 file changed, 36 insertions(+), 17 deletions(-) diff --git a/mm/sparse-vmemmap.c b/mm/sparse

[PATCH v7 1/5] mm/sparse-vmemmap: add a pgmap argument to section activation

2022-03-03 Thread Joao Martins
/ Signed-off-by: Joao Martins Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- include/linux/memory_hotplug.h | 5 - include/linux/mm.h | 3 ++- mm/memory_hotplug.c| 3 ++- mm/sparse-vmemmap.c| 3 ++- mm/sparse.c| 26

[PATCH v7 0/5] sparse-vmemmap: memory savings for compound devmaps (device-dax)

2022-03-03 Thread Joao Martins
cle.com/ [5] https://lore.kernel.org/linux-mm/20220210193345.23628-1-joao.m.mart...@oracle.com/ [6] https://lore.kernel.org/linux-mm/20210827145819.16471-15-joao.m.mart...@oracle.com/ [7] https://lore.kernel.org/linux-mm/20220223194807.12070-1-joao.m.mart...@oracle.com/ Joao Martins (5): mm/sp

Re: [PATCH v6 5/5] mm/page_alloc: reuse tail struct pages for compound devmaps

2022-02-24 Thread Joao Martins
On 2/24/22 15:41, Muchun Song wrote: > On Thu, Feb 24, 2022 at 3:48 AM Joao Martins > wrote: >> >> Currently memmap_init_zone_device() ends up initializing 32768 pages >> when it only needs to initialize 128 given tail page reuse. That >> number is worse with 1GB

Re: [PATCH v6 5/5] mm/page_alloc: reuse tail struct pages for compound devmaps

2022-02-24 Thread Joao Martins
On 2/24/22 05:57, Muchun Song wrote: > On Thu, Feb 24, 2022 at 3:48 AM Joao Martins > wrote: >> --- a/mm/page_alloc.c >> +++ b/mm/page_alloc.c >> @@ -6653,6 +6653,20 @@ static void __ref __init_zone_device_page(struct page >> *page, unsigned long pfn, >>

Re: [PATCH v6 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps

2022-02-24 Thread Joao Martins
On 2/24/22 05:54, Muchun Song wrote: > On Thu, Feb 24, 2022 at 3:48 AM Joao Martins > wrote: >> diff --git a/include/linux/mm.h b/include/linux/mm.h >> index 5f549cf6a4e8..b0798b9c6a6a 100644 >> --- a/include/linux/mm.h >> +++ b/include/linux/mm.h

Re: [PATCH v6 2/5] mm/sparse-vmemmap: refactor core of vmemmap_populate_basepages() to helper

2022-02-24 Thread Joao Martins
On 2/24/22 03:10, Muchun Song wrote: > On Thu, Feb 24, 2022 at 3:48 AM Joao Martins > wrote: >> >> In preparation for describing a memmap with compound pages, move the >> actual pte population logic into a separate function >> vmemmap_populate_address() and have vme

Re: [PATCH v6 1/5] mm/sparse-vmemmap: add a pgmap argument to section activation

2022-02-24 Thread Joao Martins
On 2/24/22 03:02, Muchun Song wrote: > On Thu, Feb 24, 2022 at 3:48 AM Joao Martins > wrote: >> >> In support of using compound pages for devmap mappings, plumb the pgmap >> down to the vmemmap_populate implementation. Note that while altmap is >> retrievable

[PATCH v6 3/5] mm/hugetlb_vmemmap: move comment block to Documentation/vm

2022-02-23 Thread Joao Martins
In preparation for device-dax for using hugetlbfs compound page tail deduplication technique, move the comment block explanation into a common place in Documentation/vm. Cc: Muchun Song Cc: Mike Kravetz Suggested-by: Dan Williams Signed-off-by: Joao Martins Reviewed-by: Muchun Song Reviewed

[PATCH v6 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps

2022-02-23 Thread Joao Martins
to the already in use vmemmap_populate(). It is worth noting that altmap for devmap mappings was there to relieve the pressure of inordinate amounts of memmap space to map terabytes of pmem. With compound pages the motivation for altmaps for pmem gets reduced. Signed-off-by: Joao Martins ---

[PATCH v6 1/5] mm/sparse-vmemmap: add a pgmap argument to section activation

2022-02-23 Thread Joao Martins
/ Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- include/linux/memory_hotplug.h | 5 - include/linux/mm.h | 3 ++- mm/memory_hotplug.c| 3 ++- mm/sparse-vmemmap.c| 3 ++- mm/sparse.c| 26 -- 5 files

[PATCH v6 2/5] mm/sparse-vmemmap: refactor core of vmemmap_populate_basepages() to helper

2022-02-23 Thread Joao Martins
return value, rather than an hardcoded errno of 0 or -ENOMEM. Signed-off-by: Joao Martins --- mm/sparse-vmemmap.c | 46 - 1 file changed, 29 insertions(+), 17 deletions(-) diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index c506f77cff23

[PATCH v6 0/5] sparse-vmemmap: memory savings for compound devmaps (device-dax)

2022-02-23 Thread Joao Martins
ernel.org/linux-mm/20210617184507.3662-1-joao.m.mart...@oracle.com/ [3] https://lore.kernel.org/linux-mm/20210714193542.21857-1-joao.m.mart...@oracle.com/ [4] https://lore.kernel.org/linux-mm/20210827145819.16471-1-joao.m.mart...@oracle.com/ [5] https://lore.kernel.org/linux-mm/20220210193345.

[PATCH v6 5/5] mm/page_alloc: reuse tail struct pages for compound devmaps

2022-02-23 Thread Joao Martins
s a lot smaller that the total amount of struct pages being mapped. The altmap path is left alone since it does not support memory savings based on compound pages devmap. Signed-off-by: Joao Martins --- mm/page_alloc.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --

Re: [PATCH v5 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps

2022-02-14 Thread Joao Martins
On 2/12/22 10:08, Muchun Song wrote: > On Fri, Feb 11, 2022 at 8:37 PM Joao Martins > wrote: >> On 2/11/22 07:54, Muchun Song wrote: >>> On Fri, Feb 11, 2022 at 3:34 AM Joao Martins >>> wrote: >>>> @@ -609,7 +624,8 @@ pgd_t * __meminit vmemmap_pgd_p

Re: [PATCH v5 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps

2022-02-14 Thread Joao Martins
On 2/12/22 14:49, Muchun Song wrote: > On Sat, Feb 12, 2022 at 6:08 PM Muchun Song wrote: >> On Fri, Feb 11, 2022 at 8:37 PM Joao Martins >> wrote: >>> On 2/11/22 07:54, Muchun Song wrote: >>>> On Fri, Feb 11, 2022 at 3:34 AM Joao Martins >>>>

Re: [PATCH v5 5/5] mm/page_alloc: reuse tail struct pages for compound devmaps

2022-02-11 Thread Joao Martins
On 2/11/22 05:07, Muchun Song wrote: > On Fri, Feb 11, 2022 at 3:34 AM Joao Martins > wrote: >> diff --git a/mm/page_alloc.c b/mm/page_alloc.c >> index cface1d38093..c10df2fd0ec2 100644 >> --- a/mm/page_alloc.c >> +++ b/mm/page_alloc.c >> @@ -

Re: [PATCH v5 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps

2022-02-11 Thread Joao Martins
On 2/11/22 07:54, Muchun Song wrote: > On Fri, Feb 11, 2022 at 3:34 AM Joao Martins > wrote: > [...] >> pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int >> node, >> -

Re: [PATCH v5 1/5] mm/sparse-vmemmap: add a pgmap argument to section activation

2022-02-11 Thread Joao Martins
On 2/11/22 08:03, Muchun Song wrote: > On Fri, Feb 11, 2022 at 3:34 AM Joao Martins > wrote: >> >> In support of using compound pages for devmap mappings, plumb the pgmap >> down to the vmemmap_populate implementation. Note that while altmap is >> retrievable from

[PATCH v5 5/5] mm/page_alloc: reuse tail struct pages for compound devmaps

2022-02-10 Thread Joao Martins
s a lot smaller that the total amount of struct pages being mapped. The altmap path is left alone since it does not support memory savings based on compound pages devmap. Signed-off-by: Joao Martins --- mm/page_alloc.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --

[PATCH v5 3/5] mm/hugetlb_vmemmap: move comment block to Documentation/vm

2022-02-10 Thread Joao Martins
In preparation for device-dax for using hugetlbfs compound page tail deduplication technique, move the comment block explanation into a common place in Documentation/vm. Cc: Muchun Song Cc: Mike Kravetz Suggested-by: Dan Williams Signed-off-by: Joao Martins Reviewed-by: Muchun Song Reviewed

[PATCH v5 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps

2022-02-10 Thread Joao Martins
to the already in use vmemmap_populate(). It is worth noting that altmap for devmap mappings was there to relieve the pressure of inordinate amounts of memmap space to map terabytes of pmem. With compound pages the motivation for altmaps for pmem gets reduced. Signed-off-by: Joao Martins ---

[PATCH v5 2/5] mm/sparse-vmemmap: refactor core of vmemmap_populate_basepages() to helper

2022-02-10 Thread Joao Martins
In preparation for describing a memmap with compound pages, move the actual pte population logic into a separate function vmemmap_populate_address() and have vmemmap_populate_basepages() walk through all base pages it needs to populate. Signed-off-by: Joao Martins --- mm/sparse-vmemmap.c | 51

[PATCH v5 0/5] sparse-vmemmap: memory savings for compound devmaps (device-dax)

2022-02-10 Thread Joao Martins
@oracle.com/ [3] https://lore.kernel.org/linux-mm/20210617184507.3662-1-joao.m.mart...@oracle.com/ [4] https://lore.kernel.org/linux-mm/20210827145819.16471-1-joao.m.mart...@oracle.com/ Joao Martins (5): mm/sparse-vmemmap: add a pgmap argument to section activation mm/sparse-vmemmap: refactor cor

[PATCH v5 1/5] mm/sparse-vmemmap: add a pgmap argument to section activation

2022-02-10 Thread Joao Martins
/ Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- include/linux/memory_hotplug.h | 5 - include/linux/mm.h | 3 ++- mm/memory_hotplug.c| 3 ++- mm/sparse-vmemmap.c| 3 ++- mm/sparse.c| 26 -- 5 files

[PATCH v7 11/11] device-dax: compound devmap support

2021-12-02 Thread Joao Martins
art...@oracle.com/ Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- drivers/dax/device.c | 9 + 1 file changed, 9 insertions(+) diff --git a/drivers/dax/device.c b/drivers/dax/device.c index 914368164e05..6ef8f374e27b 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c

[PATCH v7 10/11] device-dax: remove pfn from __dev_dax_{pte,pmd,pud}_fault()

2021-12-02 Thread Joao Martins
After moving the page mapping to be set prior to pte insertion, the pfn in dev_dax_huge_fault() no longer is necessary. Remove it, as well as the @pfn argument passed to the internal fault handler helpers. Suggested-by: Christoph Hellwig Signed-off-by: Joao Martins --- drivers/dax/device.c

[PATCH v7 09/11] device-dax: set mapping prior to vmf_insert_pfn{,_pmd,pud}()

2021-12-02 Thread Joao Martins
page mapping is set once but cleared when the struct pages are removed/freed (i.e. after {devm_}memunmap_pages()). Suggested-by: Jason Gunthorpe Signed-off-by: Joao Martins --- drivers/dax/device.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/dax

[PATCH v7 08/11] device-dax: factor out page mapping initialization

2021-12-02 Thread Joao Martins
Move initialization of page->mapping into a separate helper. This is in preparation to move the mapping set to be prior to inserting the page table entry and also for tidying up compound page handling into one helper. Signed-off-by: Joao Martins --- drivers/dax/device.c |

[PATCH v7 07/11] device-dax: ensure dev_dax->pgmap is valid for dynamic devices

2021-12-02 Thread Joao Martins
take the opportunity to document the differences between static and dynamic da regions. Suggested-by: Dan Williams Signed-off-by: Joao Martins --- drivers/dax/bus.c| 32 drivers/dax/bus.h| 1 + drivers/dax/device.c | 29 + 3

[PATCH v7 05/11] device-dax: use ALIGN() for determining pgoff

2021-12-02 Thread Joao Martins
Rather than calculating @pgoff manually, switch to ALIGN() instead. Suggested-by: Dan Williams Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- drivers/dax/device.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c

[PATCH v7 03/11] mm/page_alloc: refactor memmap_init_zone_device() page init

2021-12-02 Thread Joao Martins
Move struct page init to an helper function __init_zone_device_page(). This is in preparation for sharing the storage for compound page metadata. Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- mm/page_alloc.c | 74 +++-- 1 file changed

[PATCH v7 06/11] device-dax: use struct_size()

2021-12-02 Thread Joao Martins
Use the struct_size() helper for the size of a struct with variable array member at the end, rather than manually calculating it. Suggested-by: Dan Williams Signed-off-by: Joao Martins --- drivers/dax/device.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/dax

[PATCH v7 00/11] mm, device-dax: Introduce compound pages in devmap

2021-12-02 Thread Joao Martins
https://lore.kernel.org/linux-mm/20211018182559.gc3686...@ziepe.ca/ [6] https://lore.kernel.org/linux-mm/499043a0-b3d8-7a42-4aee-84b81f5b6...@oracle.com/ [7] https://lore.kernel.org/linux-mm/20210827145819.16471-9-joao.m.mart...@oracle.com/ [8] https://lore.kernel.org/linux-mm/20210827145819.16

[PATCH v7 01/11] memory-failure: fetch compound_head after pgmap_pfn_valid()

2021-12-02 Thread Joao Martins
memory_failure_dev_pagemap() to keep working. Reported-by: Jane Chu Signed-off-by: Joao Martins Reviewed-by: Naoya Horiguchi Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- mm/memory-failure.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index

[PATCH v7 02/11] mm/page_alloc: split prep_compound_page into head and tail subparts

2021-12-02 Thread Joao Martins
Split the utility function prep_compound_page() into head and tail counterparts, and use them accordingly. This is in preparation for sharing the storage for compound page metadata. Signed-off-by: Joao Martins Acked-by: Mike Kravetz Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- mm

[PATCH v7 04/11] mm/memremap: add ZONE_DEVICE support for compound pages

2021-12-02 Thread Joao Martins
_page") removed set_page_count() because the setting of page ref count to zero was redundant. devmap pages don't come from page allocator though and only head page refcount is used for compound pages, hence initialize tail page count to zero. Signed-off-by: Joao Martins Reviewed-by: Dan William

Re: [PATCH v6 09/10] device-dax: set mapping prior to vmf_insert_pfn{,_pmd,pud}()

2021-11-29 Thread Joao Martins
On 11/29/21 15:49, Joao Martins wrote: > On 11/29/21 07:32, Christoph Hellwig wrote: >> On Fri, Nov 26, 2021 at 06:39:39PM +0000, Joao Martins wrote: >> Aso it seems like pfn is only an input >> parameter now and doesn't need to be passed by reference. >> > It&

Re: [PATCH v6 09/10] device-dax: set mapping prior to vmf_insert_pfn{,_pmd,pud}()

2021-11-29 Thread Joao Martins
On 11/29/21 07:32, Christoph Hellwig wrote: > On Fri, Nov 26, 2021 at 06:39:39PM +0000, Joao Martins wrote: >> @@ -230,23 +235,18 @@ static vm_fault_t dev_dax_huge_fault(struct vm_fault >> *vmf, >> id = dax_read_lock(); >> switch (pe_size)

Re: [PATCH v6 09/10] device-dax: set mapping prior to vmf_insert_pfn{,_pmd,pud}()

2021-11-26 Thread Joao Martins
On 11/25/21 11:42, Joao Martins wrote: > On 11/24/21 19:10, Joao Martins wrote: >> @@ -245,8 +251,6 @@ static vm_fault_t dev_dax_huge_fault(struct vm_fault >> *vmf, >> rc = VM_FAULT_SIGBUS; >> } >> >> -if (rc == VM_FAULT_NOPAGE) &

Re: [PATCH v6 09/10] device-dax: set mapping prior to vmf_insert_pfn{,_pmd,pud}()

2021-11-25 Thread Joao Martins
On 11/24/21 19:10, Joao Martins wrote: > Normally, the @page mapping is set prior to inserting the page into a > page table entry. Make device-dax adhere to the same ordering, rather > than setting mapping after the PTE is inserted. > > The address_space never changes and it is al

Re: [PATCH v6 04/10] mm/memremap: add ZONE_DEVICE support for compound pages

2021-11-25 Thread Joao Martins
On 11/25/21 06:11, Christoph Hellwig wrote: > On Wed, Nov 24, 2021 at 07:09:59PM +0000, Joao Martins wrote: >> Add a new @vmemmap_shift property for struct dev_pagemap which specifies >> that a >> devmap is composed of a set of compound pages of order @vmemmap_shift, >&g

[PATCH v6 10/10] device-dax: compound devmap support

2021-11-24 Thread Joao Martins
art...@oracle.com/ Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- drivers/dax/device.c | 9 + 1 file changed, 9 insertions(+) diff --git a/drivers/dax/device.c b/drivers/dax/device.c index 0ef9fecec005..9b51108aea91 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c

[PATCH v6 09/10] device-dax: set mapping prior to vmf_insert_pfn{,_pmd,pud}()

2021-11-24 Thread Joao Martins
page mapping is set once but cleared when the struct pages are removed/freed (i.e. after {devm_}memunmap_pages()). Suggested-by: Jason Gunthorpe Signed-off-by: Joao Martins --- drivers/dax/device.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/dax/device.c

[PATCH v6 06/10] device-dax: use struct_size()

2021-11-24 Thread Joao Martins
Use the struct_size() helper for the size of a struct with variable array member at the end, rather than manually calculating it. Suggested-by: Dan Williams Signed-off-by: Joao Martins --- drivers/dax/device.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/dax

[PATCH v6 08/10] device-dax: factor out page mapping initialization

2021-11-24 Thread Joao Martins
Move initialization of page->mapping into a separate helper. This is in preparation to move the mapping set to be prior to inserting the page table entry and also for tidying up compound page handling into one helper. Signed-off-by: Joao Martins --- drivers/dax/device.c |

[PATCH v6 07/10] device-dax: ensure dev_dax->pgmap is valid for dynamic devices

2021-11-24 Thread Joao Martins
take the opportunity to document the differences between static and dynamic da regions. Suggested-by: Dan Williams Signed-off-by: Joao Martins --- drivers/dax/bus.c| 32 drivers/dax/bus.h| 1 + drivers/dax/device.c | 29 + 3

[PATCH v6 05/10] device-dax: use ALIGN() for determining pgoff

2021-11-24 Thread Joao Martins
Rather than calculating @pgoff manually, switch to ALIGN() instead. Suggested-by: Dan Williams Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- drivers/dax/device.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c

[PATCH v6 03/10] mm/page_alloc: refactor memmap_init_zone_device() page init

2021-11-24 Thread Joao Martins
Move struct page init to an helper function __init_zone_device_page(). This is in preparation for sharing the storage for compound page metadata. Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- mm/page_alloc.c | 74 +++-- 1 file changed

[PATCH v6 00/10] mm, device-dax: Introduce compound pages in devmap

2021-11-24 Thread Joao Martins
@oracle.com/ [7] https://lore.kernel.org/linux-mm/20210827145819.16471-9-joao.m.mart...@oracle.com/ [8] https://lore.kernel.org/linux-mm/20210827145819.16471-13-joao.m.mart...@oracle.com/ [9] https://lore.kernel.org/linux-mm/2022150824.11028-1-joao.m.mart...@oracle.com/ Joao Martins (10)

[PATCH v6 04/10] mm/memremap: add ZONE_DEVICE support for compound pages

2021-11-24 Thread Joao Martins
_page") removed set_page_count() because the setting of page ref count to zero was redundant. devmap pages don't come from page allocator though and only head page refcount is used for compound pages, hence initialize tail page count to zero. Signed-off-by: Joao Martins Reviewed-by: Dan William

[PATCH v6 02/10] mm/page_alloc: split prep_compound_page into head and tail subparts

2021-11-24 Thread Joao Martins
Split the utility function prep_compound_page() into head and tail counterparts, and use them accordingly. This is in preparation for sharing the storage for compound page metadata. Signed-off-by: Joao Martins Acked-by: Mike Kravetz Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- mm

[PATCH v6 01/10] memory-failure: fetch compound_head after pgmap_pfn_valid()

2021-11-24 Thread Joao Martins
memory_failure_dev_pagemap() to keep working. Reported-by: Jane Chu Signed-off-by: Joao Martins Reviewed-by: Naoya Horiguchi Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- mm/memory-failure.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index

Re: [PATCH v5 8/8] device-dax: compound devmap support

2021-11-19 Thread Joao Martins
On 11/19/21 19:53, Jason Gunthorpe wrote: > On Fri, Nov 19, 2021 at 07:26:44PM +0000, Joao Martins wrote: >> On 11/19/21 16:55, Jason Gunthorpe wrote: >>> On Fri, Nov 19, 2021 at 04:12:18PM +, Joao Martins wrote: >>> >>>>> Dan, any thoughts (se

Re: [PATCH v5 8/8] device-dax: compound devmap support

2021-11-19 Thread Joao Martins
On 11/19/21 16:55, Jason Gunthorpe wrote: > On Fri, Nov 19, 2021 at 04:12:18PM +0000, Joao Martins wrote: > >>> Dan, any thoughts (see also below) ? You probably hold all that >>> history since its inception on commit 2232c6382a4 ("device-dax: Enable >&g

Re: [PATCH v5 8/8] device-dax: compound devmap support

2021-11-19 Thread Joao Martins
On 11/16/21 16:38, Joao Martins wrote: > On 11/15/21 17:49, Jason Gunthorpe wrote: >> On Mon, Nov 15, 2021 at 01:11:32PM +0100, Joao Martins wrote: >>> On 11/12/21 16:34, Jason Gunthorpe wrote: >>>> On Fri, Nov 12, 2021 at 04:08:24PM +0100, Joao Martins wrote:

Re: [PATCH v5 8/8] device-dax: compound devmap support

2021-11-17 Thread Joao Martins
On 11/17/21 10:43, Christoph Hellwig wrote: > On Fri, Nov 12, 2021 at 04:08:24PM +0100, Joao Martins wrote: >> Use the newly added compound devmap facility which maps the assigned dax >> ranges as compound pages at a page size of @align. >> >> dax devices are created wit

Re: [PATCH v5 7/8] device-dax: ensure dev_dax->pgmap is valid for dynamic devices

2021-11-17 Thread Joao Martins
On 11/17/21 10:37, Christoph Hellwig wrote: >> +bool static_dev_dax(struct dev_dax *dev_dax) >> +{ >> +return is_static(dev_dax->region); >> +} >> +EXPORT_SYMBOL_GPL(static_dev_dax); > > This function would massively benefit from documentic what a static > DAX region is and why someone woul

Re: [PATCH v5 6/8] device-dax: use struct_size()

2021-11-17 Thread Joao Martins
On 11/17/21 10:37, Christoph Hellwig wrote: >> +pgmap = devm_kzalloc( >> + dev, struct_size(pgmap, ranges, dev_dax->nr_range - >> 1), >> + GFP_KERNEL); > > Keeping the dev argument on the previous line would not only make this > much more re

Re: [PATCH v5 8/8] device-dax: compound devmap support

2021-11-16 Thread Joao Martins
On 11/15/21 17:49, Jason Gunthorpe wrote: > On Mon, Nov 15, 2021 at 01:11:32PM +0100, Joao Martins wrote: >> On 11/12/21 16:34, Jason Gunthorpe wrote: >>> On Fri, Nov 12, 2021 at 04:08:24PM +0100, Joao Martins wrote: >>>> diff --git a/drivers/dax/device.c b/d

Re: [PATCH v5 8/8] device-dax: compound devmap support

2021-11-15 Thread Joao Martins
On 11/12/21 16:34, Jason Gunthorpe wrote: > On Fri, Nov 12, 2021 at 04:08:24PM +0100, Joao Martins wrote: > >> diff --git a/drivers/dax/device.c b/drivers/dax/device.c >> index a65c67ab5ee0..0c2ac97d397d 100644 >> +++ b/drivers/dax/device.c >> @@ -192,

Re: [PATCH v5 0/8] mm, dax: Introduce compound pages in devmap

2021-11-15 Thread Joao Martins
On 11/12/21 16:40, Jason Gunthorpe wrote: > On Fri, Nov 12, 2021 at 04:08:16PM +0100, Joao Martins wrote: > >> This series converts device-dax to use compound pages, and moves away from >> the >> 'struct page per basepage on PMD/PUD' that is done today. Doi

[PATCH v5 8/8] device-dax: compound devmap support

2021-11-12 Thread Joao Martins
art...@oracle.com/ Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- drivers/dax/device.c | 57 ++-- 1 file changed, 44 insertions(+), 13 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c index a65c67ab5ee0..0c2ac97d397d 100644

[PATCH v5 7/8] device-dax: ensure dev_dax->pgmap is valid for dynamic devices

2021-11-12 Thread Joao Martins
ested-by: Dan Williams Signed-off-by: Joao Martins --- drivers/dax/bus.c| 14 ++ drivers/dax/bus.h| 1 + drivers/dax/device.c | 26 +++--- 3 files changed, 34 insertions(+), 7 deletions(-) diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index 6cc4da4

[PATCH v5 5/8] device-dax: use ALIGN() for determining pgoff

2021-11-12 Thread Joao Martins
Rather than calculating @pgoff manually, switch to ALIGN() instead. Suggested-by: Dan Williams Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- drivers/dax/device.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c

[PATCH v5 6/8] device-dax: use struct_size()

2021-11-12 Thread Joao Martins
Use the struct_size() helper for the size of a struct with variable array member at the end, rather than manually calculating it. Suggested-by: Dan Williams Signed-off-by: Joao Martins --- drivers/dax/device.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/dax

[PATCH v5 0/8] mm, dax: Introduce compound pages in devmap

2021-11-12 Thread Joao Martins
mm/20210827145819.16471-1-joao.m.mart...@oracle.com/ [5] https://lore.kernel.org/linux-mm/20211018182559.gc3686...@ziepe.ca/ [6] https://lore.kernel.org/linux-mm/499043a0-b3d8-7a42-4aee-84b81f5b6...@oracle.com/ [7] https://lore.kernel.org/linux-mm/20210827145819.16471-9-joao.m.mart...@oracl

[PATCH v5 4/8] mm/memremap: add ZONE_DEVICE support for compound pages

2021-11-12 Thread Joao Martins
_page") removed set_page_count() because the setting of page ref count to zero was redundant. devmap pages don't come from page allocator though and only head page refcount is used for compound pages, hence initialize tail page count to zero. Signed-off-by: Joao Martins Reviewed-by: Dan William

[PATCH v5 1/8] memory-failure: fetch compound_head after pgmap_pfn_valid()

2021-11-12 Thread Joao Martins
memory_failure_dev_pagemap() to keep working. Reported-by: Jane Chu Signed-off-by: Joao Martins Reviewed-by: Naoya Horiguchi Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- mm/memory-failure.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index

[PATCH v5 2/8] mm/page_alloc: split prep_compound_page into head and tail subparts

2021-11-12 Thread Joao Martins
Split the utility function prep_compound_page() into head and tail counterparts, and use them accordingly. This is in preparation for sharing the storage for compound page metadata. Signed-off-by: Joao Martins Acked-by: Mike Kravetz Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- mm

[PATCH v5 3/8] mm/page_alloc: refactor memmap_init_zone_device() page init

2021-11-12 Thread Joao Martins
Move struct page init to an helper function __init_zone_device_page(). This is in preparation for sharing the storage for compound page metadata. Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- mm/page_alloc.c | 74 +++-- 1 file changed

Re: [PATCH v4 06/14] device-dax: ensure dev_dax->pgmap is valid for dynamic devices

2021-11-05 Thread Joao Martins
On 11/5/21 16:46, Dan Williams wrote: > On Fri, Nov 5, 2021 at 5:10 AM Joao Martins wrote: >> >> On 11/5/21 00:31, Dan Williams wrote: >>> On Fri, Aug 27, 2021 at 7:59 AM Joao Martins >>> wrote: >>>> >>>> Right now, only static dax regio

Re: [PATCH v4 06/14] device-dax: ensure dev_dax->pgmap is valid for dynamic devices

2021-11-05 Thread Joao Martins
On 11/5/21 12:09, Joao Martins wrote: > On 11/5/21 00:31, Dan Williams wrote: >> On Fri, Aug 27, 2021 at 7:59 AM Joao Martins >> wrote: >>> >>> Right now, only static dax regions have a valid @pgmap pointer in its >>> struct dev_dax. Dynamic dax case h

Re: [PATCH v4 07/14] device-dax: compound devmap support

2021-11-05 Thread Joao Martins
On 11/5/21 00:38, Dan Williams wrote: > On Fri, Aug 27, 2021 at 7:59 AM Joao Martins > wrote: >> >> Use the newly added compound devmap facility which maps the assigned dax >> ranges as compound pages at a page size of @align. Currently, this means, >> that region/

Re: [PATCH v4 06/14] device-dax: ensure dev_dax->pgmap is valid for dynamic devices

2021-11-05 Thread Joao Martins
On 11/5/21 00:31, Dan Williams wrote: > On Fri, Aug 27, 2021 at 7:59 AM Joao Martins > wrote: >> >> Right now, only static dax regions have a valid @pgmap pointer in its >> struct dev_dax. Dynamic dax case however, do not. >> >> In preparation for device-da

Re: [PATCH v4 08/14] mm/gup: grab head page refcount once for group of subpages

2021-10-14 Thread Joao Martins
On 10/13/21 20:43, Jason Gunthorpe wrote: > On Wed, Oct 13, 2021 at 08:18:08PM +0100, Joao Martins wrote: >> On 10/13/21 18:41, Jason Gunthorpe wrote: >>> On Mon, Oct 11, 2021 at 04:53:29PM +0100, Joao Martins wrote: >>>> On 10/8/21 12:54, Jason Gunthorpe wrote: &

Re: [PATCH v4 08/14] mm/gup: grab head page refcount once for group of subpages

2021-10-13 Thread Joao Martins
On 10/13/21 18:41, Jason Gunthorpe wrote: > On Mon, Oct 11, 2021 at 04:53:29PM +0100, Joao Martins wrote: >> On 10/8/21 12:54, Jason Gunthorpe wrote: > >>> The only optimization that might work here is to grab the head, then >>> compute the extent of tail pages and

Re: [PATCH v4 08/14] mm/gup: grab head page refcount once for group of subpages

2021-10-11 Thread Joao Martins
On 10/8/21 12:54, Jason Gunthorpe wrote: > On Fri, Aug 27, 2021 at 03:58:13PM +0100, Joao Martins wrote: >> @@ -2252,16 +2265,25 @@ static int __gup_device_huge(unsigned long pfn, >> unsigned long addr, >> ret = 0; >>

Re: [PATCH v4 08/14] mm/gup: grab head page refcount once for group of subpages

2021-09-30 Thread Joao Martins
On 9/30/21 04:01, Alistair Popple wrote: > On Thursday, 30 September 2021 5:34:05 AM AEST Jason Gunthorpe wrote: >> On Wed, Sep 29, 2021 at 12:50:15PM +0100, Joao Martins wrote: >> >>>> If the get_dev_pagemap has to remain then it just means we have to >>>>

Re: [PATCH v4 08/14] mm/gup: grab head page refcount once for group of subpages

2021-09-29 Thread Joao Martins
On 9/28/21 19:01, Jason Gunthorpe wrote: > On Thu, Sep 23, 2021 at 05:51:04PM +0100, Joao Martins wrote: >> So ... if pgmap accounting was removed from gup-fast then this patch >> would be a lot simpler and we could perhaps just fallback to the regular >> hugepage case (TH

Re: [PATCH v4 08/14] mm/gup: grab head page refcount once for group of subpages

2021-09-23 Thread Joao Martins
On 8/31/21 6:05 PM, Jason Gunthorpe wrote: > On Tue, Aug 31, 2021 at 01:34:04PM +0100, Joao Martins wrote: >> On 8/30/21 2:07 PM, Jason Gunthorpe wrote: >>> On Fri, Aug 27, 2021 at 07:34:54PM +0100, Joao Martins wrote: >>>> On 8/27/21 5:25 PM, Jason Gunthorpe wrote:

  1   2   >