Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-12-04 Thread Peter Xu
On Mon, Dec 04, 2023 at 11:11:26AM +, Ryan Roberts wrote: > To be honest, while I understand pte_cont() and friends, I don't understand > their relevance (or at least potential future relevance) to GUP? GUP in general can be smarter to recognize if a pte/pmd is a cont_pte and fetch the whole p

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-12-04 Thread Ryan Roberts
On 04/12/2023 11:57, Christophe Leroy wrote: > > > Le 04/12/2023 à 12:46, Ryan Roberts a écrit : >> On 04/12/2023 11:25, Christophe Leroy wrote: >>> >>> >>> Le 04/12/2023 à 12:11, Ryan Roberts a écrit : On 03/12/2023 13:33, Christophe Leroy wrote: > > > Le 30/11/2023 à 22:30, Pet

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-12-04 Thread Christophe Leroy
Le 04/12/2023 à 12:46, Ryan Roberts a écrit : > On 04/12/2023 11:25, Christophe Leroy wrote: >> >> >> Le 04/12/2023 à 12:11, Ryan Roberts a écrit : >>> On 03/12/2023 13:33, Christophe Leroy wrote: Le 30/11/2023 à 22:30, Peter Xu a écrit : > On Fri, Nov 24, 2023 at 11:07:51AM -0

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-12-04 Thread Ryan Roberts
On 04/12/2023 11:25, Christophe Leroy wrote: > > > Le 04/12/2023 à 12:11, Ryan Roberts a écrit : >> On 03/12/2023 13:33, Christophe Leroy wrote: >>> >>> >>> Le 30/11/2023 à 22:30, Peter Xu a écrit : On Fri, Nov 24, 2023 at 11:07:51AM -0500, Peter Xu wrote: > On Fri, Nov 24, 2023 at 09:06

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-12-04 Thread Christophe Leroy
Le 04/12/2023 à 12:11, Ryan Roberts a écrit : > On 03/12/2023 13:33, Christophe Leroy wrote: >> >> >> Le 30/11/2023 à 22:30, Peter Xu a écrit : >>> On Fri, Nov 24, 2023 at 11:07:51AM -0500, Peter Xu wrote: On Fri, Nov 24, 2023 at 09:06:01AM +, Ryan Roberts wrote: > I don't have any m

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-12-04 Thread Ryan Roberts
On 03/12/2023 13:33, Christophe Leroy wrote: > > > Le 30/11/2023 à 22:30, Peter Xu a écrit : >> On Fri, Nov 24, 2023 at 11:07:51AM -0500, Peter Xu wrote: >>> On Fri, Nov 24, 2023 at 09:06:01AM +, Ryan Roberts wrote: I don't have any micro-benchmarks for GUP though, if that's your questio

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-12-03 Thread Christophe Leroy
Le 30/11/2023 à 22:30, Peter Xu a écrit : > On Fri, Nov 24, 2023 at 11:07:51AM -0500, Peter Xu wrote: >> On Fri, Nov 24, 2023 at 09:06:01AM +, Ryan Roberts wrote: >>> I don't have any micro-benchmarks for GUP though, if that's your question. >>> Is >>> there an easy-to-use test I can run to

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-30 Thread Peter Xu
On Fri, Nov 24, 2023 at 11:07:51AM -0500, Peter Xu wrote: > On Fri, Nov 24, 2023 at 09:06:01AM +, Ryan Roberts wrote: > > I don't have any micro-benchmarks for GUP though, if that's your question. > > Is > > there an easy-to-use test I can run to get some numbers? I'd be happy to > > try it o

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-24 Thread Peter Xu
Hi, Christophe, Michael, Aneesh, [I'll reply altogether here] On Fri, Nov 24, 2023 at 07:03:11AM +, Christophe Leroy wrote: > I added that code with commit e17eae2b8399 ("mm: pagewalk: fix walk for > hugepage tables") because I was getting crazy displays when dumping > /sys/kernel/debug/pag

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-24 Thread Peter Xu
On Fri, Nov 24, 2023 at 09:06:01AM +, Ryan Roberts wrote: > I don't have any micro-benchmarks for GUP though, if that's your question. Is > there an easy-to-use test I can run to get some numbers? I'd be happy to try > it out. Thanks Ryan. Then nothing is needed to be tested if gup is not ye

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-24 Thread Ryan Roberts
On 23/11/2023 19:46, Peter Xu wrote: > On Thu, Nov 23, 2023 at 07:11:19PM +, Ryan Roberts wrote: >> Hi, >> >> I'm not sure I've 100% understood the crossover between this series and my >> work >> to support arm64's contpte mappings generally for anonymous and file-backed >> memory. > > No wo

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-23 Thread Christophe Leroy
Le 23/11/2023 à 20:37, Peter Xu a écrit : > On Thu, Nov 23, 2023 at 06:22:33PM +, Christophe Leroy wrote: >>> For fast-gup I think the hugepd code is in use, however for walk_page_* >>> apis hugepd code shouldn't be reached iiuc as we have the hugetlb specific >>> handling (walk_hugetlb_range

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-23 Thread IBM
Peter Xu writes: > On Thu, Nov 23, 2023 at 06:22:33PM +, Christophe Leroy wrote: >> > For fast-gup I think the hugepd code is in use, however for walk_page_* >> > apis hugepd code shouldn't be reached iiuc as we have the hugetlb specific >> > handling (walk_hugetlb_range()), so anything withi

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-23 Thread Michael Ellerman
Peter Xu writes: > On Wed, Nov 22, 2023 at 12:00:24AM -0800, Christoph Hellwig wrote: >> On Tue, Nov 21, 2023 at 10:59:35AM -0500, Peter Xu wrote: ... >> >> If dropping the check is the right thing for now (and I think the ppc >> maintainers and willy as the large folio guy might have a more usef

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-23 Thread Peter Xu
On Thu, Nov 23, 2023 at 07:11:19PM +, Ryan Roberts wrote: > Hi, > > I'm not sure I've 100% understood the crossover between this series and my > work > to support arm64's contpte mappings generally for anonymous and file-backed > memory. No worry, there's no confliction. If you worked on t

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-23 Thread Peter Xu
On Thu, Nov 23, 2023 at 06:22:33PM +, Christophe Leroy wrote: > > For fast-gup I think the hugepd code is in use, however for walk_page_* > > apis hugepd code shouldn't be reached iiuc as we have the hugetlb specific > > handling (walk_hugetlb_range()), so anything within walk_pgd_range() to hi

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-23 Thread Ryan Roberts
On 23/11/2023 17:22, Peter Xu wrote: > On Thu, Nov 23, 2023 at 03:47:49PM +, Matthew Wilcox wrote: >> It looks like ARM (in the person of Ryan) are going to add support for >> something equivalent to hugepd. > > If it's about arm's cont_pte, then it looks ideal because this series > didn't yet

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-23 Thread Christophe Leroy
Le 22/11/2023 à 16:22, Peter Xu a écrit : > On Wed, Nov 22, 2023 at 12:00:24AM -0800, Christoph Hellwig wrote: >> On Tue, Nov 21, 2023 at 10:59:35AM -0500, Peter Xu wrote: What prevents us from ever using hugepd with file mappings? I think it would naturally fit in with how large folio

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-23 Thread Peter Xu
On Thu, Nov 23, 2023 at 03:47:49PM +, Matthew Wilcox wrote: > It looks like ARM (in the person of Ryan) are going to add support for > something equivalent to hugepd. If it's about arm's cont_pte, then it looks ideal because this series didn't yet touch cont_pte, assuming it'll just work. Fro

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-23 Thread Peter Xu
On Wed, Nov 22, 2023 at 11:21:50PM -0800, Christoph Hellwig wrote: > That alone sounds like a good reason to not bother. So unless more > qualified people have a different opinion I'm fine with this patch > as long as you leave a comment in place, and ammend the commit message > with some of the v

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-23 Thread Matthew Wilcox
On Wed, Nov 22, 2023 at 12:00:24AM -0800, Christoph Hellwig wrote: > > The other option is I can always add a comment above gup_huge_pd() > > explaining this special bit, so that when someone is adding hugepd support > > to file large folios we'll hopefully not forget it? But then that > > general

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-22 Thread Christoph Hellwig
On Wed, Nov 22, 2023 at 10:22:11AM -0500, Peter Xu wrote: > The other reason I feel like hugepd may or may not be further developed for > new features like large folio is that I saw Power9 started to shift to > radix pgtables, and afaics hugepd is only supported in hash tables > (hugepd_ok()). But

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-22 Thread Peter Xu
On Wed, Nov 22, 2023 at 12:00:24AM -0800, Christoph Hellwig wrote: > On Tue, Nov 21, 2023 at 10:59:35AM -0500, Peter Xu wrote: > > > What prevents us from ever using hugepd with file mappings? I think > > > it would naturally fit in with how large folios for the pagecache work. > > > > > > So kee

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-22 Thread Christoph Hellwig
On Tue, Nov 21, 2023 at 10:59:35AM -0500, Peter Xu wrote: > > What prevents us from ever using hugepd with file mappings? I think > > it would naturally fit in with how large folios for the pagecache work. > > > > So keeping this check and generalizing it seems like the better idea to > > me. >

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-21 Thread Peter Xu
On Mon, Nov 20, 2023 at 12:26:24AM -0800, Christoph Hellwig wrote: > On Wed, Nov 15, 2023 at 08:29:02PM -0500, Peter Xu wrote: > > Hugepd format is only used in PowerPC with hugetlbfs. In commit > > a6e79df92e4a ("mm/gup: disallow FOLL_LONGTERM GUP-fast writing to > > file-backed mappings"), we ad

Re: [PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-20 Thread Christoph Hellwig
On Wed, Nov 15, 2023 at 08:29:02PM -0500, Peter Xu wrote: > Hugepd format is only used in PowerPC with hugetlbfs. In commit > a6e79df92e4a ("mm/gup: disallow FOLL_LONGTERM GUP-fast writing to > file-backed mappings"), we added a check to fail gup-fast if there's > potential risk of violating GUP o

[PATCH RFC 06/12] mm/gup: Drop folio_fast_pin_allowed() in hugepd processing

2023-11-15 Thread Peter Xu
Hugepd format is only used in PowerPC with hugetlbfs. In commit a6e79df92e4a ("mm/gup: disallow FOLL_LONGTERM GUP-fast writing to file-backed mappings"), we added a check to fail gup-fast if there's potential risk of violating GUP over writeback file systems. That should never apply to hugepd. D