Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-15 Thread Joel Fernandes
On Mon, Oct 15, 2018 at 10:18:14AM +0200, Martin Schwidefsky wrote: > On Mon, 15 Oct 2018 09:10:53 +0200 > Christian Borntraeger wrote: > > > On 10/12/2018 03:37 AM, Joel Fernandes (Google) wrote: > > > Android needs to mremap large regions of memory during memory management > > > related operati

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-15 Thread Martin Schwidefsky
On Mon, 15 Oct 2018 09:10:53 +0200 Christian Borntraeger wrote: > On 10/12/2018 03:37 AM, Joel Fernandes (Google) wrote: > > Android needs to mremap large regions of memory during memory management > > related operations. The mremap system call can be really slow if THP is > > not enabled. The bo

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-15 Thread Christian Borntraeger
On 10/12/2018 03:37 AM, Joel Fernandes (Google) wrote: > Android needs to mremap large regions of memory during memory management > related operations. The mremap system call can be really slow if THP is > not enabled. The bottleneck is move_page_tables, which is copying each > pte at a time, an

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-13 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 07:25:08PM -0700, Daniel Colascione wrote: [...] > > But anyway, I think this runtime detection thing is not needed. THP is > > actually expected to be as fast as this anyway, so if that's available then > > we should already be as fast. > > Ah, I think the commit message

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Anton Ivanov
On 12/10/2018 22:40, Kirill A. Shutemov wrote: On Fri, Oct 12, 2018 at 05:42:24PM +0100, Anton Ivanov wrote: On 10/12/18 3:48 PM, Anton Ivanov wrote: On 12/10/2018 15:37, Kirill A. Shutemov wrote: On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: On 10/12/18 2:37 AM, Joel Fernande

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Daniel Colascione
On Fri, Oct 12, 2018 at 7:10 PM, Joel Fernandes wrote: > On Fri, Oct 12, 2018 at 06:54:33PM -0700, Daniel Colascione wrote: >> I wonder whether it makes sense to expose to userspace somehow whether >> mremap is "fast" for a particular architecture. If a feature relies on >> fast mremap, it might b

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Daniel Colascione
I wonder whether it makes sense to expose to userspace somehow whether mremap is "fast" for a particular architecture. If a feature relies on fast mremap, it might be better for some userland component to disable that feature entirely rather than blindly use mremap and end up performing very poorly

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Daniel Colascione
Not 32-bit ARM? On Fri, Oct 12, 2018 at 6:35 PM, Joel Fernandes wrote: > On Fri, Oct 12, 2018 at 11:18:36AM -0700, David Miller wrote: >> From: Joel Fernandes > [...] >> > Also, do we not flush the caches from any path when we munmap >> > address space? We do call do_munmap on the old mapping f

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 06:54:33PM -0700, Daniel Colascione wrote: > I wonder whether it makes sense to expose to userspace somehow whether > mremap is "fast" for a particular architecture. If a feature relies on > fast mremap, it might be better for some userland component to disable > that featur

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 06:39:45PM -0700, Daniel Colascione wrote: > Not 32-bit ARM? Well, I didn't want to enable every possible architecture we could in a single go. Certainly arm32 can be a follow on enablement as can be other architectures. The point of this series is to upstream this feature

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 11:18:36AM -0700, David Miller wrote: > From: Joel Fernandes [...] > > Also, do we not flush the caches from any path when we munmap > > address space? We do call do_munmap on the old mapping from mremap > > after moving to the new one. > > Sparc makes sure that shared ma

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Fri, Oct 12, 2018 at 05:42:24PM +0100, Anton Ivanov wrote: > > On 10/12/18 3:48 PM, Anton Ivanov wrote: > > On 12/10/2018 15:37, Kirill A. Shutemov wrote: > > > On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: > > > > On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: > > > > > An

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Fri, Oct 12, 2018 at 09:57:19AM -0700, Joel Fernandes wrote: > On Fri, Oct 12, 2018 at 04:19:46PM +0300, Kirill A. Shutemov wrote: > > On Fri, Oct 12, 2018 at 05:50:46AM -0700, Joel Fernandes wrote: > > > On Fri, Oct 12, 2018 at 02:30:56PM +0300, Kirill A. Shutemov wrote: > > > > On Thu, Oct 11,

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Anton Ivanov
On 12/10/2018 15:37, Kirill A. Shutemov wrote: On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: Android needs to mremap large regions of memory during memory management related operations. The mremap system call can be really slow

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Anton Ivanov
On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: Android needs to mremap large regions of memory during memory management related operations. The mremap system call can be really slow if THP is not enabled. The bottleneck is move_page_tables, which is copying each pte at a time, and can be rea

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Anton Ivanov
On 10/12/18 3:48 PM, Anton Ivanov wrote: On 12/10/2018 15:37, Kirill A. Shutemov wrote: On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: Android needs to mremap large regions of memory during memory management related operations

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Anton Ivanov
On 10/12/18 5:50 PM, Joel Fernandes wrote: On Fri, Oct 12, 2018 at 05:42:24PM +0100, Anton Ivanov wrote: On 10/12/18 3:48 PM, Anton Ivanov wrote: On 12/10/2018 15:37, Kirill A. Shutemov wrote: On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: On 10/12/18 2:37 AM, Joel Fernandes

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 05:58:40PM +0100, Anton Ivanov wrote: [...] > > > > > > If I read the code right, MIPS completely ignores the address > > > > > > argument so > > > > > > set_pmd_at there may not have the effect which this patch is trying > > > > > > to > > > > > > achieve. > > > > > Ignori

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 04:19:46PM +0300, Kirill A. Shutemov wrote: > On Fri, Oct 12, 2018 at 05:50:46AM -0700, Joel Fernandes wrote: > > On Fri, Oct 12, 2018 at 02:30:56PM +0300, Kirill A. Shutemov wrote: > > > On Thu, Oct 11, 2018 at 06:37:56PM -0700, Joel Fernandes (Google) wrote: > > > > Androi

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 05:42:24PM +0100, Anton Ivanov wrote: > > On 10/12/18 3:48 PM, Anton Ivanov wrote: > > On 12/10/2018 15:37, Kirill A. Shutemov wrote: > > > On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: > > > > On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: > > > > > An

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Fri, Oct 12, 2018 at 03:09:49PM +0100, Anton Ivanov wrote: > On 10/12/18 2:37 AM, Joel Fernandes (Google) wrote: > > Android needs to mremap large regions of memory during memory management > > related operations. The mremap system call can be really slow if THP is > > not enabled. The bottlenec

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Fri, Oct 12, 2018 at 05:50:46AM -0700, Joel Fernandes wrote: > On Fri, Oct 12, 2018 at 02:30:56PM +0300, Kirill A. Shutemov wrote: > > On Thu, Oct 11, 2018 at 06:37:56PM -0700, Joel Fernandes (Google) wrote: > > > Android needs to mremap large regions of memory during memory management > > > rel

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Joel Fernandes
On Fri, Oct 12, 2018 at 02:30:56PM +0300, Kirill A. Shutemov wrote: > On Thu, Oct 11, 2018 at 06:37:56PM -0700, Joel Fernandes (Google) wrote: > > Android needs to mremap large regions of memory during memory management > > related operations. The mremap system call can be really slow if THP is > >

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread David Miller
From: Joel Fernandes Date: Fri, 12 Oct 2018 05:50:46 -0700 > If its an issue, then how do transparent huge pages work on Sparc? I don't > see the huge page code (move_huge_pages) during mremap doing anything special > for Sparc architecture when moving PMDs.. This is because all huge pages are

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread David Miller
From: "Kirill A. Shutemov" Date: Fri, 12 Oct 2018 14:30:56 +0300 > I looked into the code more and noticed move_pte() helper called from > move_ptes(). It changes PTE entry to suite new address. > > It is only defined in non-trivial way on Sparc. I don't know much about > Sparc and it's hard for

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Fri, Oct 12, 2018 at 02:30:56PM +0300, Kirill A. Shutemov wrote: > On Thu, Oct 11, 2018 at 06:37:56PM -0700, Joel Fernandes (Google) wrote: > > @@ -239,7 +287,21 @@ unsigned long move_page_tables(struct vm_area_struct > > *vma, > > split_huge_pmd(vma, old_pmd, old_addr); > >

Re: [PATCH v2 2/2] mm: speed up mremap by 500x on large regions

2018-10-12 Thread Kirill A. Shutemov
On Thu, Oct 11, 2018 at 06:37:56PM -0700, Joel Fernandes (Google) wrote: > Android needs to mremap large regions of memory during memory management > related operations. The mremap system call can be really slow if THP is > not enabled. The bottleneck is move_page_tables, which is copying each > pt