k.msu.ru>, Al Viro <v...@zeniv.linux.org.uk>, Andy Lutomirski <l...@kernel.org>, Paul Walmsley <paul.walms...@sifive.com>, Thomas Gleixner <t...@linutronix.de>, "open list:ALPHA PORT" <linux-al...@vger.kernel.org>, Andrew Morton <a...@linux-foundation.org>, Vlastimil Babka <vba...@suse.cz>, Richard Henderson <r...@twiddle.net>, Chris Zankel <ch...@zankel.net>, Michal Simek <mon...@monstr.eu>, Thomas Bogendoerfer <tsbog...@alpha.franken.de>, "open list:PARISC ARCHITECTURE" <linux-par...@vger.kernel.org>, LKML <linux-ker...@vger.kernel.org>, Dinh Nguyen <dingu...@kernel.org>, Palmer Dabbelt <pal...@dabbelt.com>, Sven Schnelle <sv...@linux.ibm.com>, Guo Ren <guo...@kernel.org>, Borislav Petkov <b...@alien8.de>, Johannes Berg <johan...@sipsolutions.net>, linuxppc-dev@lists.ozlabs.org, "David S . Miller" <da...@davemloft.net> Errors-To: linuxppc-dev-bounces+archive=mail-archive....@lists.ozlabs.org Sender: "Linuxppc-dev" <linuxppc-dev-bounces+archive=mail-archive....@lists.ozlabs.org>
On Tue, May 24, 2022 at 4:45 PM Peter Xu <pet...@redhat.com> wrote: > > I observed that for each of the shared file-backed page faults, we're very > likely to retry one more time for the 1st write fault upon no page. It's > because we'll need to release the mmap lock for dirty rate limit purpose > with balance_dirty_pages_ratelimited() (in fault_dirty_shared_page()). > > Then after that throttling we return VM_FAULT_RETRY. > > We did that probably because VM_FAULT_RETRY is the only way we can return > to the fault handler at that time telling it we've released the mmap lock. > > However that's not ideal because it's very likely the fault does not need > to be retried at all since the pgtable was well installed before the > throttling, so the next continuous fault (including taking mmap read lock, > walk the pgtable, etc.) could be in most cases unnecessary. > > It's not only slowing down page faults for shared file-backed, but also add > more mmap lock contention which is in most cases not needed at all. > > To observe this, one could try to write to some shmem page and look at > "pgfault" value in /proc/vmstat, then we should expect 2 counts for each > shmem write simply because we retried, and vm event "pgfault" will capture > that. > > To make it more efficient, add a new VM_FAULT_COMPLETED return code just to > show that we've completed the whole fault and released the lock. It's also > a hint that we should very possibly not need another fault immediately on > this page because we've just completed it. > > This patch provides a ~12% perf boost on my aarch64 test VM with a simple > program sequentially dirtying 400MB shmem file being mmap()ed and these are > the time it needs: > > Before: 650.980 ms (+-1.94%) > After: 569.396 ms (+-1.38%) > > I believe it could help more than that. > > We need some special care on GUP and the s390 pgfault handler (for gmap > code before returning from pgfault), the rest changes in the page fault > handlers should be relatively straightforward. > > Another thing to mention is that mm_account_fault() does take this new > fault as a generic fault to be accounted, unlike VM_FAULT_RETRY. > > I explicitly didn't touch hmm_vma_fault() and break_ksm() because they do > not handle VM_FAULT_RETRY even with existing code, so I'm literally keeping > them as-is. > > Signed-off-by: Peter Xu <pet...@redhat.com> > --- > > v3: > - Rebase to akpm/mm-unstable > - Copy arch maintainers > --- For xtensa: Acked-by: Max Filippov <jcmvb...@gmail.com> -- Thanks. -- Max
