Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-04 Thread Hugh Dickins
On Fri, 4 Mar 2005, Christoph Lameter wrote: > On Fri, 4 Mar 2005, Hugh Dickins wrote: > > Nacked for the same reason as just given to earlier version. Ugly too. > > Ok. Then we could still get back the also ugly solution in the earlier > patchsets that acquired the spinlock separately before get

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-04 Thread Christoph Lameter
On Fri, 4 Mar 2005, Hugh Dickins wrote: > > Here is the unglorious solution. It also requies that > > CONFIG_ATOMIC_TABLE_OPS not be used together with CONFIG_DEBUG_PAGEALLOC > > (handled in the following patch): > > Nacked for the same reason as just given to earlier version. Ugly too. Ok. Then

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-04 Thread Hugh Dickins
Another! Sorry, you're way ahead of me... On Fri, 4 Mar 2005, Christoph Lameter wrote: > On Thu, 3 Mar 2005, Andrew Morton wrote: > > > A fix would be to restore the get_page() if CONFIG_DEBUG_PAGEALLOC. Not > > particularly glorious.. > > Here is the unglorious solution. It also requies that

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-04 Thread Christoph Lameter
Changelog: - Do not allow setting of CONFIG_DEBUG_PAGEALLOC together with CONFIG_ATOMIC_TABLE_OPS - Keep mark_page_accessed in do_swap_page The page fault handler attempts to use the page_table_lock only for short time periods. It repeatedly drops and reacquires the lock. When the lock is reacqu

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-04 Thread Christoph Lameter
On Thu, 3 Mar 2005, Andrew Morton wrote: > A fix would be to restore the get_page() if CONFIG_DEBUG_PAGEALLOC. Not > particularly glorious.. Here is the unglorious solution. It also requies that CONFIG_ATOMIC_TABLE_OPS not be used together with CONFIG_DEBUG_PAGEALLOC (handled in the following pa

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-03 Thread Andrew Morton
Christoph Lameter <[EMAIL PROTECTED]> wrote: > > On Wed, 2 Mar 2005, Andrew Morton wrote: > > > > This is not relevant since it only deals with file pages. > > > > OK. And CONFIG_DEBUG_PAGEALLOC? > > Its a debug feature that can be fixed if its broken. It's broken. A fix would be to rest

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-03 Thread Christoph Lameter
On Thu, 3 Mar 2005, Andrew Morton wrote: > Christoph Lameter <[EMAIL PROTECTED]> wrote: > > > > On Wed, 2 Mar 2005, Andrew Morton wrote: > > > > > > This is not relevant since it only deals with file pages. > > > > > > OK. And CONFIG_DEBUG_PAGEALLOC? > > > > Its a debug feature that can be

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-03 Thread David S. Miller
On Thu, 03 Mar 2005 17:30:28 +1100 Benjamin Herrenschmidt <[EMAIL PROTECTED]> wrote: > On Fri, 2005-03-04 at 04:19 +1100, Nick Piggin wrote: > > > You don't want to do that for all architectures, as I said earlier. > > eg. i386 can concurrently set the dirty bit with the MMU (which won't > > hono

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-03 Thread Christoph Lameter
On Wed, 2 Mar 2005, Andrew Morton wrote: > > This is not relevant since it only deals with file pages. > > OK. And CONFIG_DEBUG_PAGEALLOC? Its a debug feature that can be fixed if its broken. > > ptes are only > > installed atomically for anonymous memory (if CONFIG_ATOMIC_OPS > > is define

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-03 Thread Christoph Lameter
On Thu, 3 Mar 2005, Benjamin Herrenschmidt wrote: > > There is no need to provide pte_cmpxchg. If the arch does not support > > cmpxchg on ptes (CONFIG_ATOMIC_TABLE_OPS not defined) > > then it will fall back to using pte_get_and_clear while holding the > > page_table_lock to insure that the entry

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Nick Piggin
Benjamin Herrenschmidt wrote: On Fri, 2005-03-04 at 04:19 +1100, Nick Piggin wrote: You don't want to do that for all architectures, as I said earlier. eg. i386 can concurrently set the dirty bit with the MMU (which won't honour the lock). So you then need an atomic lock, atomic pte operations, an

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Christoph Lameter
On Wed, 2 Mar 2005, Andrew Morton wrote: > > Any mmap changes requires the mmapsem. > > sys_remap_file_pages() will call install_page() under down_read(mmap_sem). > It relies upon page_table_lock for pte atomicity. This is not relevant since it only deals with file pages. ptes are only installed

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Benjamin Herrenschmidt
On Wed, 2005-03-02 at 21:51 -0800, Christoph Lameter wrote: > On Wed, 2 Mar 2005, David S. Miller wrote: > > > Actually, I guess I could do the pte_cmpxchg() stuff, but only if it's > > used to "add" access. If the TLB miss handler races, we just go into > > the handle_mm_fault() path unnecessari

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Benjamin Herrenschmidt
On Fri, 2005-03-04 at 04:19 +1100, Nick Piggin wrote: > You don't want to do that for all architectures, as I said earlier. > eg. i386 can concurrently set the dirty bit with the MMU (which won't > honour the lock). > > So you then need an atomic lock, atomic pte operations, and atomic > unlock w

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Andrew Morton
Christoph Lameter <[EMAIL PROTECTED]> wrote: > > On Wed, 2 Mar 2005, Andrew Morton wrote: > > > > Any mmap changes requires the mmapsem. > > > > sys_remap_file_pages() will call install_page() under down_read(mmap_sem). > > It relies upon page_table_lock for pte atomicity. > > This is not re

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Nick Piggin
Benjamin Herrenschmidt wrote: However, if this pte_cmpxchg() thing is used for removing access, then sparc64 can't use it. In such a case a race in the TLB handler would result in using an invalid PTE. I could "spin" on some lock bit, but there is no way I'm adding instructions to the carefully c

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Christoph Lameter
On Wed, 2 Mar 2005, David S. Miller wrote: > Actually, I guess I could do the pte_cmpxchg() stuff, but only if it's > used to "add" access. If the TLB miss handler races, we just go into > the handle_mm_fault() path unnecessarily in order to synchronize. > > However, if this pte_cmpxchg() thing i

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Benjamin Herrenschmidt
> However, if this pte_cmpxchg() thing is used for removing access, then > sparc64 can't use it. In such a case a race in the TLB handler would > result in using an invalid PTE. I could "spin" on some lock bit, but > there is no way I'm adding instructions to the carefully constructed > TLB miss

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Christoph Lameter
On Wed, 2 Mar 2005, Andrew Morton wrote: > > There should be no change to these arches > > But we must at least confirm that these architectures can make these > changes in the future. If they make no changes then they haven't > benefitted from the patch. And the patch must be suitable for all >

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Andrew Morton
Christoph Lameter <[EMAIL PROTECTED]> wrote: > > On Wed, 2 Mar 2005, Andrew Morton wrote: > > > Have the ppc64 and sparc64 people reviewed and acked the change? (Not a > > facetious question - I just haven't been following the saga sufficiently > > closely to remember). > > There should be no ch

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread David S. Miller
On Thu, 3 Mar 2005 16:00:10 +1100 Paul Mackerras <[EMAIL PROTECTED]> wrote: > Andrew Morton writes: > > > But if the approach which these patches take is not suitable for these > > architectures then they have no solution to the scalability problem. The > > machines will perform suboptimally and

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Nick Piggin
Andrew Morton wrote: Christoph Lameter <[EMAIL PROTECTED]> wrote: On Wed, 2 Mar 2005, Andrew Morton wrote: Earlier releases back in September 2004 had some pte locking code (and AFAIK Nick also played around with pte locking) but that was less efficient than atomic operations. How much less effici

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Christoph Lameter
On Wed, 2 Mar 2005, Andrew Morton wrote: > Have the ppc64 and sparc64 people reviewed and acked the change? (Not a > facetious question - I just haven't been following the saga sufficiently > closely to remember). There should be no change to these arches > > Because if a pte is locked it shoul

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Christoph Lameter
On Thu, 3 Mar 2005, Paul Mackerras wrote: > More generally, I would be interested to know what sorts of > applications or benchmarks show scalability problems on large machines > due to contention on mm->page_table_lock. Number crunching apps that use vast amounts of memory through MPI or large d

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Paul Mackerras
Andrew Morton writes: > But if the approach which these patches take is not suitable for these > architectures then they have no solution to the scalability problem. The > machines will perform suboptimally and more (perhaps conflicting) > development will be needed. We can do a pte_cmpxchg on p

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Christoph Lameter
On Wed, 2 Mar 2005, Andrew Morton wrote: > > There have been extensive discussions on all aspects of this patch. > > This issue was discussed in > > http://marc.theaimsgroup.com/?t=11069449724&r=1&w=2 > > This is a difficult, intrusive and controversial patch. Things like the > above should b

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Andrew Morton
Christoph Lameter <[EMAIL PROTECTED]> wrote: > > > > The cmpxchg will fail if that happens. > > > > How about if someone does remap_file_pages() against that virtual address > > and that syscalls happens to pick the same physical page? We have the same > > physical page at the same pte slot with d

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Andrew Morton
Christoph Lameter <[EMAIL PROTECTED]> wrote: > > On Wed, 2 Mar 2005, Andrew Morton wrote: > > > > This is a related change discussed during V16 with Nick. > > > > It's worth retaining a paragraph for the changelog. > > There have been extensive discussions on all aspects of this patch. > This iss

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Christoph Lameter
On Wed, 2 Mar 2005, Andrew Morton wrote: > > This is a related change discussed during V16 with Nick. > > It's worth retaining a paragraph for the changelog. There have been extensive discussions on all aspects of this patch. This issue was discussed in http://marc.theaimsgroup.com/?t=11069449720

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Andrew Morton
Christoph Lameter <[EMAIL PROTECTED]> wrote: > > On Wed, 2 Mar 2005, Andrew Morton wrote: > > > > - if (!PageReserved(old_page)) > > > - page_cache_get(old_page); > > > > hm, this seems to be an unrelated change. You're saying that this page is > > protected from munmap() by munmap()'s do

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Christoph Lameter
On Wed, 2 Mar 2005, Andrew Morton wrote: > > - if (!PageReserved(old_page)) > > - page_cache_get(old_page); > > hm, this seems to be an unrelated change. You're saying that this page is > protected from munmap() by munmap()'s down_write(mmap_sem), yes? What > stops memory reclaim fro

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-02 Thread Andrew Morton
Christoph Lameter <[EMAIL PROTECTED]> wrote: > > ... > static int do_wp_page(struct mm_struct *mm, struct vm_area_struct * vma, > unsigned long address, pte_t *page_table, pmd_t *pmd, pte_t pte) > @@ -1306,22 +1308,25 @@ static int do_wp_page(struct mm_struct * > flush_

Re: Page fault scalability patch V18: Drop first acquisition of ptl

2005-03-01 Thread Christoph Lameter
The page fault handler attempts to use the page_table_lock only for short time periods. It repeatedly drops and reacquires the lock. When the lock is reacquired, checks are made if the underlying pte has changed before replacing the pte value. These locations are a good fit for the use of ptep_cmpx