Re: [patch 0/6] MMU Notifiers V6

2008-02-13 Thread Jack Steiner
> GRU > - Simple additional hardware TLB (possibly covering multiple instances of > Linux) > - Needs TLB shootdown when the VM unmaps pages. > - Determines page address via follow_page (from interrupt context) but can > fall back to get_user_pages(). > - No page reference possible since no page

Re: [ofa-general] Re: [patch 0/6] MMU Notifiers V6

2008-02-09 Thread Christoph Lameter
On Sat, 9 Feb 2008, Rik van Riel wrote: > PG_mlock is on the way and can easily be reused for this, too. Note that a pinned page is different from an mlocked page. A mlocked page can be moved through page migration and/or memory hotplug. A pinned page must make both fail. -- To unsubscribe fro

Re: [ofa-general] Re: [patch 0/6] MMU Notifiers V6

2008-02-09 Thread Rik van Riel
On Fri, 8 Feb 2008 18:16:16 -0800 (PST) Christoph Lameter <[EMAIL PROTECTED]> wrote: > On Sat, 9 Feb 2008, Andrea Arcangeli wrote: > > > The VM shouldn't break if try_to_unmap doesn't actually make the page > > freeable for whatever reason. Permanent pins shouldn't happen anyway, > > VM is livelo

Re: [ofa-general] Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Christoph Lameter
On Sat, 9 Feb 2008, Andrea Arcangeli wrote: > The VM shouldn't break if try_to_unmap doesn't actually make the page > freeable for whatever reason. Permanent pins shouldn't happen anyway, VM is livelocking if too many page are pinned that way right now. The higher the processors per node the hig

Re: [ofa-general] Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Andrea Arcangeli
On Fri, Feb 08, 2008 at 05:27:03PM -0800, Christoph Lameter wrote: > Pages will still be on the LRU and cycle through rmap again and again. > If page migration is used on those pages then the code may make repeated > attempt to migrate the page thinking that the page count must at some > point d

Re: [ofa-general] Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Christoph Lameter
On Sat, 9 Feb 2008, Andrea Arcangeli wrote: > > H.. that means we need something that actually pins pages for good so > > that the VM can avoid reclaiming it and so that page migration can avoid > > trying to migrate them. Something like yet another page flag. > > What's wrong with pinning

Re: [ofa-general] Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Andrea Arcangeli
On Fri, Feb 08, 2008 at 04:36:16PM -0800, Christoph Lameter wrote: > On Fri, 8 Feb 2008, Roland Dreier wrote: > > > That would of course work -- dumb adapters would just always fail, > > which might be inefficient. > > H.. that means we need something that actually pins pages for good so > t

Re: [ofa-general] Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Christoph Lameter
On Fri, 8 Feb 2008, Roland Dreier wrote: > That would of course work -- dumb adapters would just always fail, > which might be inefficient. H.. that means we need something that actually pins pages for good so that the VM can avoid reclaiming it and so that page migration can avoid trying t

Re: [ofa-general] Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Roland Dreier
> I thought the adaptor can always remove the mapping by renegotiating > with the remote side? Even if its dumb then a callback could notify the > driver that it may be required to tear down the mapping. We then hold the > pages until we get okay by the driver that the mapping has been remov

Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Christoph Lameter
On Fri, 8 Feb 2008, Andrew Morton wrote: > Quite possibly none of the infiniband developers even know about it.. Well Andrea's initial approach was even featured on LWN a couple of weeks back. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to

Re: [ofa-general] Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Christoph Lameter
On Fri, 8 Feb 2008, Roland Dreier wrote: > In general, this MMU notifier stuff will only be useful to a subset of > InfiniBand/RDMA hardware. Some adapters are smart enough to handle > changing the IO virtual -> bus/physical mapping on the fly, but some > aren't. For the dumb adapters, I think t

Re: [ofa-general] Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Roland Dreier
> We have done several rounds of discussion on linux-kernel about this so > far and the IB folks have not shown up to join in. I have tried to make > this as general as possible. Sorry, this has been on my "things to look at" list for a while, but I haven't gotten a chance to really understan

Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Andrew Morton
On Fri, 8 Feb 2008 16:05:00 -0800 (PST) Christoph Lameter <[EMAIL PROTECTED]> wrote: > On Fri, 8 Feb 2008, Andrew Morton wrote: > > > You took it correctly, and I didn't understand the answer ;) > > We have done several rounds of discussion on linux-kernel about this so > far and the IB folks

Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Christoph Lameter
On Fri, 8 Feb 2008, Andrew Morton wrote: > You took it correctly, and I didn't understand the answer ;) We have done several rounds of discussion on linux-kernel about this so far and the IB folks have not shown up to join in. I have tried to make this as general as possible. -- To unsubscribe

Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Andrew Morton
On Fri, 8 Feb 2008 17:43:02 -0600 Robin Holt <[EMAIL PROTECTED]> wrote: > On Fri, Feb 08, 2008 at 03:41:24PM -0800, Christoph Lameter wrote: > > On Fri, 8 Feb 2008, Robin Holt wrote: > > > > > > > What about ib_umem_get()? > > > > Correct. > > > > You missed the turn of the conversation to how

Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Robin Holt
On Fri, Feb 08, 2008 at 03:41:24PM -0800, Christoph Lameter wrote: > On Fri, 8 Feb 2008, Robin Holt wrote: > > > > > What about ib_umem_get()? > > Correct. > > You missed the turn of the conversation to how ib_umem_get() works. > Currently it seems to pin the same way that the SLES10 XPmem work

Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Christoph Lameter
On Fri, 8 Feb 2008, Robin Holt wrote: > > > What about ib_umem_get()? > > > > Ok. It pins using an elevated refcount. Same as XPmem right now. With that > > we effectively pin a page (page migration will fail) but we will > > continually be reclaiming the page and may repeatedly try to move it.

Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Robin Holt
On Fri, Feb 08, 2008 at 03:32:19PM -0800, Christoph Lameter wrote: > On Fri, 8 Feb 2008, Andrew Morton wrote: > > > What about ib_umem_get()? > > Ok. It pins using an elevated refcount. Same as XPmem right now. With that > we effectively pin a page (page migration will fail) but we will > conti

Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Christoph Lameter
On Fri, 8 Feb 2008, Andrew Morton wrote: > What about ib_umem_get()? Ok. It pins using an elevated refcount. Same as XPmem right now. With that we effectively pin a page (page migration will fail) but we will continually be reclaiming the page and may repeatedly try to move it. We have issues

Re: [patch 0/6] MMU Notifiers V6

2008-02-08 Thread Andrew Morton
On Fri, 08 Feb 2008 14:06:16 -0800 Christoph Lameter <[EMAIL PROTECTED]> wrote: > This is a patchset implementing MMU notifier callbacks based on Andrea's > earlier work. These are needed if Linux pages are referenced from something > else than tracked by the rmaps of the kernel (an external MMU).

[patch 0/6] MMU Notifiers V6

2008-02-08 Thread Christoph Lameter
This is a patchset implementing MMU notifier callbacks based on Andrea's earlier work. These are needed if Linux pages are referenced from something else than tracked by the rmaps of the kernel (an external MMU). MMU notifiers allow us to get rid of the page pinning for RDMA and various other purpo