Arjan van de Ven wrote:
On Mon, 2005-04-18 at 11:09 -0500, Timur Tabi wrote:
Roland Dreier wrote:
Troy> How is memory pinning handled? (I haven't had time to read
Troy> all the code, so please excuse my ignorance of something
Troy> obvious).
The userspace library calls mlock() and then the
Andy Isaacson <[EMAIL PROTECTED]> wrote:
> On Wed, Apr 20, 2005 at 10:07:45PM -0500, Timur Tabi wrote:
>> I don't know if VM_REGISTERED is a good idea or not, but it should be
>> absolutely impossible for the kernel to reclaim "registered" (aka pinned)
>> memory, no matter what. For RDMA services
Hi!
> >Timur> Why do you call mlock() and get_user_pages()? In our
> >code,
> >Timur> we only call mlock(), and the memory is pinned. We have a
> >Timur> test case that fails if only get_user_pages() is called,
> >Timur> but it passes if only mlock() is called.
> >
> >What i
On Thu, Apr 21, 2005 at 03:07:42PM -0500, Timur Tabi wrote:
> >*You* need to come up with a solution that looks good to *the community*
> >if you want it merged.
>
> True, but I'm not going to waste my time adding this support if the
> consensus I get from the kernel developers that they don't
On Thu, 2005-04-21 at 13:25 -0700, Chris Wright wrote:
> * Timur Tabi ([EMAIL PROTECTED]) wrote:
> > It works with every kernel I've tried. I'm sure there are plenty of kernel
> > configuration options that will break our driver. But as long as all the
> > distros our customers use work, as wel
* Timur Tabi ([EMAIL PROTECTED]) wrote:
> It works with every kernel I've tried. I'm sure there are plenty of kernel
> configuration options that will break our driver. But as long as all the
> distros our customers use work, as well as reasonably-configured custom
> kernels, we're happy.
>
Chris Wright wrote:
FYI, that will not work on all 2.6 kernels. Specifically anything that's
not using capabilities.
It works with every kernel I've tried. I'm sure there are plenty of kernel configuration
options that will break our driver. But as long as all the distros our customers use
wor
* Timur Tabi ([EMAIL PROTECTED]) wrote:
> Andy Isaacson wrote:
> >Do you guys simply raise RLIMIT_MEMLOCK to allow apps to lock their
> >pages? Or are you doing something more nasty?
>
> A little more nasty. I raise RLIMIT_MEMLOCK in the driver to "unlimited"
> and also set cap_raise(IPC_LOCK).
Andy Isaacson wrote:
I'm familiar with MPI 1.0 and 2.0, but I haven't been following the
development of modern messaging APIs, so I might not make sense here...
Assuming that the app calls into the library on a fairly regular basis,
Not really. The whole point is to have the adapter DMA the data d
On Thu, Apr 21, 2005 at 01:39:35PM -0500, Timur Tabi wrote:
> Andy Isaacson wrote:
> >If you take the hardline position that "the app is the only thing that
> >matters", your code is unlikely to get merged. Linux is a
> >general-purpose OS.
>
> The problem is that our driver and library implement
Andy Isaacson wrote:
If you take the hardline position that "the app is the only thing that
matters", your code is unlikely to get merged. Linux is a
general-purpose OS.
The problem is that our driver and library implement an API that we don't fully control.
The API states that the application al
On Wed, Apr 20, 2005 at 10:07:45PM -0500, Timur Tabi wrote:
> Troy Benjegerdes wrote:
> >Someone (aka Tospin, infinicon, and Amasso) should probably post a patch
> >adding '#define VM_REGISTERD 0x0100', and some extensions to
> >something like 'madvise' to set pages to be registered.
> >
> >My
Troy Benjegerdes wrote:
Someone (aka Tospin, infinicon, and Amasso) should probably post a patch
adding '#define VM_REGISTERD 0x0100', and some extensions to
something like 'madvise' to set pages to be registered.
My preference is said patch will also allow a way for the kernel to
reclaim regis
On Mon, Apr 18, 2005 at 10:07:12PM +0200, Bernhard Fischer wrote:
> On Mon, Apr 18, 2005 at 09:40:40PM +0200, Arjan van de Ven wrote:
> >On Mon, 2005-04-18 at 11:25 -0500, Timur Tabi wrote:
> >> Arjan van de Ven wrote:
> >>
> >> > this is a myth; linux is free to move the page about in physical me
Arjan van de Ven wrote:
you just said that you didn't care that it got munlock'd. So you don't
care that it gets freed either. And then reused.
Well, I can live with the app being able to call munlock(), because the apps that our
customers use don't call munlock(). What I can't live with is a bug
On Mon, Apr 18, 2005 at 09:40:40PM +0200, Arjan van de Ven wrote:
>On Mon, 2005-04-18 at 11:25 -0500, Timur Tabi wrote:
>> Arjan van de Ven wrote:
>>
>> > this is a myth; linux is free to move the page about in physical memory
>> > even if it's mlock()ed!!
darn, yes, this is true.
I know people wh
On Mon, 2005-04-18 at 15:00 -0500, Timur Tabi wrote:
> Arjan van de Ven wrote:
>
> > you should since that physical page can be reused, say by a root
> > process, and you'd be majorly screwed
>
> I don't understand what you mean by "reused". The whole point behind pinning
> the memory
> is tha
Arjan van de Ven wrote:
you should since that physical page can be reused, say by a root
process, and you'd be majorly screwed
I don't understand what you mean by "reused". The whole point behind pinning the memory
is that it stays where it is. It doesn't get moved around and it doesn't get swap
On Mon, 2005-04-18 at 11:25 -0500, Timur Tabi wrote:
> Arjan van de Ven wrote:
>
> > this is a myth; linux is free to move the page about in physical memory
> > even if it's mlock()ed!!
>
> Then Linux has a very odd definition of the word "locked".
>
> > And even then, the user can munlock the m
Libor Michalek wrote:
The problem we were seeing is that the minor fault by the app resulted
in a new physical page getting mapped for the application. The page that
had the elevated refcount was still waiting for the data to be written
to by the driver at the time that the app accessed the page ca
Roland Dreier wrote:
Timur> Why do you call mlock() and get_user_pages()? In our code,
Timur> we only call mlock(), and the memory is pinned. We have a
Timur> test case that fails if only get_user_pages() is called,
Timur> but it passes if only mlock() is called.
What if a buggy/m
Timur> Why do you call mlock() and get_user_pages()? In our code,
Timur> we only call mlock(), and the memory is pinned. We have a
Timur> test case that fails if only get_user_pages() is called,
Timur> but it passes if only mlock() is called.
What if a buggy/malicious userspace p
Christoph Hellwig wrote:
On Mon, Apr 18, 2005 at 11:22:29AM -0500, Timur Tabi wrote:
That's not what we're seeing. We have hardware that does DMA over the
network (much like the Infiniband stuff), and we have a testcase that fails
if get_user_pages() is used, but not if mlock() is used.
If you
On Mon, Apr 18, 2005 at 11:22:29AM -0500, Timur Tabi wrote:
> That's not what we're seeing. We have hardware that does DMA over the
> network (much like the Infiniband stuff), and we have a testcase that fails
> if get_user_pages() is used, but not if mlock() is used.
If you don't share your te
Arjan van de Ven wrote:
this is a myth; linux is free to move the page about in physical memory
even if it's mlock()ed!!
Then Linux has a very odd definition of the word "locked".
And even then, the user can munlock the memory from another thread etc
etc. Not a good idea.
Well, that's okay, because
Andrew Morton wrote:
Roland Dreier <[EMAIL PROTECTED]> wrote:
Troy> Do we even need the mlock in userspace then?
Yes, because the kernel may go through and unmap pages from userspace
while trying to swap. Since we have the page locked in the kernel,
the physical page won't go anywhere, but user
On Mon, 2005-04-18 at 11:09 -0500, Timur Tabi wrote:
> Roland Dreier wrote:
> > Troy> How is memory pinning handled? (I haven't had time to read
> > Troy> all the code, so please excuse my ignorance of something
> > Troy> obvious).
> >
> > The userspace library calls mlock() and then t
Roland Dreier wrote:
Troy> How is memory pinning handled? (I haven't had time to read
Troy> all the code, so please excuse my ignorance of something
Troy> obvious).
The userspace library calls mlock() and then the kernel does
get_user_pages().
Why do you call mlock() and get_user_pages(
Roland Dreier <[EMAIL PROTECTED]> wrote:
>
> OK, I'm by no means an expert on this, but Libor and I looked at
> rmap.c a little more, and there is code:
>
> if ((vma->vm_flags & (VM_LOCKED|VM_RESERVED)) ||
> ptep_clear_flush_young(vma, address, pte)) {
> r
OK, I'm by no means an expert on this, but Libor and I looked at
rmap.c a little more, and there is code:
if ((vma->vm_flags & (VM_LOCKED|VM_RESERVED)) ||
ptep_clear_flush_young(vma, address, pte)) {
ret = SWAP_FAIL;
goto out_unmap;
On Mon, Apr 11, 2005 at 05:13:47PM -0700, Andrew Morton wrote:
> Roland Dreier <[EMAIL PROTECTED]> wrote:
> >
> > Troy> Do we even need the mlock in userspace then?
> >
> > Yes, because the kernel may go through and unmap pages from userspace
> > while trying to swap. Since we have the page l
Quoting r. Roland Dreier <[EMAIL PROTECTED]>:
> Subject: Re: [PATCH][RFC][0/4] InfiniBand userspace verbs implementation
>
> Roland> Yes, because the kernel may go through and unmap pages
> Roland> from userspace while trying to swap. Since we have the
>
Roland> Yes, because the kernel may go through and unmap pages
Roland> from userspace while trying to swap. Since we have the
Roland> page locked in the kernel, the physical page won't go
Roland> anywhere, but userspace might end up with a different page
Roland> mapped at the s
Roland Dreier <[EMAIL PROTECTED]> wrote:
>
> Troy> Do we even need the mlock in userspace then?
>
> Yes, because the kernel may go through and unmap pages from userspace
> while trying to swap. Since we have the page locked in the kernel,
> the physical page won't go anywhere, but userspace m
Troy> Do we even need the mlock in userspace then?
Yes, because the kernel may go through and unmap pages from userspace
while trying to swap. Since we have the page locked in the kernel,
the physical page won't go anywhere, but userspace might end up with a
different page mapped at the same
On Mon, Apr 11, 2005 at 09:56:53AM -0700, Roland Dreier wrote:
> Troy> Is there a check in the kernel that the memory is actually
> Troy> mlock()ed?
>
> No.
>
> Troy> What if a malicious (or broken) application does
> Troy> ibv_reg_mr() but doesn't lock the memory? Does the IB car
Troy> Is there a check in the kernel that the memory is actually
Troy> mlock()ed?
No.
Troy> What if a malicious (or broken) application does
Troy> ibv_reg_mr() but doesn't lock the memory? Does the IB card
Troy> get a physical address for a page that might get swapped
Troy
On Mon, Apr 11, 2005 at 08:34:19AM -0700, Roland Dreier wrote:
> Troy> How is memory pinning handled? (I haven't had time to read
> Troy> all the code, so please excuse my ignorance of something
> Troy> obvious).
>
> The userspace library calls mlock() and then the kernel does
> get_us
Troy> How is memory pinning handled? (I haven't had time to read
Troy> all the code, so please excuse my ignorance of something
Troy> obvious).
The userspace library calls mlock() and then the kernel does
get_user_pages().
Troy> The old mellanox drivers used to have a hack to call
> In particular, the memory pinning code in in uverbs_mem.c could stand
> a looking over. In addition, a sanity check of the write()-based
> scheme for passing commands into the kernel in uverbs_main.c and
> uverbs_cmd.c is probably worthwhile.
How is memory pinning handled? (I haven't had time t
Here is an initial implementation of InfiniBand userspace verbs. I
plan to commit this code to the OpenIB repository shortly, and submit
it for inclusion during the 2.6.13 cycle, so I am posting it early for
comments.
This code, in conjunction with the libibverbs and libmthca userspace
libraries
41 matches
Mail list logo
