Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-02-06 Thread Simona Vetter
On Thu, Jan 30, 2025 at 01:42:17PM -0400, Jason Gunthorpe wrote: > On Thu, Jan 30, 2025 at 05:09:44PM +0100, Simona Vetter wrote: > > > > An optional callback is a lot less scary to me here (or redoing > > > > hmm_range_fault or whacking the migration helpers a few times) looks a > > > > lot > > >

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-02-06 Thread Simona Vetter
On Wed, Jan 29, 2025 at 09:47:57AM -0400, Jason Gunthorpe wrote: > On Wed, Jan 29, 2025 at 02:38:58PM +0100, Simona Vetter wrote: > > > > The pgmap->owner doesn't *have* to fixed, certainly during early boot > > > before > > > you hand out any page references it can be changed. I wouldn't be > >

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-02-06 Thread Simona Vetter
On Tue, Jan 28, 2025 at 01:21:23PM -0400, Jason Gunthorpe wrote: > On Tue, Jan 28, 2025 at 05:32:23PM +0100, Thomas Hellström wrote: > > > This series supports three case: > > > > > >  1) pgmap->owner == range->dev_private_owner > > >     This is "driver private fast interconnect" in this case HMM

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-02-06 Thread Simona Vetter
On Thu, Jan 30, 2025 at 09:23:17AM -0400, Jason Gunthorpe wrote: > On Thu, Jan 30, 2025 at 11:50:27AM +0100, Simona Vetter wrote: > > On Wed, Jan 29, 2025 at 09:47:57AM -0400, Jason Gunthorpe wrote: > > > On Wed, Jan 29, 2025 at 02:38:58PM +0100, Simona Vetter wrote: > > > > > > > > The pgmap->own

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-02-04 Thread Thomas Hellström
On Tue, 2025-02-04 at 15:16 -0400, Jason Gunthorpe wrote: > On Tue, Feb 04, 2025 at 03:29:48PM +0100, Thomas Hellström wrote: > > On Tue, 2025-02-04 at 09:26 -0400, Jason Gunthorpe wrote: > > > On Tue, Feb 04, 2025 at 10:32:32AM +0100, Thomas Hellström wrote: > > > > > > > > > > > 1) Existing use

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-02-04 Thread Jason Gunthorpe
On Tue, Feb 04, 2025 at 03:29:48PM +0100, Thomas Hellström wrote: > On Tue, 2025-02-04 at 09:26 -0400, Jason Gunthorpe wrote: > > On Tue, Feb 04, 2025 at 10:32:32AM +0100, Thomas Hellström wrote: > > > > > > > > 1) Existing users would never use the callback. They can still rely > > > on > > > th

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-02-04 Thread Thomas Hellström
On Tue, 2025-02-04 at 09:26 -0400, Jason Gunthorpe wrote: > On Tue, Feb 04, 2025 at 10:32:32AM +0100, Thomas Hellström wrote: > > > > > 1) Existing users would never use the callback. They can still rely > > on > > the owner check, only if that fails we check for callback > > existence. > > 2) By

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-02-04 Thread Jason Gunthorpe
On Tue, Feb 04, 2025 at 10:32:32AM +0100, Thomas Hellström wrote: > > I would not be happy to see this. Please improve pagemap directly if > > you think you need more things. > > These are mainly helpers to migrate and populate a range of cpu memory > space (struct mm_struct) with GPU device_priva

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-02-04 Thread Thomas Hellström
On Mon, 2025-02-03 at 11:08 -0400, Jason Gunthorpe wrote: > On Fri, Jan 31, 2025 at 05:59:26PM +0100, Simona Vetter wrote: > > > So one aspect where I don't like the pgmap->owner approach much is > > that > > it's a big thing to get right, and it feels a bit to me that we > > don't yet > > know th

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-02-03 Thread Jason Gunthorpe
On Fri, Jan 31, 2025 at 05:59:26PM +0100, Simona Vetter wrote: > So one aspect where I don't like the pgmap->owner approach much is that > it's a big thing to get right, and it feels a bit to me that we don't yet > know the right questions. Well, I would say it isn't really complete yet. No drive

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-01-30 Thread Jason Gunthorpe
On Thu, Jan 30, 2025 at 05:09:44PM +0100, Simona Vetter wrote: > > You could also use an integer instead of a pointer to indicate the > > cluster of interconnect, I think there are many options.. > > Hm yeah I guess an integer allocater of the atomic_inc kind plus "surely > 32bit is enough" could

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-01-30 Thread Jason Gunthorpe
On Thu, Jan 30, 2025 at 11:50:27AM +0100, Simona Vetter wrote: > On Wed, Jan 29, 2025 at 09:47:57AM -0400, Jason Gunthorpe wrote: > > On Wed, Jan 29, 2025 at 02:38:58PM +0100, Simona Vetter wrote: > > > > > > The pgmap->owner doesn't *have* to fixed, certainly during early boot > > > > before > >

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-01-29 Thread Thomas Hellström
On Wed, 2025-01-29 at 09:47 -0400, Jason Gunthorpe wrote: > On Wed, Jan 29, 2025 at 02:38:58PM +0100, Simona Vetter wrote: > > > > The pgmap->owner doesn't *have* to fixed, certainly during early > > > boot before > > > you hand out any page references it can be changed. I wouldn't be > > > surpri

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-01-29 Thread Jason Gunthorpe
On Wed, Jan 29, 2025 at 02:38:58PM +0100, Simona Vetter wrote: > > The pgmap->owner doesn't *have* to fixed, certainly during early boot before > > you hand out any page references it can be changed. I wouldn't be > > surprised if this is useful to some requirements to build up the > > private int

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-01-28 Thread Jason Gunthorpe
On Tue, Jan 28, 2025 at 05:32:23PM +0100, Thomas Hellström wrote: > > This series supports three case: > > > >  1) pgmap->owner == range->dev_private_owner > >     This is "driver private fast interconnect" in this case HMM > > should > >     immediately return the page. The calling driver underst

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-01-28 Thread Thomas Hellström
On Tue, 2025-01-28 at 11:16 -0400, Jason Gunthorpe wrote: > On Tue, Jan 28, 2025 at 03:48:54PM +0100, Thomas Hellström wrote: > > On Tue, 2025-01-28 at 09:20 -0400, Jason Gunthorpe wrote: > > > On Tue, Jan 28, 2025 at 09:51:52AM +0100, Thomas Hellström wrote: > > > > > > > How would the pgmap devi

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-01-28 Thread Jason Gunthorpe
On Tue, Jan 28, 2025 at 03:48:54PM +0100, Thomas Hellström wrote: > On Tue, 2025-01-28 at 09:20 -0400, Jason Gunthorpe wrote: > > On Tue, Jan 28, 2025 at 09:51:52AM +0100, Thomas Hellström wrote: > > > > > How would the pgmap device know whether P2P is actually possible > > > without knowing the c

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-01-28 Thread Thomas Hellström
On Tue, 2025-01-28 at 09:20 -0400, Jason Gunthorpe wrote: > On Tue, Jan 28, 2025 at 09:51:52AM +0100, Thomas Hellström wrote: > > > How would the pgmap device know whether P2P is actually possible > > without knowing the client device, (like calling > > pci_p2pdma_distance) > > and also if looking

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-01-28 Thread Jason Gunthorpe
On Tue, Jan 28, 2025 at 09:51:52AM +0100, Thomas Hellström wrote: > How would the pgmap device know whether P2P is actually possible > without knowing the client device, (like calling pci_p2pdma_distance) > and also if looking into access control, whether it is allowed? The DMA API will do this,

Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages

2025-01-28 Thread Thomas Hellström
Hi, Jonatan On Sun, 2024-12-01 at 12:36 +0200, Yonatan Maman wrote: > From: Yonatan Maman > > hmm_range_fault() by default triggered a page fault on device private > when HMM_PFN_REQ_FAULT flag was set. pages, migrating them to RAM. In > some > cases, such as with RDMA devices, the migration ov