> -----Original Message----- > From: Ruhl, Michael J <michael.j.r...@intel.com> > Sent: August 22, 2023 7:44 AM > To: Felix Kuehling <felix.kuehl...@amd.com>; Zeng, Oak <oak.z...@intel.com>; > Dave Airlie <airl...@gmail.com> > Cc: Brost, Matthew <matthew.br...@intel.com>; Thomas Hellström > <thomas.hellst...@linux.intel.com>; Philip Yang <philip.y...@amd.com>; > Welty, Brian <brian.we...@intel.com>; dri-devel@lists.freedesktop.org; > Christian > König <christian.koe...@amd.com>; Vishwanathapura, Niranjana > <niranjana.vishwanathap...@intel.com>; intel...@lists.freedesktop.org > Subject: RE: Implement svm without BO concept in xe driver > > >-----Original Message----- > >From: Felix Kuehling <felix.kuehl...@amd.com> > >Sent: Monday, August 21, 2023 4:57 PM > >To: Zeng, Oak <oak.z...@intel.com>; Dave Airlie <airl...@gmail.com> > >Cc: Brost, Matthew <matthew.br...@intel.com>; Thomas Hellström > ><thomas.hellst...@linux.intel.com>; Philip Yang <philip.y...@amd.com>; > >Welty, Brian <brian.we...@intel.com>; dri-devel@lists.freedesktop.org; > >Christian König <christian.koe...@amd.com>; Vishwanathapura, Niranjana > ><niranjana.vishwanathap...@intel.com>; intel...@lists.freedesktop.org; > >Ruhl, Michael J <michael.j.r...@intel.com> > >Subject: Re: Implement svm without BO concept in xe driver > > > > > >On 2023-08-21 15:41, Zeng, Oak wrote: > >>> I have thought about emulating BO allocation APIs on top of system SVM. > >>> This was in the context of KFD where memory management is not tied into > >>> command submissions APIs, which would add a whole other layer of > >>> complexity. The main unsolved (unsolvable?) problem I ran into was, that > >>> there is no way to share SVM memory as DMABufs. So there is no good > >way > >>> to support applications that expect to share memory in that way. > >> Great point. I also discussed the dmabuf thing with Mike (cc'ed). dmabuf > >> is a > >particular technology created specially for the BO driver (and other driver) > >to > >share buffer b/t devices. Hmm/system SVM doesn't need this technology: > >malloc'ed memory by the nature is already shared b/t different devices (in > >one process) and CPU. We just can simply submit GPU kernel to all devices > >with malloc'ed memory and let kmd decide the memory placement (such as > >map in place or migrate). No need of buffer export/import in hmm/system > >SVM world. > > > >I disagree. DMABuf can be used for sharing memory between processes. And > >it can be used for sharing memory with 3rd-party devices via PCIe P2P > >(e.g. a Mellanox NIC). You cannot easily do that with malloc'ed memory. > >POSIX IPC requires that you know that you'll be sharing the memory at > >allocation time. It adds overhead. And because it's file-backed, it's > >currently incompatible with migration. And HMM currently doesn't have a > >solution for P2P. Any access by a different device causes a migration to > >system memory. > > Hey Oak, > > I think we were discussing this solution in the context of using the P2P_DMA > feature. This has an allocation path and a device 2 device capabilities.
I was thinking sharing malloc'ed memory b/t CPU and multiple devices inside one process. I thought this should work. With Felix's words above, I looked more details. Now I agree with Felix this doesn't work with hmm. And as Felix pointed out, POSIX IPC also doesn't work with hmm. Theoretically driver can do similar migration b/t device memory and file-backed memory, just as what we did with anonymous memory. But I am not sure whether people want to do that. Anyway, buffer sharing with hmm/system SVM seems a big open. I will not try to solve this problem for now. Cheers, Oak > > Mike > > > >Regards, > > Felix > > > > > >> > >> So yes from buffer sharing perspective, the design philosophy is also very > >different. > >> > >> Thanks, > >> Oak > >>
