Re: [PATCH v2 05/10] sysfs: treewide: constify attribute callback of bin_is_visible()

2024-11-04 Thread Jason Gunthorpe
sysfs.c | 2 +- > drivers/usb/core/sysfs.c| 2 +- > include/linux/sysfs.h | 30 +++--- > 12 files changed, 27 insertions(+), 26 deletions(-) For infiniband: Acked-by: Jason Gunthorpe

Re: [PATCH 3/3] RDMA/hfi1: Use RMW accessors for changing LNKCTL2

2024-05-03 Thread Jason Gunthorpe
On Fri, May 03, 2024 at 01:18:35PM +0300, Ilpo Järvinen wrote: > On Thu, 15 Feb 2024, Ilpo Järvinen wrote: > > > Convert open coded RMW accesses for LNKCTL2 to use > > pcie_capability_clear_and_set_word() which makes its easier to > > understand what the code tries to do. > > > > LNKCTL2 is not r

Re: [PATCH V2 1/5] drm/amdkfd: ignore crat by default

2023-08-11 Thread Jason Gunthorpe
On Mon, Aug 07, 2023 at 06:05:41PM -0400, Alex Deucher wrote: > We are dropping the IOMMUv2 path, so no need to enable this. > It's often buggy on consumer platforms anyway. > > Signed-off-by: Alex Deucher > --- > drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 4 > 1 file changed, 4 deletions(-)

Re: [PATCH 1/7] mm/gup: remove unused vmas parameter from get_user_pages()

2023-04-17 Thread Jason Gunthorpe
.c | 4 ++-- > 7 files changed, 11 insertions(+), 16 deletions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [regression, bisected, pci/iommu] Bug 216865 - Black screen when amdgpu started during 6.2-rc1 boot with AMD IOMMU enabled

2023-02-15 Thread Jason Gunthorpe
On Wed, Feb 15, 2023 at 07:35:45PM -0500, Felix Kuehling wrote: > > If I understand this correctly, the HW or the BIOS is doing something wrong > about reporting ACS. I don't know what the GPU driver can do other than add > some quirk to stop using AMD IOMMUv2 on this HW/BIOS. How about this: di

Re: [regression, bisected, pci/iommu] Bug 216865 - Black screen when amdgpu started during 6.2-rc1 boot with AMD IOMMU enabled

2023-01-11 Thread Jason Gunthorpe
On Wed, Jan 11, 2023 at 11:16:32AM +0800, Baolu Lu wrote: > On 2023/1/10 21:25, Jason Gunthorpe wrote: > > > + } else { > > > + if (!pdev->bus->self || > > > + !pci_ac

Re: [regression, bisected, pci/iommu] Bug 216865 - Black screen when amdgpu started during 6.2-rc1 boot with AMD IOMMU enabled

2023-01-10 Thread Jason Gunthorpe
On Tue, Jan 10, 2023 at 10:05:44AM -0500, Felix Kuehling wrote: > Am 2023-01-10 um 08:45 schrieb Christian König: > > And I'm like 99% sure that Kabini/Wani should be identical to that. > > Kabini is not supported by KFD. There should be no calls to amd_iommu_... > functions on Kabini, at least no

Re: [regression, bisected, pci/iommu] Bug 216865 - Black screen when amdgpu started during 6.2-rc1 boot with AMD IOMMU enabled

2023-01-10 Thread Jason Gunthorpe
On Tue, Jan 10, 2023 at 01:48:39PM +0800, Baolu Lu wrote: > On 2023/1/6 22:14, Jason Gunthorpe wrote: > > On Thu, Jan 05, 2023 at 03:57:28PM +0530, Vasant Hegde wrote: > > > Matt, > > > > > > On 1/5/2023 6:39 AM, Matt Fagnani wrote: > > > > I buil

Re: [regression, bisected, pci/iommu] Bug 216865 - Black screen when amdgpu started during 6.2-rc1 boot with AMD IOMMU enabled

2023-01-10 Thread Jason Gunthorpe
On Tue, Jan 10, 2023 at 02:45:30PM +0100, Christian König wrote: > Since this is a device integrated in the CPU it could be that the ACS/ATS > functionalities are controlled by the BIOS and can be enabled/disabled > there. But this should always enable/disable both. This sounds like a GPU driver

Re: [PATCH v6 10/21] RDMA/umem: Prepare to dynamic dma-buf locking specification

2022-10-10 Thread Jason Gunthorpe
> > - dma_buf_unmap_attachment(umem_dmabuf->attach, umem_dmabuf->sgt, > > -DMA_BIDIRECTIONAL); > > + dma_buf_unmap_attachment_unlocked(umem_dmabuf->attach, umem_dmabuf->sgt, > > + DMA_BIDIRECTIONAL); > > > > umem_dmabuf->sgt = NULL; > > } > > Jason / Leon, > > Could you please ack this patch? You probably don't need it, for something so simple, but sure Acked-by: Jason Gunthorpe Jason

Re: [PATCH 2/7] mm: Free device private pages have zero refcount

2022-09-26 Thread Jason Gunthorpe
On Mon, Sep 26, 2022 at 04:03:06PM +1000, Alistair Popple wrote: > Since 27674ef6c73f ("mm: remove the extra ZONE_DEVICE struct page > refcount") device private pages have no longer had an extra reference > count when the page is in use. However before handing them back to the > owning device drive

Re: [REGRESSION 5.19.x] AMD HD-audio devices missing on 5.19

2022-08-24 Thread Jason Gunthorpe
On Tue, Aug 23, 2022 at 10:01:57PM +0100, Robin Murphy wrote: > > diff --git a/drivers/iommu/amd/iommu_v2.c b/drivers/iommu/amd/iommu_v2.c > > index 696dbe5794..6a1f02c62dffcc 100644 > > --- a/drivers/iommu/amd/iommu_v2.c > > +++ b/drivers/iommu/amd/iommu_v2.c > > @@ -777,6 +777,8 @@ int amd_i

Re: [PATCH v1 13/15] mm: handling Non-LRU pages returned by vm_normal_pages

2022-05-13 Thread Jason Gunthorpe
On Thu, May 12, 2022 at 05:33:44PM -0500, Sierra Guiza, Alejandro (Alex) wrote: > > On 5/11/2022 1:50 PM, Jason Gunthorpe wrote: > > On Thu, May 05, 2022 at 04:34:36PM -0500, Alex Sierra wrote: > > > > > diff --git a/mm/memory.c b/mm/memory.c > > > ind

Re: [PATCH v1 13/15] mm: handling Non-LRU pages returned by vm_normal_pages

2022-05-11 Thread Jason Gunthorpe
On Thu, May 05, 2022 at 04:34:36PM -0500, Alex Sierra wrote: > diff --git a/mm/memory.c b/mm/memory.c > index 76e3af9639d9..892c4cc54dc2 100644 > +++ b/mm/memory.c > @@ -621,6 +621,13 @@ struct page *vm_normal_page(struct vm_area_struct *vma, > unsigned long addr, > if (is_zero_pfn(

Re: [PATCH v2 1/3] mm: add vm_normal_lru_pages for LRU handled pages only

2022-04-04 Thread Jason Gunthorpe
On Fri, Apr 01, 2022 at 04:08:35PM -0400, Felix Kuehling wrote: > > In general I find the vm_normal_lru_page vs vm_normal_page > > API highly confusing. An explicit check for zone device pages > > in the dozen or so spots that care has a much better documentation > > value, especially if accompan

Re: [PATCH v1 1/3] mm: split vm_normal_pages for LRU and non-LRU handling

2022-03-17 Thread Jason Gunthorpe
On Thu, Mar 17, 2022 at 09:13:50AM +0100, David Hildenbrand wrote: > On 17.03.22 03:54, Alistair Popple wrote: > > Felix Kuehling writes: > > > >> On 2022-03-11 04:16, David Hildenbrand wrote: > >>> On 10.03.22 18:26, Alex Sierra wrote: > DEVICE_COHERENT pages introduce a subtle distinction

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-18 Thread Jason Gunthorpe
On Fri, Feb 18, 2022 at 02:20:45PM -0500, Felix Kuehling wrote: > Am 2022-02-17 um 19:19 schrieb Jason Gunthorpe: > > On Thu, Feb 17, 2022 at 04:12:20PM -0500, Felix Kuehling wrote: > > > > > I'm thinking of a more theoretical approach: Instead of auditing all >

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-18 Thread Jason Gunthorpe
On Thu, Feb 17, 2022 at 04:12:20PM -0500, Felix Kuehling wrote: > I'm thinking of a more theoretical approach: Instead of auditing all users, > I'd ask, what are the invariants that a vm_normal_page should have. Then > check, whether our DEVICE_COHERENT pages satisfy them. But maybe the concept >

Re: [PATCH 4/4] kunit: tool: Disable broken options for --alltests

2022-02-18 Thread Jason Gunthorpe
On Fri, Feb 18, 2022 at 03:57:27PM +0800, David Gow wrote: > There are a number of Kconfig options which break compilation under UML with > allyesconfig. As kunit_tool's --alltests option is based on allyesconfig and > UML, we need to update the list of broken options to make --alltests build > ag

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-16 Thread Jason Gunthorpe
On Wed, Feb 16, 2022 at 11:56:51AM -0500, Felix Kuehling wrote: > In the case of DEVICE_COHERENT memory, the pfns correspond to real physical > memory addresses. I don't think they have those PFN_DEV|PFN_MAP bits set. So do DAX pages. The PTE flag does several things. As this would be the first t

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-16 Thread Jason Gunthorpe
On Wed, Feb 16, 2022 at 09:31:03AM +0100, David Hildenbrand wrote: > On 16.02.22 03:36, Alistair Popple wrote: > > On Wednesday, 16 February 2022 1:03:57 PM AEDT Jason Gunthorpe wrote: > >> On Wed, Feb 16, 2022 at 12:23:44PM +1100, Alistair Popple wrote: > >> >

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-16 Thread Jason Gunthorpe
On Tue, Feb 15, 2022 at 05:49:07PM -0500, Felix Kuehling wrote: > > Userspace does > > 1) mmap(MAP_PRIVATE) to allocate anon memory > > 2) something to trigger migration to install a ZONE_DEVICE page > > 3) munmap() > > > > Who decrements the refcout on the munmap? > > > > When a ZONE_DEVI

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-16 Thread Jason Gunthorpe
On Wed, Feb 16, 2022 at 12:23:44PM +1100, Alistair Popple wrote: > Device private and device coherent pages are not marked with pte_devmap and > they > are backed by a struct page. The only way of inserting them is via > migrate_vma. > The refcount is decremented in zap_pte_range() on munmap() w

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-15 Thread Jason Gunthorpe
On Tue, Feb 15, 2022 at 04:35:56PM -0500, Felix Kuehling wrote: > > On 2022-02-15 14:41, Jason Gunthorpe wrote: > > On Tue, Feb 15, 2022 at 07:32:09PM +0100, Christoph Hellwig wrote: > > > On Tue, Feb 15, 2022 at 10:45:24AM -0400, Jason Gunthorpe wrote: > > > >

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-15 Thread Jason Gunthorpe
On Tue, Feb 15, 2022 at 07:32:09PM +0100, Christoph Hellwig wrote: > On Tue, Feb 15, 2022 at 10:45:24AM -0400, Jason Gunthorpe wrote: > > > Do you know if DEVICE_GENERIC pages would end up as PageAnon()? My > > > assumption was that they would be part of a special mapping. >

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-15 Thread Jason Gunthorpe
On Tue, Feb 15, 2022 at 01:16:43PM +0100, David Hildenbrand wrote: > > fact, the first version of our patches attempted to add migration > > support to DEVICE_GENERIC. But we were convinced to create a new > > ZONE_DEVICE page type for our use case instead. > > Do you know if DEVICE_GENERIC page

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-11 Thread Jason Gunthorpe
On Fri, Feb 11, 2022 at 05:49:08PM +0100, David Hildenbrand wrote: > On 11.02.22 17:45, Jason Gunthorpe wrote: > > On Fri, Feb 11, 2022 at 05:15:25PM +0100, David Hildenbrand wrote: > > > >> ... I'm pretty sure we cannot FOLL_PIN DEVICE_PRIVATE pages > >

Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

2022-02-11 Thread Jason Gunthorpe
On Fri, Feb 11, 2022 at 05:15:25PM +0100, David Hildenbrand wrote: > ... I'm pretty sure we cannot FOLL_PIN DEVICE_PRIVATE pages Currently the only way to get a DEVICE_PRIVATE page out of the page tables is via hmm_range_fault() and that doesn't manipulate any ref counts. Jason

Re: [PATCH 7/8] mm: remove the extra ZONE_DEVICE struct page refcount

2022-02-09 Thread Jason Gunthorpe
On Wed, Feb 09, 2022 at 02:53:51PM +0100, Christoph Hellwig wrote: > On Wed, Feb 09, 2022 at 08:29:56AM -0400, Jason Gunthorpe wrote: > > It is nice, but the other series are still impacted by the fsdax mess > > - they still stuff pages into ptes without proper refcounts and ha

Re: [PATCH 7/8] mm: remove the extra ZONE_DEVICE struct page refcount

2022-02-09 Thread Jason Gunthorpe
On Wed, Feb 09, 2022 at 07:23:45AM +0100, Christoph Hellwig wrote: > On Tue, Feb 08, 2022 at 07:30:11PM -0800, Dan Williams wrote: > > Interesting. I had expected that to really fix the refcount problem > > that fs/dax.c would need to start taking real page references as pages > > were added to a m

Re: [PATCH 8/8] fsdax: depend on ZONE_DEVICE || FS_DAX_LIMITED

2022-02-07 Thread Jason Gunthorpe
fs/Kconfig | 1 + > 1 file changed, 1 insertion(+) Makes sense, but leaves me wonder why a kconfig randomizer didn't hit this.. Or maybe it means some of the function stubs on !ZONE_DEVICE are unnecessary now.. Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH 6/8] mm: don't include in

2022-02-07 Thread Jason Gunthorpe
> include/linux/mm.h | 20 > lib/test_hmm.c | 1 + > mm/memremap.c | 6 +- > 14 files changed, 34 insertions(+), 22 deletions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH 5/8] mm: simplify freeing of devmap managed pages

2022-02-07 Thread Jason Gunthorpe
de/linux/mm.h | 34 ++ > mm/memremap.c | 20 +--- > mm/swap.c | 10 +- > 3 files changed, 20 insertions(+), 44 deletions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH 4/8] mm: move free_devmap_managed_page to memremap.c

2022-02-07 Thread Jason Gunthorpe
- > mm/memremap.c | 21 + > mm/swap.c | 23 --- > 3 files changed, 21 insertions(+), 24 deletions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH 2/8] mm: remove the __KERNEL__ guard from

2022-02-07 Thread Jason Gunthorpe
On Mon, Feb 07, 2022 at 07:32:43AM +0100, Christoph Hellwig wrote: > __KERNEL__ ifdefs don't make sense outside of include/uapi/. > > Signed-off-by: Christoph Hellwig > --- > include/linux/mm.h | 4 > 1 file changed, 4 deletions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH 1/8] mm: remove a pointless CONFIG_ZONE_DEVICE check in memremap_pages

2022-02-07 Thread Jason Gunthorpe
tions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH 7/8] mm: remove the extra ZONE_DEVICE struct page refcount

2022-02-07 Thread Jason Gunthorpe
++--- > mm/memremap.c| 57 > mm/migrate.c | 6 --- > mm/swap.c| 16 ++- > 13 files changed, 36 insertions(+), 83 deletions(-) It looks like a good next step to me Rev

Re: [PATCH 3/8] mm: remove pointless includes from

2022-02-07 Thread Jason Gunthorpe
1 + > drivers/gpu/drm/nouveau/nouveau_dmem.c | 1 + > include/linux/hmm.h | 9 ++--- > lib/test_hmm.c | 2 ++ > 4 files changed, 6 insertions(+), 7 deletions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH v4 00/10] Add MEMORY_DEVICE_COHERENT for coherent device memory mapping

2022-02-02 Thread Jason Gunthorpe
On Wed, Feb 02, 2022 at 03:57:50PM +0100, Christoph Hellwig wrote: > On Thu, Jan 27, 2022 at 02:32:58PM -0800, Andrew Morton wrote: > > On Wed, 26 Jan 2022 21:09:39 -0600 Alex Sierra wrote: > > > > > This patch series introduces MEMORY_DEVICE_COHERENT, a type of memory > > > owned by a device tha

Re: [PATCH v2 03/11] mm/gup: migrate PIN_LONGTERM dev coherent pages to system

2021-12-09 Thread Jason Gunthorpe
On Thu, Dec 09, 2021 at 12:45:24PM +1100, Alistair Popple wrote: > On Thursday, 9 December 2021 12:53:45 AM AEDT Jason Gunthorpe wrote: > > > I think a similar problem exists for device private fault handling as > > > well and > > > it has been on my list of things t

Re: [PATCH v2 03/11] mm/gup: migrate PIN_LONGTERM dev coherent pages to system

2021-12-09 Thread Jason Gunthorpe
On Wed, Dec 08, 2021 at 10:31:58PM +1100, Alistair Popple wrote: > On Tuesday, 7 December 2021 5:52:43 AM AEDT Alex Sierra wrote: > > Avoid long term pinning for Coherent device type pages. This could > > interfere with their own device memory manager. > > If caller tries to get user device coheren

Re: [PATCH v2 11/11] tools: add hmm gup test for long term pinned device pages

2021-12-07 Thread Jason Gunthorpe
On Mon, Dec 06, 2021 at 12:52:51PM -0600, Alex Sierra wrote: > The intention is to test device coherent type pages that have been > called through get user pages with PIN_LONGTERM flag set. > > Signed-off-by: Alex Sierra > tools/testing/selftests/vm/Makefile| 2 +- > tools/testing/selftests

Re: [PATCH v1 2/2] mm: remove extra ZONE_DEVICE struct page refcount

2021-10-19 Thread Jason Gunthorpe
On Tue, Oct 19, 2021 at 04:13:34PM +0100, Joao Martins wrote: > On 10/19/21 00:06, Jason Gunthorpe wrote: > > On Mon, Oct 18, 2021 at 12:37:30PM -0700, Dan Williams wrote: > > > >>> device-dax uses PUD, along with TTM, they are the only places. I'm not >

Re: [PATCH v1 2/2] mm: remove extra ZONE_DEVICE struct page refcount

2021-10-18 Thread Jason Gunthorpe
On Mon, Oct 18, 2021 at 12:37:30PM -0700, Dan Williams wrote: > > device-dax uses PUD, along with TTM, they are the only places. I'm not > > sure TTM is a real place though. > > I was setting device-dax aside because it can use Joao's changes to > get compound-page support. Ideally, but that ide

Re: [PATCH v1 2/2] mm: remove extra ZONE_DEVICE struct page refcount

2021-10-18 Thread Jason Gunthorpe
On Sun, Oct 17, 2021 at 11:35:35AM -0700, Dan Williams wrote: > > DAX is stuffing arrays of 4k pages into the PUD/PMDs. Aligning with > > THP would make using normal refconting much simpler. I looked at > > teaching the mm core to deal with page arrays - it is certainly > > doable, but it is quite

Re: [PATCH v1 2/2] mm: remove extra ZONE_DEVICE struct page refcount

2021-10-18 Thread Jason Gunthorpe
On Thu, Oct 14, 2021 at 10:45:52PM -0500, Sierra Guiza, Alejandro (Alex) wrote: > > On 10/14/2021 3:57 PM, Ralph Campbell wrote: > > > > On 10/14/21 11:01 AM, Jason Gunthorpe wrote: > > > On Thu, Oct 14, 2021 at 10:35:27AM -0700, Ralph Campbell wrote: > > >

Re: [PATCH v1 2/2] mm: remove extra ZONE_DEVICE struct page refcount

2021-10-16 Thread Jason Gunthorpe
On Thu, Oct 14, 2021 at 06:37:35PM -0700, Dan Williams wrote: > On Thu, Oct 14, 2021 at 4:06 PM Jason Gunthorpe wrote: > > > > On Thu, Oct 14, 2021 at 12:01:14PM -0700, Dan Williams wrote: > > > > > Does anyone know why devmap is pte_special anyhow? > > >

Re: [PATCH v1 1/2] ext4/xfs: add page refcount helper

2021-10-15 Thread Jason Gunthorpe
: This ref count functionality was missing on fuse/dax.c. > --- > fs/dax.c| 4 ++-- > fs/ext4/inode.c | 5 + > fs/fuse/dax.c | 4 +--- > fs/xfs/xfs_file.c | 4 +--- > include/linux/dax.h | 10 ++ > 5 files changed, 15 insertions(+), 12 deletions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH v1 2/2] mm: remove extra ZONE_DEVICE struct page refcount

2021-10-15 Thread Jason Gunthorpe
On Thu, Oct 14, 2021 at 10:35:27AM -0700, Ralph Campbell wrote: > I ran xfstests-dev using the kernel boot option to "fake" a pmem device > when I first posted this patch. The tests ran OK (or at least the same > tests passed with and without my patch). Hmm. I know nothing of xfstests but test

Re: [PATCH v1 2/2] mm: remove extra ZONE_DEVICE struct page refcount

2021-10-15 Thread Jason Gunthorpe
On Thu, Oct 14, 2021 at 10:39:28AM -0500, Alex Sierra wrote: > From: Ralph Campbell > > ZONE_DEVICE struct pages have an extra reference count that complicates the > code for put_page() and several places in the kernel that need to check the > reference count to see that a page is not being used

Re: [PATCH v1 2/2] mm: remove extra ZONE_DEVICE struct page refcount

2021-10-14 Thread Jason Gunthorpe
On Thu, Oct 14, 2021 at 12:01:14PM -0700, Dan Williams wrote: > > > Does anyone know why devmap is pte_special anyhow? > > It does not need to be special as mentioned here: > > https://lore.kernel.org/all/CAPcyv4iFeVDVPn6uc=aksyuvkiu3-fk-n16ijvzq3n8ot00...@mail.gmail.com/ I added a remark there

Re: [PATCH v1 00/12] MEMORY_DEVICE_COHERENT for CPU-accessible coherent device memory

2021-10-12 Thread Jason Gunthorpe
On Tue, Oct 12, 2021 at 11:39:57AM -0700, Andrew Morton wrote: > On Tue, 12 Oct 2021 12:12:35 -0500 Alex Sierra wrote: > > > This patch series introduces MEMORY_DEVICE_COHERENT, a type of memory > > owned by a device that can be mapped into CPU page tables like > > MEMORY_DEVICE_GENERIC and can a

Re: [PATCH v6 2/2] habanalabs: add support for dma-buf exporter

2021-10-01 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 12:17:35AM +0300, Oded Gabbay wrote: > On Tue, Sep 28, 2021 at 8:36 PM Jason Gunthorpe wrote: > > > > On Sun, Sep 12, 2021 at 07:53:09PM +0300, Oded Gabbay wrote: > > > From: Tomer Tayar > > > > > > Implement the calls to

Re: [PATCH v6 2/2] habanalabs: add support for dma-buf exporter

2021-10-01 Thread Jason Gunthorpe
On Thu, Sep 30, 2021 at 03:46:35PM +0300, Oded Gabbay wrote: > After reading the kernel iommu code, I think this is not relevant > here, and I'll add a comment appropriately but I'll also write it > here, and please correct me if my understanding is wrong. > > The memory behind this specific dma-

Re: [PATCH v6 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-09-28 Thread Jason Gunthorpe
On Sun, Sep 12, 2021 at 07:53:08PM +0300, Oded Gabbay wrote: > /* HL_MEM_OP_* */ > __u32 op; > - /* HL_MEM_* flags */ > + /* HL_MEM_* flags. > + * For the HL_MEM_OP_EXPORT_DMABUF_FD opcode, this field holds the > + * DMA-BUF file/FD flags. > + */ > __u32 fla

Re: [PATCH v6 2/2] habanalabs: add support for dma-buf exporter

2021-09-28 Thread Jason Gunthorpe
On Sun, Sep 12, 2021 at 07:53:09PM +0300, Oded Gabbay wrote: > From: Tomer Tayar > > Implement the calls to the dma-buf kernel api to create a dma-buf > object backed by FD. > > We block the option to mmap the DMA-BUF object because we don't support > DIRECT_IO and implicit P2P. This statement

Re: [PATCH v6 0/2] Add p2p via dmabuf to habanalabs

2021-09-16 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 02:31:34PM +0200, Daniel Vetter wrote: > On Wed, Sep 15, 2021 at 10:45:36AM +0300, Oded Gabbay wrote: > > On Tue, Sep 14, 2021 at 7:12 PM Jason Gunthorpe wrote: > > > > > > On Tue, Sep 14, 2021 at 04:18:31PM +0200, Daniel Vetter wrote: > >

Re: [PATCH v6 0/2] Add p2p via dmabuf to habanalabs

2021-09-14 Thread Jason Gunthorpe
On Tue, Sep 14, 2021 at 04:18:31PM +0200, Daniel Vetter wrote: > On Sun, Sep 12, 2021 at 07:53:07PM +0300, Oded Gabbay wrote: > > Hi, > > Re-sending this patch-set following the release of our user-space TPC > > compiler and runtime library. > > > > I would appreciate a review on this. > > I thin

Re: [PATCH v4 10/13] lib: test_hmm add module param for zone device type

2021-07-22 Thread Jason Gunthorpe
On Thu, Jul 22, 2021 at 11:59:17AM -0500, Sierra Guiza, Alejandro (Alex) wrote: > > On 7/22/2021 7:23 AM, Jason Gunthorpe wrote: > > On Sat, Jul 17, 2021 at 02:21:32PM -0500, Alex Sierra wrote: > > > In order to configure device generic in test_hmm, two > > > modu

Re: [PATCH v4 10/13] lib: test_hmm add module param for zone device type

2021-07-22 Thread Jason Gunthorpe
On Sat, Jul 17, 2021 at 02:21:32PM -0500, Alex Sierra wrote: > In order to configure device generic in test_hmm, two > module parameters should be passed, which correspon to the > SP start address of each device (2) spm_addr_dev0 & > spm_addr_dev1. If no parameters are passed, private device > type

Re: [PATCH v4 0/2] Add p2p via dmabuf to habanalabs

2021-07-06 Thread Jason Gunthorpe
On Tue, Jul 06, 2021 at 07:35:55PM +0200, Daniel Vetter wrote: > Yup. We dont care about any of the fancy pieces you build on top, nor > does the compiler need to be the optimizing one. Just something that's > good enough to drive the hw in some demons to see how it works and all > that. Generally

Re: [PATCH v4 0/2] Add p2p via dmabuf to habanalabs

2021-07-06 Thread Jason Gunthorpe
On Tue, Jul 06, 2021 at 07:31:37PM +0200, Christoph Hellwig wrote: > On Tue, Jul 06, 2021 at 02:28:28PM -0300, Jason Gunthorpe wrote: > > > Also on your claim that drivers/gpu is a non-upstream disaster: I've > > > also learned that that for drivers/rdma there's the

Re: [PATCH v4 0/2] Add p2p via dmabuf to habanalabs

2021-07-06 Thread Jason Gunthorpe
On Tue, Jul 06, 2021 at 06:07:17PM +0200, Daniel Vetter wrote: > Also on your claim that drivers/gpu is a non-upstream disaster: I've > also learned that that for drivers/rdma there's the upstream driver, > and then there's the out-of-tree hackjob the vendor actually > supports. In the enterprise

Re: [PATCH v4 0/2] Add p2p via dmabuf to habanalabs

2021-07-06 Thread Jason Gunthorpe
On Tue, Jul 06, 2021 at 04:09:25PM +0200, Daniel Vetter wrote: > Anyway, for anything that works like a gpu accelerator, like 3d accel, > or parallel compute accel (aka gpgpu) or spatial compute accel (aka > NN/AI) or maybe even fpga accel most of the magic to use the hardware > is in this backend

Re: [PATCH v4 0/2] Add p2p via dmabuf to habanalabs

2021-07-06 Thread Jason Gunthorpe
On Tue, Jul 06, 2021 at 12:36:51PM +0200, Daniel Vetter wrote: > If that means AI companies don't want to open our their hw specs > enough to allow that, so be it - all you get in that case is > offloading the kernel side of the stack for convenience, with zero > long term prospects to ever make

Re: [PATCH v4 0/2] Add p2p via dmabuf to habanalabs

2021-07-06 Thread Jason Gunthorpe
On Tue, Jul 06, 2021 at 04:39:19PM +0200, Daniel Vetter wrote: > On Tue, Jul 6, 2021 at 4:23 PM Jason Gunthorpe wrote: > > > > On Tue, Jul 06, 2021 at 12:36:51PM +0200, Daniel Vetter wrote: > > > > > If that means AI companies don't want to open our their hw s

Re: [PATCH v4 0/2] Add p2p via dmabuf to habanalabs

2021-07-06 Thread Jason Gunthorpe
On Tue, Jul 06, 2021 at 05:49:01PM +0200, Daniel Vetter wrote: > The other thing to keep in mind is that one of these drivers supports > 25 years of product generations, and the other one doesn't. Sure, but that is the point, isn't it? To have an actually useful thing you need all of this mess

Re: [PATCH v4 2/2] habanalabs: add support for dma-buf exporter

2021-07-06 Thread Jason Gunthorpe
On Tue, Jul 06, 2021 at 12:44:49PM +0300, Oded Gabbay wrote: > > > + /* In case we got a large memory area to export, we need to divide > > > it > > > + * to smaller areas because each entry in the dmabuf sgt can only > > > + * describe unsigned int. > > > + */ > > > > Huh? Thi

Re: [PATCH v4 0/2] Add p2p via dmabuf to habanalabs

2021-07-06 Thread Jason Gunthorpe
On Tue, Jul 06, 2021 at 02:07:16PM +0200, Daniel Vetter wrote: > On the "rdma-core" idea, afaik rdma NIC do not have fully programmable > cores in their hw, for which you'd need some kind of compiler to make > use of the hardware and the interfaces the kernel provides? So not > really compareable,

Re: [PATCH v4 2/2] habanalabs: add support for dma-buf exporter

2021-07-06 Thread Jason Gunthorpe
On Mon, Jul 05, 2021 at 04:03:14PM +0300, Oded Gabbay wrote: > + rc = sg_alloc_table(*sgt, nents, GFP_KERNEL | __GFP_ZERO); > + if (rc) > + goto error_free; If you are not going to include a CPU list then I suggest setting sg_table->orig_nents == 0 And using only the nents wh

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-24 Thread Jason Gunthorpe
On Wed, Jun 23, 2021 at 10:39:48PM +0300, Oded Gabbay wrote: > On Wed, Jun 23, 2021 at 10:34 PM Jason Gunthorpe wrote: > > > > On Wed, Jun 23, 2021 at 10:00:29PM +0300, Oded Gabbay wrote: > > > On Wed, Jun 23, 2021 at 9:50 PM Jason Gunthorpe wrote: > > > > &

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-24 Thread Jason Gunthorpe
On Wed, Jun 23, 2021 at 10:00:29PM +0300, Oded Gabbay wrote: > On Wed, Jun 23, 2021 at 9:50 PM Jason Gunthorpe wrote: > > > > On Wed, Jun 23, 2021 at 09:43:04PM +0300, Oded Gabbay wrote: > > > > > Can you please explain why it is so important to (allow) acc

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-24 Thread Jason Gunthorpe
On Wed, Jun 23, 2021 at 10:57:35AM +0200, Christian König wrote: > > > No it isn't. It makes devices depend on allocating struct pages for their > > > BARs which is not necessary nor desired. > > Which dramatically reduces the cost of establishing DMA mappings, a > > loop of dma_map_resource() is

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-24 Thread Jason Gunthorpe
On Wed, Jun 23, 2021 at 09:43:04PM +0300, Oded Gabbay wrote: > Can you please explain why it is so important to (allow) access them > through the CPU ? It is not so much important, as it reflects significant design choices that are already tightly baked into alot of our stacks. A SGL is CPU acc

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-22 Thread Jason Gunthorpe
On Tue, Jun 22, 2021 at 02:23:03PM +0200, Christian König wrote: > Am 22.06.21 um 14:01 schrieb Jason Gunthorpe: > > On Tue, Jun 22, 2021 at 11:42:27AM +0300, Oded Gabbay wrote: > > > On Tue, Jun 22, 2021 at 9:37 AM Christian König > > > wrote: > > > > Am 22

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-22 Thread Jason Gunthorpe
On Tue, Jun 22, 2021 at 05:48:10PM +0200, Christian König wrote: > Am 22.06.21 um 17:40 schrieb Jason Gunthorpe: > > On Tue, Jun 22, 2021 at 05:29:01PM +0200, Christian König wrote: > > > [SNIP] > > > No absolutely not. NVidia GPUs work exactly the same way. > &

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-22 Thread Jason Gunthorpe
On Tue, Jun 22, 2021 at 05:29:01PM +0200, Christian König wrote: > Am 22.06.21 um 17:23 schrieb Jason Gunthorpe: > > On Tue, Jun 22, 2021 at 02:23:03PM +0200, Christian König wrote: > > > Am 22.06.21 um 14:01 schrieb Jason Gunthorpe: > > > > On Tue, Jun 22, 2021 at

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-22 Thread Jason Gunthorpe
On Tue, Jun 22, 2021 at 06:24:28PM +0300, Oded Gabbay wrote: > On Tue, Jun 22, 2021 at 6:11 PM Jason Gunthorpe wrote: > > > > On Tue, Jun 22, 2021 at 04:12:26PM +0300, Oded Gabbay wrote: > > > > > > 1) Setting sg_page to NULL > > > > 2) 'mapping&

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-22 Thread Jason Gunthorpe
On Tue, Jun 22, 2021 at 04:12:26PM +0300, Oded Gabbay wrote: > > 1) Setting sg_page to NULL > > 2) 'mapping' pages for P2P DMA without going through the iommu > > 3) Allowing P2P DMA without using the p2p dma API to validate that it > >can work at all in the first place. > > > > All of these r

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-22 Thread Jason Gunthorpe
On Tue, Jun 22, 2021 at 05:24:08PM +0200, Christian König wrote: > > > I will take two GAUDI devices and use one as an exporter and one as an > > > importer. I want to see that the solution works end-to-end, with real > > > device DMA from importer to exporter. > > I can tell you it doesn't. Stuff

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-22 Thread Jason Gunthorpe
On Tue, Jun 22, 2021 at 11:42:27AM +0300, Oded Gabbay wrote: > On Tue, Jun 22, 2021 at 9:37 AM Christian König > wrote: > > > > Am 22.06.21 um 01:29 schrieb Jason Gunthorpe: > > > On Mon, Jun 21, 2021 at 10:24:16PM +0300, Oded Gabbay wrote: > > > > > &g

Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-22 Thread Jason Gunthorpe
On Tue, Jun 22, 2021 at 03:04:30PM +0300, Oded Gabbay wrote: > On Tue, Jun 22, 2021 at 3:01 PM Jason Gunthorpe wrote: > > > > On Tue, Jun 22, 2021 at 11:42:27AM +0300, Oded Gabbay wrote: > > > On Tue, Jun 22, 2021 at 9:37 AM Christian König > > > wrote: >

Re: [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-21 Thread Jason Gunthorpe
On Mon, Jun 21, 2021 at 10:24:16PM +0300, Oded Gabbay wrote: > Another thing I want to emphasize is that we are doing p2p only > through the export/import of the FD. We do *not* allow the user to > mmap the dma-buf as we do not support direct IO. So there is no access > to these pages through the

Re: [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-21 Thread Jason Gunthorpe
On Mon, Jun 21, 2021 at 07:26:14PM +0300, Oded Gabbay wrote: > On Mon, Jun 21, 2021 at 5:12 PM Jason Gunthorpe wrote: > > > > On Mon, Jun 21, 2021 at 03:02:10PM +0200, Greg KH wrote: > > > On Mon, Jun 21, 2021 at 02:28:48PM +0200, Daniel Vetter wrote: > > > >

Re: [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-21 Thread Jason Gunthorpe
On Mon, Jun 21, 2021 at 04:20:35PM +0200, Daniel Vetter wrote: > Also unless we're actually doing this properly there's zero incentive for > me to review the kernel code and check whether it follows the rules > correctly, so you have excellent chances that you just break the rules. > And dma_buf/f

Re: [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-21 Thread Jason Gunthorpe
On Mon, Jun 21, 2021 at 02:28:48PM +0200, Daniel Vetter wrote: > Mission acomplished, we've gone full circle, and the totally-not-a-gpu > driver is now trying to use gpu infrastructure. And seems to have > gained vram meanwhile too. Next up is going to be synchronization > using dma_fence so you c

Re: [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF

2021-06-21 Thread Jason Gunthorpe
On Mon, Jun 21, 2021 at 03:02:10PM +0200, Greg KH wrote: > On Mon, Jun 21, 2021 at 02:28:48PM +0200, Daniel Vetter wrote: > > Also I'm wondering which is the other driver that we share buffers > > with. The gaudi stuff doesn't have real struct pages as backing > > storage, it only fills out the dm

Re: [PATCH] treewide: Add missing semicolons to __assign_str uses

2021-06-07 Thread Jason Gunthorpe
micolon from the __assign_str() macro definition. Acked-by: Jason Gunthorpe Jason ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [RFC PATCH 0/5] Support DEVICE_GENERIC memory in migrate_vma_*

2021-05-29 Thread Jason Gunthorpe
On Thu, May 27, 2021 at 07:08:04PM -0400, Felix Kuehling wrote: > Now we're trying to migrate data to and from that memory using the > migrate_vma_* helpers so we can support page-based migration in our > unified memory allocations, while also supporting CPU access to those > pages. So you have co

Re: [PATCH 00/34] Add HMM-based SVM memory manager to KFD v4

2021-04-08 Thread Jason Gunthorpe
nk: > https://github.com/RadeonOpenCompute/ROCK-Kernel-Driver/tree/fxkamd/hmm-wip > Link: > https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/tree/fxkamd/hmm-wip > Link: https://patchwork.freedesktop.org/series/85563/ > CC: Jérôme Glisse > CC: Jason Gunthorpe This series

Re: [PATCH 000/141] Fix fall-through warnings for Clang

2020-11-23 Thread Jason Gunthorpe
On Fri, Nov 20, 2020 at 12:21:39PM -0600, Gustavo A. R. Silva wrote: > IB/hfi1: Fix fall-through warnings for Clang > IB/mlx4: Fix fall-through warnings for Clang > IB/qedr: Fix fall-through warnings for Clang > RDMA/mlx5: Fix fall-through warnings for Clang I picked these four to the rdm

Re: [RFC] treewide: cleanup unreachable breaks

2020-10-19 Thread Jason Gunthorpe
On Mon, Oct 19, 2020 at 12:42:15PM -0700, Nick Desaulniers wrote: > On Sat, Oct 17, 2020 at 10:43 PM Greg KH wrote: > > > > On Sat, Oct 17, 2020 at 09:09:28AM -0700, t...@redhat.com wrote: > > > From: Tom Rix > > > > > > This is a upcoming change to clean up a new warning treewide. > > > I am won

Re: [PATCH 02/25] dma-fence: prime lockdep annotations

2020-07-10 Thread Jason Gunthorpe
On Fri, Jul 10, 2020 at 03:01:10PM +0200, Christian König wrote: > Am 10.07.20 um 14:54 schrieb Jason Gunthorpe: > > On Fri, Jul 10, 2020 at 02:48:16PM +0200, Christian König wrote: > > > Am 10.07.20 um 14:43 schrieb Jason Gunthorpe: > > > > On Thu, Jul 09, 2020 at 1

Re: [PATCH 02/25] dma-fence: prime lockdep annotations

2020-07-10 Thread Jason Gunthorpe
On Thu, Jul 09, 2020 at 10:09:11AM +0200, Daniel Vetter wrote: > Hi Jason, > > Below the paragraph I've added after our discussions around dma-fences > outside of drivers/gpu. Good enough for an ack on this, or want something > changed? > > Thanks, Daniel > > > + * Note that only GPU drivers hav

Re: [PATCH 02/25] dma-fence: prime lockdep annotations

2020-07-10 Thread Jason Gunthorpe
On Fri, Jul 10, 2020 at 02:48:16PM +0200, Christian König wrote: > Am 10.07.20 um 14:43 schrieb Jason Gunthorpe: > > On Thu, Jul 09, 2020 at 10:09:11AM +0200, Daniel Vetter wrote: > > > Hi Jason, > > > > > > Below the paragraph I've added after our discu

Re: [PATCH 02/25] dma-fence: prime lockdep annotations

2020-07-10 Thread Jason Gunthorpe
On Fri, Jul 10, 2020 at 04:02:35PM +0200, Daniel Vetter wrote: > > dma_fence only possibly makes some sense if you intend to expose the > > completion outside a single driver. > > > > The prefered kernel design pattern for this is to connect things with > > a function callback. > > > > So the actu

Re: [Linaro-mm-sig] [PATCH 04/18] dma-fence: prime lockdep annotations

2020-06-23 Thread Jason Gunthorpe
On Fri, Jun 19, 2020 at 04:10:11PM -0400, Jerome Glisse wrote: > Maybe we can audit how user ptr buffer are use today and see if > we can define a usage pattern that would allow to cut corner in > kernel. For instance we could use mmu notifier just to block CPU > pte update while we do GUP and th

Re: [Linaro-mm-sig] [PATCH 04/18] dma-fence: prime lockdep annotations

2020-06-23 Thread Jason Gunthorpe
On Mon, Jun 22, 2020 at 04:15:40PM -0400, Jerome Glisse wrote: > On Mon, Jun 22, 2020 at 08:46:17AM -0300, Jason Gunthorpe wrote: > > On Fri, Jun 19, 2020 at 04:31:47PM -0400, Jerome Glisse wrote: > > > Not doable as page refcount can change for things unrelated to GUP, with >

Re: [Linaro-mm-sig] [PATCH 04/18] dma-fence: prime lockdep annotations

2020-06-22 Thread Jason Gunthorpe
On Fri, Jun 19, 2020 at 04:31:47PM -0400, Jerome Glisse wrote: > Not doable as page refcount can change for things unrelated to GUP, with > John changes we can identify GUP and we could potentialy copy GUPed page > instead of COW but this can potentialy slow down fork() and i am not sure > how acce

Re: [Linaro-mm-sig] [PATCH 04/18] dma-fence: prime lockdep annotations

2020-06-19 Thread Jason Gunthorpe
On Fri, Jun 19, 2020 at 03:30:32PM -0400, Felix Kuehling wrote: > We have a potential problem with CPU updating page tables while the GPU > is retrying on page table entries because 64 bit CPU transactions don't > arrive in device memory atomically. Except for 32 bit platforms atomicity is guarent

  1   2   3   4   5   >