Am 16.04.25 um 06:45 schrieb Felix Kuehling:
> Pinning of VRAM is for peer devices that don't support dynamic attachment
> and move notifiers. But it requires that all such peer devices are able to
> access VRAM via PCIe P2P. Any device without P2P access requires migration
> to GTT, which fails if the memory is already pinned for another peer
> device.
>
> Sharing between GPUs should not require pinning in VRAM. However, if
> DMABUF_MOVE_NOTIFY is disabled in the kernel build, even DMABufs shared
> between GPUs must be pinned, which can lead to failures and functional
> regressions on systems where some peer GPUs are not P2P accessible.
>
> Disable VRAM pinning if move notifiers are disabled in the kernel build
> to fix regressions when sharing BOs between GPUs.
>
> Signed-off-by: Felix Kuehling <felix.kuehl...@amd.com>
Reviewed-by: Christian König <christian.koe...@amd.com> for this one here.
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 17 ++++++++++++-----
> 1 file changed, 12 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> index 667080cc9ae1c..9abe592968ab3 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> @@ -81,14 +81,21 @@ static int amdgpu_dma_buf_pin(struct dma_buf_attachment
> *attach)
>
> dma_resv_assert_held(dmabuf->resv);
>
> - /*
> - * Try pinning into VRAM to allow P2P with RDMA NICs without ODP
> + /* Try pinning into VRAM to allow P2P with RDMA NICs without ODP
> * support if all attachments can do P2P. If any attachment can't do
> * P2P just pin into GTT instead.
> + *
> + * To avoid with conflicting pinnings between GPUs and RDMA when move
> + * notifiers are disabled, only allow pinning in VRAM when move
> + * notiers are enabled.
> */
> - list_for_each_entry(attach, &dmabuf->attachments, node)
> - if (!attach->peer2peer)
> - domains &= ~AMDGPU_GEM_DOMAIN_VRAM;
> + if (!IS_ENABLED(CONFIG_DMABUF_MOVE_NOTIFY)) {
> + domains &= ~AMDGPU_GEM_DOMAIN_VRAM;
> + } else {
> + list_for_each_entry(attach, &dmabuf->attachments, node)
> + if (!attach->peer2peer)
> + domains &= ~AMDGPU_GEM_DOMAIN_VRAM;
> + }
>
> if (domains & AMDGPU_GEM_DOMAIN_VRAM)
> bo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
