Re: [OMPI users] CUDA IPC/RDMA Not Working

2016-03-30 Thread Justin Luitjens
We have figured this out. It turns out that the first call to each MPI_Isend/Irecv is staged through the host but subsequent calls are not. Thanks, Justin From: Justin Luitjens Sent: Wednesday, March 30, 2016 9:37 AM To: us...@open-mpi.org Subject: CUDA IPC/RDMA Not Working Hello, I have inst

[OMPI users] CUDA IPC/RDMA Not Working

2016-03-30 Thread Justin Luitjens
Hello, I have installed OpenMPI 1.10.2 with cuda support: [jluitjens@dt03 repro]$ ompi_info --parsable --all | grep mpi_built_with_cuda_support:value mca:mpi:base:param:mpi_built_with_cuda_support:value:true I'm trying to verify that GPU direct is working and that messages aren't traversing t