Re: [OMPI users] CUDA Buffers: Enforce asynchronous memcpy's

2015-08-12 Thread Rolf vandeVaart
paul...@us.ibm.com> www.ibm.com<http://www.ibm.com> - Original message - From: Rolf vandeVaart mailto:rvandeva...@nvidia.com>> Sent by: "users" mailto:users-boun...@open-mpi.org>> To: Open MPI Users mailto:us...@open-mpi.org>> Cc: Subject: Re: [OMPI

Re: [OMPI users] CUDA Buffers: Enforce asynchronous memcpy's

2015-08-12 Thread Geoffrey Paulsen
I'm confused why this application needs an asynchronous cuMemcpyAsync()in a blocking MPI call.   Rolf could you please explain?And how does is a call to cuMemcpyAsync() followed by a syncronization any different than a cuMemcpy() in this use case?   I would still expect that if the MPI_Send / Recv

Re: [OMPI users] CUDA Buffers: Enforce asynchronous memcpy's

2015-08-11 Thread Rolf vandeVaart
I talked with Jeremia off list and we figured out what was going on. There is the ability to use the cuMemcpyAsync/cuStreamSynchronize rather than the cuMemcpy but it was never made the default for Open MPI 1.8 series. So, to get that behavior you need the following: --mca mpi_common_cuda_cum