Dear Matthieu, Rolf, Thank you!
But normally CUDA device selection is based on MPI process index. So, cuda context must exist where MPI index is not yet available. What is the best practice of process<->GPU mapping in this case? Or can I select any device prior to MPI_Init and later change to another device? - D. 2011/12/14 Rolf vandeVaart <rvandeva...@nvidia.com>: > To add to this, yes, we recommend that the CUDA context exists prior to a > call to MPI_Init. That is because a CUDA context needs to exist prior to > MPI_Init as the library attempts to register some internal buffers with the > CUDA library that require a CUDA context exists already. Note that this is > only relevant if you plan to send and receive CUDA device memory directly > from MPI calls. There is a little more about this in the FAQ here. > > > > http://www.open-mpi.org/faq/?category=running#mpi-cuda-support > > > > > > Rolf > > > > From: Matthieu Brucher [mailto:matthieu.bruc...@gmail.com] > Sent: Wednesday, December 14, 2011 10:47 AM > To: Open MPI Users > Cc: Rolf vandeVaart > Subject: Re: [OMPI users] How "CUDA Init prior to MPI_Init" co-exists with > unique GPU for each MPI process? > > > > Hi, > > > > Processes are not spawned by MPI_Init. They are spawned before by some > applications between your mpirun call and when your program starts. When it > does, you already have all MPI processes (you can check by adding a sleep or > something like that), but they are not synchronized and do not know each > other. This is what MPI_Init is used for. > > > > Matthieu Brucher > > 2011/12/14 Dmitry N. Mikushin <maemar...@gmail.com> > > Dear colleagues, > > For GPU Winter School powered by Moscow State University cluster > "Lomonosov", the OpenMPI 1.7 was built to test and popularize CUDA > capabilities of MPI. There is one strange warning I cannot understand: > OpenMPI runtime suggests to initialize CUDA prior to MPI_Init. Sorry, > but how could it be? I thought processes are spawned during MPI_Init, > and such context will be created on the very first root process. Why > do we need existing CUDA context before MPI_Init? I think there was no > such error in previous versions. > > Thanks, > - D. > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > > > > > > -- > Information System Engineer, Ph.D. > Blog: http://matt.eifelle.com > LinkedIn: http://www.linkedin.com/in/matthieubrucher > > ________________________________ > This email message is for the sole use of the intended recipient(s) and may > contain confidential information. Any unauthorized review, use, disclosure > or distribution is prohibited. If you are not the intended recipient, > please contact the sender by reply email and destroy all copies of the > original message. > ________________________________ > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
