On Mar 25, 2020, at 4:49 AM, Raut, S Biplab via users <users@lists.open-mpi.org> wrote: > > Let’s say the application is running with 128 ranks. > Each rank is doing send() msg to rest of 127 ranks where the msg length sent > is under question. > Now after all the sends are completed, each rank will recv() msg from rest of > 127 ranks. > Unless the msg length in the sending part is within eager_limit (4K size), > this program will hang. > So, based on the above scenario, my questions are:- > • Can each of the rank send message upto 4K size successfully, i.e all > 128 ranks sending (128 * 4K) bytes simultaneously? > • If application has bigger msg to be sent by each rank, then how to > derive the send message size? Is it equal to eager_limit and each rank needs > to send multiple chunks of this size?
I think you're asking the wrong questions. MPI, as a standard, intentionally hides all the transport details from you. You have some additional controls from implementations such as Open MPI (e.g., MCA params that control the eager limit sizes), but that's not really the point -- those are really intended for optimizations on specific hardware. From an application perspective, you should *not* be writing your algorithms tuned to the eager limit size of a given MPI implementation. If you have blocking issues (potentially due to eager limit issues), then you should be using non-blocking MPI communication. Or, in your case, perhaps you should be using MPI collective operations (since everyone is sending to everyone). -- Jeff Squyres jsquy...@cisco.com
