Open MPI is based on a component architecture, where plugins are used for various types of back-end functionality.
For MPI point-to-point communication, there are three types of plugins that are used: - pml: Point to Point Messaging Layer (what is directly invoked behind MPI_SEND and friends). This layer understands MPI semantics. - btl: Byte Transfer Layer (lowest layer "network driver" for communicating across specific networks). This layer is completely MPI-agnostic; it just pushes bytes between endpoints (usually processes). - bml: BTL Management Layer (for loading, unloading, and generally run-time managing multiple BTL plugins). Note that previous generations of Open MPI used "ptl" plugins instead of "btl" plugins. I would suggest reading the "Open MPI: A Flexible High Performance MPI" paper on the Open MPI web site (see the "Publications" section) to get an overview of the OB1 pml plugin. It describes the basis and top-level design of how the pml, bml, and btls interact at run-time. Also read "InfiniBand Scalability in Open MPI" (also in the publications section). It is much more recent and, although somewhat IB-specific, also gives a good overview of how things work. Finally, look at the MPI Developer's Workshop slides. They probably provide the best description of what is happening for point-to-point messaging (there's whole sections devoted to this). If you simply want to implement a new network driver (e.g., raw ethernet), you will want to write a new BTL plugin. If you want to play with various scheduling policies, you probably want to modify / write a new PML plugin. This code is not well documented other than comments in the code and the description of the plugin interfaces; your best bet is to ask questions here on the mailing list. I strongly recommend working with a Subversion checkout -- things have changed since the 1.0 release. See http://www.open-mpi.org/svn/ for details on getting a subversion checkout. Within the code base, you'll be working within the ompi/ tree (the MPI layer). You'll find the MPI C API bindings in ompi/mpi/c/*.c (e.g., MPI_SEND is in ompi/mpi/c/send.c). Also in ompi/ is the mca/ directory which, in turn, has subdirectories for each kind of plugin (a.k.a. "frameworks"), so you'll see subdirectories for pml/ and bml/ and btl/. In each framework's directory are subdirectories for each plugin (a.k.a. "components"). So you'll see ompi/mca/pml/ob1 (code for the ob1 PML plugin), ompi/mca/bml/r2 (code for the r2 BML plugin), ompi/mca/btl/tcp (code for the tcp BTL plugin), etc. This is probably enough to get you started. Let us know what your next questions are. > -----Original Message----- > From: users-boun...@open-mpi.org > [mailto:users-boun...@open-mpi.org] On Behalf Of Mahesh Barve > Sent: Tuesday, May 23, 2006 1:24 AM > To: us...@open-mpi.org > Subject: [OMPI users] Help regarding send/recv code > > Hi, > I am a novice to openmpi. Just managed to get openmpi > running on my system. > I would like to modify the code for send and recv. > The target lower level device will be ethernet and > infiniband. I would like to know the files/functions > to look for. Could you please guide me in this. > thanks, > -Mahesh > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users >
