Thanks. That's what I expected and hoped. But is there a pointer about how to get started? If I've got an existing OpenMPI build, what's the process to get a new MCA plugin built with a new set of header files?
(I'm a bit surprised only header files are necessary. Shouldn't the plugin require at least runtime linking with a low-level transport library?) -Sean -- Sean Ahern Computational Engineering International 919-363-0883 On Fri, Oct 28, 2016 at 3:40 PM, r...@open-mpi.org <r...@open-mpi.org> wrote: > You don’t need any of the hardware - you just need the headers. Things > like libfabric and libibverbs are all publicly available, and so you can > build all that support even if you cannot run it on your machine. > > Once your customer installs the binary, the various plugins will check for > their required library and hardware and disqualify themselves if it isn’t > found. > > On Oct 28, 2016, at 12:33 PM, Sean Ahern <s...@ensight.com> wrote: > > There's been discussion on the OpenMPI list recently about static linking > of OpenMPI with all of the desired MCAs in it. I've got the opposite > question. I'd like to add MCAs later on to an already-compiled version of > OpenMPI and am not quite sure how to do it. > > Let me summarize. We've got a commercial code that we deploy on customer > machines in binary form. We're working to integrate OpenMPI into the > installer, and things seem to be progressing well. (Note: because we're a > commercial code, making the customer compile something doesn't work for us > like it can for open source or research codes.) > > Now, we want to take advantage of OpenMPI's ability to find MCAs at > runtime, pointing to the various plugins that might apply to a deployed > system. I've configured and compiled OpenMPI on one of our build machines, > one that doesn't have any special interconnect hardware or software > installed. We take this compiled version of OpenMPI and use it on all of > our machines. (Yes, I've read Building FAQ #39 > <https://www.open-mpi.org/faq/?category=building#installdirs> about > relocating OpenMPI. Useful, that.) I'd like to take our pre-compiled > version of OpenMPI and add MCA libraries to it, giving OpenMPI the ability > to communicate via transport mechanisms that weren't available on the > original build machine. Things like InfiniBand, OmniPath, or one of Cray's > interconnects. > > How would I go about doing this? And what are the limitations? > > I'm guessing that I need to go configure and compile the same version of > OpenMPI on a machine that has the desired interconnect installation > (headers and libraries), then go grab the corresponding > lib/openmpi/mca_*{la,so} files. Take those files and drop them in our > pre-built OpenMPI from our build machine in the same relative plugin > location (lib/openmpi). If I stick with the same compiler (gcc, in this > case), I'm hoping that symbols will all resolve themselves at runtime. (I > probably will have to do some LD_LIBRARY_PATH games to be sure to find the > appropriate underlying libraries unless OpenMPI's process for building MCAs > links them in statically somehow.) > > Am I even on the right track here? (The various system-level FAQs (here > <https://www.open-mpi.org/faq/?category=supported-systems>, here > <https://www.open-mpi.org/faq/?category=developers>, and especially here > <https://www.open-mpi.org/faq/?category=sysadmin>) seem to suggest that I > am.) > > Our first test platform will be getting OpenMPI via IB working on our > cluster, where we have IB (and TCP/IP) functional and not OpenMPI. This > will be a great stand-in for a customer that has an IB cluster and wants to > just run our binary installation. > > Thanks. > > -Sean > > -- > Sean Ahern > Computational Engineering International > 919-363-0883 > _______________________________________________ > users mailing list > users@lists.open-mpi.org > https://rfd.newmexicoconsortium.org/mailman/listinfo/users > > > > _______________________________________________ > users mailing list > users@lists.open-mpi.org > https://rfd.newmexicoconsortium.org/mailman/listinfo/users >
_______________________________________________ users mailing list users@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/users
