There's been discussion on the OpenMPI list recently about static linking of OpenMPI with all of the desired MCAs in it. I've got the opposite question. I'd like to add MCAs later on to an already-compiled version of OpenMPI and am not quite sure how to do it.
Let me summarize. We've got a commercial code that we deploy on customer machines in binary form. We're working to integrate OpenMPI into the installer, and things seem to be progressing well. (Note: because we're a commercial code, making the customer compile something doesn't work for us like it can for open source or research codes.) Now, we want to take advantage of OpenMPI's ability to find MCAs at runtime, pointing to the various plugins that might apply to a deployed system. I've configured and compiled OpenMPI on one of our build machines, one that doesn't have any special interconnect hardware or software installed. We take this compiled version of OpenMPI and use it on all of our machines. (Yes, I've read Building FAQ #39 <https://www.open-mpi.org/faq/?category=building#installdirs> about relocating OpenMPI. Useful, that.) I'd like to take our pre-compiled version of OpenMPI and add MCA libraries to it, giving OpenMPI the ability to communicate via transport mechanisms that weren't available on the original build machine. Things like InfiniBand, OmniPath, or one of Cray's interconnects. How would I go about doing this? And what are the limitations? I'm guessing that I need to go configure and compile the same version of OpenMPI on a machine that has the desired interconnect installation (headers and libraries), then go grab the corresponding lib/openmpi/mca_*{la,so} files. Take those files and drop them in our pre-built OpenMPI from our build machine in the same relative plugin location (lib/openmpi). If I stick with the same compiler (gcc, in this case), I'm hoping that symbols will all resolve themselves at runtime. (I probably will have to do some LD_LIBRARY_PATH games to be sure to find the appropriate underlying libraries unless OpenMPI's process for building MCAs links them in statically somehow.) Am I even on the right track here? (The various system-level FAQs (here <https://www.open-mpi.org/faq/?category=supported-systems>, here <https://www.open-mpi.org/faq/?category=developers>, and especially here <https://www.open-mpi.org/faq/?category=sysadmin>) seem to suggest that I am.) Our first test platform will be getting OpenMPI via IB working on our cluster, where we have IB (and TCP/IP) functional and not OpenMPI. This will be a great stand-in for a customer that has an IB cluster and wants to just run our binary installation. Thanks. -Sean -- Sean Ahern Computational Engineering International 919-363-0883
_______________________________________________ users mailing list users@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/users
