On Oct 28, 2016, at 8:12 AM, Mahesh Nanavalla <[email protected]>
wrote:
>
> i have configured as below for arm
>
> ./configure --enable-orterun-prefix-by-default
> --prefix="/home/nmahesh/Workspace/ARM_MPI/openmpi"
> CC=arm-openwrt-linux-muslgnueabi-gcc CXX=arm-openwrt-linux-muslgnueabi-g++
> --host=arm-openwrt-linux-muslgnueabi --enable-script-wrapper-compilers
> --disable-mpi-fortran --enable-dlopen --enable-shared --disable-vt
> --disable-java --disable-libompitrace --disable-static
Note that there is a tradeoff here: --enable-dlopen will reduce the size of
libmpi.so by splitting out all the plugins into separate DSOs (dynamic shared
objects -- i.e., individual .so plugin files). But note that some of plugins
are quite small in terms of code. I mention this because when you dlopen a
DSO, it will load in DSOs in units of pages. So even if a DSO only has 1KB of
code, it will use <page_size> of bytes in your running process (e.g., 4KB -- or
whatever the page size is on your system).
On the other hand, if you --disable-dlopen, then all of Open MPI's plugins are
slurped into libmpi.so (and friends). Meaning: no DSOs, no dlopen, no
page-boundary-loading behavior. This allows the compiler/linker to pack in all
the plugins into memory more efficiently (because they'll be compiled as part
of libmpi.so, and all the code is packed in there -- just like any other
library). Your total memory usage in the process may be smaller.
Sidenote: if you run more than one MPI process per node, then libmpi.so (and
friends) will be shared between processes. You're assumedly running in an
embedded environment, so I don't know if this factor matters (i.e., I don't
know if you'll run with ppn>1), but I thought I'd mention it anyway.
On the other hand (that's your third hand, for those at home counting...), you
may not want to include *all* the plugins. I.e., there may be a bunch of
plugins that you're not actually using, and therefore if they are compiled in
as part of libmpi.so (and friends), they're consuming space that you don't
want/need. So the dlopen mechanism might actually be better -- because Open
MPI may dlopen a plugin at run time, determine that it won't be used, and then
dlclose it (i.e., release the memory that would have been used for it).
On the other (fourth!) hand, you can actually tell Open MPI to *not* build
specific plugins with the --enable-dso-no-build=LIST configure option. I.e.,
if you know exactly what plugins you want to use, you can negate the ones that
you *don't* want to use on the configure line, use --disable-static and
--disable-dlopen, and you'll likely use the least amount of memory. This is
admittedly a bit clunky, but Open MPI's configure process was (obviously) not
optimized for this use case -- it's much more optimized to the "build
everything possible, and figure out which to use at run time" use case.
If you really want to hit rock bottom on MPI process size in your embedded
environment, you can do some experimentation to figure out exactly which
components you need. You can use repeated runs with "mpirun --mca
ABC_base_verbose 100 ...", where "ABC" is each of Open MPI's framework names
("framework" = collection of plugins of the same type). This verbose output
will show you exactly which components are opened, which ones are used, and
which ones are discarded. You can build up a list of all the discarded
components and --enable-mca-no-build them.
> While i am running the using mpirun
> am getting following errror..
> root@OpenWrt:~# /usr/bin/mpirun --allow-run-as-root -np 1
> /usr/bin/openmpiWiFiBulb
> --------------------------------------------------------------------------
> Sorry! You were supposed to get help about:
> opal_init:startup:internal-failure
> But I couldn't open the help file:
>
> /home/nmahesh/Workspace/ARM_MPI/openmpi/share/openmpi/help-opal-runtime.txt:
> No such file or directory. Sorry!
So this is really two errors:
1. The help message file is not being found.
2. Something is obviously going wrong during opal_init() (which is one of Open
MPI's startup functions).
For #1, when I do a default build of Open MPI 1.10.3, that file *is* installed.
Are you trimming the installation tree, perchance? If so, if you can put at
least that one file back in its installation location (it's in the Open MPI
source tarball), it might reveal more information on exactly what is failing.
Additionally, I wonder if shared memory is not getting setup right. Try
running with "mpirun --mca shmem_base_verbose 100 ..." and see if it's
reporting an error.
--
Jeff Squyres
[email protected]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/
_______________________________________________
users mailing list
[email protected]
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
