It looks to me like the network is losing connections - your error messages all 
state "no route to host", which implies that the network interface dropped out.

On Sep 3, 2012, at 1:39 PM, Andrea Negri <negri.an...@gmail.com> wrote:

> I have asked to my admin and he said that no log messages were present
> in /var/log, apart my login on the compute node.
> No killed processes, no full stack errors, the memory is ok, 1GB is
> used and 2GB are free.
> Actually I don't know what kind of problem coud be, does someone have
> ideas? Or at least a suspect?
> 
> Please, don't let me alone!
> 
> Sorry for the trouble with the mail
> 
> 2012/9/1  <users-requ...@open-mpi.org>:
>> Send users mailing list submissions to
>>        us...@open-mpi.org
>> 
>> To subscribe or unsubscribe via the World Wide Web, visit
>>        http://www.open-mpi.org/mailman/listinfo.cgi/users
>> or, via email, send a message with subject or body 'help' to
>>        users-requ...@open-mpi.org
>> 
>> You can reach the person managing the list at
>>        users-ow...@open-mpi.org
>> 
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of users digest..."
>> 
>> 
>> Today's Topics:
>> 
>>   1. Re: some mpi processes "disappear" on a cluster of        servers
>>      (John Hearns)
>>   2. Re: users Digest, Vol 2339, Issue 5 (Andrea Negri)
>> 
>> 
>> ----------------------------------------------------------------------
>> 
>> Message: 1
>> Date: Sat, 1 Sep 2012 08:48:56 +0100
>> From: John Hearns <hear...@googlemail.com>
>> Subject: Re: [OMPI users] some mpi processes "disappear" on a cluster
>>        of      servers
>> To: Open MPI Users <us...@open-mpi.org>
>> Message-ID:
>>        <capqne2wo3bgefjiyfums6yquoduhjj75zjoeeyjrdq60fmz...@mail.gmail.com>
>> Content-Type: text/plain; charset=ISO-8859-1
>> 
>> Apologies, I have not taken the time to read your comprehensive diagnostics!
>> 
>> As Gus says, this sounds like a memory problem.
>> My suspicion would be the kernel Out Of Memory (OOM) killer.
>> Log into those nodes (or ask your systems manager to do this). Look
>> closely at /var/log/messages where there will be notifications when
>> the OOM Killer kicks in and - well - kills large memory processes!
>> Grep for "killed process" in /var/log/messages
>> 
>> http://linux-mm.org/OOM_Killer
>> 
>> 
>> ------------------------------
>> 
>> Message: 2
>> Date: Sat, 1 Sep 2012 11:50:59 +0200
>> From: Andrea Negri <negri.an...@gmail.com>
>> Subject: Re: [OMPI users] users Digest, Vol 2339, Issue 5
>> To: us...@open-mpi.org
>> Message-ID:
>>        <CAPUxaiQ4RFqSK1kz7fM7U9XRxjQh8N+=98paqm2ikvr7bv-...@mail.gmail.com>
>> Content-Type: text/plain; charset=ISO-8859-1
>> 
>> Hi, Gus and John,
>> 
>> my code (zeusmp2) is a F77 code ported in F95, and the very first time
>> I have launched it, the processed disappears almost immediately.
>> I checked the code with valgrind and some unallocated arrays were
>> passed wrongly to some subroutines.
>> After having corrected this bug, the code runs for a while and after
>> occour all the stuff described in my first post.
>> However, the code still performs a lot of main temporal cycle before
>> "die" (I don't know if thies piece of information is useful).
>> 
>> Now I'm going to check the memory usage, (I also have a lot of unused
>> variables in this pretty large code, maybe I shoud comment them).
>> 
>> uname -a returns
>> Linux cloud 2.6.9-42.0.3.ELsmp #1 SMP Thu Oct 5 16:29:37 CDT 2006
>> x86_64 x86_64 x86_64 GNU/Linux
>> 
>> ulimit -a returns:
>> core file size                        (blocks, -c) 0
>> data seg size                       (kbytes, -d) unlimited
>> file size                               (blocks, -f) unlimited
>> pending signals                    (-i) 1024
>> max locked memory             (kbytes, -l) 32
>> max memory size                (kbytes, -m) unlimited
>> open files                           (-n) 1024
>> pipe size                            (512 bytes, -p) 8
>> POSIX message queues     (bytes, -q) 819200
>> stack size                           (kbytes, -s) 10240
>> cpu time                             (seconds, -t) unlimited
>> max user processes             (-u) 36864
>> virtual memory                   (kbytes, -v) unlimited
>> file locks                            (-x) unlimited
>> 
>> 
>> I can log on the logins nodes, but unfortunately the command ls
>> /var/log/messages return
>> acpid               cron.4              messages.3         secure.4
>> anaconda.log        cups                messages.4         spooler
>> anaconda.syslog     dmesg               mpi_uninstall.log  spooler.1
>> anaconda.xlog       gdm                 ppp                spooler.2
>> audit               httpd               prelink.log        spooler.3
>> boot.log            itac_uninstall.log  rpmpkgs            spooler.4
>> boot.log.1          lastlog             rpmpkgs.1          vbox
>> boot.log.2          mail                rpmpkgs.2          wtmp
>> boot.log.3          maillog             rpmpkgs.3          wtmp.1
>> boot.log.4          maillog.1           rpmpkgs.4          Xorg.0.log
>> cmkl_install.log    maillog.2           samba              Xorg.0.log.old
>> cmkl_uninstall.log  maillog.3           scrollkeeper.log   yum.log
>> cron                maillog.4           secure             yum.log.1
>> cron.1              messages            secure.1
>> cron.2              messages.1          secure.2
>> cron.3              messages.2          secure.3
>> 
>> so, the log should be in some of these files (I don't have read
>> permission on these files). I'll contact the admin for that.
>> 
>> 2012/9/1  <users-requ...@open-mpi.org>:
>>> Send users mailing list submissions to
>>>        us...@open-mpi.org
>>> 
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>        http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> or, via email, send a message with subject or body 'help' to
>>>        users-requ...@open-mpi.org
>>> 
>>> You can reach the person managing the list at
>>>        users-ow...@open-mpi.org
>>> 
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of users digest..."
>>> 
>>> 
>>> Today's Topics:
>>> 
>>>   1. Re: some mpi processes "disappear" on a cluster of        servers
>>>      (Gus Correa)
>>> 
>>> 
>>> ----------------------------------------------------------------------
>>> 
>>> Message: 1
>>> Date: Fri, 31 Aug 2012 20:11:41 -0400
>>> From: Gus Correa <g...@ldeo.columbia.edu>
>>> Subject: Re: [OMPI users] some mpi processes "disappear" on a cluster
>>>        of      servers
>>> To: Open MPI Users <us...@open-mpi.org>
>>> Message-ID: <504152bd.3000...@ldeo.columbia.edu>
>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>> 
>>> Hi Andrea
>>> 
>>> I would guess this is a memory problem.
>>> Do you know how much memory each node has?
>>> Do you know the memory that
>>> each MPI process in the CFD code requires?
>>> If the program starts swapping/paging into disk, because of
>>> low memory, those interesting things that you described can happen.
>>> 
>>> I would login to the compute nodes and monitor the
>>> amount of memory being used with "top" right after the program
>>> starts to run.  If there is a pattern of which node tends to fail,
>>> track login to that fail-prone node and monitor it.
>>> 
>>> Alternatively, if you cluster is running Ganglia,
>>> you can see the memory use graphically,
>>> in the Ganglia web page in a web browser.
>>> 
>>> If your cluster
>>> doesn't allow direct user logins to compute nodes,
>>> you can ask the system administrator to do this for you.
>>> 
>>> It may well be that the code has a memory leak, or that
>>> it has a memory request spike, which may not be caught by
>>> OpenMPI.
>>> [Jeff and Ralph will probably correct me soon for
>>> saying this, and I know the OpenMPI safeguards against
>>> process misbehavior are great, but ...]
>>> 
>>> Anyway, we had several codes here that would make a node go south
>>> because of either type of memory problem, and subsequently the
>>> program would die, or the other processes in other nodes would
>>> continue "running" [i.e. mostly waiting for MPI calls to the
>>> dead node that would never return] as you described.
>>> 
>>> If the problem is benign, i.e., if it is just that the
>>> memory-per-processor is not large enough to run in 10 processors,
>>> you can get around it by running in, say, 20 processors.
>>> 
>>> Yet another issue that you may check is the stacksize in the
>>> compute nodes.  Many codes require a large stacksize, i.e.,
>>> they create large arrays in subroutines, etc, and
>>> the default stacksize in standard Linux distributions
>>> may not be as large as needed.
>>> We use ulimited stacksize in our compute nodes.
>>> 
>>> You can ask the system administrator to check this for you,
>>> and perhaps change it in /etc/security/limits.conf to make it
>>> unlimited or at least larger than the default.
>>> The Linux shell command "ulimit -a" [bash] or
>>> "limit" [tcsh] will tell what the limits are.
>>> 
>>> I hope this helps,
>>> Gus Correa
>>> 
>>> On 08/31/2012 07:15 PM, Andrea Negri wrote:
>>>> Hi, I have been in trouble for a year.
>>>> 
>>>> I run a pure MPI (no openMP) Fortran fluid dynamical code on a cluster
>>>> of server, and I obtain a strange behaviour by running the code on
>>>> multiple nodes.
>>>> The cluster is formed by 16 pc (1 pc is a node) with a dual core processor.
>>>> Basically, I'm able to run the code from the login node with the command:
>>>> mpirun  --mca btl_base_verbose 100 --mca backtrace_base_verbose 100
>>>> --mca memory_base_verbose 100 --mca sysinfo_base_verbose 100  -nolocal
>>>> -hostfile ./host_file -n 10  ./zeusmp2.x>>  zmp_errors 2>&1
>>>> by selecting one process per core (i.e. in this case I use 5 nodes)
>>>> 
>>>> The code starts, and it runs correctely for some time.
>>>> After that, an entire node (sometimes two) "disappears" and it cannot
>>>> be reached with the ssh command, which returns: No route to host.
>>>> Sometimes the node is still reachable, but the two processes that was
>>>> running on the node are disappears.
>>>> In addition, on the other nodes, the others processes are still running.
>>>> 
>>>> If I have a look on the output and error file of mpirun, the following
>>>> error is present: [btl_tcp_frag.c:215:mca_btl_tcp_frag_recv]
>>>> mca_btl_tcp_frag_recv: readv failed: No route to host (113)
>>>> 
>>>> PS: I'm not the admin of the cluster, I've installed the gcc and
>>>> openmpi on my own because the complier aviable on that machine are 8
>>>> years old.
>>>> 
>>>> 
>>>> I post here some information, if you want other info, you have only to
>>>> tell me which command I have to type on the bash and I will
>>>> immediately reply.
>>>> 
>>>> 
>>>> complier: gcc 4.7 (which was also used to compile openmpi)
>>>> openmpi version: 1.6
>>>> 
>>>> output of "ompi_info --all" from the node where I launch mpirun (which
>>>> is also the login node of the cluster)
>>>> 
>>>>   Package: Open MPI and...@cloud.bo.astro.it Distribution
>>>>                 Open MPI: 1.6
>>>>    Open MPI SVN revision: r26429
>>>>    Open MPI release date: May 10, 2012
>>>>                 Open RTE: 1.6
>>>>    Open RTE SVN revision: r26429
>>>>    Open RTE release date: May 10, 2012
>>>>                     OPAL: 1.6
>>>>        OPAL SVN revision: r26429
>>>>        OPAL release date: May 10, 2012
>>>>                  MPI API: 2.1
>>>>             Ident string: 1.6
>>>>            MCA backtrace: execinfo (MCA v2.0, API v2.0, Component v1.6)
>>>>               MCA memory: linux (MCA v2.0, API v2.0, Component v1.6)
>>>>            MCA paffinity: hwloc (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA carto: auto_detect (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA carto: file (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA shmem: mmap (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA shmem: posix (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA shmem: sysv (MCA v2.0, API v2.0, Component v1.6)
>>>>            MCA maffinity: first_use (MCA v2.0, API v2.0, Component v1.6)
>>>>            MCA maffinity: hwloc (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA timer: linux (MCA v2.0, API v2.0, Component v1.6)
>>>>          MCA installdirs: env (MCA v2.0, API v2.0, Component v1.6)
>>>>          MCA installdirs: config (MCA v2.0, API v2.0, Component v1.6)
>>>>              MCA sysinfo: linux (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA hwloc: hwloc132 (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA dpm: orte (MCA v2.0, API v2.0, Component v1.6)
>>>>               MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.6)
>>>>            MCA allocator: basic (MCA v2.0, API v2.0, Component v1.6)
>>>>            MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.6)
>>>>                 MCA coll: basic (MCA v2.0, API v2.0, Component v1.6)
>>>>                 MCA coll: hierarch (MCA v2.0, API v2.0, Component v1.6)
>>>>                 MCA coll: inter (MCA v2.0, API v2.0, Component v1.6)
>>>>                 MCA coll: self (MCA v2.0, API v2.0, Component v1.6)
>>>>                 MCA coll: sm (MCA v2.0, API v2.0, Component v1.6)
>>>>                 MCA coll: sync (MCA v2.0, API v2.0, Component v1.6)
>>>>                 MCA coll: tuned (MCA v2.0, API v2.0, Component v1.6)
>>>>                   MCA io: romio (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA mpool: fake (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA mpool: sm (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA pml: bfo (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA pml: csum (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA pml: ob1 (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA pml: v (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA bml: r2 (MCA v2.0, API v2.0, Component v1.6)
>>>>               MCA rcache: vma (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA btl: self (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA btl: sm (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA btl: tcp (MCA v2.0, API v2.0, Component v1.6)
>>>>                 MCA topo: unity (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA osc: pt2pt (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA osc: rdma (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA iof: hnp (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA iof: orted (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA iof: tool (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA oob: tcp (MCA v2.0, API v2.0, Component v1.6)
>>>>                 MCA odls: default (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA ras: cm (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA ras: loadleveler (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA ras: slurm (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA rmaps: load_balance (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA rmaps: rank_file (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA rmaps: resilient (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA rmaps: round_robin (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA rmaps: seq (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA rmaps: topo (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA rml: oob (MCA v2.0, API v2.0, Component v1.6)
>>>>               MCA routed: binomial (MCA v2.0, API v2.0, Component v1.6)
>>>>               MCA routed: cm (MCA v2.0, API v2.0, Component v1.6)
>>>>               MCA routed: direct (MCA v2.0, API v2.0, Component v1.6)
>>>>               MCA routed: linear (MCA v2.0, API v2.0, Component v1.6)
>>>>               MCA routed: radix (MCA v2.0, API v2.0, Component v1.6)
>>>>               MCA routed: slave (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA plm: rsh (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA plm: slurm (MCA v2.0, API v2.0, Component v1.6)
>>>>                MCA filem: rsh (MCA v2.0, API v2.0, Component v1.6)
>>>>               MCA errmgr: default (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA ess: env (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA ess: hnp (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA ess: singleton (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA ess: slave (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA ess: slurm (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA ess: slurmd (MCA v2.0, API v2.0, Component v1.6)
>>>>                  MCA ess: tool (MCA v2.0, API v2.0, Component v1.6)
>>>>              MCA grpcomm: bad (MCA v2.0, API v2.0, Component v1.6)
>>>>              MCA grpcomm: basic (MCA v2.0, API v2.0, Component v1.6)
>>>>              MCA grpcomm: hier (MCA v2.0, API v2.0, Component v1.6)
>>>>             MCA notifier: command (MCA v2.0, API v1.0, Component v1.6)
>>>>             MCA notifier: syslog (MCA v2.0, API v1.0, Component v1.6)
>>>>                   Prefix: /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7
>>>>              Exec_prefix: /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7
>>>>                   Bindir: 
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/bin
>>>>                  Sbindir: 
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/sbin
>>>>                   Libdir: 
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib
>>>>                   Incdir:
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/include
>>>>                   Mandir:
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/man
>>>>                Pkglibdir:
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib/openmpi
>>>>               Libexecdir:
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/libexec
>>>>              Datarootdir: 
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share
>>>>                  Datadir: 
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share
>>>>               Sysconfdir: 
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/etc
>>>>           Sharedstatedir: 
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/com
>>>>            Localstatedir: 
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/var
>>>>                  Infodir:
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/info
>>>>               Pkgdatadir:
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/openmpi
>>>>                Pkglibdir:
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib/openmpi
>>>>            Pkgincludedir:
>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/include/openmpi
>>>>  Configured architecture: x86_64-unknown-linux-gnu
>>>>           Configure host: cloud.bo.astro.it
>>>>            Configured by: andrea
>>>>            Configured on: Tue Jul 31 10:53:46 CEST 2012
>>>>           Configure host: cloud.bo.astro.it
>>>>                 Built by: andrea
>>>>                 Built on: Tue Jul 31 11:08:33 CEST 2012
>>>>               Built host: cloud.bo.astro.it
>>>>               C bindings: yes
>>>>             C++ bindings: yes
>>>>       Fortran77 bindings: yes (all)
>>>>       Fortran90 bindings: yes
>>>>  Fortran90 bindings size: medium
>>>>               C compiler: /home/andrea/library/gcc/gcc-objects/bin/gcc
>>>>      C compiler absolute:
>>>>   C compiler family name: GNU
>>>>       C compiler version: 4.7.1
>>>>              C char size: 1
>>>>              C bool size: 1
>>>>             C short size: 2
>>>>               C int size: 4
>>>>              C long size: 8
>>>>             C float size: 4
>>>>            C double size: 8
>>>>           C pointer size: 8
>>>>             C char align: 1
>>>>             C bool align: 1
>>>>              C int align: 4
>>>>            C float align: 4
>>>>           C double align: 8
>>>>             C++ compiler: /home/andrea/library/gcc/gcc-objects/bin/g++
>>>>    C++ compiler absolute: none
>>>>       Fortran77 compiler: /home/andrea/library/gcc/gcc-objects/bin/gfortran
>>>>   Fortran77 compiler abs:
>>>>       Fortran90 compiler: /home/andrea/library/gcc/gcc-objects/bin/gfortran
>>>>   Fortran90 compiler abs:
>>>>        Fort integer size: 4
>>>>        Fort logical size: 4
>>>>  Fort logical value true: 1
>>>>       Fort have integer1: yes
>>>>       Fort have integer2: yes
>>>>       Fort have integer4: yes
>>>>       Fort have integer8: yes
>>>>      Fort have integer16: no
>>>>          Fort have real4: yes
>>>>          Fort have real8: yes
>>>>         Fort have real16: no
>>>>       Fort have complex8: yes
>>>>      Fort have complex16: yes
>>>>      Fort have complex32: no
>>>>       Fort integer1 size: 1
>>>>       Fort integer2 size: 2
>>>>       Fort integer4 size: 4
>>>>       Fort integer8 size: 8
>>>>      Fort integer16 size: -1
>>>>           Fort real size: 4
>>>>          Fort real4 size: 4
>>>>          Fort real8 size: 8
>>>>         Fort real16 size: 16
>>>>       Fort dbl prec size: 8
>>>>           Fort cplx size: 8
>>>>       Fort dbl cplx size: 16
>>>>          Fort cplx8 size: 8
>>>>         Fort cplx16 size: 16
>>>>         Fort cplx32 size: 32
>>>>       Fort integer align: 4
>>>>      Fort integer1 align: 1
>>>>      Fort integer2 align: 2
>>>>      Fort integer4 align: 4
>>>>      Fort integer8 align: 8
>>>>     Fort integer16 align: -1
>>>>          Fort real align: 4
>>>>         Fort real4 align: 4
>>>>         Fort real8 align: 8
>>>>        Fort real16 align: 16
>>>>      Fort dbl prec align: 8
>>>>          Fort cplx align: 4
>>>>      Fort dbl cplx align: 8
>>>>         Fort cplx8 align: 4
>>>>        Fort cplx16 align: 8
>>>>        Fort cplx32 align: 16
>>>>              C profiling: yes
>>>>            C++ profiling: yes
>>>>      Fortran77 profiling: yes
>>>>      Fortran90 profiling: yes
>>>>           C++ exceptions: no
>>>>           Thread support: posix (MPI_THREAD_MULTIPLE: no, progress: no)
>>>>            Sparse Groups: no
>>>>             Build CFLAGS: -DNDEBUG -g -O2 -finline-functions
>>>> -fno-strict-aliasing
>>>>                           -pthread
>>>>           Build CXXFLAGS: -O3 -DNDEBUG -finline-functions -pthread
>>>>             Build FFLAGS:
>>>>            Build FCFLAGS:
>>>>            Build LDFLAGS: -Wl,--rpath
>>>> -Wl,/home/andrea/library/gcc/gcc-objects/lib64
>>>>               Build LIBS: -lrt -lnsl  -lutil -lm
>>>>     Wrapper extra CFLAGS: -pthread
>>>>   Wrapper extra CXXFLAGS: -pthread
>>>>     Wrapper extra FFLAGS: -pthread
>>>>    Wrapper extra FCFLAGS: -pthread
>>>>    Wrapper extra LDFLAGS:
>>>>       Wrapper extra LIBS: -ldl  -lm -lnuma -lrt -lnsl  -lutil -lm
>>>>   Internal debug support: no
>>>>   MPI interface warnings: yes
>>>>      MPI parameter check: runtime
>>>> Memory profiling support: no
>>>> Memory debugging support: no
>>>>          libltdl support: no
>>>>    Heterogeneous support: no
>>>>  mpirun default --prefix: yes
>>>>          MPI I/O support: yes
>>>>        MPI_WTIME support: gettimeofday
>>>>      Symbol vis. support: yes
>>>>    Host topology support: yes
>>>>           MPI extensions: affinity example
>>>>    FT Checkpoint support: no (checkpoint thread: no)
>>>>      VampirTrace support: yes
>>>>   MPI_MAX_PROCESSOR_NAME: 256
>>>>     MPI_MAX_ERROR_STRING: 256
>>>>      MPI_MAX_OBJECT_NAME: 64
>>>>         MPI_MAX_INFO_KEY: 36
>>>>         MPI_MAX_INFO_VAL: 256
>>>>        MPI_MAX_PORT_NAME: 1024
>>>>   MPI_MAX_DATAREP_STRING: 128
>>>>                  MCA mca: parameter "mca_param_files" (current value:
>>>> 
>>>> </home/andrea/.openmpi/mca-params.conf:/home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/etc/openmpi-mca-params.conf>,
>>>>                           data source: default value)
>>>>                           Path for MCA configuration files containing
>>>> default parameter
>>>>                           values
>>>>                  MCA mca: parameter "mca_base_param_file_prefix"
>>>> (current value:<none>,
>>>>                           data source: default value)
>>>>                           Aggregate MCA parameter file sets
>>>>                  MCA mca: parameter "mca_base_param_file_path" (current 
>>>> value:
>>>> 
>>>> </home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/openmpi/amca-param-sets:/home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/bin>,
>>>>                           data source: default value)
>>>>                           Aggregate MCA parameter Search path
>>>>                  MCA mca: parameter "mca_base_param_file_path_force"
>>>> (current value:
>>>>                           <none>, data source: default value)
>>>>                           Forced Aggregate MCA parameter Search path
>>>>                  MCA mca: parameter "mca_component_path" (current value:
>>>> 
>>>> </home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib/openmpi:/home/andrea/.openmpi/components>,
>>>>                           data source: default value)
>>>>                           Path where to look for Open MPI and ORTE 
>>>> components
>>>>                  MCA mca: parameter "mca_component_show_load_errors"
>>>> (current value:<1>,
>>>>                           data source: default value)
>>>>                           Whether to show errors for components that
>>>> failed to load or
>>>>                           not
>>>>                  MCA mca: parameter "mca_component_disable_dlopen"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Whether to attempt to disable opening
>>>> dynamic components or not
>>>>                  MCA mca: parameter "mca_verbose" (current value:
>>>> <stderr>, data source:
>>>>                           default value)
>>>>                           Specifies where the default error output
>>>> stream goes (this is
>>>>                           separate from distinct help messages).  Accepts a
>>>>                           comma-delimited list of: stderr, stdout, syslog,
>>>>                           syslogpri:<notice|info|debug>,
>>>> syslogid:<str>  (where str is the
>>>>                           prefix string for all syslog notices),
>>>> file[:filename] (if
>>>>                           filename is not specified, a default
>>>> filename is used),
>>>>                           fileappend (if not specified, the file is opened 
>>>> for
>>>>                           truncation), level[:N] (if specified,
>>>> integer verbose level;
>>>>                           otherwise, 0 is implied)
>>>>                  MCA mpi: parameter "mpi_paffinity_alone" (current
>>>> value:<0>, data
>>>>                           source: default value, synonym of:
>>>> opal_paffinity_alone)
>>>>                           If nonzero, assume that this job is the only 
>>>> (set of)
>>>>                           process(es) running on each node and bind 
>>>> processes to
>>>>                           processors, starting with processor ID 0
>>>>                  MCA mpi: parameter "mpi_param_check" (current value:
>>>> <1>, data source:
>>>>                           default value)
>>>>                           Whether you want MPI API parameters checked
>>>> at run-time or not.
>>>>                            Possible values are 0 (no checking) and 1
>>>> (perform checking at
>>>>                           run-time)
>>>>                  MCA mpi: parameter "mpi_yield_when_idle" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                           Yield the processor when waiting for MPI
>>>> communication (for MPI
>>>>                           processes, will default to 1 when
>>>> oversubscribing nodes)
>>>>                  MCA mpi: parameter "mpi_event_tick_rate" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                           How often to progress TCP communications (0
>>>> = never, otherwise
>>>>                           specified in microseconds)
>>>>                  MCA mpi: parameter "mpi_show_handle_leaks" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Whether MPI_FINALIZE shows all MPI handles
>>>> that were not freed
>>>>                           or not
>>>>                  MCA mpi: parameter "mpi_no_free_handles" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Whether to actually free MPI objects when
>>>> their handles are
>>>>                           freed
>>>>                  MCA mpi: parameter "mpi_show_mpi_alloc_mem_leaks"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           If>0, MPI_FINALIZE will show up to this
>>>> many instances of
>>>>                           memory allocated by MPI_ALLOC_MEM that was
>>>> not freed by
>>>>                           MPI_FREE_MEM
>>>>                  MCA mpi: parameter "mpi_show_mca_params" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           Whether to show all MCA parameter values
>>>> during MPI_INIT or not
>>>>                           (good for reproducability of MPI jobs for
>>>> debug purposes).
>>>>                           Accepted values are all, default, file, api,
>>>> and enviro - or a
>>>>                           comma delimited combination of them
>>>>                  MCA mpi: parameter "mpi_show_mca_params_file"
>>>> (current value:<none>,
>>>>                           data source: default value)
>>>>                           If mpi_show_mca_params is true, setting this
>>>> string to a valid
>>>>                           filename tells Open MPI to dump all the MCA
>>>> parameter values
>>>>                           into a file suitable for reading via the
>>>> mca_param_files
>>>>                           parameter (good for reproducability of MPI jobs)
>>>>                  MCA mpi: parameter "mpi_keep_peer_hostnames" (current
>>>> value:<1>, data
>>>>                           source: default value)
>>>>                           If nonzero, save the string hostnames of all
>>>> MPI peer processes
>>>>                           (mostly for error / debugging output
>>>> messages).  This can add
>>>>                           quite a bit of memory usage to each MPI process.
>>>>                  MCA mpi: parameter "mpi_abort_delay" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           If nonzero, print out an identifying message
>>>> when MPI_ABORT is
>>>>                           invoked (hostname, PID of the process that
>>>> called MPI_ABORT)
>>>>                           and delay for that many seconds before
>>>> exiting (a negative
>>>>                           delay value means to never abort).  This
>>>> allows attaching of a
>>>>                           debugger before quitting the job.
>>>>                  MCA mpi: parameter "mpi_abort_print_stack" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           If nonzero, print out a stack trace when
>>>> MPI_ABORT is invoked
>>>>                  MCA mpi: parameter "mpi_preconnect_mpi" (current
>>>> value:<0>, data
>>>>                           source: default value, synonyms: 
>>>> mpi_preconnect_all)
>>>>                           Whether to force MPI processes to fully
>>>> wire-up the MPI
>>>>                           connections between MPI processes during
>>>> MPI_INIT (vs. making
>>>>                           connections lazily -- upon the first MPI
>>>> traffic between each
>>>>                           process peer pair)
>>>>                  MCA mpi: parameter "mpi_preconnect_all" (current
>>>> value:<0>, data
>>>>                           source: default value, deprecated, synonym of:
>>>>                           mpi_preconnect_mpi)
>>>>                           Whether to force MPI processes to fully
>>>> wire-up the MPI
>>>>                           connections between MPI processes during
>>>> MPI_INIT (vs. making
>>>>                           connections lazily -- upon the first MPI
>>>> traffic between each
>>>>                           process peer pair)
>>>>                  MCA mpi: parameter "mpi_leave_pinned" (current value:
>>>> <-1>, data source:
>>>>                           default value)
>>>>                           Whether to use the "leave pinned" protocol
>>>> or not.  Enabling
>>>>                           this setting can help bandwidth performance
>>>> when repeatedly
>>>>                           sending and receiving large messages with
>>>> the same buffers over
>>>>                           RDMA-based networks (0 = do not use "leave
>>>> pinned" protocol, 1
>>>>                           = use "leave pinned" protocol, -1 = allow
>>>> network to choose at
>>>>                           runtime).
>>>>                  MCA mpi: parameter "mpi_leave_pinned_pipeline"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>                           Whether to use the "leave pinned pipeline"
>>>> protocol or not.
>>>>                  MCA mpi: parameter "mpi_warn_on_fork" (current value:
>>>> <1>, data source:
>>>>                           default value)
>>>>                           If nonzero, issue a warning if program forks
>>>> under conditions
>>>>                           that could cause system errors
>>>>                  MCA mpi: information "mpi_have_sparse_group_storage"
>>>> (value:<0>, data
>>>>                           source: default value)
>>>>                           Whether this Open MPI installation supports
>>>> storing of data in
>>>>                           MPI groups in "sparse" formats (good for
>>>> extremely large
>>>>                           process count MPI jobs that create many
>>>> communicators/groups)
>>>>                  MCA mpi: parameter "mpi_use_sparse_group_storage"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Whether to use "sparse" storage formats for
>>>> MPI groups (only
>>>>                           relevant if mpi_have_sparse_group_storage is 1)
>>>>                  MCA mpi: parameter "mpi_notify_init_finalize"
>>>> (current value:<1>, data
>>>>                           source: default value)
>>>>                           If nonzero, send two notifications during
>>>> MPI_INIT: one near
>>>>                           when MPI_INIT starts, and another right
>>>> before MPI_INIT
>>>>                           finishes, and send 2 notifications during
>>>> MPI_FINALIZE: one
>>>>                           right when MPI_FINALIZE starts, and another near 
>>>> when
>>>>                           MPI_FINALIZE finishes.
>>>>                 MCA orte: parameter "orte_base_help_aggregate"
>>>> (current value:<1>, data
>>>>                           source: default value)
>>>>                           If orte_base_help_aggregate is true,
>>>> duplicate help messages
>>>>                           will be aggregated rather than displayed
>>>> individually.  This
>>>>                           can be helpful for parallel jobs that
>>>> experience multiple
>>>>                           identical failures; rather than print out
>>>> the same help/failure
>>>>                           message N times, display it once with a
>>>> count of how many
>>>>                           processes sent the same message.
>>>>                 MCA orte: parameter "orte_tmpdir_base" (current value:
>>>> <none>, data
>>>>                           source: default value)
>>>>                           Base of the session directory tree
>>>>                 MCA orte: parameter "orte_no_session_dirs" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           Prohibited locations for session directories 
>>>> (multiple
>>>>                           locations separated by ',', default=NULL)
>>>>                 MCA orte: parameter "orte_send_profile" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                           Send profile info in launch message
>>>>                 MCA orte: parameter "orte_debug" (current value:<0>,
>>>> data source:
>>>>                           default value)
>>>>                           Top-level ORTE debug switch (default verbosity: 
>>>> 1)
>>>>                 MCA orte: parameter "orte_debug_verbose" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                           Verbosity level for ORTE debug messages 
>>>> (default: 1)
>>>>                 MCA orte: parameter "orte_debug_daemons" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Whether to debug the ORTE daemons or not
>>>>                 MCA orte: parameter "orte_debug_daemons_file" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Whether want stdout/stderr of daemons to go
>>>> to a file or not
>>>>                 MCA orte: parameter "orte_daemon_bootstrap" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Bootstrap the connection to the HNP
>>>>                 MCA orte: parameter "orte_leave_session_attached"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Whether applications and/or daemons should
>>>> leave their sessions
>>>>                           attached so that any output can be received
>>>> - this allows X
>>>>                           forwarding without all the attendant debugging 
>>>> output
>>>>                 MCA orte: parameter "orte_output_debugger_proctable"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Whether or not to output the debugger
>>>> proctable after launch
>>>>                           (default: false)
>>>>                 MCA orte: parameter "orte_debugger_test_daemon"
>>>> (current value:<none>,
>>>>                           data source: default value)
>>>>                           Name of the executable to be used to
>>>> simulate a debugger
>>>>                           colaunch (relative or absolute path)
>>>>                 MCA orte: parameter "orte_debugger_test_attach"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>                           Test debugger colaunch after debugger attachment
>>>>                 MCA orte: parameter "orte_debugger_check_rate"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>                           Set rate (in secs) for auto-detect of
>>>> debugger attachment (0 =>
>>>>                           do not check)
>>>>                 MCA orte: parameter "orte_do_not_launch" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Perform all necessary operations to prepare
>>>> to launch the
>>>>                           application, but do not actually launch it
>>>>                 MCA orte: parameter "orte_daemon_spin" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Have any orteds spin until we can connect a
>>>> debugger to them
>>>>                 MCA orte: parameter "orte_daemon_fail" (current value:
>>>> <-1>, data source:
>>>>                           default value)
>>>>                           Have the specified orted fail after init for
>>>> debugging purposes
>>>>                 MCA orte: parameter "orte_daemon_fail_delay" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Have the specified orted fail after
>>>> specified number of seconds
>>>>                           (default: 0 =>  no delay)
>>>>                 MCA orte: parameter "orte_heartbeat_rate" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Seconds between checks for daemon
>>>> state-of-health (default: 0
>>>>                           =>  do not check)
>>>>                 MCA orte: parameter "orte_startup_timeout" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Milliseconds/daemon to wait for startup
>>>> before declaring
>>>>                           failed_to_start (default: 0 =>  do not check)
>>>>                 MCA orte: parameter "orte_timing" (current value:<0>,
>>>> data source:
>>>>                           default value)
>>>>                           Request that critical timing loops be measured
>>>>                 MCA orte: parameter "orte_timing_details" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Request that detailed timing data by reported
>>>>                 MCA orte: parameter "orte_base_user_debugger" (current
>>>> value:<totalview
>>>>                           @mpirun@ -a @mpirun_args@ : ddt -n @np@
>>>> -start @executable@
>>>>                           @executable_argv@ @single_app@ : fxp @mpirun@ -a
>>>>                           @mpirun_args@>, data source: default value)
>>>>                           Sequence of user-level debuggers to search
>>>> for in orterun
>>>>                 MCA orte: parameter "orte_abort_timeout" (current
>>>> value:<1>, data
>>>>                           source: default value)
>>>>                           Max time to wait [in secs] before aborting
>>>> an ORTE operation
>>>>                           (default: 1sec)
>>>>                 MCA orte: parameter "orte_timeout_step" (current
>>>> value:<1000>, data
>>>>                           source: default value)
>>>>                           Time to wait [in usecs/proc] before aborting
>>>> an ORTE operation
>>>>                           (default: 1000 usec/proc)
>>>>                 MCA orte: parameter "orte_default_hostfile" (current value:
>>>> 
>>>> </home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/etc/openmpi-default-hostfile>,
>>>>                           data source: default value)
>>>>                           Name of the default hostfile (relative or
>>>> absolute path, "none"
>>>>                           to ignore environmental or default MCA param 
>>>> setting)
>>>>                 MCA orte: parameter "orte_rankfile" (current value:
>>>> <none>, data source:
>>>>                           default value, synonyms: rmaps_rank_file_path)
>>>>                           Name of the rankfile to be used for mapping
>>>> processes (relative
>>>>                           or absolute path)
>>>>                 MCA orte: parameter "orte_keep_fqdn_hostnames"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>                           Whether or not to keep FQDN hostnames [default: 
>>>> no]
>>>>                 MCA orte: parameter "orte_use_regexp" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Whether or not to use regular expressions
>>>> for launch [default:
>>>>                           no]
>>>>                 MCA orte: parameter "orte_tag_output" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Tag all output with [job,rank] (default: false)
>>>>                 MCA orte: parameter "orte_xml_output" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Display all output in XML format (default: false)
>>>>                 MCA orte: parameter "orte_xml_file" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           Provide all output in XML format to the 
>>>> specified file
>>>>                 MCA orte: parameter "orte_timestamp_output" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Timestamp all application process output
>>>> (default: false)
>>>>                 MCA orte: parameter "orte_output_filename" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           Redirect output from application processes
>>>> into filename.rank
>>>>                           [default: NULL]
>>>>                 MCA orte: parameter "orte_show_resolved_nodenames"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Display any node names that are resolved to
>>>> a different name
>>>>                           (default: false)
>>>>                 MCA orte: parameter "orte_hetero_apps" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Indicates that multiple app_contexts are
>>>> being provided that
>>>>                           are a mix of 32/64 bit binaries (default: false)
>>>>                 MCA orte: parameter "orte_launch_agent" (current
>>>> value:<orted>, data
>>>>                           source: default value)
>>>>                           Command used to start processes on remote
>>>> nodes (default:
>>>>                           orted)
>>>>                 MCA orte: parameter "orte_allocation_required"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>                           Whether or not an allocation by a resource
>>>> manager is required
>>>>                           [default: no]
>>>>                 MCA orte: parameter "orte_xterm" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           Create a new xterm window and display output
>>>> from the specified
>>>>                           ranks there [default: none]
>>>>                 MCA orte: parameter "orte_forward_job_control"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>                           Forward SIGTSTP (after converting to
>>>> SIGSTOP) and SIGCONT
>>>>                           signals to the application procs [default: no]
>>>>                 MCA orte: parameter "orte_rsh_agent" (current value:
>>>> <ssh : rsh>, data
>>>>                           source: default value, synonyms:
>>>> pls_rsh_agent, plm_rsh_agent)
>>>>                           The command used to launch executables on remote 
>>>> nodes
>>>>                           (typically either "ssh" or "rsh")
>>>>                 MCA orte: parameter "orte_assume_same_shell" (current
>>>> value:<1>, data
>>>>                           source: default value, synonyms:
>>>> plm_rsh_assume_same_shell)
>>>>                           If set to 1, assume that the shell on the
>>>> remote node is the
>>>>                           same as the shell on the local node.
>>>> Otherwise, probe for what
>>>>                           the remote shell [default: 1]
>>>>                 MCA orte: parameter "orte_report_launch_progress"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Output a brief periodic report on launch
>>>> progress [default: no]
>>>>                 MCA orte: parameter "orte_num_boards" (current value:
>>>> <1>, data source:
>>>>                           default value)
>>>>                           Number of processor boards/node (1-256) 
>>>> [default: 1]
>>>>                 MCA orte: parameter "orte_num_sockets" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Number of sockets/board (1-256)
>>>>                 MCA orte: parameter "orte_num_cores" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Number of cores/socket (1-256)
>>>>                 MCA orte: parameter "orte_cpu_set" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           Comma-separated list of ranges specifying 
>>>> logical cpus
>>>>                           allocated to this job [default: none]
>>>>                 MCA orte: parameter "orte_process_binding" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           Policy for binding processes [none | core |
>>>> socket | board]
>>>>                           (supported qualifier: if-avail)
>>>>                 MCA orte: parameter "orte_report_bindings" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Report bindings
>>>>                 MCA orte: parameter "orte_report_events" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           URI to which events are to be reported
>>>> (default: NULL)]
>>>>                 MCA opal: parameter "opal_net_private_ipv4" (current value:
>>>> 
>>>> <10.0.0.0/8;172.16.0.0/12;192.168.0.0/16;169.254.0.0/16>, data
>>>>                           source: default value)
>>>>                           Semicolon-delimited list of CIDR notation
>>>> entries specifying
>>>>                           what networks are considered "private"
>>>> (default value based on
>>>>                           RFC1918 and RFC3330)
>>>>                 MCA opal: parameter "opal_signal" (current value:
>>>> <6,7,8,11>, data
>>>>                           source: default value)
>>>>                           Comma-delimited list of integer signal
>>>> numbers to Open MPI to
>>>>                           attempt to intercept.  Upon receipt of the
>>>> intercepted signal,
>>>>                           Open MPI will display a stack trace and
>>>> abort.  Open MPI will
>>>>                           *not* replace signals if handlers are
>>>> already installed by the
>>>>                           time MPI_INIT is invoked.  Optionally append
>>>> ":complain" to any
>>>>                           signal number in the comma-delimited list to
>>>> make Open MPI
>>>>                           complain if it detects another signal
>>>> handler (and therefore
>>>>                           does not insert its own).
>>>>                 MCA opal: parameter "opal_profile" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Set to non-zero to profile component selections
>>>>                 MCA opal: parameter "opal_profile_file" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           Name of the file containing the cluster 
>>>> configuration
>>>>                           information
>>>>                 MCA opal: parameter "opal_paffinity_alone" (current
>>>> value:<0>, data
>>>>                           source: default value, synonyms: 
>>>> mpi_paffinity_alone)
>>>>                           If nonzero, assume that this job is the only 
>>>> (set of)
>>>>                           process(es) running on each node and bind 
>>>> processes to
>>>>                           processors, starting with processor ID 0
>>>>                 MCA opal: parameter "opal_set_max_sys_limits" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Set to non-zero to automatically set any
>>>> system-imposed limits
>>>>                           to the maximum allowed
>>>>                 MCA opal: parameter "opal_event_include" (current
>>>> value:<poll>, data
>>>>                           source: default value)
>>>>                           Comma-delimited list of libevent subsystems
>>>> to use (epoll,
>>>>                           poll, select -- available on your platform)
>>>>            MCA backtrace: parameter "backtrace" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           Default selection set of components for the
>>>> backtrace framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>            MCA backtrace: parameter "backtrace_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the backtrace framework
>>>> (0 = no verbosity)
>>>>            MCA backtrace: parameter "backtrace_execinfo_priority"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>           MCA memchecker: parameter "memchecker" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           Default selection set of components for the 
>>>> memchecker
>>>>                           framework (<none>  means use all components
>>>> that can be found)
>>>>               MCA memory: parameter "memory" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> memory framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>               MCA memory: parameter "memory_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the memory framework (0
>>>> = no verbosity)
>>>>               MCA memory: information
>>>> "memory_linux_ptmalloc2_available" (value:<1>,
>>>>                           data source: default value)
>>>>                           Whether ptmalloc2 support is included in
>>>> Open MPI or not (1 =
>>>>                           yes, 0 = no)
>>>>               MCA memory: information
>>>> "memory_linux_ummunotify_available" (value:<0>,
>>>>                           data source: default value)
>>>>                           Whether ummunotify support is included in
>>>> Open MPI or not (1 =
>>>>                           yes, 0 = no)
>>>>               MCA memory: parameter "memory_linux_ptmalloc2_enable"
>>>> (current value:<-1>,
>>>>                           data source: default value)
>>>>                           Whether to enable ptmalloc2 support or not
>>>> (negative = try to
>>>>                           enable, but continue even if support is not
>>>> available, 0 = do
>>>>                           not enable support, positive = try to enable
>>>> and fail if
>>>>                           support is not available)
>>>>               MCA memory: parameter "memory_linux_ummunotify_enable"
>>>> (current value:
>>>>                           <-1>, data source: default value)
>>>>                           Whether to enable ummunotify support or not
>>>> (negative = try to
>>>>                           enable, but continue even if support is not
>>>> available, 0 = do
>>>>                           not enable support, positive = try to enable
>>>> and fail if
>>>>                           support is not available)
>>>>               MCA memory: parameter "memory_linux_disable" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           If this MCA parameter is set to 1 **VIA
>>>> ENVIRONMENT VARIABLE
>>>>                           ONLY*** (this MCA parameter *CANNOT* be set
>>>> in a file or on the
>>>>                           mpirun command line!), this component will
>>>> be disabled and will
>>>>                           not attempt to use either ummunotify or
>>>> memory hook support
>>>>               MCA memory: parameter "memory_linux_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>            MCA paffinity: parameter "paffinity_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level of the paffinity framework
>>>>            MCA paffinity: parameter "paffinity" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           Default selection set of components for the
>>>> paffinity framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>            MCA paffinity: parameter "paffinity_hwloc_priority"
>>>> (current value:<40>, data
>>>>                           source: default value)
>>>>                           Priority of the hwloc paffinity component
>>>>                MCA carto: parameter "carto_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level of the carto framework
>>>>                MCA carto: parameter "carto" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> carto framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                MCA carto: parameter "carto_auto_detect_priority"
>>>> (current value:<11>,
>>>>                           data source: default value)
>>>>                           Priority of the auto_detect carto component
>>>>                MCA carto: parameter "carto_file_path" (current value:
>>>> <none>, data
>>>>                           source: default value)
>>>>                           The path to the cartography file
>>>>                MCA carto: parameter "carto_file_priority" (current
>>>> value:<10>, data
>>>>                           source: default value)
>>>>                           Priority of the file carto component
>>>>                MCA shmem: parameter "shmem_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level of the shmem framework
>>>>                MCA shmem: parameter "shmem" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> shmem framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                MCA shmem: parameter "shmem_mmap_enable_nfs_warning"
>>>> (current value:<1>,
>>>>                           data source: default value)
>>>>                           Enable the warning emitted when Open MPI
>>>> detects that its
>>>>                           shared memory backing file is located on a
>>>> network filesystem
>>>>                           (1 = enabled, 0 = disabled).
>>>>                MCA shmem: parameter "shmem_mmap_priority" (current
>>>> value:<50>, data
>>>>                           source: default value)
>>>>                           Priority of the mmap shmem component
>>>>                MCA shmem: parameter "shmem_mmap_relocate_backing_file"
>>>> (current value:
>>>>                           <0>, data source: default value)
>>>>                           Whether to change the default placement of
>>>> backing files or not
>>>>                           (Negative = try to relocate backing files to
>>>> an area rooted at
>>>>                           the path specified by
>>>> 
>>>> shmem_mmap_opal_shmem_mmap_backing_file_base_dir, but continue
>>>>                           with the default path if the relocation
>>>> fails, 0 = do not
>>>>                           relocate, Positive = same as the negative
>>>> option, but will fail
>>>>                           if the relocation fails.
>>>>                MCA shmem: parameter "shmem_mmap_backing_file_base_dir"
>>>> (current value:
>>>>                           </dev/shm>, data source: default value)
>>>>                           Specifies where backing files will be created 
>>>> when
>>>>                           shmem_mmap_relocate_backing_file is in use.
>>>>                MCA shmem: parameter "shmem_posix_priority" (current
>>>> value:<40>, data
>>>>                           source: default value)
>>>>                           Priority of the posix shmem component
>>>>                MCA shmem: parameter "shmem_sysv_priority" (current
>>>> value:<30>, data
>>>>                           source: default value)
>>>>                           Priority of the sysv shmem component
>>>>            MCA maffinity: parameter "maffinity_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level of the maffinity framework
>>>>            MCA maffinity: parameter "maffinity" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           Default selection set of components for the
>>>> maffinity framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>            MCA maffinity: parameter "maffinity_first_use_priority"
>>>> (current value:<10>,
>>>>                           data source: default value)
>>>>                           Priority of the first_use maffinity component
>>>>            MCA maffinity: parameter "maffinity_hwloc_priority"
>>>> (current value:<40>, data
>>>>                           source: default value)
>>>>                           Priority of the hwloc maffinity component
>>>>                MCA timer: parameter "timer" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> timer framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                MCA timer: parameter "timer_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the timer framework (0 =
>>>> no verbosity)
>>>>                MCA timer: parameter "timer_linux_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>              MCA sysinfo: parameter "sysinfo" (current value:<none>,
>>>> data source:
>>>>                           default value)
>>>>                           Default selection set of components for the
>>>> sysinfo framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>              MCA sysinfo: parameter "sysinfo_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the sysinfo framework (0
>>>> = no verbosity)
>>>>              MCA sysinfo: parameter "sysinfo_linux_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                MCA hwloc: parameter "hwloc_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level of the hwloc framework
>>>>                MCA hwloc: parameter "hwloc_base_mem_alloc_policy"
>>>> (current value:<none>,
>>>>                           data source: default value)
>>>>                           Policy that determines how general memory
>>>> allocations are bound
>>>>                           after MPI_INIT.  A value of "none" means
>>>> that no memory policy
>>>>                           is applied.  A value of "local_only" means
>>>> that all memory
>>>>                           allocations will be restricted to the local
>>>> NUMA node where
>>>>                           each process is placed.  Note that operating
>>>> system paging
>>>>                           policies are unaffected by this setting.
>>>> For example, if
>>>>                           "local_only" is used and local NUMA node
>>>> memory is exhausted, a
>>>>                           new memory allocation may cause paging.
>>>>                MCA hwloc: parameter
>>>> "hwloc_base_mem_bind_failure_action" (current value:
>>>>                           <error>, data source: default value)
>>>>                           What Open MPI will do if it explicitly tries
>>>> to bind memory to
>>>>                           a specific NUMA location, and fails.  Note
>>>> that this is a
>>>>                           different case than the general allocation
>>>> policy described by
>>>>                           hwloc_base_alloc_policy.  A value of "warn"
>>>> means that Open MPI
>>>>                           will warn the first time this happens, but
>>>> allow the job to
>>>>                           continue (possibly with degraded
>>>> performance).  A value of
>>>>                           "error" means that Open MPI will abort the
>>>> job if this happens.
>>>>                MCA hwloc: parameter "hwloc" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> hwloc framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                MCA hwloc: parameter "hwloc_hwloc132_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA dpm: parameter "dpm" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> dpm framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA dpm: parameter "dpm_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the dpm framework (0 =
>>>> no verbosity)
>>>>                  MCA dpm: parameter "dpm_orte_priority" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>               MCA pubsub: parameter "pubsub" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> pubsub framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>               MCA pubsub: parameter "pubsub_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the pubsub framework (0
>>>> = no verbosity)
>>>>               MCA pubsub: parameter "pubsub_orte_priority" (current
>>>> value:<50>, data
>>>>                           source: default value)
>>>>                           Priority of the pubsub pmi component
>>>>            MCA allocator: parameter "allocator" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           Default selection set of components for the
>>>> allocator framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>            MCA allocator: parameter "allocator_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the allocator framework
>>>> (0 = no verbosity)
>>>>            MCA allocator: parameter "allocator_basic_priority"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>            MCA allocator: parameter "allocator_bucket_num_buckets"
>>>> (current value:<30>,
>>>>                           data source: default value)
>>>>            MCA allocator: parameter "allocator_bucket_priority"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>                 MCA coll: parameter "coll" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> coll framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                 MCA coll: parameter "coll_base_verbose" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the coll framework (0 =
>>>> no verbosity)
>>>>                 MCA coll: parameter "coll_basic_priority" (current
>>>> value:<10>, data
>>>>                           source: default value)
>>>>                           Priority of the basic coll component
>>>>                 MCA coll: parameter "coll_basic_crossover" (current
>>>> value:<4>, data
>>>>                           source: default value)
>>>>                           Minimum number of processes in a
>>>> communicator before using the
>>>>                           logarithmic algorithms
>>>>                 MCA coll: parameter "coll_hierarch_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Priority of the hierarchical coll component
>>>>                 MCA coll: parameter "coll_hierarch_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Turn verbose message of the hierarchical
>>>> coll component on/off
>>>>                 MCA coll: parameter "coll_hierarch_use_rdma" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Switch from the send btl list used to detect
>>>> hierarchies to the
>>>>                           rdma btl list
>>>>                 MCA coll: parameter "coll_hierarch_ignore_sm" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Ignore sm protocol when detecting
>>>> hierarchies. Required to
>>>>                           enable the usage of protocol specific
>>>> collective operations
>>>>                 MCA coll: parameter "coll_hierarch_detection_alg"
>>>> (current value:<2>,
>>>>                           data source: default value)
>>>>                           Used to specify the algorithm for detecting
>>>> Hierarchy.Choose
>>>>                           between all or two levels of hierarchy
>>>>                 MCA coll: parameter "coll_hierarch_bcast_alg" (current
>>>> value:<4>, data
>>>>                           source: default value)
>>>>                           Used to specify the algorithm used for bcast
>>>> operations.
>>>>                 MCA coll: parameter "coll_hierarch_segment_size"
>>>> (current value:<32768>,
>>>>                           data source: default value)
>>>>                           Used to specify the segment size for
>>>> segmented algorithms.
>>>>                 MCA coll: parameter "coll_inter_priority" (current
>>>> value:<40>, data
>>>>                           source: default value)
>>>>                           Priority of the inter coll component
>>>>                 MCA coll: parameter "coll_inter_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Turn verbose message of the inter coll
>>>> component on/off
>>>>                 MCA coll: parameter "coll_self_priority" (current
>>>> value:<75>, data
>>>>                           source: default value)
>>>>                 MCA coll: parameter "coll_sm_priority" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Priority of the sm coll component
>>>>                 MCA coll: parameter "coll_sm_control_size" (current
>>>> value:<4096>, data
>>>>                           source: default value)
>>>>                           Length of the control data -- should usually
>>>> be either the
>>>>                           length of a cache line on most SMPs, or the
>>>> size of a page on
>>>>                           machines that support direct memory affinity
>>>> page placement (in
>>>>                           bytes)
>>>>                 MCA coll: parameter "coll_sm_fragment_size" (current
>>>> value:<8192>, data
>>>>                           source: default value)
>>>>                           Fragment size (in bytes) used for passing
>>>> data through shared
>>>>                           memory (will be rounded up to the nearest
>>>> control_size size)
>>>>                 MCA coll: parameter "coll_sm_comm_in_use_flags"
>>>> (current value:<2>, data
>>>>                           source: default value)
>>>>                           Number of "in use" flags, used to mark a
>>>> message passing area
>>>>                           segment as currently being used or not (must
>>>> be>= 2 and<=
>>>>                           comm_num_segments)
>>>>                 MCA coll: parameter "coll_sm_comm_num_segments"
>>>> (current value:<8>, data
>>>>                           source: default value)
>>>>                           Number of segments in each communicator's
>>>> shared memory message
>>>>                           passing area (must be>= 2, and must be a 
>>>> multiple of
>>>>                           comm_in_use_flags)
>>>>                 MCA coll: parameter "coll_sm_tree_degree" (current
>>>> value:<4>, data
>>>>                           source: default value)
>>>>                           Degree of the tree for tree-based operations
>>>> (must be =>  1 and
>>>>                           <= min(control_size, 255))
>>>>                 MCA coll: parameter "coll_sm_info_num_procs" (current
>>>> value:<4>, data
>>>>                           source: default value)
>>>>                           Number of processes to use for the calculation 
>>>> of the
>>>>                           shared_mem_size MCA information parameter
>>>> (must be =>  2)
>>>>                 MCA coll: information "coll_sm_shared_mem_used_data"
>>>> (value:<548864>,
>>>>                           data source: default value)
>>>>                           Amount of shared memory used, per
>>>> communicator, in the shared
>>>>                           memory data area for info_num_procs
>>>> processes (in bytes)
>>>>                 MCA coll: parameter "coll_sync_priority" (current
>>>> value:<50>, data
>>>>                           source: default value)
>>>>                           Priority of the sync coll component; only 
>>>> relevant if
>>>>                           barrier_before or barrier_after is>  0
>>>>                 MCA coll: parameter "coll_sync_barrier_before"
>>>> (current value:<1000>,
>>>>                           data source: default value)
>>>>                           Do a synchronization before each Nth collective
>>>>                 MCA coll: parameter "coll_sync_barrier_after" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Do a synchronization after each Nth collective
>>>>                 MCA coll: parameter "coll_tuned_priority" (current
>>>> value:<30>, data
>>>>                           source: default value)
>>>>                           Priority of the tuned coll component
>>>>                 MCA coll: parameter
>>>> "coll_tuned_pre_allocate_memory_comm_size_limit"
>>>>                           (current value:<32768>, data source: default 
>>>> value)
>>>>                           Size of communicator were we stop
>>>> pre-allocating memory for the
>>>>                           fixed internal buffer used for message
>>>> requests etc that is
>>>>                           hung off the communicator data segment. I.e.
>>>> if you have a
>>>>                           100'000 nodes you might not want to
>>>> pre-allocate 200'000
>>>>                           request handle slots per communicator instance!
>>>>                 MCA coll: parameter "coll_tuned_init_tree_fanout"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                           Inital fanout used in the tree topologies for 
>>>> each
>>>>                           communicator. This is only an initial guess,
>>>> if a tuned
>>>>                           collective needs a different fanout for an
>>>> operation, it build
>>>>                           it dynamically. This parameter is only for
>>>> the first guess and
>>>>                           might save a little time
>>>>                 MCA coll: parameter "coll_tuned_init_chain_fanout"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                           Inital fanout used in the chain (fanout
>>>> followed by pipeline)
>>>>                           topologies for each communicator. This is
>>>> only an initial
>>>>                           guess, if a tuned collective needs a
>>>> different fanout for an
>>>>                           operation, it build it dynamically. This
>>>> parameter is only for
>>>>                           the first guess and might save a little time
>>>>                 MCA coll: parameter "coll_tuned_use_dynamic_rules"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Switch used to decide if we use static
>>>> (compiled/if statements)
>>>>                           or dynamic (built at runtime) decision function 
>>>> rules
>>>>                   MCA io: parameter "io_base_freelist_initial_size"
>>>> (current value:<16>,
>>>>                           data source: default value)
>>>>                           Initial MPI-2 IO request freelist size
>>>>                   MCA io: parameter "io_base_freelist_max_size"
>>>> (current value:<64>,
>>>>                           data source: default value)
>>>>                           Max size of the MPI-2 IO request freelist
>>>>                   MCA io: parameter "io_base_freelist_increment"
>>>> (current value:<16>,
>>>>                           data source: default value)
>>>>                           Increment size of the MPI-2 IO request freelist
>>>>                   MCA io: parameter "io" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> io framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                   MCA io: parameter "io_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the io framework (0 = no
>>>> verbosity)
>>>>                   MCA io: parameter "io_romio_priority" (current
>>>> value:<10>, data
>>>>                           source: default value)
>>>>                           Priority of the io romio component
>>>>                   MCA io: parameter "io_romio_delete_priority"
>>>> (current value:<10>, data
>>>>                           source: default value)
>>>>                           Delete priority of the io romio component
>>>>                   MCA io: information "io_romio_version" (value:<from
>>>> MPICH2 v1.3.1 with
>>>>                           an additional patch from
>>>> romio-ma...@mcs.anl.gov about an
>>>>                           attribute issue>, data source: default value)
>>>>                           Version of ROMIO
>>>>                   MCA io: information "io_romio_user_configure_params"
>>>> (value:<none>,
>>>>                           data source: default value)
>>>>                           User-specified command line parameters
>>>> passed to ROMIO's
>>>>                           configure script
>>>>                   MCA io: information
>>>> "io_romio_complete_configure_params" (value:<
>>>>                           CFLAGS='-DNDEBUG -g -O2 -finline-functions
>>>> -fno-strict-aliasing
>>>>                           -pthread' CPPFLAGS='
>>>> 
>>>> -I/home/andrea/library/openmpi/openmpi-1.6/opal/mca/hwloc/hwloc132/hwloc/include
>>>>                             -I/usr/include/infiniband 
>>>> -I/usr/include/infiniband'
>>>>                           FFLAGS='' LDFLAGS='-Wl,--rpath
>>>>                           -Wl,/home/andrea/library/gcc/gcc-objects/lib64 '
>>>>                           --enable-shared --enable-static
>>>> 
>>>> --prefix=/home/andrea/library/openmpi/openmpi-1.6-gnu-4.7
>>>>                           --with-mpi=open_mpi --disable-aio>, data
>>>> source: default value)
>>>>                           Complete set of command line parameters
>>>> passed to ROMIO's
>>>>                           configure script
>>>>                MCA mpool: parameter "mpool" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> mpool framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                MCA mpool: parameter "mpool_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the mpool framework (0 =
>>>> no verbosity)
>>>>                MCA mpool: parameter "mpool_fake_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                MCA mpool: parameter "mpool_rdma_rcache_name" (current
>>>> value:<vma>, data
>>>>                           source: default value)
>>>>                           The name of the registration cache the mpool
>>>> should use
>>>>                MCA mpool: parameter "mpool_rdma_rcache_size_limit"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           the maximum size of registration cache in
>>>> bytes. 0 is unlimited
>>>>                           (default 0)
>>>>                MCA mpool: parameter "mpool_rdma_print_stats" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           print pool usage statistics at the end of the run
>>>>                MCA mpool: parameter "mpool_rdma_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                MCA mpool: parameter "mpool_sm_allocator" (current
>>>> value:<bucket>, data
>>>>                           source: default value)
>>>>                           Name of allocator component to use with sm mpool
>>>>                MCA mpool: parameter "mpool_sm_min_size" (current
>>>> value:<67108864>, data
>>>>                           source: default value)
>>>>                           Minimum size of the sm mpool shared memory file
>>>>                MCA mpool: parameter "mpool_sm_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Enable verbose output for mpool sm component
>>>>                MCA mpool: parameter "mpool_sm_priority" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                  MCA pml: parameter "pml_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level of the PML framework
>>>>                  MCA pml: parameter "pml" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> pml framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA pml: parameter "pml_bfo_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                  MCA pml: parameter "pml_bfo_free_list_num" (current
>>>> value:<4>, data
>>>>                           source: default value)
>>>>                  MCA pml: parameter "pml_bfo_free_list_max" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                  MCA pml: parameter "pml_bfo_free_list_inc" (current
>>>> value:<64>, data
>>>>                           source: default value)
>>>>                  MCA pml: parameter "pml_bfo_priority" (current value:
>>>> <5>, data source:
>>>>                           default value)
>>>>                  MCA pml: parameter "pml_bfo_send_pipeline_depth"
>>>> (current value:<3>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_bfo_recv_pipeline_depth"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_bfo_rdma_put_retries_limit"
>>>> (current value:<5>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_bfo_max_rdma_per_request"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_bfo_max_send_per_range"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_bfo_unexpected_limit"
>>>> (current value:<128>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_bfo_allocator" (current
>>>> value:<bucket>, data
>>>>                           source: default value)
>>>>                           Name of allocator component for unexpected 
>>>> messages
>>>>                  MCA pml: parameter "pml_cm_free_list_num" (current
>>>> value:<4>, data
>>>>                           source: default value)
>>>>                           Initial size of request free lists
>>>>                  MCA pml: parameter "pml_cm_free_list_max" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                           Maximum size of request free lists
>>>>                  MCA pml: parameter "pml_cm_free_list_inc" (current
>>>> value:<64>, data
>>>>                           source: default value)
>>>>                           Number of elements to add when growing
>>>> request free lists
>>>>                  MCA pml: parameter "pml_cm_priority" (current value:
>>>> <10>, data source:
>>>>                           default value)
>>>>                           CM PML selection priority
>>>>                  MCA pml: parameter "pml_csum_free_list_num" (current
>>>> value:<4>, data
>>>>                           source: default value)
>>>>                  MCA pml: parameter "pml_csum_free_list_max" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                  MCA pml: parameter "pml_csum_free_list_inc" (current
>>>> value:<64>, data
>>>>                           source: default value)
>>>>                  MCA pml: parameter "pml_csum_send_pipeline_depth"
>>>> (current value:<3>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_csum_recv_pipeline_depth"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_csum_rdma_put_retries_limit"
>>>> (current value:
>>>>                           <5>, data source: default value)
>>>>                  MCA pml: parameter "pml_csum_max_rdma_per_request"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_csum_max_send_per_range"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_csum_unexpected_limit"
>>>> (current value:<128>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_csum_allocator" (current
>>>> value:<bucket>, data
>>>>                           source: default value)
>>>>                           Name of allocator component for unexpected 
>>>> messages
>>>>                  MCA pml: parameter "pml_csum_priority" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                  MCA pml: parameter "pml_ob1_free_list_num" (current
>>>> value:<4>, data
>>>>                           source: default value)
>>>>                  MCA pml: parameter "pml_ob1_free_list_max" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                  MCA pml: parameter "pml_ob1_free_list_inc" (current
>>>> value:<64>, data
>>>>                           source: default value)
>>>>                  MCA pml: parameter "pml_ob1_priority" (current value:
>>>> <20>, data source:
>>>>                           default value)
>>>>                  MCA pml: parameter "pml_ob1_send_pipeline_depth"
>>>> (current value:<3>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_ob1_recv_pipeline_depth"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_ob1_rdma_put_retries_limit"
>>>> (current value:<5>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_ob1_max_rdma_per_request"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_ob1_max_send_per_range"
>>>> (current value:<4>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_ob1_unexpected_limit"
>>>> (current value:<128>,
>>>>                           data source: default value)
>>>>                  MCA pml: parameter "pml_ob1_allocator" (current
>>>> value:<bucket>, data
>>>>                           source: default value)
>>>>                           Name of allocator component for unexpected 
>>>> messages
>>>>                  MCA pml: parameter "pml_v_priority" (current value:
>>>> <-1>, data source:
>>>>                           default value)
>>>>                  MCA pml: parameter "pml_v_output" (current value:
>>>> <stderr>, data source:
>>>>                           default value)
>>>>                  MCA pml: parameter "pml_v_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                  MCA bml: parameter "bml" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> bml framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA bml: parameter "bml_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the bml framework (0 =
>>>> no verbosity)
>>>>                  MCA bml: parameter "bml_r2_show_unreach_errors"
>>>> (current value:<1>,
>>>>                           data source: default value)
>>>>                           Show error message when procs are unreachable
>>>>                  MCA bml: parameter "bml_r2_priority" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>               MCA rcache: parameter "rcache" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> rcache framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>               MCA rcache: parameter "rcache_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the rcache framework (0
>>>> = no verbosity)
>>>>               MCA rcache: parameter "rcache_vma_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level of the BTL framework
>>>>                  MCA btl: parameter "btl" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> btl framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA btl: parameter "btl_self_free_list_num" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Number of fragments by default
>>>>                  MCA btl: parameter "btl_self_free_list_max" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                           Maximum number of fragments
>>>>                  MCA btl: parameter "btl_self_free_list_inc" (current
>>>> value:<32>, data
>>>>                           source: default value)
>>>>                           Increment by this number of fragments
>>>>                  MCA btl: parameter "btl_self_exclusivity" (current
>>>> value:<65536>, data
>>>>                           source: default value)
>>>>                           BTL exclusivity (must be>= 0)
>>>>                  MCA btl: parameter "btl_self_flags" (current value:
>>>> <10>, data source:
>>>>                           default value)
>>>>                           BTL bit flags (general flags: SEND=1, PUT=2, 
>>>> GET=4,
>>>>                           SEND_INPLACE=8, RDMA_MATCHED=64,
>>>> HETEROGENEOUS_RDMA=256; flags
>>>>                           only used by the "dr" PML (ignored by others): 
>>>> ACK=16,
>>>>                           CHECKSUM=32, RDMA_COMPLETION=128; flags only
>>>> used by the "bfo"
>>>>                           PML (ignored by others): FAILOVER_SUPPORT=512)
>>>>                  MCA btl: parameter "btl_self_rndv_eager_limit"
>>>> (current value:<131072>,
>>>>                           data source: default value)
>>>>                           Size (in bytes) of "phase 1" fragment sent
>>>> for all large
>>>>                           messages (must be>= 0 and<= eager_limit)
>>>>                  MCA btl: parameter "btl_self_eager_limit" (current
>>>> value:<131072>, data
>>>>                           source: default value)
>>>>                           Maximum size (in bytes) of "short" messages
>>>> (must be>= 1).
>>>>                  MCA btl: parameter "btl_self_max_send_size" (current
>>>> value:<262144>,
>>>>                           data source: default value)
>>>>                           Maximum size (in bytes) of a single "phase
>>>> 2" fragment of a
>>>>                           long message when using the pipeline
>>>> protocol (must be>= 1)
>>>>                  MCA btl: parameter
>>>> "btl_self_rdma_pipeline_send_length" (current value:
>>>>                           <2147483647>, data source: default value)
>>>>                           Length of the "phase 2" portion of a large
>>>> message (in bytes)
>>>>                           when using the pipeline protocol.  This part
>>>> of the message
>>>>                           will be split into fragments of size
>>>> max_send_size and sent
>>>>                           using send/receive semantics (must be>= 0;
>>>> only relevant when
>>>>                           the PUT flag is set)
>>>>                  MCA btl: parameter "btl_self_rdma_pipeline_frag_size"
>>>> (current value:
>>>>                           <2147483647>, data source: default value)
>>>>                           Maximum size (in bytes) of a single "phase
>>>> 3" fragment from a
>>>>                           long message when using the pipeline
>>>> protocol.  These fragments
>>>>                           will be sent using RDMA semantics (must be
>>>>> = 1; only relevant
>>>>                           when the PUT flag is set)
>>>>                  MCA btl: parameter "btl_self_min_rdma_pipeline_size"
>>>> (current value:
>>>>                           <0>, data source: default value)
>>>>                           Messages smaller than this size (in bytes)
>>>> will not use the
>>>>                           RDMA pipeline protocol.  Instead, they will
>>>> be split into
>>>>                           fragments of max_send_size and sent using 
>>>> send/receive
>>>>                           semantics (must be>=0, and is automatically
>>>> adjusted up to at
>>>>                           least
>>>> (eager_limit+btl_rdma_pipeline_send_length); only
>>>>                           relevant when the PUT flag is set)
>>>>                  MCA btl: parameter "btl_self_bandwidth" (current
>>>> value:<100>, data
>>>>                           source: default value)
>>>>                           Approximate maximum bandwidth of
>>>> interconnect(0 = auto-detect
>>>>                           value at run-time [not supported in all BTL
>>>> modules],>= 1 =
>>>>                           bandwidth in Mbps)
>>>>                  MCA btl: parameter "btl_self_latency" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Approximate latency of interconnect (must be>= 0)
>>>>                  MCA btl: parameter "btl_self_priority" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                  MCA btl: information "btl_sm_have_knem_support"
>>>> (value:<0>, data
>>>>                           source: default value)
>>>>                           Whether this component supports the knem
>>>> Linux kernel module or
>>>>                           not
>>>>                  MCA btl: parameter "btl_sm_use_knem" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Whether knem support is desired or not
>>>> (negative = try to
>>>>                           enable knem support, but continue even if it
>>>> is not available,
>>>>                           0 = do not enable knem support, positive =
>>>> try to enable knem
>>>>                           support and fail if it is not available)
>>>>                  MCA btl: parameter "btl_sm_knem_dma_min" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Minimum message size (in bytes) to use the
>>>> knem DMA mode;
>>>>                           ignored if knem does not support DMA mode (0
>>>> = do not use the
>>>>                           knem DMA mode)
>>>>                  MCA btl: parameter "btl_sm_knem_max_simultaneous"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Max number of simultaneous ongoing knem
>>>> operations to support
>>>>                           (0 = do everything synchronously, which
>>>> probably gives the best
>>>>                           large message latency;>0 means to do all 
>>>> operations
>>>>                           asynchronously, which supports better
>>>> overlap for simultaneous
>>>>                           large message sends)
>>>>                  MCA btl: parameter "btl_sm_free_list_num" (current
>>>> value:<8>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_sm_free_list_max" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_sm_free_list_inc" (current
>>>> value:<64>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_sm_max_procs" (current value:
>>>> <-1>, data source:
>>>>                           default value)
>>>>                  MCA btl: parameter "btl_sm_mpool" (current value:
>>>> <sm>, data source:
>>>>                           default value)
>>>>                  MCA btl: parameter "btl_sm_fifo_size" (current value:
>>>> <4096>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_sm_num_fifos" (current value:
>>>> <1>, data source:
>>>>                           default value)
>>>>                  MCA btl: parameter "btl_sm_fifo_lazy_free" (current
>>>> value:<120>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_sm_sm_extra_procs" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_sm_exclusivity" (current
>>>> value:<65535>, data
>>>>                           source: default value)
>>>>                           BTL exclusivity (must be>= 0)
>>>>                  MCA btl: parameter "btl_sm_flags" (current value:
>>>> <1>, data source:
>>>>                           default value)
>>>>                           BTL bit flags (general flags: SEND=1, PUT=2, 
>>>> GET=4,
>>>>                           SEND_INPLACE=8, RDMA_MATCHED=64,
>>>> HETEROGENEOUS_RDMA=256; flags
>>>>                           only used by the "dr" PML (ignored by others): 
>>>> ACK=16,
>>>>                           CHECKSUM=32, RDMA_COMPLETION=128; flags only
>>>> used by the "bfo"
>>>>                           PML (ignored by others): FAILOVER_SUPPORT=512)
>>>>                  MCA btl: parameter "btl_sm_rndv_eager_limit" (current
>>>> value:<4096>,
>>>>                           data source: default value)
>>>>                           Size (in bytes) of "phase 1" fragment sent
>>>> for all large
>>>>                           messages (must be>= 0 and<= eager_limit)
>>>>                  MCA btl: parameter "btl_sm_eager_limit" (current
>>>> value:<4096>, data
>>>>                           source: default value)
>>>>                           Maximum size (in bytes) of "short" messages
>>>> (must be>= 1).
>>>>                  MCA btl: parameter "btl_sm_max_send_size" (current
>>>> value:<32768>, data
>>>>                           source: default value)
>>>>                           Maximum size (in bytes) of a single "phase
>>>> 2" fragment of a
>>>>                           long message when using the pipeline
>>>> protocol (must be>= 1)
>>>>                  MCA btl: parameter "btl_sm_bandwidth" (current value:
>>>> <9000>, data
>>>>                           source: default value)
>>>>                           Approximate maximum bandwidth of
>>>> interconnect(0 = auto-detect
>>>>                           value at run-time [not supported in all BTL
>>>> modules],>= 1 =
>>>>                           bandwidth in Mbps)
>>>>                  MCA btl: parameter "btl_sm_latency" (current value:
>>>> <1>, data source:
>>>>                           default value)
>>>>                           Approximate latency of interconnect (must be>= 0)
>>>>                  MCA btl: parameter "btl_sm_priority" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                  MCA btl: parameter "btl_tcp_links" (current value:
>>>> <1>, data source:
>>>>                           default value)
>>>>                  MCA btl: parameter "btl_tcp_if_include" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           Comma-delimited list of devices or CIDR
>>>> notation of networks to
>>>>                           use for MPI communication (e.g., "eth0,eth1" or
>>>>                           "192.168.0.0/16,10.1.4.0/24").  Mutually
>>>> exclusive with
>>>>                           btl_tcp_if_exclude.
>>>>                  MCA btl: parameter "btl_tcp_if_exclude" (current
>>>> value:<lo,sppp>, data
>>>>                           source: default value)
>>>>                           Comma-delimited list of devices or CIDR
>>>> notation of networks to
>>>>                           NOT use for MPI communication -- all devices
>>>> not matching these
>>>>                           specifications will be used (e.g., "eth0,eth1" or
>>>>                           "192.168.0.0/16,10.1.4.0/24").  Mutually
>>>> exclusive with
>>>>                           btl_tcp_if_include.
>>>>                  MCA btl: parameter "btl_tcp_free_list_num" (current
>>>> value:<8>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_tcp_free_list_max" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_tcp_free_list_inc" (current
>>>> value:<32>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_tcp_sndbuf" (current value:
>>>> <131072>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_tcp_rcvbuf" (current value:
>>>> <131072>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_tcp_endpoint_cache" (current
>>>> value:<30720>,
>>>>                           data source: default value)
>>>>                           The size of the internal cache for each TCP
>>>> connection. This
>>>>                           cache is used to reduce the number of
>>>> syscalls, by replacing
>>>>                           them with memcpy. Every read will read the
>>>> expected data plus
>>>>                           the amount of the endpoint_cache
>>>>                  MCA btl: parameter "btl_tcp_use_nagle" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                           Whether to use Nagle's algorithm or not (using 
>>>> Nagle's
>>>>                           algorithm may increase short message latency)
>>>>                  MCA btl: parameter "btl_tcp_port_min_v4" (current
>>>> value:<1024>, data
>>>>                           source: default value)
>>>>                           The minimum port where the TCP BTL will try
>>>> to bind (default
>>>>                           1024)
>>>>                  MCA btl: parameter "btl_tcp_port_range_v4" (current
>>>> value:<64511>, data
>>>>                           source: default value)
>>>>                           The number of ports where the TCP BTL will
>>>> try to bind (default
>>>>                           64511). This parameter together with the
>>>> port min, define a
>>>>                           range of ports where Open MPI will open sockets.
>>>>                  MCA btl: parameter "btl_tcp_exclusivity" (current
>>>> value:<100>, data
>>>>                           source: default value)
>>>>                           BTL exclusivity (must be>= 0)
>>>>                  MCA btl: parameter "btl_tcp_flags" (current value:
>>>> <314>, data source:
>>>>                           default value)
>>>>                           BTL bit flags (general flags: SEND=1, PUT=2, 
>>>> GET=4,
>>>>                           SEND_INPLACE=8, RDMA_MATCHED=64,
>>>> HETEROGENEOUS_RDMA=256; flags
>>>>                           only used by the "dr" PML (ignored by others): 
>>>> ACK=16,
>>>>                           CHECKSUM=32, RDMA_COMPLETION=128; flags only
>>>> used by the "bfo"
>>>>                           PML (ignored by others): FAILOVER_SUPPORT=512)
>>>>                  MCA btl: parameter "btl_tcp_rndv_eager_limit"
>>>> (current value:<65536>,
>>>>                           data source: default value)
>>>>                           Size (in bytes) of "phase 1" fragment sent
>>>> for all large
>>>>                           messages (must be>= 0 and<= eager_limit)
>>>>                  MCA btl: parameter "btl_tcp_eager_limit" (current
>>>> value:<65536>, data
>>>>                           source: default value)
>>>>                           Maximum size (in bytes) of "short" messages
>>>> (must be>= 1).
>>>>                  MCA btl: parameter "btl_tcp_max_send_size" (current
>>>> value:<131072>,
>>>>                           data source: default value)
>>>>                           Maximum size (in bytes) of a single "phase
>>>> 2" fragment of a
>>>>                           long message when using the pipeline
>>>> protocol (must be>= 1)
>>>>                  MCA btl: parameter
>>>> "btl_tcp_rdma_pipeline_send_length" (current value:
>>>>                           <131072>, data source: default value)
>>>>                           Length of the "phase 2" portion of a large
>>>> message (in bytes)
>>>>                           when using the pipeline protocol.  This part
>>>> of the message
>>>>                           will be split into fragments of size
>>>> max_send_size and sent
>>>>                           using send/receive semantics (must be>= 0;
>>>> only relevant when
>>>>                           the PUT flag is set)
>>>>                  MCA btl: parameter "btl_tcp_rdma_pipeline_frag_size"
>>>> (current value:
>>>>                           <2147483647>, data source: default value)
>>>>                           Maximum size (in bytes) of a single "phase
>>>> 3" fragment from a
>>>>                           long message when using the pipeline
>>>> protocol.  These fragments
>>>>                           will be sent using RDMA semantics (must be
>>>>> = 1; only relevant
>>>>                           when the PUT flag is set)
>>>>                  MCA btl: parameter "btl_tcp_min_rdma_pipeline_size"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Messages smaller than this size (in bytes)
>>>> will not use the
>>>>                           RDMA pipeline protocol.  Instead, they will
>>>> be split into
>>>>                           fragments of max_send_size and sent using 
>>>> send/receive
>>>>                           semantics (must be>=0, and is automatically
>>>> adjusted up to at
>>>>                           least
>>>> (eager_limit+btl_rdma_pipeline_send_length); only
>>>>                           relevant when the PUT flag is set)
>>>>                  MCA btl: parameter "btl_tcp_bandwidth" (current
>>>> value:<100>, data
>>>>                           source: default value)
>>>>                           Approximate maximum bandwidth of
>>>> interconnect(0 = auto-detect
>>>>                           value at run-time [not supported in all BTL
>>>> modules],>= 1 =
>>>>                           bandwidth in Mbps)
>>>>                  MCA btl: parameter "btl_tcp_latency" (current value:
>>>> <100>, data source:
>>>>                           default value)
>>>>                           Approximate latency of interconnect (must be>= 0)
>>>>                  MCA btl: parameter "btl_tcp_disable_family" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_tcp_if_seq" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           If specified, a comma-delimited list of TCP
>>>> interfaces.
>>>>                           Interfaces will be assigned, one to each MPI
>>>> process, in a
>>>>                           round-robin fashion on each server.  For
>>>> example, if the list
>>>>                           is "eth0,eth1" and four MPI processes are
>>>> run on a single
>>>>                           server, then local ranks 0 and 2 will use
>>>> eth0 and local ranks
>>>>                           1 and 3 will use eth1.
>>>>                  MCA btl: parameter "btl_tcp_priority" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                  MCA btl: parameter "btl_base_include" (current value:
>>>> <none>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_base_exclude" (current value:
>>>> <none>, data
>>>>                           source: default value)
>>>>                  MCA btl: parameter "btl_base_warn_component_unused"
>>>> (current value:<1>,
>>>>                           data source: default value)
>>>>                           This parameter is used to turn on warning
>>>> messages when certain
>>>>                           NICs are not used
>>>>                  MCA mtl: parameter "mtl" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> mtl framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA mtl: parameter "mtl_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the mtl framework (0 =
>>>> no verbosity)
>>>>                 MCA topo: parameter "topo" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> topo framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                 MCA topo: parameter "topo_base_verbose" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the topo framework (0 =
>>>> no verbosity)
>>>>                 MCA topo: parameter "topo_unity_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA osc: parameter "osc" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> osc framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA osc: parameter "osc_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the osc framework (0 =
>>>> no verbosity)
>>>>                  MCA osc: parameter "osc_pt2pt_no_locks" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Enable optimizations available only if
>>>> MPI_LOCK is not used.
>>>>                  MCA osc: parameter "osc_pt2pt_eager_limit" (current
>>>> value:<16384>, data
>>>>                           source: default value)
>>>>                           Max size of eagerly sent data
>>>>                  MCA osc: parameter "osc_pt2pt_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA osc: parameter "osc_rdma_eager_send" (current
>>>> value:<1>, data
>>>>                           source: default value)
>>>>                           Attempt to start data movement during
>>>> communication call,
>>>>                           instead of at synchrnoization time.  Info
>>>> key of same name
>>>>                           overrides this value.
>>>>                  MCA osc: parameter "osc_rdma_use_buffers" (current
>>>> value:<1>, data
>>>>                           source: default value)
>>>>                           Coalesce messages during an epoch to reduce 
>>>> network
>>>>                           utilization.  Info key of same name
>>>> overrides this value.
>>>>                  MCA osc: parameter "osc_rdma_use_rdma" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                           Use real RDMA operations to transfer data.
>>>> Info key of same
>>>>                           name overrides this value.
>>>>                  MCA osc: parameter "osc_rdma_rdma_completion_wait"
>>>> (current value:<1>,
>>>>                           data source: default value)
>>>>                           Wait for all completion of rdma events before 
>>>> sending
>>>>                           acknowledgment.  Info key of same name
>>>> overrides this value.
>>>>                  MCA osc: parameter "osc_rdma_no_locks" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                           Enable optimizations available only if
>>>> MPI_LOCK is not used.
>>>>                           Info key of same name overrides this value.
>>>>                  MCA osc: parameter "osc_rdma_priority" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                   MCA op: parameter "op_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level of the op framework
>>>>                  MCA iof: parameter "iof" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> iof framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA iof: parameter "iof_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the iof framework (0 =
>>>> no verbosity)
>>>>                  MCA iof: parameter "iof_hnp_priority" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                  MCA iof: parameter "iof_orted_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA iof: parameter "iof_tool_priority" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                  MCA oob: parameter "oob" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> oob framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA oob: parameter "oob_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the oob framework (0 =
>>>> no verbosity)
>>>>                  MCA oob: parameter "oob_tcp_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbose level for the OOB tcp component
>>>>                  MCA oob: parameter "oob_tcp_peer_limit" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                           Maximum number of peer connections to
>>>> simultaneously maintain
>>>>                           (-1 = infinite)
>>>>                  MCA oob: parameter "oob_tcp_peer_retries" (current
>>>> value:<60>, data
>>>>                           source: default value)
>>>>                           Number of times to try shutting down a
>>>> connection before giving
>>>>                           up
>>>>                  MCA oob: parameter "oob_tcp_debug" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Enable (1) / disable (0) debugging output
>>>> for this component
>>>>                  MCA oob: parameter "oob_tcp_sndbuf" (current value:
>>>> <131072>, data
>>>>                           source: default value)
>>>>                           TCP socket send buffering size (in bytes)
>>>>                  MCA oob: parameter "oob_tcp_rcvbuf" (current value:
>>>> <131072>, data
>>>>                           source: default value)
>>>>                           TCP socket receive buffering size (in bytes)
>>>>                  MCA oob: parameter "oob_tcp_if_include" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           Comma-delimited list of TCP interfaces to use
>>>>                  MCA oob: parameter "oob_tcp_if_exclude" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           Comma-delimited list of TCP interfaces to exclude
>>>>                  MCA oob: parameter "oob_tcp_connect_sleep" (current
>>>> value:<1>, data
>>>>                           source: default value)
>>>>                           Enable (1) / disable (0) random sleep for
>>>> connection wireup.
>>>>                  MCA oob: parameter "oob_tcp_listen_mode" (current
>>>> value:<event>, data
>>>>                           source: default value)
>>>>                           Mode for HNP to accept incoming connections: 
>>>> event,
>>>>                           listen_thread.
>>>>                  MCA oob: parameter "oob_tcp_listen_thread_max_queue"
>>>> (current value:
>>>>                           <10>, data source: default value)
>>>>                           High water mark for queued accepted socket
>>>> list size.  Used
>>>>                           only when listen_mode is listen_thread.
>>>>                  MCA oob: parameter "oob_tcp_listen_thread_wait_time"
>>>> (current value:
>>>>                           <10>, data source: default value)
>>>>                           Time in milliseconds to wait before actively
>>>> checking for new
>>>>                           connections when listen_mode is listen_thread.
>>>>                  MCA oob: parameter "oob_tcp_static_ports" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           Static ports for daemons and procs (IPv4)
>>>>                  MCA oob: parameter "oob_tcp_dynamic_ports" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           Range of ports to be dynamically used by
>>>> daemons and procs
>>>>                           (IPv4)
>>>>                  MCA oob: parameter "oob_tcp_disable_family" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Disable IPv4 (4) or IPv6 (6)
>>>>                  MCA oob: parameter "oob_tcp_priority" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                 MCA odls: parameter "odls_base_sigkill_timeout"
>>>> (current value:<1>, data
>>>>                           source: default value)
>>>>                           Time to wait for a process to die after
>>>> issuing a kill signal
>>>>                           to it
>>>>                 MCA odls: parameter "odls" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> odls framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                 MCA odls: parameter "odls_base_verbose" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the odls framework (0 =
>>>> no verbosity)
>>>>                 MCA odls: parameter "odls_default_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA ras: parameter "ras_base_display_alloc" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Whether to display the allocation after it
>>>> is determined
>>>>                  MCA ras: parameter "ras_base_display_devel_alloc"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Whether to display a developer-detail
>>>> allocation after it is
>>>>                           determined
>>>>                  MCA ras: parameter "ras" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> ras framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA ras: parameter "ras_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the ras framework (0 =
>>>> no verbosity)
>>>>                  MCA ras: parameter "ras_cm_priority" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                  MCA ras: parameter "ras_loadleveler_priority"
>>>> (current value:<90>, data
>>>>                           source: default value)
>>>>                           Priority of the loadleveler ras component
>>>>                  MCA ras: parameter "ras_slurm_priority" (current
>>>> value:<75>, data
>>>>                           source: default value)
>>>>                           Priority of the slurm ras component
>>>>                MCA rmaps: parameter "rmaps_rank_file_path" (current
>>>> value:<none>, data
>>>>                           source: default value, synonym of: orte_rankfile)
>>>>                           Name of the rankfile to be used for mapping
>>>> processes (relative
>>>>                           or absolute path)
>>>>                MCA rmaps: parameter "rmaps_base_schedule_policy"
>>>> (current value:<slot>,
>>>>                           data source: default value)
>>>>                           Scheduling Policy for RMAPS. [slot
>>>> (alias:core) | socket |
>>>>                           board | node]
>>>>                MCA rmaps: parameter "rmaps_base_pernode" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Launch one ppn as directed
>>>>                MCA rmaps: parameter "rmaps_base_n_pernode" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                           Launch n procs/node
>>>>                MCA rmaps: parameter "rmaps_base_n_perboard" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                           Launch n procs/board
>>>>                MCA rmaps: parameter "rmaps_base_n_persocket" (current
>>>> value:<-1>, data
>>>>                           source: default value)
>>>>                           Launch n procs/socket
>>>>                MCA rmaps: parameter "rmaps_base_loadbalance" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Balance total number of procs across all
>>>> allocated nodes
>>>>                MCA rmaps: parameter "rmaps_base_cpus_per_proc"
>>>> (current value:<1>, data
>>>>                           source: default value, synonyms:
>>>> rmaps_base_cpus_per_rank)
>>>>                           Number of cpus to use for each rank [1-2**15
>>>> (default=1)]
>>>>                MCA rmaps: parameter "rmaps_base_cpus_per_rank"
>>>> (current value:<1>, data
>>>>                           source: default value, synonym of:
>>>> rmaps_base_cpus_per_proc)
>>>>                           Number of cpus to use for each rank [1-2**15
>>>> (default=1)]
>>>>                MCA rmaps: parameter "rmaps_base_stride" (current
>>>> value:<1>, data source:
>>>>                           default value)
>>>>                           When binding multiple cores to a rank, the
>>>> step size to use
>>>>                           between cores [1-2**15 (default: 1)]
>>>>                MCA rmaps: parameter "rmaps_base_slot_list" (current
>>>> value:<none>, data
>>>>                           source: default value)
>>>>                           List of processor IDs to bind MPI processes
>>>> to (e.g., used in
>>>>                           conjunction with rank files) [default=NULL]
>>>>                MCA rmaps: parameter "rmaps_base_no_schedule_local"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           If false, allow scheduling MPI applications
>>>> on the same node as
>>>>                           mpirun (default).  If true, do not schedule any 
>>>> MPI
>>>>                           applications on the same node as mpirun
>>>>                MCA rmaps: parameter "rmaps_base_no_oversubscribe"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           If true, then do not allow oversubscription
>>>> of nodes - mpirun
>>>>                           will return an error if there aren't enough
>>>> nodes to launch all
>>>>                           processes without oversubscribing
>>>>                MCA rmaps: parameter "rmaps_base_display_map" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Whether to display the process map after it
>>>> is computed
>>>>                MCA rmaps: parameter "rmaps_base_display_devel_map"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                           Whether to display a developer-detail
>>>> process map after it is
>>>>                           computed
>>>>                MCA rmaps: parameter "rmaps" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> rmaps framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                MCA rmaps: parameter "rmaps_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the rmaps framework (0 =
>>>> no verbosity)
>>>>                MCA rmaps: parameter "rmaps_load_balance_priority"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                MCA rmaps: parameter "rmaps_rank_file_priority"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>                MCA rmaps: parameter "rmaps_resilient_fault_grp_file"
>>>> (current value:
>>>>                           <none>, data source: default value)
>>>>                           Filename that contains a description of
>>>> fault groups for this
>>>>                           system
>>>>                MCA rmaps: parameter "rmaps_resilient_priority"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>                MCA rmaps: parameter "rmaps_round_robin_priority"
>>>> (current value:<0>,
>>>>                           data source: default value)
>>>>                MCA rmaps: parameter "rmaps_seq_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                MCA rmaps: parameter "rmaps_topo_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA rml: parameter "rml_wrapper" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           Use a Wrapper component around the selected
>>>> RML component
>>>>                  MCA rml: parameter "rml" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> rml framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA rml: parameter "rml_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the rml framework (0 =
>>>> no verbosity)
>>>>                  MCA rml: parameter "rml_oob_priority" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>               MCA routed: parameter "routed" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> routed framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>               MCA routed: parameter "routed_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the routed framework (0
>>>> = no verbosity)
>>>>               MCA routed: parameter "routed_binomial_priority"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>               MCA routed: parameter "routed_cm_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>               MCA routed: parameter "routed_direct_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>               MCA routed: parameter "routed_linear_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>               MCA routed: parameter "routed_radix_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>               MCA routed: parameter "routed_slave_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA plm: parameter "plm_rsh_agent" (current value:
>>>> <ssh : rsh>, data
>>>>                           source: default value, deprecated, synonym
>>>> of: orte_rsh_agent)
>>>>                           The command used to launch executables on remote 
>>>> nodes
>>>>                           (typically either "ssh" or "rsh")
>>>>                  MCA plm: parameter "plm_rsh_assume_same_shell"
>>>> (current value:<1>, data
>>>>                           source: default value, deprecated, synonym of:
>>>>                           orte_assume_same_shell)
>>>>                           If set to 1, assume that the shell on the
>>>> remote node is the
>>>>                           same as the shell on the local node.
>>>> Otherwise, probe for what
>>>>                           the remote shell [default: 1]
>>>>                  MCA plm: parameter "plm" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> plm framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA plm: parameter "plm_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the plm framework (0 =
>>>> no verbosity)
>>>>                  MCA plm: parameter "plm_rsh_num_concurrent" (current
>>>> value:<128>, data
>>>>                           source: default value)
>>>>                           How many plm_rsh_agent instances to invoke
>>>> concurrently (must
>>>>                           be>  0)
>>>>                  MCA plm: parameter "plm_rsh_force_rsh" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                           Force the launcher to always use rsh
>>>>                  MCA plm: parameter "plm_rsh_disable_qrsh" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Disable the launcher to use qrsh when under
>>>> the SGE parallel
>>>>                           environment
>>>>                  MCA plm: parameter "plm_rsh_daemonize_qrsh" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Daemonize the orted under the SGE parallel 
>>>> environment
>>>>                  MCA plm: parameter "plm_rsh_disable_llspawn" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Disable the use of llspawn when under the 
>>>> LoadLeveler
>>>>                           environment
>>>>                  MCA plm: parameter "plm_rsh_daemonize_llspawn"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>>                           Daemonize the orted when under the
>>>> LoadLeveler environment
>>>>                  MCA plm: parameter "plm_rsh_priority" (current value:
>>>> <10>, data source:
>>>>                           default value)
>>>>                           Priority of the rsh plm component
>>>>                  MCA plm: parameter "plm_rsh_delay" (current value:
>>>> <1>, data source:
>>>>                           default value)
>>>>                           Delay (in seconds) between invocations of
>>>> the remote agent, but
>>>>                           only used when the "debug" MCA parameter is
>>>> true, or the
>>>>                           top-level MCA debugging is enabled
>>>> (otherwise this value is
>>>>                           ignored)
>>>>                  MCA plm: parameter "plm_rsh_tree_spawn" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           If set to 1, launch via a tree-based topology
>>>>                  MCA plm: parameter "plm_slurm_args" (current value:
>>>> <none>, data source:
>>>>                           default value)
>>>>                           Custom arguments to srun
>>>>                  MCA plm: parameter "plm_slurm_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                MCA filem: parameter "filem" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Which Filem component to use (empty = 
>>>> auto-select)
>>>>                MCA filem: parameter "filem_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the filem framework (0 =
>>>> no verbosity)
>>>>                MCA filem: parameter "filem_rsh_priority" (current
>>>> value:<20>, data
>>>>                           source: default value)
>>>>                           Priority of the FILEM rsh component
>>>>                MCA filem: parameter "filem_rsh_verbose" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>                           Verbose level for the FILEM rsh component
>>>>                MCA filem: parameter "filem_rsh_rcp" (current value:
>>>> <scp>, data source:
>>>>                           default value)
>>>>                           The rsh cp command for the FILEM rsh component
>>>>                MCA filem: parameter "filem_rsh_cp" (current value:
>>>> <cp>, data source:
>>>>                           default value)
>>>>                           The Unix cp command for the FILEM rsh component
>>>>                MCA filem: parameter "filem_rsh_rsh" (current value:
>>>> <ssh>, data source:
>>>>                           default value)
>>>>                           The remote shell command for the FILEM rsh 
>>>> component
>>>>                MCA filem: parameter "filem_rsh_max_incomming" (current
>>>> value:<10>, data
>>>>                           source: default value)
>>>>                           Maximum number of incomming connections (0 = any)
>>>>                MCA filem: parameter "filem_rsh_max_outgoing" (current
>>>> value:<10>, data
>>>>                           source: default value)
>>>>                           Maximum number of out going connections (0 = any)
>>>>               MCA errmgr: parameter "errmgr" (current value:<none>,
>>>> data source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> errmgr framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>               MCA errmgr: parameter "errmgr_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the errmgr framework (0
>>>> = no verbosity)
>>>>               MCA errmgr: parameter "errmgr_default_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA ess: parameter "ess" (current value:<none>, data
>>>> source: default
>>>>                           value)
>>>>                           Default selection set of components for the
>>>> ess framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>                  MCA ess: parameter "ess_base_verbose" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                           Verbosity level for the ess framework (0 =
>>>> no verbosity)
>>>>                  MCA ess: parameter "ess_env_priority" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                  MCA ess: parameter "ess_hnp_priority" (current value:
>>>> <0>, data source:
>>>>                           default value)
>>>>                  MCA ess: parameter "ess_singleton_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA ess: parameter "ess_slave_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA ess: parameter "ess_slurm_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA ess: parameter "ess_slurmd_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                  MCA ess: parameter "ess_tool_priority" (current
>>>> value:<0>, data source:
>>>>                           default value)
>>>>              MCA grpcomm: parameter "grpcomm" (current value:<none>,
>>>> data source:
>>>>                           default value)
>>>>                           Default selection set of components for the
>>>> grpcomm framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>              MCA grpcomm: parameter "grpcomm_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the grpcomm framework (0
>>>> = no verbosity)
>>>>              MCA grpcomm: parameter "grpcomm_bad_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>              MCA grpcomm: parameter "grpcomm_basic_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>              MCA grpcomm: parameter "grpcomm_hier_priority" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>             MCA notifier: parameter "notifier_threshold_severity"
>>>> (current value:
>>>>                           <critical>, data source: default value)
>>>>                           Report all events at or above this severity
>>>> [default: critical]
>>>>             MCA notifier: parameter "notifier" (current value:<none>,
>>>> data source:
>>>>                           default value)
>>>>                           Default selection set of components for the
>>>> notifier framework
>>>>                           (<none>  means use all components that can be 
>>>> found)
>>>>             MCA notifier: parameter "notifier_base_verbose" (current
>>>> value:<0>, data
>>>>                           source: default value)
>>>>                           Verbosity level for the notifier framework
>>>> (0 = no verbosity)
>>>>             MCA notifier: parameter "notifier_command_cmd" (current
>>>> value:</sbin/initlog
>>>>                           -f $s -n "Open MPI" -s "$S: $m (errorcode:
>>>> $e)">, data source:
>>>>                           default value)
>>>>                           Command to execute, with substitution.  $s =
>>>> integer severity;
>>>>                           $S = string severity; $e = integer error
>>>> code; $m = string
>>>>                           message
>>>>             MCA notifier: parameter "notifier_command_timeout"
>>>> (current value:<30>, data
>>>>                           source: default value)
>>>>                           Timeout (in seconds) of the command
>>>>             MCA notifier: parameter "notifier_command_priority"
>>>> (current value:<10>,
>>>>                           data source: default value)
>>>>                           Priority of this component
>>>>             MCA notifier: parameter "notifier_syslog_priority"
>>>> (current value:<0>, data
>>>>                           source: default value)
>>>> 
>>>> ====================================================================================================
>>>> 
>>>> output of cat /proc/cpuinfo
>>>> processor     : 0
>>>> vendor_id     : AuthenticAMD
>>>> cpu family    : 15
>>>> model         : 75
>>>> model name    : AMD Athlon(tm) 64 X2 Dual Core Processor 3800+
>>>> stepping      : 2
>>>> cpu MHz               : 1002.094
>>>> cache size    : 512 KB
>>>> physical id   : 0
>>>> siblings      : 2
>>>> core id               : 0
>>>> cpu cores     : 2
>>>> fpu           : yes
>>>> fpu_exception : yes
>>>> cpuid level   : 1
>>>> wp            : yes
>>>> flags         : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
>>>> cmov
>>>> pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext
>>>> 3dnow pni cx16
>>>> bogomips      : 2003.90
>>>> TLB size      : 1088 4K pages
>>>> clflush size  : 64
>>>> cache_alignment       : 64
>>>> address sizes : 40 bits physical, 48 bits virtual
>>>> power management: ts fid vid ttp [4] [5]
>>>> 
>>>> processor     : 1
>>>> vendor_id     : AuthenticAMD
>>>> cpu family    : 15
>>>> model         : 75
>>>> model name    : AMD Athlon(tm) 64 X2 Dual Core Processor 3800+
>>>> stepping      : 2
>>>> cpu MHz               : 1002.094
>>>> cache size    : 512 KB
>>>> physical id   : 0
>>>> siblings      : 2
>>>> core id               : 1
>>>> cpu cores     : 2
>>>> fpu           : yes
>>>> fpu_exception : yes
>>>> cpuid level   : 1
>>>> wp            : yes
>>>> flags         : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
>>>> cmov
>>>> pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext
>>>> 3dnow pni cx16
>>>> bogomips      : 2003.90
>>>> TLB size      : 1088 4K pages
>>>> clflush size  : 64
>>>> cache_alignment       : 64
>>>> address sizes : 40 bits physical, 48 bits virtual
>>>> power management: ts fid vid ttp [4] [5]
>>>> 
>>>> 
>>>> ====================================================================================================
>>>> output of ifconfig -a from a compute node
>>>> eth0      Link encap:Ethernet  HWaddr 00:18:F3:3F:84:A1
>>>>           inet addr:192.168.0.2  Bcast:192.168.0.255  Mask:255.255.255.0
>>>>           inet6 addr: fe80::218:f3ff:fe3f:84a1/64 Scope:Link
>>>>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>           RX packets:2006 errors:0 dropped:0 overruns:0 frame:0
>>>>           TX packets:2064 errors:0 dropped:0 overruns:0 carrier:0
>>>>           collisions:0 txqueuelen:1000
>>>>           RX bytes:242685 (236.9 KiB)  TX bytes:0 (0.0 b)
>>>>           Interrupt:11 Base address:0x8000
>>>> 
>>>> lo        Link encap:Local Loopback
>>>>           inet addr:127.0.0.1  Mask:255.0.0.0
>>>>           inet6 addr: ::1/128 Scope:Host
>>>>           UP LOOPBACK RUNNING  MTU:16436  Metric:1
>>>>           RX packets:60 errors:0 dropped:0 overruns:0 frame:0
>>>>           TX packets:60 errors:0 dropped:0 overruns:0 carrier:0
>>>>           collisions:0 txqueuelen:0
>>>>           RX bytes:4440 (4.3 KiB)  TX bytes:4440 (4.3 KiB)
>>>> 
>>>> sit0      Link encap:IPv6-in-IPv4
>>>>           NOARP  MTU:1480  Metric:1
>>>>           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>           collisions:0 txqueuelen:0
>>>>           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
>>>> 
>>>> 
>>>> ====================================================================================================
>>>> output of ifconfig -a from the login node, where I run mpirun
>>>> 
>>>> eth0      Link encap:Ethernet  HWaddr 00:18:F3:51:B3:6E
>>>>           inet addr:192.168.0.1  Bcast:192.168.0.255  Mask:255.255.255.0
>>>>           inet6 addr: fe80::218:f3ff:fe51:b36e/64 Scope:Link
>>>>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>           RX packets:7180758 errors:0 dropped:0 overruns:0 frame:0
>>>>           TX packets:4989496 errors:0 dropped:0 overruns:0 carrier:0
>>>>           collisions:0 txqueuelen:1000
>>>>           RX bytes:6045614452 (5.6 GiB)  TX bytes:0 (0.0 b)
>>>>           Interrupt:201 Base address:0xe000
>>>> 
>>>> eth1      Link encap:Ethernet  HWaddr 00:01:02:13:AA:3C
>>>>           inet addr:137.204.66.188  Bcast:137.204.66.255  
>>>> Mask:255.255.255.0
>>>>           inet6 addr: fe80::201:2ff:fe13:aa3c/64 Scope:Link
>>>>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>           RX packets:4750212 errors:0 dropped:0 overruns:0 frame:0
>>>>           TX packets:405027 errors:0 dropped:0 overruns:0 carrier:0
>>>>           collisions:0 txqueuelen:1000
>>>>           RX bytes:629146679 (600.0 MiB)  TX bytes:332118265 (316.7 MiB)
>>>>           Interrupt:177 Base address:0x9c00
>>>> 
>>>> lo        Link encap:Local Loopback
>>>>           inet addr:127.0.0.1  Mask:255.0.0.0
>>>>           inet6 addr: ::1/128 Scope:Host
>>>>           UP LOOPBACK RUNNING  MTU:16436  Metric:1
>>>>           RX packets:288455 errors:0 dropped:0 overruns:0 frame:0
>>>>           TX packets:288455 errors:0 dropped:0 overruns:0 carrier:0
>>>>           collisions:0 txqueuelen:0
>>>>           RX bytes:35908038 (34.2 MiB)  TX bytes:35908038 (34.2 MiB)
>>>> 
>>>> sit0      Link encap:IPv6-in-IPv4
>>>>           NOARP  MTU:1480  Metric:1
>>>>           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>           collisions:0 txqueuelen:0
>>>>           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
>>>> 
>>>> ====================================================================================================
>>>> output of mpirun --bynode --hostfile my_hostfile.txt --tag-output
>>>> ompi_info -v ompi full --parsable
>>>> 
>>>> [1,0]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,0]<stdout>:ompi:version:full:1.6
>>>> [1,0]<stdout>:ompi:version:svn:r26429
>>>> [1,0]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,0]<stdout>:orte:version:full:1.6
>>>> [1,0]<stdout>:orte:version:svn:r26429
>>>> [1,0]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,0]<stdout>:opal:version:full:1.6
>>>> [1,0]<stdout>:opal:version:svn:r26429
>>>> [1,0]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,0]<stdout>:mpi-api:version:full:2.1
>>>> [1,0]<stdout>:ident:1.6
>>>> [1,6]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,6]<stdout>:ompi:version:full:1.6
>>>> [1,6]<stdout>:ompi:version:svn:r26429
>>>> [1,6]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,6]<stdout>:orte:version:full:1.6
>>>> [1,6]<stdout>:orte:version:svn:r26429
>>>> [1,6]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,6]<stdout>:opal:version:full:1.6
>>>> [1,6]<stdout>:opal:version:svn:r26429
>>>> [1,6]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,6]<stdout>:mpi-api:version:full:2.1
>>>> [1,6]<stdout>:ident:1.6
>>>> [1,9]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,10]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,3]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,3]<stdout>:ompi:version:full:1.6
>>>> [1,3]<stdout>:ompi:version:svn:r26429
>>>> [1,3]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,3]<stdout>:orte:version:full:1.6
>>>> [1,3]<stdout>:orte:version:svn:r26429
>>>> [1,3]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,3]<stdout>:opal:version:full:1.6
>>>> [1,3]<stdout>:opal:version:svn:r26429
>>>> [1,3]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,3]<stdout>:mpi-api:version:full:2.1
>>>> [1,3]<stdout>:ident:1.6
>>>> [1,4]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,4]<stdout>:ompi:version:full:1.6
>>>> [1,4]<stdout>:ompi:version:svn:r26429
>>>> [1,4]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,4]<stdout>:orte:version:full:1.6
>>>> [1,4]<stdout>:orte:version:svn:r26429
>>>> [1,4]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,4]<stdout>:opal:version:full:1.6
>>>> [1,9]<stdout>:ompi:version:full:1.6
>>>> [1,4]<stdout>:opal:version:svn:r26429
>>>> [1,4]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,4]<stdout>:mpi-api:version:full:2.1
>>>> [1,4]<stdout>:ident:1.6
>>>> [1,9]<stdout>:ompi:version:svn:r26429
>>>> [1,10]<stdout>:ompi:version:full:1.6
>>>> [1,9]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,10]<stdout>:ompi:version:svn:r26429
>>>> [1,9]<stdout>:orte:version:full:1.6
>>>> [1,10]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,9]<stdout>:orte:version:svn:r26429
>>>> [1,10]<stdout>:orte:version:full:1.6
>>>> [1,10]<stdout>:orte:version:svn:r26429
>>>> [1,9]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,10]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,9]<stdout>:opal:version:full:1.6
>>>> [1,10]<stdout>:opal:version:full:1.6
>>>> [1,9]<stdout>:opal:version:svn:r26429
>>>> [1,10]<stdout>:opal:version:svn:r26429
>>>> [1,9]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,10]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,9]<stdout>:mpi-api:version:full:2.1
>>>> [1,9]<stdout>:ident:1.6
>>>> [1,10]<stdout>:mpi-api:version:full:2.1
>>>> [1,10]<stdout>:ident:1.6
>>>> [1,2]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,2]<stdout>:ompi:version:full:1.6
>>>> [1,2]<stdout>:ompi:version:svn:r26429
>>>> [1,2]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,2]<stdout>:orte:version:full:1.6
>>>> [1,2]<stdout>:orte:version:svn:r26429
>>>> [1,2]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,2]<stdout>:opal:version:full:1.6
>>>> [1,2]<stdout>:opal:version:svn:r26429
>>>> [1,2]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,2]<stdout>:mpi-api:version:full:2.1
>>>> [1,2]<stdout>:ident:1.6
>>>> [1,8]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,8]<stdout>:ompi:version:full:1.6
>>>> [1,8]<stdout>:ompi:version:svn:r26429
>>>> [1,8]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,8]<stdout>:orte:version:full:1.6
>>>> [1,8]<stdout>:orte:version:svn:r26429
>>>> [1,8]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,8]<stdout>:opal:version:full:1.6
>>>> [1,8]<stdout>:opal:version:svn:r26429
>>>> [1,8]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,8]<stdout>:mpi-api:version:full:2.1
>>>> [1,8]<stdout>:ident:1.6
>>>> [1,11]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,11]<stdout>:ompi:version:full:1.6
>>>> [1,11]<stdout>:ompi:version:svn:r26429
>>>> [1,11]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,11]<stdout>:orte:version:full:1.6
>>>> [1,11]<stdout>:orte:version:svn:r26429
>>>> [1,11]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,11]<stdout>:opal:version:full:1.6
>>>> [1,11]<stdout>:opal:version:svn:r26429
>>>> [1,11]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,11]<stdout>:mpi-api:version:full:2.1
>>>> [1,11]<stdout>:ident:1.6
>>>> [1,5]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,5]<stdout>:ompi:version:full:1.6
>>>> [1,5]<stdout>:ompi:version:svn:r26429
>>>> [1,5]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,5]<stdout>:orte:version:full:1.6
>>>> [1,5]<stdout>:orte:version:svn:r26429
>>>> [1,5]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,5]<stdout>:opal:version:full:1.6
>>>> [1,5]<stdout>:opal:version:svn:r26429
>>>> [1,5]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,5]<stdout>:mpi-api:version:full:2.1
>>>> [1,5]<stdout>:ident:1.6
>>>> [1,1]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,7]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>> [1,7]<stdout>:ompi:version:full:1.6
>>>> [1,7]<stdout>:ompi:version:svn:r26429
>>>> [1,7]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,7]<stdout>:orte:version:full:1.6
>>>> [1,7]<stdout>:orte:version:svn:r26429
>>>> [1,7]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,7]<stdout>:opal:version:full:1.6
>>>> [1,7]<stdout>:opal:version:svn:r26429
>>>> [1,7]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,7]<stdout>:mpi-api:version:full:2.1
>>>> [1,7]<stdout>:ident:1.6
>>>> [1,1]<stdout>:ompi:version:full:1.6
>>>> [1,1]<stdout>:ompi:version:svn:r26429
>>>> [1,1]<stdout>:ompi:version:release_date:May 10, 2012
>>>> [1,1]<stdout>:orte:version:full:1.6
>>>> [1,1]<stdout>:orte:version:svn:r26429
>>>> [1,1]<stdout>:orte:version:release_date:May 10, 2012
>>>> [1,1]<stdout>:opal:version:full:1.6
>>>> [1,1]<stdout>:opal:version:svn:r26429
>>>> [1,1]<stdout>:opal:version:release_date:May 10, 2012
>>>> [1,1]<stdout>:mpi-api:version:full:2.1
>>>> [1,1]<stdout>:ident:1.6
>>>> _______________________________________________
>>>> users mailing list
>>>> us...@open-mpi.org
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> 
>>> 
>>> 
>>> ------------------------------
>>> 
>>> _______________________________________________
>>> users mailing list
>>> us...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> 
>>> End of users Digest, Vol 2339, Issue 5
>>> **************************************
>> 
>> 
>> ------------------------------
>> 
>> _______________________________________________
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>> 
>> End of users Digest, Vol 2340, Issue 1
>> **************************************
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users


Reply via email to