I have tried with these flags (I use gcc 4.7 and open mpi 1.6), but
the program doesn't crash, a node go down and the rest of them remain
to wait a signal (there is an ALLREDUCE in the code).

Anyway, yesterday some processes died (without a log) on the node 10,
I logged almost immediately in the node and I found the process

/usr/sbin/hal_lpadmin -x /org/freedesktop/Hal/devices/pci_10de_267

What is it? I know that hal is a device demon, but hal_lpadmin?
In any case, this is the output of lsb_release -a
LSB Version:    
:core-3.0-amd64:core-3.0-noarch:graphics-3.0-amd64:graphics-3.0-noarch
andrea@cloud ~/run_zeus/CR15_100pc/exe90$ lsb_release -a
LSB Version:    
:core-3.0-amd64:core-3.0-noarch:graphics-3.0-amd64:graphics-3.0-noarch
Distributor ID: ScientificSL
Description:    Scientific Linux SL release 4.4 (Beryllium)
Release:        4.4
Codename:       Beryllium


PS: What is the correct method to reply in this mailing list? I use
gmail and I usually I hit the reply butt, replace the object, but here
it seems the I opening a new thread each time I post.


2012/9/4  <users-requ...@open-mpi.org>:
> Send users mailing list submissions to
>         us...@open-mpi.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://www.open-mpi.org/mailman/listinfo.cgi/users
> or, via email, send a message with subject or body 'help' to
>         users-requ...@open-mpi.org
>
> You can reach the person managing the list at
>         users-ow...@open-mpi.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of users digest..."
>
>
> Today's Topics:
>
>    1. Re: OMPI 1.6.x Hang on khugepaged 100% CPU time (Yong Qin)
>    2. Re: some mpi processes "disappear" on a cluster of        servers
>       (David Warren)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 4 Sep 2012 09:21:27 -0700
> From: Yong Qin <yong....@gmail.com>
> Subject: Re: [OMPI users] OMPI 1.6.x Hang on khugepaged 100% CPU time
> To: klit...@dev.mellanox.co.il
> Cc: Open MPI Users <us...@open-mpi.org>
> Message-ID:
>         <CADEJBEVwU=n9OmU5i4YY=ufcnigyc_jn_mvwm-8e-lkyc4e...@mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> On Tue, Sep 4, 2012 at 5:42 AM, Yevgeny Kliteynik
> <klit...@dev.mellanox.co.il> wrote:
>> On 8/30/2012 10:28 PM, Yong Qin wrote:
>>> On Thu, Aug 30, 2012 at 5:12 AM, Jeff Squyres<jsquy...@cisco.com>  wrote:
>>>> On Aug 29, 2012, at 2:25 PM, Yong Qin wrote:
>>>>
>>>>> This issue has been observed on OMPI 1.6 and 1.6.1 with openib btl but
>>>>> not on 1.4.5 (tcp btl is always fine). The application is VASP and
>>>>> only one specific dataset is identified during the testing, and the OS
>>>>> is SL 6.2 with kernel 2.6.32-220.23.1.el6.x86_64. The issue is that
>>>>> when a certain type of load is put on OMPI 1.6.x, khugepaged thread
>>>>> always runs with 100% CPU load, and it looks to me like that OMPI is
>>>>> waiting for some memory to be available thus appears to be hung.
>>>>> Reducing the per node processes would sometimes ease the problem a bit
>>>>> but not always. So I did some further testing by playing around with
>>>>> the kernel transparent hugepage support.
>>>>>
>>>>> 1. Disable transparent hugepage support completely (echo never
>>>>>> /sys/kernel/mm/redhat_transparent_hugepage/enabled). This would allow
>>>>> the program to progress as normal (as in 1.4.5). Total run time for an
>>>>> iteration is 3036.03 s.
>>>>
>>>> I'll admit that we have not tested using transparent hugepages.  I wonder 
>>>> if there's some kind of bad interaction going on here...
>>>
>>> The transparent hugepage is "transparent", which means it is
>>> automatically applied to all applications unless it is explicitly told
>>> otherwise. I highly suspect that it is not working properly in this
>>> case.
>>
>> Like Jeff said - I don't think we've ever tested OMPI with transparent
>> huge pages.
>>
>
> Thanks. But have you tested OMPI under RHEL 6 or its variants (CentOS
> 6, SL 6)? THP is on by default in RHEL 6 so no matter you want it or
> not it's there.
>
>>>>
>>>> What exactly does changing this setting do?
>>>
>>> Here (http://lwn.net/Articles/423592/)  is a pretty good documentation
>>> on what these settings would do to the behaviour of the THP. I don't
>>> think I can explain it better than the article so I will leave it to
>>> you to digest. :)
>>>
>>>>
>>>>> 2. Disable VM defrag effort (echo never
>>>>>> /sys/kernel/mm/redhat_transparent_hugepage/defrag). This allows the
>>>>> program to run as well, but the performance is horrible. The same
>>>>> iteration takes 4967.40 s.
>>>>>
>>>>> 3. Disable defrag in khugepaged (echo no
>>>>>> /sys/kernel/mm/redhat_transparent_hugepage/khugepaged/defrag). This
>>>>> allows the program to run, and the performance is worse than #1 but
>>>>> better than #2. The same iteration takes 3348.10 s.
>>>>>
>>>>> 4. Disable both VM defrag and khugepaged defrag (#2 + #3). Similar
>>>>> performance as #3.
>>>>>
>>>>> So my question is, looks to me like this has to do with the memory
>>>>> management in the openib btl, are we using huge pages in 1.6.x? If
>>>>> that is true, is there a better way to resolve or workaround it within
>>>>> OMPI itself without disabling transparent hugepage support? We'd like
>>>>> to keep the hugepage support if possible.
>>>>
>>>> Mellanox -- can you comment on this?
>>
>> Actually, I don't think that THP were really tested with OFED.
>> I can think of lots of ways thing can go wrong there.
>> This might be a good question to address to Linux-RDMA mailing list.
>>
>
> This is quite useful information. I guess we will just turn off THP
> support for now.
>
>> -- YK
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 04 Sep 2012 10:31:05 -0700
> From: David Warren <war...@atmos.washington.edu>
> Subject: Re: [OMPI users] some mpi processes "disappear" on a cluster
>         of      servers
> To: us...@open-mpi.org
> Message-ID: <50463ad9.3030...@atmos.washington.edu>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Which FORTRAN compiler are you using? I believe that most of them allow
> you to compile with -g and optimization and then force a stack dump on
> crash. I have found this to work on code that seems to vanish on random
> processors. Also, you might look at the FORTRAN options and see if it
> lets you allocate memory for temporary arrays from the heap instead of
> the stack. Sometime this will help, even if you have stacksize set to
> unlimited. One last thing to check is are you running with 32bit  or 64
> bit memory models.
>
> If you are using ifort, the options you want are
>   -O[2,3] -g -traceback -heap-arrays -mcmodel=medium
> You only need -mcmodel=medium if you might exceed 2 gig in a process.
>
> For gfortran try
> -O -g -fbacktrace -mcmodel=medium
>
> On 09/03/12 13:39, Andrea Negri wrote:
>> I have asked to my admin and he said that no log messages were present
>> in /var/log, apart my login on the compute node.
>> No killed processes, no full stack errors, the memory is ok, 1GB is
>> used and 2GB are free.
>> Actually I don't know what kind of problem coud be, does someone have
>> ideas? Or at least a suspect?
>>
>> Please, don't let me alone!
>>
>> Sorry for the trouble with the mail
>>
>> 2012/9/1<users-requ...@open-mpi.org>:
>>> Send users mailing list submissions to
>>>          us...@open-mpi.org
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>          http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> or, via email, send a message with subject or body 'help' to
>>>          users-requ...@open-mpi.org
>>>
>>> You can reach the person managing the list at
>>>          users-ow...@open-mpi.org
>>>
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of users digest..."
>>>
>>>
>>> Today's Topics:
>>>
>>>     1. Re: some mpi processes "disappear" on a cluster of        servers
>>>        (John Hearns)
>>>     2. Re: users Digest, Vol 2339, Issue 5 (Andrea Negri)
>>>
>>>
>>> ----------------------------------------------------------------------
>>>
>>> Message: 1
>>> Date: Sat, 1 Sep 2012 08:48:56 +0100
>>> From: John Hearns<hear...@googlemail.com>
>>> Subject: Re: [OMPI users] some mpi processes "disappear" on a cluster
>>>          of      servers
>>> To: Open MPI Users<us...@open-mpi.org>
>>> Message-ID:
>>>          
>>> <capqne2wo3bgefjiyfums6yquoduhjj75zjoeeyjrdq60fmz...@mail.gmail.com>
>>> Content-Type: text/plain; charset=ISO-8859-1
>>>
>>> Apologies, I have not taken the time to read your comprehensive diagnostics!
>>>
>>> As Gus says, this sounds like a memory problem.
>>> My suspicion would be the kernel Out Of Memory (OOM) killer.
>>> Log into those nodes (or ask your systems manager to do this). Look
>>> closely at /var/log/messages where there will be notifications when
>>> the OOM Killer kicks in and - well - kills large memory processes!
>>> Grep for "killed process" in /var/log/messages
>>>
>>> http://linux-mm.org/OOM_Killer
>>>
>>>
>>> ------------------------------
>>>
>>> Message: 2
>>> Date: Sat, 1 Sep 2012 11:50:59 +0200
>>> From: Andrea Negri<negri.an...@gmail.com>
>>> Subject: Re: [OMPI users] users Digest, Vol 2339, Issue 5
>>> To: us...@open-mpi.org
>>> Message-ID:
>>>          
>>> <CAPUxaiQ4RFqSK1kz7fM7U9XRxjQh8N+=98paqm2ikvr7bv-...@mail.gmail.com>
>>> Content-Type: text/plain; charset=ISO-8859-1
>>>
>>> Hi, Gus and John,
>>>
>>> my code (zeusmp2) is a F77 code ported in F95, and the very first time
>>> I have launched it, the processed disappears almost immediately.
>>> I checked the code with valgrind and some unallocated arrays were
>>> passed wrongly to some subroutines.
>>> After having corrected this bug, the code runs for a while and after
>>> occour all the stuff described in my first post.
>>> However, the code still performs a lot of main temporal cycle before
>>> "die" (I don't know if thies piece of information is useful).
>>>
>>> Now I'm going to check the memory usage, (I also have a lot of unused
>>> variables in this pretty large code, maybe I shoud comment them).
>>>
>>> uname -a returns
>>> Linux cloud 2.6.9-42.0.3.ELsmp #1 SMP Thu Oct 5 16:29:37 CDT 2006
>>> x86_64 x86_64 x86_64 GNU/Linux
>>>
>>> ulimit -a returns:
>>> core file size                        (blocks, -c) 0
>>> data seg size                       (kbytes, -d) unlimited
>>> file size                               (blocks, -f) unlimited
>>> pending signals                    (-i) 1024
>>> max locked memory             (kbytes, -l) 32
>>> max memory size                (kbytes, -m) unlimited
>>> open files                           (-n) 1024
>>> pipe size                            (512 bytes, -p) 8
>>> POSIX message queues     (bytes, -q) 819200
>>> stack size                           (kbytes, -s) 10240
>>> cpu time                             (seconds, -t) unlimited
>>> max user processes             (-u) 36864
>>> virtual memory                   (kbytes, -v) unlimited
>>> file locks                            (-x) unlimited
>>>
>>>
>>> I can log on the logins nodes, but unfortunately the command ls
>>> /var/log/messages return
>>> acpid               cron.4              messages.3         secure.4
>>> anaconda.log        cups                messages.4         spooler
>>> anaconda.syslog     dmesg               mpi_uninstall.log  spooler.1
>>> anaconda.xlog       gdm                 ppp                spooler.2
>>> audit               httpd               prelink.log        spooler.3
>>> boot.log            itac_uninstall.log  rpmpkgs            spooler.4
>>> boot.log.1          lastlog             rpmpkgs.1          vbox
>>> boot.log.2          mail                rpmpkgs.2          wtmp
>>> boot.log.3          maillog             rpmpkgs.3          wtmp.1
>>> boot.log.4          maillog.1           rpmpkgs.4          Xorg.0.log
>>> cmkl_install.log    maillog.2           samba              Xorg.0.log.old
>>> cmkl_uninstall.log  maillog.3           scrollkeeper.log   yum.log
>>> cron                maillog.4           secure             yum.log.1
>>> cron.1              messages            secure.1
>>> cron.2              messages.1          secure.2
>>> cron.3              messages.2          secure.3
>>>
>>> so, the log should be in some of these files (I don't have read
>>> permission on these files). I'll contact the admin for that.
>>>
>>> 2012/9/1<users-requ...@open-mpi.org>:
>>>> Send users mailing list submissions to
>>>>          us...@open-mpi.org
>>>>
>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>          http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>> or, via email, send a message with subject or body 'help' to
>>>>          users-requ...@open-mpi.org
>>>>
>>>> You can reach the person managing the list at
>>>>          users-ow...@open-mpi.org
>>>>
>>>> When replying, please edit your Subject line so it is more specific
>>>> than "Re: Contents of users digest..."
>>>>
>>>>
>>>> Today's Topics:
>>>>
>>>>     1. Re: some mpi processes "disappear" on a cluster of        servers
>>>>        (Gus Correa)
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>>
>>>> Message: 1
>>>> Date: Fri, 31 Aug 2012 20:11:41 -0400
>>>> From: Gus Correa<g...@ldeo.columbia.edu>
>>>> Subject: Re: [OMPI users] some mpi processes "disappear" on a cluster
>>>>          of      servers
>>>> To: Open MPI Users<us...@open-mpi.org>
>>>> Message-ID:<504152bd.3000...@ldeo.columbia.edu>
>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>
>>>> Hi Andrea
>>>>
>>>> I would guess this is a memory problem.
>>>> Do you know how much memory each node has?
>>>> Do you know the memory that
>>>> each MPI process in the CFD code requires?
>>>> If the program starts swapping/paging into disk, because of
>>>> low memory, those interesting things that you described can happen.
>>>>
>>>> I would login to the compute nodes and monitor the
>>>> amount of memory being used with "top" right after the program
>>>> starts to run.  If there is a pattern of which node tends to fail,
>>>> track login to that fail-prone node and monitor it.
>>>>
>>>> Alternatively, if you cluster is running Ganglia,
>>>> you can see the memory use graphically,
>>>> in the Ganglia web page in a web browser.
>>>>
>>>> If your cluster
>>>> doesn't allow direct user logins to compute nodes,
>>>> you can ask the system administrator to do this for you.
>>>>
>>>> It may well be that the code has a memory leak, or that
>>>> it has a memory request spike, which may not be caught by
>>>> OpenMPI.
>>>> [Jeff and Ralph will probably correct me soon for
>>>> saying this, and I know the OpenMPI safeguards against
>>>> process misbehavior are great, but ...]
>>>>
>>>> Anyway, we had several codes here that would make a node go south
>>>> because of either type of memory problem, and subsequently the
>>>> program would die, or the other processes in other nodes would
>>>> continue "running" [i.e. mostly waiting for MPI calls to the
>>>> dead node that would never return] as you described.
>>>>
>>>> If the problem is benign, i.e., if it is just that the
>>>> memory-per-processor is not large enough to run in 10 processors,
>>>> you can get around it by running in, say, 20 processors.
>>>>
>>>> Yet another issue that you may check is the stacksize in the
>>>> compute nodes.  Many codes require a large stacksize, i.e.,
>>>> they create large arrays in subroutines, etc, and
>>>> the default stacksize in standard Linux distributions
>>>> may not be as large as needed.
>>>> We use ulimited stacksize in our compute nodes.
>>>>
>>>> You can ask the system administrator to check this for you,
>>>> and perhaps change it in /etc/security/limits.conf to make it
>>>> unlimited or at least larger than the default.
>>>> The Linux shell command "ulimit -a" [bash] or
>>>> "limit" [tcsh] will tell what the limits are.
>>>>
>>>> I hope this helps,
>>>> Gus Correa
>>>>
>>>> On 08/31/2012 07:15 PM, Andrea Negri wrote:
>>>>> Hi, I have been in trouble for a year.
>>>>>
>>>>> I run a pure MPI (no openMP) Fortran fluid dynamical code on a cluster
>>>>> of server, and I obtain a strange behaviour by running the code on
>>>>> multiple nodes.
>>>>> The cluster is formed by 16 pc (1 pc is a node) with a dual core 
>>>>> processor.
>>>>> Basically, I'm able to run the code from the login node with the command:
>>>>> mpirun  --mca btl_base_verbose 100 --mca backtrace_base_verbose 100
>>>>> --mca memory_base_verbose 100 --mca sysinfo_base_verbose 100  -nolocal
>>>>> -hostfile ./host_file -n 10  ./zeusmp2.x>>   zmp_errors 2>&1
>>>>> by selecting one process per core (i.e. in this case I use 5 nodes)
>>>>>
>>>>> The code starts, and it runs correctely for some time.
>>>>> After that, an entire node (sometimes two) "disappears" and it cannot
>>>>> be reached with the ssh command, which returns: No route to host.
>>>>> Sometimes the node is still reachable, but the two processes that was
>>>>> running on the node are disappears.
>>>>> In addition, on the other nodes, the others processes are still running.
>>>>>
>>>>> If I have a look on the output and error file of mpirun, the following
>>>>> error is present: [btl_tcp_frag.c:215:mca_btl_tcp_frag_recv]
>>>>> mca_btl_tcp_frag_recv: readv failed: No route to host (113)
>>>>>
>>>>> PS: I'm not the admin of the cluster, I've installed the gcc and
>>>>> openmpi on my own because the complier aviable on that machine are 8
>>>>> years old.
>>>>>
>>>>>
>>>>> I post here some information, if you want other info, you have only to
>>>>> tell me which command I have to type on the bash and I will
>>>>> immediately reply.
>>>>>
>>>>>
>>>>> complier: gcc 4.7 (which was also used to compile openmpi)
>>>>> openmpi version: 1.6
>>>>>
>>>>> output of "ompi_info --all" from the node where I launch mpirun (which
>>>>> is also the login node of the cluster)
>>>>>
>>>>>     Package: Open MPI and...@cloud.bo.astro.it Distribution
>>>>>                   Open MPI: 1.6
>>>>>      Open MPI SVN revision: r26429
>>>>>      Open MPI release date: May 10, 2012
>>>>>                   Open RTE: 1.6
>>>>>      Open RTE SVN revision: r26429
>>>>>      Open RTE release date: May 10, 2012
>>>>>                       OPAL: 1.6
>>>>>          OPAL SVN revision: r26429
>>>>>          OPAL release date: May 10, 2012
>>>>>                    MPI API: 2.1
>>>>>               Ident string: 1.6
>>>>>              MCA backtrace: execinfo (MCA v2.0, API v2.0, Component v1.6)
>>>>>                 MCA memory: linux (MCA v2.0, API v2.0, Component v1.6)
>>>>>              MCA paffinity: hwloc (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA carto: auto_detect (MCA v2.0, API v2.0, Component 
>>>>> v1.6)
>>>>>                  MCA carto: file (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA shmem: mmap (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA shmem: posix (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA shmem: sysv (MCA v2.0, API v2.0, Component v1.6)
>>>>>              MCA maffinity: first_use (MCA v2.0, API v2.0, Component v1.6)
>>>>>              MCA maffinity: hwloc (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA timer: linux (MCA v2.0, API v2.0, Component v1.6)
>>>>>            MCA installdirs: env (MCA v2.0, API v2.0, Component v1.6)
>>>>>            MCA installdirs: config (MCA v2.0, API v2.0, Component v1.6)
>>>>>                MCA sysinfo: linux (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA hwloc: hwloc132 (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA dpm: orte (MCA v2.0, API v2.0, Component v1.6)
>>>>>                 MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.6)
>>>>>              MCA allocator: basic (MCA v2.0, API v2.0, Component v1.6)
>>>>>              MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.6)
>>>>>                   MCA coll: basic (MCA v2.0, API v2.0, Component v1.6)
>>>>>                   MCA coll: hierarch (MCA v2.0, API v2.0, Component v1.6)
>>>>>                   MCA coll: inter (MCA v2.0, API v2.0, Component v1.6)
>>>>>                   MCA coll: self (MCA v2.0, API v2.0, Component v1.6)
>>>>>                   MCA coll: sm (MCA v2.0, API v2.0, Component v1.6)
>>>>>                   MCA coll: sync (MCA v2.0, API v2.0, Component v1.6)
>>>>>                   MCA coll: tuned (MCA v2.0, API v2.0, Component v1.6)
>>>>>                     MCA io: romio (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA mpool: fake (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA mpool: sm (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA pml: bfo (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA pml: csum (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA pml: ob1 (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA pml: v (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA bml: r2 (MCA v2.0, API v2.0, Component v1.6)
>>>>>                 MCA rcache: vma (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA btl: self (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA btl: sm (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA btl: tcp (MCA v2.0, API v2.0, Component v1.6)
>>>>>                   MCA topo: unity (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA osc: pt2pt (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA osc: rdma (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA iof: hnp (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA iof: orted (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA iof: tool (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA oob: tcp (MCA v2.0, API v2.0, Component v1.6)
>>>>>                   MCA odls: default (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA ras: cm (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA ras: loadleveler (MCA v2.0, API v2.0, Component 
>>>>> v1.6)
>>>>>                    MCA ras: slurm (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA rmaps: load_balance (MCA v2.0, API v2.0, Component 
>>>>> v1.6)
>>>>>                  MCA rmaps: rank_file (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA rmaps: resilient (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA rmaps: round_robin (MCA v2.0, API v2.0, Component 
>>>>> v1.6)
>>>>>                  MCA rmaps: seq (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA rmaps: topo (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA rml: oob (MCA v2.0, API v2.0, Component v1.6)
>>>>>                 MCA routed: binomial (MCA v2.0, API v2.0, Component v1.6)
>>>>>                 MCA routed: cm (MCA v2.0, API v2.0, Component v1.6)
>>>>>                 MCA routed: direct (MCA v2.0, API v2.0, Component v1.6)
>>>>>                 MCA routed: linear (MCA v2.0, API v2.0, Component v1.6)
>>>>>                 MCA routed: radix (MCA v2.0, API v2.0, Component v1.6)
>>>>>                 MCA routed: slave (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA plm: rsh (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA plm: slurm (MCA v2.0, API v2.0, Component v1.6)
>>>>>                  MCA filem: rsh (MCA v2.0, API v2.0, Component v1.6)
>>>>>                 MCA errmgr: default (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA ess: env (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA ess: hnp (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA ess: singleton (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA ess: slave (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA ess: slurm (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA ess: slurmd (MCA v2.0, API v2.0, Component v1.6)
>>>>>                    MCA ess: tool (MCA v2.0, API v2.0, Component v1.6)
>>>>>                MCA grpcomm: bad (MCA v2.0, API v2.0, Component v1.6)
>>>>>                MCA grpcomm: basic (MCA v2.0, API v2.0, Component v1.6)
>>>>>                MCA grpcomm: hier (MCA v2.0, API v2.0, Component v1.6)
>>>>>               MCA notifier: command (MCA v2.0, API v1.0, Component v1.6)
>>>>>               MCA notifier: syslog (MCA v2.0, API v1.0, Component v1.6)
>>>>>                     Prefix: 
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7
>>>>>                Exec_prefix: 
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7
>>>>>                     Bindir: 
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/bin
>>>>>                    Sbindir: 
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/sbin
>>>>>                     Libdir: 
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib
>>>>>                     Incdir:
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/include
>>>>>                     Mandir:
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/man
>>>>>                  Pkglibdir:
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib/openmpi
>>>>>                 Libexecdir:
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/libexec
>>>>>                Datarootdir: 
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share
>>>>>                    Datadir: 
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share
>>>>>                 Sysconfdir: 
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/etc
>>>>>             Sharedstatedir: 
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/com
>>>>>              Localstatedir: 
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/var
>>>>>                    Infodir:
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/info
>>>>>                 Pkgdatadir:
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/openmpi
>>>>>                  Pkglibdir:
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib/openmpi
>>>>>              Pkgincludedir:
>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/include/openmpi
>>>>>    Configured architecture: x86_64-unknown-linux-gnu
>>>>>             Configure host: cloud.bo.astro.it
>>>>>              Configured by: andrea
>>>>>              Configured on: Tue Jul 31 10:53:46 CEST 2012
>>>>>             Configure host: cloud.bo.astro.it
>>>>>                   Built by: andrea
>>>>>                   Built on: Tue Jul 31 11:08:33 CEST 2012
>>>>>                 Built host: cloud.bo.astro.it
>>>>>                 C bindings: yes
>>>>>               C++ bindings: yes
>>>>>         Fortran77 bindings: yes (all)
>>>>>         Fortran90 bindings: yes
>>>>>    Fortran90 bindings size: medium
>>>>>                 C compiler: /home/andrea/library/gcc/gcc-objects/bin/gcc
>>>>>        C compiler absolute:
>>>>>     C compiler family name: GNU
>>>>>         C compiler version: 4.7.1
>>>>>                C char size: 1
>>>>>                C bool size: 1
>>>>>               C short size: 2
>>>>>                 C int size: 4
>>>>>                C long size: 8
>>>>>               C float size: 4
>>>>>              C double size: 8
>>>>>             C pointer size: 8
>>>>>               C char align: 1
>>>>>               C bool align: 1
>>>>>                C int align: 4
>>>>>              C float align: 4
>>>>>             C double align: 8
>>>>>               C++ compiler: /home/andrea/library/gcc/gcc-objects/bin/g++
>>>>>      C++ compiler absolute: none
>>>>>         Fortran77 compiler: 
>>>>> /home/andrea/library/gcc/gcc-objects/bin/gfortran
>>>>>     Fortran77 compiler abs:
>>>>>         Fortran90 compiler: 
>>>>> /home/andrea/library/gcc/gcc-objects/bin/gfortran
>>>>>     Fortran90 compiler abs:
>>>>>          Fort integer size: 4
>>>>>          Fort logical size: 4
>>>>>    Fort logical value true: 1
>>>>>         Fort have integer1: yes
>>>>>         Fort have integer2: yes
>>>>>         Fort have integer4: yes
>>>>>         Fort have integer8: yes
>>>>>        Fort have integer16: no
>>>>>            Fort have real4: yes
>>>>>            Fort have real8: yes
>>>>>           Fort have real16: no
>>>>>         Fort have complex8: yes
>>>>>        Fort have complex16: yes
>>>>>        Fort have complex32: no
>>>>>         Fort integer1 size: 1
>>>>>         Fort integer2 size: 2
>>>>>         Fort integer4 size: 4
>>>>>         Fort integer8 size: 8
>>>>>        Fort integer16 size: -1
>>>>>             Fort real size: 4
>>>>>            Fort real4 size: 4
>>>>>            Fort real8 size: 8
>>>>>           Fort real16 size: 16
>>>>>         Fort dbl prec size: 8
>>>>>             Fort cplx size: 8
>>>>>         Fort dbl cplx size: 16
>>>>>            Fort cplx8 size: 8
>>>>>           Fort cplx16 size: 16
>>>>>           Fort cplx32 size: 32
>>>>>         Fort integer align: 4
>>>>>        Fort integer1 align: 1
>>>>>        Fort integer2 align: 2
>>>>>        Fort integer4 align: 4
>>>>>        Fort integer8 align: 8
>>>>>       Fort integer16 align: -1
>>>>>            Fort real align: 4
>>>>>           Fort real4 align: 4
>>>>>           Fort real8 align: 8
>>>>>          Fort real16 align: 16
>>>>>        Fort dbl prec align: 8
>>>>>            Fort cplx align: 4
>>>>>        Fort dbl cplx align: 8
>>>>>           Fort cplx8 align: 4
>>>>>          Fort cplx16 align: 8
>>>>>          Fort cplx32 align: 16
>>>>>                C profiling: yes
>>>>>              C++ profiling: yes
>>>>>        Fortran77 profiling: yes
>>>>>        Fortran90 profiling: yes
>>>>>             C++ exceptions: no
>>>>>             Thread support: posix (MPI_THREAD_MULTIPLE: no, progress: no)
>>>>>              Sparse Groups: no
>>>>>               Build CFLAGS: -DNDEBUG -g -O2 -finline-functions
>>>>> -fno-strict-aliasing
>>>>>                             -pthread
>>>>>             Build CXXFLAGS: -O3 -DNDEBUG -finline-functions -pthread
>>>>>               Build FFLAGS:
>>>>>              Build FCFLAGS:
>>>>>              Build LDFLAGS: -Wl,--rpath
>>>>> -Wl,/home/andrea/library/gcc/gcc-objects/lib64
>>>>>                 Build LIBS: -lrt -lnsl  -lutil -lm
>>>>>       Wrapper extra CFLAGS: -pthread
>>>>>     Wrapper extra CXXFLAGS: -pthread
>>>>>       Wrapper extra FFLAGS: -pthread
>>>>>      Wrapper extra FCFLAGS: -pthread
>>>>>      Wrapper extra LDFLAGS:
>>>>>         Wrapper extra LIBS: -ldl  -lm -lnuma -lrt -lnsl  -lutil -lm
>>>>>     Internal debug support: no
>>>>>     MPI interface warnings: yes
>>>>>        MPI parameter check: runtime
>>>>> Memory profiling support: no
>>>>> Memory debugging support: no
>>>>>            libltdl support: no
>>>>>      Heterogeneous support: no
>>>>>    mpirun default --prefix: yes
>>>>>            MPI I/O support: yes
>>>>>          MPI_WTIME support: gettimeofday
>>>>>        Symbol vis. support: yes
>>>>>      Host topology support: yes
>>>>>             MPI extensions: affinity example
>>>>>      FT Checkpoint support: no (checkpoint thread: no)
>>>>>        VampirTrace support: yes
>>>>>     MPI_MAX_PROCESSOR_NAME: 256
>>>>>       MPI_MAX_ERROR_STRING: 256
>>>>>        MPI_MAX_OBJECT_NAME: 64
>>>>>           MPI_MAX_INFO_KEY: 36
>>>>>           MPI_MAX_INFO_VAL: 256
>>>>>          MPI_MAX_PORT_NAME: 1024
>>>>>     MPI_MAX_DATAREP_STRING: 128
>>>>>                    MCA mca: parameter "mca_param_files" (current value:
>>>>>
>>>>> </home/andrea/.openmpi/mca-params.conf:/home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/etc/openmpi-mca-params.conf>,
>>>>>                             data source: default value)
>>>>>                             Path for MCA configuration files containing
>>>>> default parameter
>>>>>                             values
>>>>>                    MCA mca: parameter "mca_base_param_file_prefix"
>>>>> (current value:<none>,
>>>>>                             data source: default value)
>>>>>                             Aggregate MCA parameter file sets
>>>>>                    MCA mca: parameter "mca_base_param_file_path" (current 
>>>>> value:
>>>>>
>>>>> </home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/openmpi/amca-param-sets:/home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/bin>,
>>>>>                             data source: default value)
>>>>>                             Aggregate MCA parameter Search path
>>>>>                    MCA mca: parameter "mca_base_param_file_path_force"
>>>>> (current value:
>>>>>                             <none>, data source: default value)
>>>>>                             Forced Aggregate MCA parameter Search path
>>>>>                    MCA mca: parameter "mca_component_path" (current value:
>>>>>
>>>>> </home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib/openmpi:/home/andrea/.openmpi/components>,
>>>>>                             data source: default value)
>>>>>                             Path where to look for Open MPI and ORTE 
>>>>> components
>>>>>                    MCA mca: parameter "mca_component_show_load_errors"
>>>>> (current value:<1>,
>>>>>                             data source: default value)
>>>>>                             Whether to show errors for components that
>>>>> failed to load or
>>>>>                             not
>>>>>                    MCA mca: parameter "mca_component_disable_dlopen"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Whether to attempt to disable opening
>>>>> dynamic components or not
>>>>>                    MCA mca: parameter "mca_verbose" (current value:
>>>>> <stderr>, data source:
>>>>>                             default value)
>>>>>                             Specifies where the default error output
>>>>> stream goes (this is
>>>>>                             separate from distinct help messages).  
>>>>> Accepts a
>>>>>                             comma-delimited list of: stderr, stdout, 
>>>>> syslog,
>>>>>                             syslogpri:<notice|info|debug>,
>>>>> syslogid:<str>   (where str is the
>>>>>                             prefix string for all syslog notices),
>>>>> file[:filename] (if
>>>>>                             filename is not specified, a default
>>>>> filename is used),
>>>>>                             fileappend (if not specified, the file is 
>>>>> opened for
>>>>>                             truncation), level[:N] (if specified,
>>>>> integer verbose level;
>>>>>                             otherwise, 0 is implied)
>>>>>                    MCA mpi: parameter "mpi_paffinity_alone" (current
>>>>> value:<0>, data
>>>>>                             source: default value, synonym of:
>>>>> opal_paffinity_alone)
>>>>>                             If nonzero, assume that this job is the only 
>>>>> (set of)
>>>>>                             process(es) running on each node and bind 
>>>>> processes to
>>>>>                             processors, starting with processor ID 0
>>>>>                    MCA mpi: parameter "mpi_param_check" (current value:
>>>>> <1>, data source:
>>>>>                             default value)
>>>>>                             Whether you want MPI API parameters checked
>>>>> at run-time or not.
>>>>>                              Possible values are 0 (no checking) and 1
>>>>> (perform checking at
>>>>>                             run-time)
>>>>>                    MCA mpi: parameter "mpi_yield_when_idle" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                             Yield the processor when waiting for MPI
>>>>> communication (for MPI
>>>>>                             processes, will default to 1 when
>>>>> oversubscribing nodes)
>>>>>                    MCA mpi: parameter "mpi_event_tick_rate" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                             How often to progress TCP communications (0
>>>>> = never, otherwise
>>>>>                             specified in microseconds)
>>>>>                    MCA mpi: parameter "mpi_show_handle_leaks" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether MPI_FINALIZE shows all MPI handles
>>>>> that were not freed
>>>>>                             or not
>>>>>                    MCA mpi: parameter "mpi_no_free_handles" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether to actually free MPI objects when
>>>>> their handles are
>>>>>                             freed
>>>>>                    MCA mpi: parameter "mpi_show_mpi_alloc_mem_leaks"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             If>0, MPI_FINALIZE will show up to this
>>>>> many instances of
>>>>>                             memory allocated by MPI_ALLOC_MEM that was
>>>>> not freed by
>>>>>                             MPI_FREE_MEM
>>>>>                    MCA mpi: parameter "mpi_show_mca_params" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             Whether to show all MCA parameter values
>>>>> during MPI_INIT or not
>>>>>                             (good for reproducability of MPI jobs for
>>>>> debug purposes).
>>>>>                             Accepted values are all, default, file, api,
>>>>> and enviro - or a
>>>>>                             comma delimited combination of them
>>>>>                    MCA mpi: parameter "mpi_show_mca_params_file"
>>>>> (current value:<none>,
>>>>>                             data source: default value)
>>>>>                             If mpi_show_mca_params is true, setting this
>>>>> string to a valid
>>>>>                             filename tells Open MPI to dump all the MCA
>>>>> parameter values
>>>>>                             into a file suitable for reading via the
>>>>> mca_param_files
>>>>>                             parameter (good for reproducability of MPI 
>>>>> jobs)
>>>>>                    MCA mpi: parameter "mpi_keep_peer_hostnames" (current
>>>>> value:<1>, data
>>>>>                             source: default value)
>>>>>                             If nonzero, save the string hostnames of all
>>>>> MPI peer processes
>>>>>                             (mostly for error / debugging output
>>>>> messages).  This can add
>>>>>                             quite a bit of memory usage to each MPI 
>>>>> process.
>>>>>                    MCA mpi: parameter "mpi_abort_delay" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             If nonzero, print out an identifying message
>>>>> when MPI_ABORT is
>>>>>                             invoked (hostname, PID of the process that
>>>>> called MPI_ABORT)
>>>>>                             and delay for that many seconds before
>>>>> exiting (a negative
>>>>>                             delay value means to never abort).  This
>>>>> allows attaching of a
>>>>>                             debugger before quitting the job.
>>>>>                    MCA mpi: parameter "mpi_abort_print_stack" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             If nonzero, print out a stack trace when
>>>>> MPI_ABORT is invoked
>>>>>                    MCA mpi: parameter "mpi_preconnect_mpi" (current
>>>>> value:<0>, data
>>>>>                             source: default value, synonyms: 
>>>>> mpi_preconnect_all)
>>>>>                             Whether to force MPI processes to fully
>>>>> wire-up the MPI
>>>>>                             connections between MPI processes during
>>>>> MPI_INIT (vs. making
>>>>>                             connections lazily -- upon the first MPI
>>>>> traffic between each
>>>>>                             process peer pair)
>>>>>                    MCA mpi: parameter "mpi_preconnect_all" (current
>>>>> value:<0>, data
>>>>>                             source: default value, deprecated, synonym of:
>>>>>                             mpi_preconnect_mpi)
>>>>>                             Whether to force MPI processes to fully
>>>>> wire-up the MPI
>>>>>                             connections between MPI processes during
>>>>> MPI_INIT (vs. making
>>>>>                             connections lazily -- upon the first MPI
>>>>> traffic between each
>>>>>                             process peer pair)
>>>>>                    MCA mpi: parameter "mpi_leave_pinned" (current value:
>>>>> <-1>, data source:
>>>>>                             default value)
>>>>>                             Whether to use the "leave pinned" protocol
>>>>> or not.  Enabling
>>>>>                             this setting can help bandwidth performance
>>>>> when repeatedly
>>>>>                             sending and receiving large messages with
>>>>> the same buffers over
>>>>>                             RDMA-based networks (0 = do not use "leave
>>>>> pinned" protocol, 1
>>>>>                             = use "leave pinned" protocol, -1 = allow
>>>>> network to choose at
>>>>>                             runtime).
>>>>>                    MCA mpi: parameter "mpi_leave_pinned_pipeline"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether to use the "leave pinned pipeline"
>>>>> protocol or not.
>>>>>                    MCA mpi: parameter "mpi_warn_on_fork" (current value:
>>>>> <1>, data source:
>>>>>                             default value)
>>>>>                             If nonzero, issue a warning if program forks
>>>>> under conditions
>>>>>                             that could cause system errors
>>>>>                    MCA mpi: information "mpi_have_sparse_group_storage"
>>>>> (value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether this Open MPI installation supports
>>>>> storing of data in
>>>>>                             MPI groups in "sparse" formats (good for
>>>>> extremely large
>>>>>                             process count MPI jobs that create many
>>>>> communicators/groups)
>>>>>                    MCA mpi: parameter "mpi_use_sparse_group_storage"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Whether to use "sparse" storage formats for
>>>>> MPI groups (only
>>>>>                             relevant if mpi_have_sparse_group_storage is 
>>>>> 1)
>>>>>                    MCA mpi: parameter "mpi_notify_init_finalize"
>>>>> (current value:<1>, data
>>>>>                             source: default value)
>>>>>                             If nonzero, send two notifications during
>>>>> MPI_INIT: one near
>>>>>                             when MPI_INIT starts, and another right
>>>>> before MPI_INIT
>>>>>                             finishes, and send 2 notifications during
>>>>> MPI_FINALIZE: one
>>>>>                             right when MPI_FINALIZE starts, and another 
>>>>> near when
>>>>>                             MPI_FINALIZE finishes.
>>>>>                   MCA orte: parameter "orte_base_help_aggregate"
>>>>> (current value:<1>, data
>>>>>                             source: default value)
>>>>>                             If orte_base_help_aggregate is true,
>>>>> duplicate help messages
>>>>>                             will be aggregated rather than displayed
>>>>> individually.  This
>>>>>                             can be helpful for parallel jobs that
>>>>> experience multiple
>>>>>                             identical failures; rather than print out
>>>>> the same help/failure
>>>>>                             message N times, display it once with a
>>>>> count of how many
>>>>>                             processes sent the same message.
>>>>>                   MCA orte: parameter "orte_tmpdir_base" (current value:
>>>>> <none>, data
>>>>>                             source: default value)
>>>>>                             Base of the session directory tree
>>>>>                   MCA orte: parameter "orte_no_session_dirs" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             Prohibited locations for session directories 
>>>>> (multiple
>>>>>                             locations separated by ',', default=NULL)
>>>>>                   MCA orte: parameter "orte_send_profile" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                             Send profile info in launch message
>>>>>                   MCA orte: parameter "orte_debug" (current value:<0>,
>>>>> data source:
>>>>>                             default value)
>>>>>                             Top-level ORTE debug switch (default 
>>>>> verbosity: 1)
>>>>>                   MCA orte: parameter "orte_debug_verbose" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for ORTE debug messages 
>>>>> (default: 1)
>>>>>                   MCA orte: parameter "orte_debug_daemons" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether to debug the ORTE daemons or not
>>>>>                   MCA orte: parameter "orte_debug_daemons_file" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether want stdout/stderr of daemons to go
>>>>> to a file or not
>>>>>                   MCA orte: parameter "orte_daemon_bootstrap" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Bootstrap the connection to the HNP
>>>>>                   MCA orte: parameter "orte_leave_session_attached"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Whether applications and/or daemons should
>>>>> leave their sessions
>>>>>                             attached so that any output can be received
>>>>> - this allows X
>>>>>                             forwarding without all the attendant 
>>>>> debugging output
>>>>>                   MCA orte: parameter "orte_output_debugger_proctable"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Whether or not to output the debugger
>>>>> proctable after launch
>>>>>                             (default: false)
>>>>>                   MCA orte: parameter "orte_debugger_test_daemon"
>>>>> (current value:<none>,
>>>>>                             data source: default value)
>>>>>                             Name of the executable to be used to
>>>>> simulate a debugger
>>>>>                             colaunch (relative or absolute path)
>>>>>                   MCA orte: parameter "orte_debugger_test_attach"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                             Test debugger colaunch after debugger 
>>>>> attachment
>>>>>                   MCA orte: parameter "orte_debugger_check_rate"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                             Set rate (in secs) for auto-detect of
>>>>> debugger attachment (0 =>
>>>>>                             do not check)
>>>>>                   MCA orte: parameter "orte_do_not_launch" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Perform all necessary operations to prepare
>>>>> to launch the
>>>>>                             application, but do not actually launch it
>>>>>                   MCA orte: parameter "orte_daemon_spin" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Have any orteds spin until we can connect a
>>>>> debugger to them
>>>>>                   MCA orte: parameter "orte_daemon_fail" (current value:
>>>>> <-1>, data source:
>>>>>                             default value)
>>>>>                             Have the specified orted fail after init for
>>>>> debugging purposes
>>>>>                   MCA orte: parameter "orte_daemon_fail_delay" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Have the specified orted fail after
>>>>> specified number of seconds
>>>>>                             (default: 0 =>   no delay)
>>>>>                   MCA orte: parameter "orte_heartbeat_rate" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Seconds between checks for daemon
>>>>> state-of-health (default: 0
>>>>>                             =>   do not check)
>>>>>                   MCA orte: parameter "orte_startup_timeout" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Milliseconds/daemon to wait for startup
>>>>> before declaring
>>>>>                             failed_to_start (default: 0 =>   do not check)
>>>>>                   MCA orte: parameter "orte_timing" (current value:<0>,
>>>>> data source:
>>>>>                             default value)
>>>>>                             Request that critical timing loops be measured
>>>>>                   MCA orte: parameter "orte_timing_details" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Request that detailed timing data by reported
>>>>>                   MCA orte: parameter "orte_base_user_debugger" (current
>>>>> value:<totalview
>>>>>                             @mpirun@ -a @mpirun_args@ : ddt -n @np@
>>>>> -start @executable@
>>>>>                             @executable_argv@ @single_app@ : fxp @mpirun@ 
>>>>> -a
>>>>>                             @mpirun_args@>, data source: default value)
>>>>>                             Sequence of user-level debuggers to search
>>>>> for in orterun
>>>>>                   MCA orte: parameter "orte_abort_timeout" (current
>>>>> value:<1>, data
>>>>>                             source: default value)
>>>>>                             Max time to wait [in secs] before aborting
>>>>> an ORTE operation
>>>>>                             (default: 1sec)
>>>>>                   MCA orte: parameter "orte_timeout_step" (current
>>>>> value:<1000>, data
>>>>>                             source: default value)
>>>>>                             Time to wait [in usecs/proc] before aborting
>>>>> an ORTE operation
>>>>>                             (default: 1000 usec/proc)
>>>>>                   MCA orte: parameter "orte_default_hostfile" (current 
>>>>> value:
>>>>>
>>>>> </home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/etc/openmpi-default-hostfile>,
>>>>>                             data source: default value)
>>>>>                             Name of the default hostfile (relative or
>>>>> absolute path, "none"
>>>>>                             to ignore environmental or default MCA param 
>>>>> setting)
>>>>>                   MCA orte: parameter "orte_rankfile" (current value:
>>>>> <none>, data source:
>>>>>                             default value, synonyms: rmaps_rank_file_path)
>>>>>                             Name of the rankfile to be used for mapping
>>>>> processes (relative
>>>>>                             or absolute path)
>>>>>                   MCA orte: parameter "orte_keep_fqdn_hostnames"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether or not to keep FQDN hostnames 
>>>>> [default: no]
>>>>>                   MCA orte: parameter "orte_use_regexp" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Whether or not to use regular expressions
>>>>> for launch [default:
>>>>>                             no]
>>>>>                   MCA orte: parameter "orte_tag_output" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Tag all output with [job,rank] (default: 
>>>>> false)
>>>>>                   MCA orte: parameter "orte_xml_output" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Display all output in XML format (default: 
>>>>> false)
>>>>>                   MCA orte: parameter "orte_xml_file" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             Provide all output in XML format to the 
>>>>> specified file
>>>>>                   MCA orte: parameter "orte_timestamp_output" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Timestamp all application process output
>>>>> (default: false)
>>>>>                   MCA orte: parameter "orte_output_filename" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             Redirect output from application processes
>>>>> into filename.rank
>>>>>                             [default: NULL]
>>>>>                   MCA orte: parameter "orte_show_resolved_nodenames"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Display any node names that are resolved to
>>>>> a different name
>>>>>                             (default: false)
>>>>>                   MCA orte: parameter "orte_hetero_apps" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Indicates that multiple app_contexts are
>>>>> being provided that
>>>>>                             are a mix of 32/64 bit binaries (default: 
>>>>> false)
>>>>>                   MCA orte: parameter "orte_launch_agent" (current
>>>>> value:<orted>, data
>>>>>                             source: default value)
>>>>>                             Command used to start processes on remote
>>>>> nodes (default:
>>>>>                             orted)
>>>>>                   MCA orte: parameter "orte_allocation_required"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether or not an allocation by a resource
>>>>> manager is required
>>>>>                             [default: no]
>>>>>                   MCA orte: parameter "orte_xterm" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             Create a new xterm window and display output
>>>>> from the specified
>>>>>                             ranks there [default: none]
>>>>>                   MCA orte: parameter "orte_forward_job_control"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                             Forward SIGTSTP (after converting to
>>>>> SIGSTOP) and SIGCONT
>>>>>                             signals to the application procs [default: no]
>>>>>                   MCA orte: parameter "orte_rsh_agent" (current value:
>>>>> <ssh : rsh>, data
>>>>>                             source: default value, synonyms:
>>>>> pls_rsh_agent, plm_rsh_agent)
>>>>>                             The command used to launch executables on 
>>>>> remote nodes
>>>>>                             (typically either "ssh" or "rsh")
>>>>>                   MCA orte: parameter "orte_assume_same_shell" (current
>>>>> value:<1>, data
>>>>>                             source: default value, synonyms:
>>>>> plm_rsh_assume_same_shell)
>>>>>                             If set to 1, assume that the shell on the
>>>>> remote node is the
>>>>>                             same as the shell on the local node.
>>>>> Otherwise, probe for what
>>>>>                             the remote shell [default: 1]
>>>>>                   MCA orte: parameter "orte_report_launch_progress"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Output a brief periodic report on launch
>>>>> progress [default: no]
>>>>>                   MCA orte: parameter "orte_num_boards" (current value:
>>>>> <1>, data source:
>>>>>                             default value)
>>>>>                             Number of processor boards/node (1-256) 
>>>>> [default: 1]
>>>>>                   MCA orte: parameter "orte_num_sockets" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Number of sockets/board (1-256)
>>>>>                   MCA orte: parameter "orte_num_cores" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Number of cores/socket (1-256)
>>>>>                   MCA orte: parameter "orte_cpu_set" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             Comma-separated list of ranges specifying 
>>>>> logical cpus
>>>>>                             allocated to this job [default: none]
>>>>>                   MCA orte: parameter "orte_process_binding" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             Policy for binding processes [none | core |
>>>>> socket | board]
>>>>>                             (supported qualifier: if-avail)
>>>>>                   MCA orte: parameter "orte_report_bindings" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Report bindings
>>>>>                   MCA orte: parameter "orte_report_events" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             URI to which events are to be reported
>>>>> (default: NULL)]
>>>>>                   MCA opal: parameter "opal_net_private_ipv4" (current 
>>>>> value:
>>>>>
>>>>> <10.0.0.0/8;172.16.0.0/12;192.168.0.0/16;169.254.0.0/16>, data
>>>>>                             source: default value)
>>>>>                             Semicolon-delimited list of CIDR notation
>>>>> entries specifying
>>>>>                             what networks are considered "private"
>>>>> (default value based on
>>>>>                             RFC1918 and RFC3330)
>>>>>                   MCA opal: parameter "opal_signal" (current value:
>>>>> <6,7,8,11>, data
>>>>>                             source: default value)
>>>>>                             Comma-delimited list of integer signal
>>>>> numbers to Open MPI to
>>>>>                             attempt to intercept.  Upon receipt of the
>>>>> intercepted signal,
>>>>>                             Open MPI will display a stack trace and
>>>>> abort.  Open MPI will
>>>>>                             *not* replace signals if handlers are
>>>>> already installed by the
>>>>>                             time MPI_INIT is invoked.  Optionally append
>>>>> ":complain" to any
>>>>>                             signal number in the comma-delimited list to
>>>>> make Open MPI
>>>>>                             complain if it detects another signal
>>>>> handler (and therefore
>>>>>                             does not insert its own).
>>>>>                   MCA opal: parameter "opal_profile" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Set to non-zero to profile component 
>>>>> selections
>>>>>                   MCA opal: parameter "opal_profile_file" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             Name of the file containing the cluster 
>>>>> configuration
>>>>>                             information
>>>>>                   MCA opal: parameter "opal_paffinity_alone" (current
>>>>> value:<0>, data
>>>>>                             source: default value, synonyms: 
>>>>> mpi_paffinity_alone)
>>>>>                             If nonzero, assume that this job is the only 
>>>>> (set of)
>>>>>                             process(es) running on each node and bind 
>>>>> processes to
>>>>>                             processors, starting with processor ID 0
>>>>>                   MCA opal: parameter "opal_set_max_sys_limits" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Set to non-zero to automatically set any
>>>>> system-imposed limits
>>>>>                             to the maximum allowed
>>>>>                   MCA opal: parameter "opal_event_include" (current
>>>>> value:<poll>, data
>>>>>                             source: default value)
>>>>>                             Comma-delimited list of libevent subsystems
>>>>> to use (epoll,
>>>>>                             poll, select -- available on your platform)
>>>>>              MCA backtrace: parameter "backtrace" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             Default selection set of components for the
>>>>> backtrace framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>              MCA backtrace: parameter "backtrace_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the backtrace framework
>>>>> (0 = no verbosity)
>>>>>              MCA backtrace: parameter "backtrace_execinfo_priority"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>             MCA memchecker: parameter "memchecker" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             Default selection set of components for the 
>>>>> memchecker
>>>>>                             framework (<none>   means use all components
>>>>> that can be found)
>>>>>                 MCA memory: parameter "memory" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> memory framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                 MCA memory: parameter "memory_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the memory framework (0
>>>>> = no verbosity)
>>>>>                 MCA memory: information
>>>>> "memory_linux_ptmalloc2_available" (value:<1>,
>>>>>                             data source: default value)
>>>>>                             Whether ptmalloc2 support is included in
>>>>> Open MPI or not (1 =
>>>>>                             yes, 0 = no)
>>>>>                 MCA memory: information
>>>>> "memory_linux_ummunotify_available" (value:<0>,
>>>>>                             data source: default value)
>>>>>                             Whether ummunotify support is included in
>>>>> Open MPI or not (1 =
>>>>>                             yes, 0 = no)
>>>>>                 MCA memory: parameter "memory_linux_ptmalloc2_enable"
>>>>> (current value:<-1>,
>>>>>                             data source: default value)
>>>>>                             Whether to enable ptmalloc2 support or not
>>>>> (negative = try to
>>>>>                             enable, but continue even if support is not
>>>>> available, 0 = do
>>>>>                             not enable support, positive = try to enable
>>>>> and fail if
>>>>>                             support is not available)
>>>>>                 MCA memory: parameter "memory_linux_ummunotify_enable"
>>>>> (current value:
>>>>>                             <-1>, data source: default value)
>>>>>                             Whether to enable ummunotify support or not
>>>>> (negative = try to
>>>>>                             enable, but continue even if support is not
>>>>> available, 0 = do
>>>>>                             not enable support, positive = try to enable
>>>>> and fail if
>>>>>                             support is not available)
>>>>>                 MCA memory: parameter "memory_linux_disable" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             If this MCA parameter is set to 1 **VIA
>>>>> ENVIRONMENT VARIABLE
>>>>>                             ONLY*** (this MCA parameter *CANNOT* be set
>>>>> in a file or on the
>>>>>                             mpirun command line!), this component will
>>>>> be disabled and will
>>>>>                             not attempt to use either ummunotify or
>>>>> memory hook support
>>>>>                 MCA memory: parameter "memory_linux_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>              MCA paffinity: parameter "paffinity_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level of the paffinity framework
>>>>>              MCA paffinity: parameter "paffinity" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             Default selection set of components for the
>>>>> paffinity framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>              MCA paffinity: parameter "paffinity_hwloc_priority"
>>>>> (current value:<40>, data
>>>>>                             source: default value)
>>>>>                             Priority of the hwloc paffinity component
>>>>>                  MCA carto: parameter "carto_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level of the carto framework
>>>>>                  MCA carto: parameter "carto" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> carto framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                  MCA carto: parameter "carto_auto_detect_priority"
>>>>> (current value:<11>,
>>>>>                             data source: default value)
>>>>>                             Priority of the auto_detect carto component
>>>>>                  MCA carto: parameter "carto_file_path" (current value:
>>>>> <none>, data
>>>>>                             source: default value)
>>>>>                             The path to the cartography file
>>>>>                  MCA carto: parameter "carto_file_priority" (current
>>>>> value:<10>, data
>>>>>                             source: default value)
>>>>>                             Priority of the file carto component
>>>>>                  MCA shmem: parameter "shmem_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level of the shmem framework
>>>>>                  MCA shmem: parameter "shmem" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> shmem framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                  MCA shmem: parameter "shmem_mmap_enable_nfs_warning"
>>>>> (current value:<1>,
>>>>>                             data source: default value)
>>>>>                             Enable the warning emitted when Open MPI
>>>>> detects that its
>>>>>                             shared memory backing file is located on a
>>>>> network filesystem
>>>>>                             (1 = enabled, 0 = disabled).
>>>>>                  MCA shmem: parameter "shmem_mmap_priority" (current
>>>>> value:<50>, data
>>>>>                             source: default value)
>>>>>                             Priority of the mmap shmem component
>>>>>                  MCA shmem: parameter "shmem_mmap_relocate_backing_file"
>>>>> (current value:
>>>>>                             <0>, data source: default value)
>>>>>                             Whether to change the default placement of
>>>>> backing files or not
>>>>>                             (Negative = try to relocate backing files to
>>>>> an area rooted at
>>>>>                             the path specified by
>>>>>
>>>>> shmem_mmap_opal_shmem_mmap_backing_file_base_dir, but continue
>>>>>                             with the default path if the relocation
>>>>> fails, 0 = do not
>>>>>                             relocate, Positive = same as the negative
>>>>> option, but will fail
>>>>>                             if the relocation fails.
>>>>>                  MCA shmem: parameter "shmem_mmap_backing_file_base_dir"
>>>>> (current value:
>>>>>                             </dev/shm>, data source: default value)
>>>>>                             Specifies where backing files will be created 
>>>>> when
>>>>>                             shmem_mmap_relocate_backing_file is in use.
>>>>>                  MCA shmem: parameter "shmem_posix_priority" (current
>>>>> value:<40>, data
>>>>>                             source: default value)
>>>>>                             Priority of the posix shmem component
>>>>>                  MCA shmem: parameter "shmem_sysv_priority" (current
>>>>> value:<30>, data
>>>>>                             source: default value)
>>>>>                             Priority of the sysv shmem component
>>>>>              MCA maffinity: parameter "maffinity_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level of the maffinity framework
>>>>>              MCA maffinity: parameter "maffinity" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             Default selection set of components for the
>>>>> maffinity framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>              MCA maffinity: parameter "maffinity_first_use_priority"
>>>>> (current value:<10>,
>>>>>                             data source: default value)
>>>>>                             Priority of the first_use maffinity component
>>>>>              MCA maffinity: parameter "maffinity_hwloc_priority"
>>>>> (current value:<40>, data
>>>>>                             source: default value)
>>>>>                             Priority of the hwloc maffinity component
>>>>>                  MCA timer: parameter "timer" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> timer framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                  MCA timer: parameter "timer_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the timer framework (0 =
>>>>> no verbosity)
>>>>>                  MCA timer: parameter "timer_linux_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                MCA sysinfo: parameter "sysinfo" (current value:<none>,
>>>>> data source:
>>>>>                             default value)
>>>>>                             Default selection set of components for the
>>>>> sysinfo framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                MCA sysinfo: parameter "sysinfo_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the sysinfo framework (0
>>>>> = no verbosity)
>>>>>                MCA sysinfo: parameter "sysinfo_linux_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                  MCA hwloc: parameter "hwloc_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level of the hwloc framework
>>>>>                  MCA hwloc: parameter "hwloc_base_mem_alloc_policy"
>>>>> (current value:<none>,
>>>>>                             data source: default value)
>>>>>                             Policy that determines how general memory
>>>>> allocations are bound
>>>>>                             after MPI_INIT.  A value of "none" means
>>>>> that no memory policy
>>>>>                             is applied.  A value of "local_only" means
>>>>> that all memory
>>>>>                             allocations will be restricted to the local
>>>>> NUMA node where
>>>>>                             each process is placed.  Note that operating
>>>>> system paging
>>>>>                             policies are unaffected by this setting.
>>>>> For example, if
>>>>>                             "local_only" is used and local NUMA node
>>>>> memory is exhausted, a
>>>>>                             new memory allocation may cause paging.
>>>>>                  MCA hwloc: parameter
>>>>> "hwloc_base_mem_bind_failure_action" (current value:
>>>>>                             <error>, data source: default value)
>>>>>                             What Open MPI will do if it explicitly tries
>>>>> to bind memory to
>>>>>                             a specific NUMA location, and fails.  Note
>>>>> that this is a
>>>>>                             different case than the general allocation
>>>>> policy described by
>>>>>                             hwloc_base_alloc_policy.  A value of "warn"
>>>>> means that Open MPI
>>>>>                             will warn the first time this happens, but
>>>>> allow the job to
>>>>>                             continue (possibly with degraded
>>>>> performance).  A value of
>>>>>                             "error" means that Open MPI will abort the
>>>>> job if this happens.
>>>>>                  MCA hwloc: parameter "hwloc" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> hwloc framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                  MCA hwloc: parameter "hwloc_hwloc132_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA dpm: parameter "dpm" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> dpm framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA dpm: parameter "dpm_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the dpm framework (0 =
>>>>> no verbosity)
>>>>>                    MCA dpm: parameter "dpm_orte_priority" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                 MCA pubsub: parameter "pubsub" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> pubsub framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                 MCA pubsub: parameter "pubsub_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the pubsub framework (0
>>>>> = no verbosity)
>>>>>                 MCA pubsub: parameter "pubsub_orte_priority" (current
>>>>> value:<50>, data
>>>>>                             source: default value)
>>>>>                             Priority of the pubsub pmi component
>>>>>              MCA allocator: parameter "allocator" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             Default selection set of components for the
>>>>> allocator framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>              MCA allocator: parameter "allocator_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the allocator framework
>>>>> (0 = no verbosity)
>>>>>              MCA allocator: parameter "allocator_basic_priority"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>              MCA allocator: parameter "allocator_bucket_num_buckets"
>>>>> (current value:<30>,
>>>>>                             data source: default value)
>>>>>              MCA allocator: parameter "allocator_bucket_priority"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                   MCA coll: parameter "coll" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> coll framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                   MCA coll: parameter "coll_base_verbose" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the coll framework (0 =
>>>>> no verbosity)
>>>>>                   MCA coll: parameter "coll_basic_priority" (current
>>>>> value:<10>, data
>>>>>                             source: default value)
>>>>>                             Priority of the basic coll component
>>>>>                   MCA coll: parameter "coll_basic_crossover" (current
>>>>> value:<4>, data
>>>>>                             source: default value)
>>>>>                             Minimum number of processes in a
>>>>> communicator before using the
>>>>>                             logarithmic algorithms
>>>>>                   MCA coll: parameter "coll_hierarch_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Priority of the hierarchical coll component
>>>>>                   MCA coll: parameter "coll_hierarch_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Turn verbose message of the hierarchical
>>>>> coll component on/off
>>>>>                   MCA coll: parameter "coll_hierarch_use_rdma" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Switch from the send btl list used to detect
>>>>> hierarchies to the
>>>>>                             rdma btl list
>>>>>                   MCA coll: parameter "coll_hierarch_ignore_sm" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Ignore sm protocol when detecting
>>>>> hierarchies. Required to
>>>>>                             enable the usage of protocol specific
>>>>> collective operations
>>>>>                   MCA coll: parameter "coll_hierarch_detection_alg"
>>>>> (current value:<2>,
>>>>>                             data source: default value)
>>>>>                             Used to specify the algorithm for detecting
>>>>> Hierarchy.Choose
>>>>>                             between all or two levels of hierarchy
>>>>>                   MCA coll: parameter "coll_hierarch_bcast_alg" (current
>>>>> value:<4>, data
>>>>>                             source: default value)
>>>>>                             Used to specify the algorithm used for bcast
>>>>> operations.
>>>>>                   MCA coll: parameter "coll_hierarch_segment_size"
>>>>> (current value:<32768>,
>>>>>                             data source: default value)
>>>>>                             Used to specify the segment size for
>>>>> segmented algorithms.
>>>>>                   MCA coll: parameter "coll_inter_priority" (current
>>>>> value:<40>, data
>>>>>                             source: default value)
>>>>>                             Priority of the inter coll component
>>>>>                   MCA coll: parameter "coll_inter_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Turn verbose message of the inter coll
>>>>> component on/off
>>>>>                   MCA coll: parameter "coll_self_priority" (current
>>>>> value:<75>, data
>>>>>                             source: default value)
>>>>>                   MCA coll: parameter "coll_sm_priority" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Priority of the sm coll component
>>>>>                   MCA coll: parameter "coll_sm_control_size" (current
>>>>> value:<4096>, data
>>>>>                             source: default value)
>>>>>                             Length of the control data -- should usually
>>>>> be either the
>>>>>                             length of a cache line on most SMPs, or the
>>>>> size of a page on
>>>>>                             machines that support direct memory affinity
>>>>> page placement (in
>>>>>                             bytes)
>>>>>                   MCA coll: parameter "coll_sm_fragment_size" (current
>>>>> value:<8192>, data
>>>>>                             source: default value)
>>>>>                             Fragment size (in bytes) used for passing
>>>>> data through shared
>>>>>                             memory (will be rounded up to the nearest
>>>>> control_size size)
>>>>>                   MCA coll: parameter "coll_sm_comm_in_use_flags"
>>>>> (current value:<2>, data
>>>>>                             source: default value)
>>>>>                             Number of "in use" flags, used to mark a
>>>>> message passing area
>>>>>                             segment as currently being used or not (must
>>>>> be>= 2 and<=
>>>>>                             comm_num_segments)
>>>>>                   MCA coll: parameter "coll_sm_comm_num_segments"
>>>>> (current value:<8>, data
>>>>>                             source: default value)
>>>>>                             Number of segments in each communicator's
>>>>> shared memory message
>>>>>                             passing area (must be>= 2, and must be a 
>>>>> multiple of
>>>>>                             comm_in_use_flags)
>>>>>                   MCA coll: parameter "coll_sm_tree_degree" (current
>>>>> value:<4>, data
>>>>>                             source: default value)
>>>>>                             Degree of the tree for tree-based operations
>>>>> (must be =>   1 and
>>>>>                             <= min(control_size, 255))
>>>>>                   MCA coll: parameter "coll_sm_info_num_procs" (current
>>>>> value:<4>, data
>>>>>                             source: default value)
>>>>>                             Number of processes to use for the 
>>>>> calculation of the
>>>>>                             shared_mem_size MCA information parameter
>>>>> (must be =>   2)
>>>>>                   MCA coll: information "coll_sm_shared_mem_used_data"
>>>>> (value:<548864>,
>>>>>                             data source: default value)
>>>>>                             Amount of shared memory used, per
>>>>> communicator, in the shared
>>>>>                             memory data area for info_num_procs
>>>>> processes (in bytes)
>>>>>                   MCA coll: parameter "coll_sync_priority" (current
>>>>> value:<50>, data
>>>>>                             source: default value)
>>>>>                             Priority of the sync coll component; only 
>>>>> relevant if
>>>>>                             barrier_before or barrier_after is>   0
>>>>>                   MCA coll: parameter "coll_sync_barrier_before"
>>>>> (current value:<1000>,
>>>>>                             data source: default value)
>>>>>                             Do a synchronization before each Nth 
>>>>> collective
>>>>>                   MCA coll: parameter "coll_sync_barrier_after" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Do a synchronization after each Nth collective
>>>>>                   MCA coll: parameter "coll_tuned_priority" (current
>>>>> value:<30>, data
>>>>>                             source: default value)
>>>>>                             Priority of the tuned coll component
>>>>>                   MCA coll: parameter
>>>>> "coll_tuned_pre_allocate_memory_comm_size_limit"
>>>>>                             (current value:<32768>, data source: default 
>>>>> value)
>>>>>                             Size of communicator were we stop
>>>>> pre-allocating memory for the
>>>>>                             fixed internal buffer used for message
>>>>> requests etc that is
>>>>>                             hung off the communicator data segment. I.e.
>>>>> if you have a
>>>>>                             100'000 nodes you might not want to
>>>>> pre-allocate 200'000
>>>>>                             request handle slots per communicator 
>>>>> instance!
>>>>>                   MCA coll: parameter "coll_tuned_init_tree_fanout"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                             Inital fanout used in the tree topologies for 
>>>>> each
>>>>>                             communicator. This is only an initial guess,
>>>>> if a tuned
>>>>>                             collective needs a different fanout for an
>>>>> operation, it build
>>>>>                             it dynamically. This parameter is only for
>>>>> the first guess and
>>>>>                             might save a little time
>>>>>                   MCA coll: parameter "coll_tuned_init_chain_fanout"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                             Inital fanout used in the chain (fanout
>>>>> followed by pipeline)
>>>>>                             topologies for each communicator. This is
>>>>> only an initial
>>>>>                             guess, if a tuned collective needs a
>>>>> different fanout for an
>>>>>                             operation, it build it dynamically. This
>>>>> parameter is only for
>>>>>                             the first guess and might save a little time
>>>>>                   MCA coll: parameter "coll_tuned_use_dynamic_rules"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Switch used to decide if we use static
>>>>> (compiled/if statements)
>>>>>                             or dynamic (built at runtime) decision 
>>>>> function rules
>>>>>                     MCA io: parameter "io_base_freelist_initial_size"
>>>>> (current value:<16>,
>>>>>                             data source: default value)
>>>>>                             Initial MPI-2 IO request freelist size
>>>>>                     MCA io: parameter "io_base_freelist_max_size"
>>>>> (current value:<64>,
>>>>>                             data source: default value)
>>>>>                             Max size of the MPI-2 IO request freelist
>>>>>                     MCA io: parameter "io_base_freelist_increment"
>>>>> (current value:<16>,
>>>>>                             data source: default value)
>>>>>                             Increment size of the MPI-2 IO request 
>>>>> freelist
>>>>>                     MCA io: parameter "io" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> io framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                     MCA io: parameter "io_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the io framework (0 = no
>>>>> verbosity)
>>>>>                     MCA io: parameter "io_romio_priority" (current
>>>>> value:<10>, data
>>>>>                             source: default value)
>>>>>                             Priority of the io romio component
>>>>>                     MCA io: parameter "io_romio_delete_priority"
>>>>> (current value:<10>, data
>>>>>                             source: default value)
>>>>>                             Delete priority of the io romio component
>>>>>                     MCA io: information "io_romio_version" (value:<from
>>>>> MPICH2 v1.3.1 with
>>>>>                             an additional patch from
>>>>> romio-ma...@mcs.anl.gov about an
>>>>>                             attribute issue>, data source: default value)
>>>>>                             Version of ROMIO
>>>>>                     MCA io: information "io_romio_user_configure_params"
>>>>> (value:<none>,
>>>>>                             data source: default value)
>>>>>                             User-specified command line parameters
>>>>> passed to ROMIO's
>>>>>                             configure script
>>>>>                     MCA io: information
>>>>> "io_romio_complete_configure_params" (value:<
>>>>>                             CFLAGS='-DNDEBUG -g -O2 -finline-functions
>>>>> -fno-strict-aliasing
>>>>>                             -pthread' CPPFLAGS='
>>>>>
>>>>> -I/home/andrea/library/openmpi/openmpi-1.6/opal/mca/hwloc/hwloc132/hwloc/include
>>>>>                               -I/usr/include/infiniband 
>>>>> -I/usr/include/infiniband'
>>>>>                             FFLAGS='' LDFLAGS='-Wl,--rpath
>>>>>                             
>>>>> -Wl,/home/andrea/library/gcc/gcc-objects/lib64 '
>>>>>                             --enable-shared --enable-static
>>>>>
>>>>> --prefix=/home/andrea/library/openmpi/openmpi-1.6-gnu-4.7
>>>>>                             --with-mpi=open_mpi --disable-aio>, data
>>>>> source: default value)
>>>>>                             Complete set of command line parameters
>>>>> passed to ROMIO's
>>>>>                             configure script
>>>>>                  MCA mpool: parameter "mpool" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> mpool framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                  MCA mpool: parameter "mpool_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the mpool framework (0 =
>>>>> no verbosity)
>>>>>                  MCA mpool: parameter "mpool_fake_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                  MCA mpool: parameter "mpool_rdma_rcache_name" (current
>>>>> value:<vma>, data
>>>>>                             source: default value)
>>>>>                             The name of the registration cache the mpool
>>>>> should use
>>>>>                  MCA mpool: parameter "mpool_rdma_rcache_size_limit"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             the maximum size of registration cache in
>>>>> bytes. 0 is unlimited
>>>>>                             (default 0)
>>>>>                  MCA mpool: parameter "mpool_rdma_print_stats" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             print pool usage statistics at the end of the 
>>>>> run
>>>>>                  MCA mpool: parameter "mpool_rdma_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                  MCA mpool: parameter "mpool_sm_allocator" (current
>>>>> value:<bucket>, data
>>>>>                             source: default value)
>>>>>                             Name of allocator component to use with sm 
>>>>> mpool
>>>>>                  MCA mpool: parameter "mpool_sm_min_size" (current
>>>>> value:<67108864>, data
>>>>>                             source: default value)
>>>>>                             Minimum size of the sm mpool shared memory 
>>>>> file
>>>>>                  MCA mpool: parameter "mpool_sm_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Enable verbose output for mpool sm component
>>>>>                  MCA mpool: parameter "mpool_sm_priority" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                    MCA pml: parameter "pml_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level of the PML framework
>>>>>                    MCA pml: parameter "pml" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> pml framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA pml: parameter "pml_bfo_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                    MCA pml: parameter "pml_bfo_free_list_num" (current
>>>>> value:<4>, data
>>>>>                             source: default value)
>>>>>                    MCA pml: parameter "pml_bfo_free_list_max" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                    MCA pml: parameter "pml_bfo_free_list_inc" (current
>>>>> value:<64>, data
>>>>>                             source: default value)
>>>>>                    MCA pml: parameter "pml_bfo_priority" (current value:
>>>>> <5>, data source:
>>>>>                             default value)
>>>>>                    MCA pml: parameter "pml_bfo_send_pipeline_depth"
>>>>> (current value:<3>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_bfo_recv_pipeline_depth"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_bfo_rdma_put_retries_limit"
>>>>> (current value:<5>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_bfo_max_rdma_per_request"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_bfo_max_send_per_range"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_bfo_unexpected_limit"
>>>>> (current value:<128>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_bfo_allocator" (current
>>>>> value:<bucket>, data
>>>>>                             source: default value)
>>>>>                             Name of allocator component for unexpected 
>>>>> messages
>>>>>                    MCA pml: parameter "pml_cm_free_list_num" (current
>>>>> value:<4>, data
>>>>>                             source: default value)
>>>>>                             Initial size of request free lists
>>>>>                    MCA pml: parameter "pml_cm_free_list_max" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                             Maximum size of request free lists
>>>>>                    MCA pml: parameter "pml_cm_free_list_inc" (current
>>>>> value:<64>, data
>>>>>                             source: default value)
>>>>>                             Number of elements to add when growing
>>>>> request free lists
>>>>>                    MCA pml: parameter "pml_cm_priority" (current value:
>>>>> <10>, data source:
>>>>>                             default value)
>>>>>                             CM PML selection priority
>>>>>                    MCA pml: parameter "pml_csum_free_list_num" (current
>>>>> value:<4>, data
>>>>>                             source: default value)
>>>>>                    MCA pml: parameter "pml_csum_free_list_max" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                    MCA pml: parameter "pml_csum_free_list_inc" (current
>>>>> value:<64>, data
>>>>>                             source: default value)
>>>>>                    MCA pml: parameter "pml_csum_send_pipeline_depth"
>>>>> (current value:<3>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_csum_recv_pipeline_depth"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_csum_rdma_put_retries_limit"
>>>>> (current value:
>>>>>                             <5>, data source: default value)
>>>>>                    MCA pml: parameter "pml_csum_max_rdma_per_request"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_csum_max_send_per_range"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_csum_unexpected_limit"
>>>>> (current value:<128>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_csum_allocator" (current
>>>>> value:<bucket>, data
>>>>>                             source: default value)
>>>>>                             Name of allocator component for unexpected 
>>>>> messages
>>>>>                    MCA pml: parameter "pml_csum_priority" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                    MCA pml: parameter "pml_ob1_free_list_num" (current
>>>>> value:<4>, data
>>>>>                             source: default value)
>>>>>                    MCA pml: parameter "pml_ob1_free_list_max" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                    MCA pml: parameter "pml_ob1_free_list_inc" (current
>>>>> value:<64>, data
>>>>>                             source: default value)
>>>>>                    MCA pml: parameter "pml_ob1_priority" (current value:
>>>>> <20>, data source:
>>>>>                             default value)
>>>>>                    MCA pml: parameter "pml_ob1_send_pipeline_depth"
>>>>> (current value:<3>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_ob1_recv_pipeline_depth"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_ob1_rdma_put_retries_limit"
>>>>> (current value:<5>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_ob1_max_rdma_per_request"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_ob1_max_send_per_range"
>>>>> (current value:<4>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_ob1_unexpected_limit"
>>>>> (current value:<128>,
>>>>>                             data source: default value)
>>>>>                    MCA pml: parameter "pml_ob1_allocator" (current
>>>>> value:<bucket>, data
>>>>>                             source: default value)
>>>>>                             Name of allocator component for unexpected 
>>>>> messages
>>>>>                    MCA pml: parameter "pml_v_priority" (current value:
>>>>> <-1>, data source:
>>>>>                             default value)
>>>>>                    MCA pml: parameter "pml_v_output" (current value:
>>>>> <stderr>, data source:
>>>>>                             default value)
>>>>>                    MCA pml: parameter "pml_v_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                    MCA bml: parameter "bml" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> bml framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA bml: parameter "bml_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the bml framework (0 =
>>>>> no verbosity)
>>>>>                    MCA bml: parameter "bml_r2_show_unreach_errors"
>>>>> (current value:<1>,
>>>>>                             data source: default value)
>>>>>                             Show error message when procs are unreachable
>>>>>                    MCA bml: parameter "bml_r2_priority" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                 MCA rcache: parameter "rcache" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> rcache framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                 MCA rcache: parameter "rcache_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the rcache framework (0
>>>>> = no verbosity)
>>>>>                 MCA rcache: parameter "rcache_vma_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level of the BTL framework
>>>>>                    MCA btl: parameter "btl" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> btl framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA btl: parameter "btl_self_free_list_num" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Number of fragments by default
>>>>>                    MCA btl: parameter "btl_self_free_list_max" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                             Maximum number of fragments
>>>>>                    MCA btl: parameter "btl_self_free_list_inc" (current
>>>>> value:<32>, data
>>>>>                             source: default value)
>>>>>                             Increment by this number of fragments
>>>>>                    MCA btl: parameter "btl_self_exclusivity" (current
>>>>> value:<65536>, data
>>>>>                             source: default value)
>>>>>                             BTL exclusivity (must be>= 0)
>>>>>                    MCA btl: parameter "btl_self_flags" (current value:
>>>>> <10>, data source:
>>>>>                             default value)
>>>>>                             BTL bit flags (general flags: SEND=1, PUT=2, 
>>>>> GET=4,
>>>>>                             SEND_INPLACE=8, RDMA_MATCHED=64,
>>>>> HETEROGENEOUS_RDMA=256; flags
>>>>>                             only used by the "dr" PML (ignored by 
>>>>> others): ACK=16,
>>>>>                             CHECKSUM=32, RDMA_COMPLETION=128; flags only
>>>>> used by the "bfo"
>>>>>                             PML (ignored by others): FAILOVER_SUPPORT=512)
>>>>>                    MCA btl: parameter "btl_self_rndv_eager_limit"
>>>>> (current value:<131072>,
>>>>>                             data source: default value)
>>>>>                             Size (in bytes) of "phase 1" fragment sent
>>>>> for all large
>>>>>                             messages (must be>= 0 and<= eager_limit)
>>>>>                    MCA btl: parameter "btl_self_eager_limit" (current
>>>>> value:<131072>, data
>>>>>                             source: default value)
>>>>>                             Maximum size (in bytes) of "short" messages
>>>>> (must be>= 1).
>>>>>                    MCA btl: parameter "btl_self_max_send_size" (current
>>>>> value:<262144>,
>>>>>                             data source: default value)
>>>>>                             Maximum size (in bytes) of a single "phase
>>>>> 2" fragment of a
>>>>>                             long message when using the pipeline
>>>>> protocol (must be>= 1)
>>>>>                    MCA btl: parameter
>>>>> "btl_self_rdma_pipeline_send_length" (current value:
>>>>>                             <2147483647>, data source: default value)
>>>>>                             Length of the "phase 2" portion of a large
>>>>> message (in bytes)
>>>>>                             when using the pipeline protocol.  This part
>>>>> of the message
>>>>>                             will be split into fragments of size
>>>>> max_send_size and sent
>>>>>                             using send/receive semantics (must be>= 0;
>>>>> only relevant when
>>>>>                             the PUT flag is set)
>>>>>                    MCA btl: parameter "btl_self_rdma_pipeline_frag_size"
>>>>> (current value:
>>>>>                             <2147483647>, data source: default value)
>>>>>                             Maximum size (in bytes) of a single "phase
>>>>> 3" fragment from a
>>>>>                             long message when using the pipeline
>>>>> protocol.  These fragments
>>>>>                             will be sent using RDMA semantics (must be
>>>>>> = 1; only relevant
>>>>>                             when the PUT flag is set)
>>>>>                    MCA btl: parameter "btl_self_min_rdma_pipeline_size"
>>>>> (current value:
>>>>>                             <0>, data source: default value)
>>>>>                             Messages smaller than this size (in bytes)
>>>>> will not use the
>>>>>                             RDMA pipeline protocol.  Instead, they will
>>>>> be split into
>>>>>                             fragments of max_send_size and sent using 
>>>>> send/receive
>>>>>                             semantics (must be>=0, and is automatically
>>>>> adjusted up to at
>>>>>                             least
>>>>> (eager_limit+btl_rdma_pipeline_send_length); only
>>>>>                             relevant when the PUT flag is set)
>>>>>                    MCA btl: parameter "btl_self_bandwidth" (current
>>>>> value:<100>, data
>>>>>                             source: default value)
>>>>>                             Approximate maximum bandwidth of
>>>>> interconnect(0 = auto-detect
>>>>>                             value at run-time [not supported in all BTL
>>>>> modules],>= 1 =
>>>>>                             bandwidth in Mbps)
>>>>>                    MCA btl: parameter "btl_self_latency" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Approximate latency of interconnect (must 
>>>>> be>= 0)
>>>>>                    MCA btl: parameter "btl_self_priority" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                    MCA btl: information "btl_sm_have_knem_support"
>>>>> (value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether this component supports the knem
>>>>> Linux kernel module or
>>>>>                             not
>>>>>                    MCA btl: parameter "btl_sm_use_knem" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Whether knem support is desired or not
>>>>> (negative = try to
>>>>>                             enable knem support, but continue even if it
>>>>> is not available,
>>>>>                             0 = do not enable knem support, positive =
>>>>> try to enable knem
>>>>>                             support and fail if it is not available)
>>>>>                    MCA btl: parameter "btl_sm_knem_dma_min" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Minimum message size (in bytes) to use the
>>>>> knem DMA mode;
>>>>>                             ignored if knem does not support DMA mode (0
>>>>> = do not use the
>>>>>                             knem DMA mode)
>>>>>                    MCA btl: parameter "btl_sm_knem_max_simultaneous"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Max number of simultaneous ongoing knem
>>>>> operations to support
>>>>>                             (0 = do everything synchronously, which
>>>>> probably gives the best
>>>>>                             large message latency;>0 means to do all 
>>>>> operations
>>>>>                             asynchronously, which supports better
>>>>> overlap for simultaneous
>>>>>                             large message sends)
>>>>>                    MCA btl: parameter "btl_sm_free_list_num" (current
>>>>> value:<8>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_sm_free_list_max" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_sm_free_list_inc" (current
>>>>> value:<64>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_sm_max_procs" (current value:
>>>>> <-1>, data source:
>>>>>                             default value)
>>>>>                    MCA btl: parameter "btl_sm_mpool" (current value:
>>>>> <sm>, data source:
>>>>>                             default value)
>>>>>                    MCA btl: parameter "btl_sm_fifo_size" (current value:
>>>>> <4096>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_sm_num_fifos" (current value:
>>>>> <1>, data source:
>>>>>                             default value)
>>>>>                    MCA btl: parameter "btl_sm_fifo_lazy_free" (current
>>>>> value:<120>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_sm_sm_extra_procs" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_sm_exclusivity" (current
>>>>> value:<65535>, data
>>>>>                             source: default value)
>>>>>                             BTL exclusivity (must be>= 0)
>>>>>                    MCA btl: parameter "btl_sm_flags" (current value:
>>>>> <1>, data source:
>>>>>                             default value)
>>>>>                             BTL bit flags (general flags: SEND=1, PUT=2, 
>>>>> GET=4,
>>>>>                             SEND_INPLACE=8, RDMA_MATCHED=64,
>>>>> HETEROGENEOUS_RDMA=256; flags
>>>>>                             only used by the "dr" PML (ignored by 
>>>>> others): ACK=16,
>>>>>                             CHECKSUM=32, RDMA_COMPLETION=128; flags only
>>>>> used by the "bfo"
>>>>>                             PML (ignored by others): FAILOVER_SUPPORT=512)
>>>>>                    MCA btl: parameter "btl_sm_rndv_eager_limit" (current
>>>>> value:<4096>,
>>>>>                             data source: default value)
>>>>>                             Size (in bytes) of "phase 1" fragment sent
>>>>> for all large
>>>>>                             messages (must be>= 0 and<= eager_limit)
>>>>>                    MCA btl: parameter "btl_sm_eager_limit" (current
>>>>> value:<4096>, data
>>>>>                             source: default value)
>>>>>                             Maximum size (in bytes) of "short" messages
>>>>> (must be>= 1).
>>>>>                    MCA btl: parameter "btl_sm_max_send_size" (current
>>>>> value:<32768>, data
>>>>>                             source: default value)
>>>>>                             Maximum size (in bytes) of a single "phase
>>>>> 2" fragment of a
>>>>>                             long message when using the pipeline
>>>>> protocol (must be>= 1)
>>>>>                    MCA btl: parameter "btl_sm_bandwidth" (current value:
>>>>> <9000>, data
>>>>>                             source: default value)
>>>>>                             Approximate maximum bandwidth of
>>>>> interconnect(0 = auto-detect
>>>>>                             value at run-time [not supported in all BTL
>>>>> modules],>= 1 =
>>>>>                             bandwidth in Mbps)
>>>>>                    MCA btl: parameter "btl_sm_latency" (current value:
>>>>> <1>, data source:
>>>>>                             default value)
>>>>>                             Approximate latency of interconnect (must 
>>>>> be>= 0)
>>>>>                    MCA btl: parameter "btl_sm_priority" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                    MCA btl: parameter "btl_tcp_links" (current value:
>>>>> <1>, data source:
>>>>>                             default value)
>>>>>                    MCA btl: parameter "btl_tcp_if_include" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             Comma-delimited list of devices or CIDR
>>>>> notation of networks to
>>>>>                             use for MPI communication (e.g., "eth0,eth1" 
>>>>> or
>>>>>                             "192.168.0.0/16,10.1.4.0/24").  Mutually
>>>>> exclusive with
>>>>>                             btl_tcp_if_exclude.
>>>>>                    MCA btl: parameter "btl_tcp_if_exclude" (current
>>>>> value:<lo,sppp>, data
>>>>>                             source: default value)
>>>>>                             Comma-delimited list of devices or CIDR
>>>>> notation of networks to
>>>>>                             NOT use for MPI communication -- all devices
>>>>> not matching these
>>>>>                             specifications will be used (e.g., 
>>>>> "eth0,eth1" or
>>>>>                             "192.168.0.0/16,10.1.4.0/24").  Mutually
>>>>> exclusive with
>>>>>                             btl_tcp_if_include.
>>>>>                    MCA btl: parameter "btl_tcp_free_list_num" (current
>>>>> value:<8>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_tcp_free_list_max" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_tcp_free_list_inc" (current
>>>>> value:<32>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_tcp_sndbuf" (current value:
>>>>> <131072>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_tcp_rcvbuf" (current value:
>>>>> <131072>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_tcp_endpoint_cache" (current
>>>>> value:<30720>,
>>>>>                             data source: default value)
>>>>>                             The size of the internal cache for each TCP
>>>>> connection. This
>>>>>                             cache is used to reduce the number of
>>>>> syscalls, by replacing
>>>>>                             them with memcpy. Every read will read the
>>>>> expected data plus
>>>>>                             the amount of the endpoint_cache
>>>>>                    MCA btl: parameter "btl_tcp_use_nagle" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                             Whether to use Nagle's algorithm or not 
>>>>> (using Nagle's
>>>>>                             algorithm may increase short message latency)
>>>>>                    MCA btl: parameter "btl_tcp_port_min_v4" (current
>>>>> value:<1024>, data
>>>>>                             source: default value)
>>>>>                             The minimum port where the TCP BTL will try
>>>>> to bind (default
>>>>>                             1024)
>>>>>                    MCA btl: parameter "btl_tcp_port_range_v4" (current
>>>>> value:<64511>, data
>>>>>                             source: default value)
>>>>>                             The number of ports where the TCP BTL will
>>>>> try to bind (default
>>>>>                             64511). This parameter together with the
>>>>> port min, define a
>>>>>                             range of ports where Open MPI will open 
>>>>> sockets.
>>>>>                    MCA btl: parameter "btl_tcp_exclusivity" (current
>>>>> value:<100>, data
>>>>>                             source: default value)
>>>>>                             BTL exclusivity (must be>= 0)
>>>>>                    MCA btl: parameter "btl_tcp_flags" (current value:
>>>>> <314>, data source:
>>>>>                             default value)
>>>>>                             BTL bit flags (general flags: SEND=1, PUT=2, 
>>>>> GET=4,
>>>>>                             SEND_INPLACE=8, RDMA_MATCHED=64,
>>>>> HETEROGENEOUS_RDMA=256; flags
>>>>>                             only used by the "dr" PML (ignored by 
>>>>> others): ACK=16,
>>>>>                             CHECKSUM=32, RDMA_COMPLETION=128; flags only
>>>>> used by the "bfo"
>>>>>                             PML (ignored by others): FAILOVER_SUPPORT=512)
>>>>>                    MCA btl: parameter "btl_tcp_rndv_eager_limit"
>>>>> (current value:<65536>,
>>>>>                             data source: default value)
>>>>>                             Size (in bytes) of "phase 1" fragment sent
>>>>> for all large
>>>>>                             messages (must be>= 0 and<= eager_limit)
>>>>>                    MCA btl: parameter "btl_tcp_eager_limit" (current
>>>>> value:<65536>, data
>>>>>                             source: default value)
>>>>>                             Maximum size (in bytes) of "short" messages
>>>>> (must be>= 1).
>>>>>                    MCA btl: parameter "btl_tcp_max_send_size" (current
>>>>> value:<131072>,
>>>>>                             data source: default value)
>>>>>                             Maximum size (in bytes) of a single "phase
>>>>> 2" fragment of a
>>>>>                             long message when using the pipeline
>>>>> protocol (must be>= 1)
>>>>>                    MCA btl: parameter
>>>>> "btl_tcp_rdma_pipeline_send_length" (current value:
>>>>>                             <131072>, data source: default value)
>>>>>                             Length of the "phase 2" portion of a large
>>>>> message (in bytes)
>>>>>                             when using the pipeline protocol.  This part
>>>>> of the message
>>>>>                             will be split into fragments of size
>>>>> max_send_size and sent
>>>>>                             using send/receive semantics (must be>= 0;
>>>>> only relevant when
>>>>>                             the PUT flag is set)
>>>>>                    MCA btl: parameter "btl_tcp_rdma_pipeline_frag_size"
>>>>> (current value:
>>>>>                             <2147483647>, data source: default value)
>>>>>                             Maximum size (in bytes) of a single "phase
>>>>> 3" fragment from a
>>>>>                             long message when using the pipeline
>>>>> protocol.  These fragments
>>>>>                             will be sent using RDMA semantics (must be
>>>>>> = 1; only relevant
>>>>>                             when the PUT flag is set)
>>>>>                    MCA btl: parameter "btl_tcp_min_rdma_pipeline_size"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Messages smaller than this size (in bytes)
>>>>> will not use the
>>>>>                             RDMA pipeline protocol.  Instead, they will
>>>>> be split into
>>>>>                             fragments of max_send_size and sent using 
>>>>> send/receive
>>>>>                             semantics (must be>=0, and is automatically
>>>>> adjusted up to at
>>>>>                             least
>>>>> (eager_limit+btl_rdma_pipeline_send_length); only
>>>>>                             relevant when the PUT flag is set)
>>>>>                    MCA btl: parameter "btl_tcp_bandwidth" (current
>>>>> value:<100>, data
>>>>>                             source: default value)
>>>>>                             Approximate maximum bandwidth of
>>>>> interconnect(0 = auto-detect
>>>>>                             value at run-time [not supported in all BTL
>>>>> modules],>= 1 =
>>>>>                             bandwidth in Mbps)
>>>>>                    MCA btl: parameter "btl_tcp_latency" (current value:
>>>>> <100>, data source:
>>>>>                             default value)
>>>>>                             Approximate latency of interconnect (must 
>>>>> be>= 0)
>>>>>                    MCA btl: parameter "btl_tcp_disable_family" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_tcp_if_seq" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             If specified, a comma-delimited list of TCP
>>>>> interfaces.
>>>>>                             Interfaces will be assigned, one to each MPI
>>>>> process, in a
>>>>>                             round-robin fashion on each server.  For
>>>>> example, if the list
>>>>>                             is "eth0,eth1" and four MPI processes are
>>>>> run on a single
>>>>>                             server, then local ranks 0 and 2 will use
>>>>> eth0 and local ranks
>>>>>                             1 and 3 will use eth1.
>>>>>                    MCA btl: parameter "btl_tcp_priority" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                    MCA btl: parameter "btl_base_include" (current value:
>>>>> <none>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_base_exclude" (current value:
>>>>> <none>, data
>>>>>                             source: default value)
>>>>>                    MCA btl: parameter "btl_base_warn_component_unused"
>>>>> (current value:<1>,
>>>>>                             data source: default value)
>>>>>                             This parameter is used to turn on warning
>>>>> messages when certain
>>>>>                             NICs are not used
>>>>>                    MCA mtl: parameter "mtl" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> mtl framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA mtl: parameter "mtl_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the mtl framework (0 =
>>>>> no verbosity)
>>>>>                   MCA topo: parameter "topo" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> topo framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                   MCA topo: parameter "topo_base_verbose" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the topo framework (0 =
>>>>> no verbosity)
>>>>>                   MCA topo: parameter "topo_unity_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA osc: parameter "osc" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> osc framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA osc: parameter "osc_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the osc framework (0 =
>>>>> no verbosity)
>>>>>                    MCA osc: parameter "osc_pt2pt_no_locks" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Enable optimizations available only if
>>>>> MPI_LOCK is not used.
>>>>>                    MCA osc: parameter "osc_pt2pt_eager_limit" (current
>>>>> value:<16384>, data
>>>>>                             source: default value)
>>>>>                             Max size of eagerly sent data
>>>>>                    MCA osc: parameter "osc_pt2pt_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA osc: parameter "osc_rdma_eager_send" (current
>>>>> value:<1>, data
>>>>>                             source: default value)
>>>>>                             Attempt to start data movement during
>>>>> communication call,
>>>>>                             instead of at synchrnoization time.  Info
>>>>> key of same name
>>>>>                             overrides this value.
>>>>>                    MCA osc: parameter "osc_rdma_use_buffers" (current
>>>>> value:<1>, data
>>>>>                             source: default value)
>>>>>                             Coalesce messages during an epoch to reduce 
>>>>> network
>>>>>                             utilization.  Info key of same name
>>>>> overrides this value.
>>>>>                    MCA osc: parameter "osc_rdma_use_rdma" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                             Use real RDMA operations to transfer data.
>>>>> Info key of same
>>>>>                             name overrides this value.
>>>>>                    MCA osc: parameter "osc_rdma_rdma_completion_wait"
>>>>> (current value:<1>,
>>>>>                             data source: default value)
>>>>>                             Wait for all completion of rdma events before 
>>>>> sending
>>>>>                             acknowledgment.  Info key of same name
>>>>> overrides this value.
>>>>>                    MCA osc: parameter "osc_rdma_no_locks" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                             Enable optimizations available only if
>>>>> MPI_LOCK is not used.
>>>>>                             Info key of same name overrides this value.
>>>>>                    MCA osc: parameter "osc_rdma_priority" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                     MCA op: parameter "op_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level of the op framework
>>>>>                    MCA iof: parameter "iof" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> iof framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA iof: parameter "iof_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the iof framework (0 =
>>>>> no verbosity)
>>>>>                    MCA iof: parameter "iof_hnp_priority" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                    MCA iof: parameter "iof_orted_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA iof: parameter "iof_tool_priority" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                    MCA oob: parameter "oob" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> oob framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA oob: parameter "oob_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the oob framework (0 =
>>>>> no verbosity)
>>>>>                    MCA oob: parameter "oob_tcp_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbose level for the OOB tcp component
>>>>>                    MCA oob: parameter "oob_tcp_peer_limit" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                             Maximum number of peer connections to
>>>>> simultaneously maintain
>>>>>                             (-1 = infinite)
>>>>>                    MCA oob: parameter "oob_tcp_peer_retries" (current
>>>>> value:<60>, data
>>>>>                             source: default value)
>>>>>                             Number of times to try shutting down a
>>>>> connection before giving
>>>>>                             up
>>>>>                    MCA oob: parameter "oob_tcp_debug" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Enable (1) / disable (0) debugging output
>>>>> for this component
>>>>>                    MCA oob: parameter "oob_tcp_sndbuf" (current value:
>>>>> <131072>, data
>>>>>                             source: default value)
>>>>>                             TCP socket send buffering size (in bytes)
>>>>>                    MCA oob: parameter "oob_tcp_rcvbuf" (current value:
>>>>> <131072>, data
>>>>>                             source: default value)
>>>>>                             TCP socket receive buffering size (in bytes)
>>>>>                    MCA oob: parameter "oob_tcp_if_include" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             Comma-delimited list of TCP interfaces to use
>>>>>                    MCA oob: parameter "oob_tcp_if_exclude" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             Comma-delimited list of TCP interfaces to 
>>>>> exclude
>>>>>                    MCA oob: parameter "oob_tcp_connect_sleep" (current
>>>>> value:<1>, data
>>>>>                             source: default value)
>>>>>                             Enable (1) / disable (0) random sleep for
>>>>> connection wireup.
>>>>>                    MCA oob: parameter "oob_tcp_listen_mode" (current
>>>>> value:<event>, data
>>>>>                             source: default value)
>>>>>                             Mode for HNP to accept incoming connections: 
>>>>> event,
>>>>>                             listen_thread.
>>>>>                    MCA oob: parameter "oob_tcp_listen_thread_max_queue"
>>>>> (current value:
>>>>>                             <10>, data source: default value)
>>>>>                             High water mark for queued accepted socket
>>>>> list size.  Used
>>>>>                             only when listen_mode is listen_thread.
>>>>>                    MCA oob: parameter "oob_tcp_listen_thread_wait_time"
>>>>> (current value:
>>>>>                             <10>, data source: default value)
>>>>>                             Time in milliseconds to wait before actively
>>>>> checking for new
>>>>>                             connections when listen_mode is listen_thread.
>>>>>                    MCA oob: parameter "oob_tcp_static_ports" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             Static ports for daemons and procs (IPv4)
>>>>>                    MCA oob: parameter "oob_tcp_dynamic_ports" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             Range of ports to be dynamically used by
>>>>> daemons and procs
>>>>>                             (IPv4)
>>>>>                    MCA oob: parameter "oob_tcp_disable_family" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Disable IPv4 (4) or IPv6 (6)
>>>>>                    MCA oob: parameter "oob_tcp_priority" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                   MCA odls: parameter "odls_base_sigkill_timeout"
>>>>> (current value:<1>, data
>>>>>                             source: default value)
>>>>>                             Time to wait for a process to die after
>>>>> issuing a kill signal
>>>>>                             to it
>>>>>                   MCA odls: parameter "odls" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> odls framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                   MCA odls: parameter "odls_base_verbose" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the odls framework (0 =
>>>>> no verbosity)
>>>>>                   MCA odls: parameter "odls_default_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA ras: parameter "ras_base_display_alloc" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether to display the allocation after it
>>>>> is determined
>>>>>                    MCA ras: parameter "ras_base_display_devel_alloc"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Whether to display a developer-detail
>>>>> allocation after it is
>>>>>                             determined
>>>>>                    MCA ras: parameter "ras" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> ras framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA ras: parameter "ras_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the ras framework (0 =
>>>>> no verbosity)
>>>>>                    MCA ras: parameter "ras_cm_priority" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                    MCA ras: parameter "ras_loadleveler_priority"
>>>>> (current value:<90>, data
>>>>>                             source: default value)
>>>>>                             Priority of the loadleveler ras component
>>>>>                    MCA ras: parameter "ras_slurm_priority" (current
>>>>> value:<75>, data
>>>>>                             source: default value)
>>>>>                             Priority of the slurm ras component
>>>>>                  MCA rmaps: parameter "rmaps_rank_file_path" (current
>>>>> value:<none>, data
>>>>>                             source: default value, synonym of: 
>>>>> orte_rankfile)
>>>>>                             Name of the rankfile to be used for mapping
>>>>> processes (relative
>>>>>                             or absolute path)
>>>>>                  MCA rmaps: parameter "rmaps_base_schedule_policy"
>>>>> (current value:<slot>,
>>>>>                             data source: default value)
>>>>>                             Scheduling Policy for RMAPS. [slot
>>>>> (alias:core) | socket |
>>>>>                             board | node]
>>>>>                  MCA rmaps: parameter "rmaps_base_pernode" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Launch one ppn as directed
>>>>>                  MCA rmaps: parameter "rmaps_base_n_pernode" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                             Launch n procs/node
>>>>>                  MCA rmaps: parameter "rmaps_base_n_perboard" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                             Launch n procs/board
>>>>>                  MCA rmaps: parameter "rmaps_base_n_persocket" (current
>>>>> value:<-1>, data
>>>>>                             source: default value)
>>>>>                             Launch n procs/socket
>>>>>                  MCA rmaps: parameter "rmaps_base_loadbalance" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Balance total number of procs across all
>>>>> allocated nodes
>>>>>                  MCA rmaps: parameter "rmaps_base_cpus_per_proc"
>>>>> (current value:<1>, data
>>>>>                             source: default value, synonyms:
>>>>> rmaps_base_cpus_per_rank)
>>>>>                             Number of cpus to use for each rank [1-2**15
>>>>> (default=1)]
>>>>>                  MCA rmaps: parameter "rmaps_base_cpus_per_rank"
>>>>> (current value:<1>, data
>>>>>                             source: default value, synonym of:
>>>>> rmaps_base_cpus_per_proc)
>>>>>                             Number of cpus to use for each rank [1-2**15
>>>>> (default=1)]
>>>>>                  MCA rmaps: parameter "rmaps_base_stride" (current
>>>>> value:<1>, data source:
>>>>>                             default value)
>>>>>                             When binding multiple cores to a rank, the
>>>>> step size to use
>>>>>                             between cores [1-2**15 (default: 1)]
>>>>>                  MCA rmaps: parameter "rmaps_base_slot_list" (current
>>>>> value:<none>, data
>>>>>                             source: default value)
>>>>>                             List of processor IDs to bind MPI processes
>>>>> to (e.g., used in
>>>>>                             conjunction with rank files) [default=NULL]
>>>>>                  MCA rmaps: parameter "rmaps_base_no_schedule_local"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             If false, allow scheduling MPI applications
>>>>> on the same node as
>>>>>                             mpirun (default).  If true, do not schedule 
>>>>> any MPI
>>>>>                             applications on the same node as mpirun
>>>>>                  MCA rmaps: parameter "rmaps_base_no_oversubscribe"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             If true, then do not allow oversubscription
>>>>> of nodes - mpirun
>>>>>                             will return an error if there aren't enough
>>>>> nodes to launch all
>>>>>                             processes without oversubscribing
>>>>>                  MCA rmaps: parameter "rmaps_base_display_map" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Whether to display the process map after it
>>>>> is computed
>>>>>                  MCA rmaps: parameter "rmaps_base_display_devel_map"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                             Whether to display a developer-detail
>>>>> process map after it is
>>>>>                             computed
>>>>>                  MCA rmaps: parameter "rmaps" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> rmaps framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                  MCA rmaps: parameter "rmaps_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the rmaps framework (0 =
>>>>> no verbosity)
>>>>>                  MCA rmaps: parameter "rmaps_load_balance_priority"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                  MCA rmaps: parameter "rmaps_rank_file_priority"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                  MCA rmaps: parameter "rmaps_resilient_fault_grp_file"
>>>>> (current value:
>>>>>                             <none>, data source: default value)
>>>>>                             Filename that contains a description of
>>>>> fault groups for this
>>>>>                             system
>>>>>                  MCA rmaps: parameter "rmaps_resilient_priority"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                  MCA rmaps: parameter "rmaps_round_robin_priority"
>>>>> (current value:<0>,
>>>>>                             data source: default value)
>>>>>                  MCA rmaps: parameter "rmaps_seq_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                  MCA rmaps: parameter "rmaps_topo_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA rml: parameter "rml_wrapper" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             Use a Wrapper component around the selected
>>>>> RML component
>>>>>                    MCA rml: parameter "rml" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> rml framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA rml: parameter "rml_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the rml framework (0 =
>>>>> no verbosity)
>>>>>                    MCA rml: parameter "rml_oob_priority" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                 MCA routed: parameter "routed" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> routed framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                 MCA routed: parameter "routed_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the routed framework (0
>>>>> = no verbosity)
>>>>>                 MCA routed: parameter "routed_binomial_priority"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                 MCA routed: parameter "routed_cm_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                 MCA routed: parameter "routed_direct_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                 MCA routed: parameter "routed_linear_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                 MCA routed: parameter "routed_radix_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                 MCA routed: parameter "routed_slave_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA plm: parameter "plm_rsh_agent" (current value:
>>>>> <ssh : rsh>, data
>>>>>                             source: default value, deprecated, synonym
>>>>> of: orte_rsh_agent)
>>>>>                             The command used to launch executables on 
>>>>> remote nodes
>>>>>                             (typically either "ssh" or "rsh")
>>>>>                    MCA plm: parameter "plm_rsh_assume_same_shell"
>>>>> (current value:<1>, data
>>>>>                             source: default value, deprecated, synonym of:
>>>>>                             orte_assume_same_shell)
>>>>>                             If set to 1, assume that the shell on the
>>>>> remote node is the
>>>>>                             same as the shell on the local node.
>>>>> Otherwise, probe for what
>>>>>                             the remote shell [default: 1]
>>>>>                    MCA plm: parameter "plm" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> plm framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA plm: parameter "plm_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the plm framework (0 =
>>>>> no verbosity)
>>>>>                    MCA plm: parameter "plm_rsh_num_concurrent" (current
>>>>> value:<128>, data
>>>>>                             source: default value)
>>>>>                             How many plm_rsh_agent instances to invoke
>>>>> concurrently (must
>>>>>                             be>   0)
>>>>>                    MCA plm: parameter "plm_rsh_force_rsh" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                             Force the launcher to always use rsh
>>>>>                    MCA plm: parameter "plm_rsh_disable_qrsh" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Disable the launcher to use qrsh when under
>>>>> the SGE parallel
>>>>>                             environment
>>>>>                    MCA plm: parameter "plm_rsh_daemonize_qrsh" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Daemonize the orted under the SGE parallel 
>>>>> environment
>>>>>                    MCA plm: parameter "plm_rsh_disable_llspawn" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Disable the use of llspawn when under the 
>>>>> LoadLeveler
>>>>>                             environment
>>>>>                    MCA plm: parameter "plm_rsh_daemonize_llspawn"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>                             Daemonize the orted when under the
>>>>> LoadLeveler environment
>>>>>                    MCA plm: parameter "plm_rsh_priority" (current value:
>>>>> <10>, data source:
>>>>>                             default value)
>>>>>                             Priority of the rsh plm component
>>>>>                    MCA plm: parameter "plm_rsh_delay" (current value:
>>>>> <1>, data source:
>>>>>                             default value)
>>>>>                             Delay (in seconds) between invocations of
>>>>> the remote agent, but
>>>>>                             only used when the "debug" MCA parameter is
>>>>> true, or the
>>>>>                             top-level MCA debugging is enabled
>>>>> (otherwise this value is
>>>>>                             ignored)
>>>>>                    MCA plm: parameter "plm_rsh_tree_spawn" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             If set to 1, launch via a tree-based topology
>>>>>                    MCA plm: parameter "plm_slurm_args" (current value:
>>>>> <none>, data source:
>>>>>                             default value)
>>>>>                             Custom arguments to srun
>>>>>                    MCA plm: parameter "plm_slurm_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                  MCA filem: parameter "filem" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Which Filem component to use (empty = 
>>>>> auto-select)
>>>>>                  MCA filem: parameter "filem_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the filem framework (0 =
>>>>> no verbosity)
>>>>>                  MCA filem: parameter "filem_rsh_priority" (current
>>>>> value:<20>, data
>>>>>                             source: default value)
>>>>>                             Priority of the FILEM rsh component
>>>>>                  MCA filem: parameter "filem_rsh_verbose" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                             Verbose level for the FILEM rsh component
>>>>>                  MCA filem: parameter "filem_rsh_rcp" (current value:
>>>>> <scp>, data source:
>>>>>                             default value)
>>>>>                             The rsh cp command for the FILEM rsh component
>>>>>                  MCA filem: parameter "filem_rsh_cp" (current value:
>>>>> <cp>, data source:
>>>>>                             default value)
>>>>>                             The Unix cp command for the FILEM rsh 
>>>>> component
>>>>>                  MCA filem: parameter "filem_rsh_rsh" (current value:
>>>>> <ssh>, data source:
>>>>>                             default value)
>>>>>                             The remote shell command for the FILEM rsh 
>>>>> component
>>>>>                  MCA filem: parameter "filem_rsh_max_incomming" (current
>>>>> value:<10>, data
>>>>>                             source: default value)
>>>>>                             Maximum number of incomming connections (0 = 
>>>>> any)
>>>>>                  MCA filem: parameter "filem_rsh_max_outgoing" (current
>>>>> value:<10>, data
>>>>>                             source: default value)
>>>>>                             Maximum number of out going connections (0 = 
>>>>> any)
>>>>>                 MCA errmgr: parameter "errmgr" (current value:<none>,
>>>>> data source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> errmgr framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                 MCA errmgr: parameter "errmgr_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the errmgr framework (0
>>>>> = no verbosity)
>>>>>                 MCA errmgr: parameter "errmgr_default_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA ess: parameter "ess" (current value:<none>, data
>>>>> source: default
>>>>>                             value)
>>>>>                             Default selection set of components for the
>>>>> ess framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                    MCA ess: parameter "ess_base_verbose" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                             Verbosity level for the ess framework (0 =
>>>>> no verbosity)
>>>>>                    MCA ess: parameter "ess_env_priority" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                    MCA ess: parameter "ess_hnp_priority" (current value:
>>>>> <0>, data source:
>>>>>                             default value)
>>>>>                    MCA ess: parameter "ess_singleton_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA ess: parameter "ess_slave_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA ess: parameter "ess_slurm_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA ess: parameter "ess_slurmd_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                    MCA ess: parameter "ess_tool_priority" (current
>>>>> value:<0>, data source:
>>>>>                             default value)
>>>>>                MCA grpcomm: parameter "grpcomm" (current value:<none>,
>>>>> data source:
>>>>>                             default value)
>>>>>                             Default selection set of components for the
>>>>> grpcomm framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>                MCA grpcomm: parameter "grpcomm_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the grpcomm framework (0
>>>>> = no verbosity)
>>>>>                MCA grpcomm: parameter "grpcomm_bad_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                MCA grpcomm: parameter "grpcomm_basic_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                MCA grpcomm: parameter "grpcomm_hier_priority" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>               MCA notifier: parameter "notifier_threshold_severity"
>>>>> (current value:
>>>>>                             <critical>, data source: default value)
>>>>>                             Report all events at or above this severity
>>>>> [default: critical]
>>>>>               MCA notifier: parameter "notifier" (current value:<none>,
>>>>> data source:
>>>>>                             default value)
>>>>>                             Default selection set of components for the
>>>>> notifier framework
>>>>>                             (<none>   means use all components that can 
>>>>> be found)
>>>>>               MCA notifier: parameter "notifier_base_verbose" (current
>>>>> value:<0>, data
>>>>>                             source: default value)
>>>>>                             Verbosity level for the notifier framework
>>>>> (0 = no verbosity)
>>>>>               MCA notifier: parameter "notifier_command_cmd" (current
>>>>> value:</sbin/initlog
>>>>>                             -f $s -n "Open MPI" -s "$S: $m (errorcode:
>>>>> $e)">, data source:
>>>>>                             default value)
>>>>>                             Command to execute, with substitution.  $s =
>>>>> integer severity;
>>>>>                             $S = string severity; $e = integer error
>>>>> code; $m = string
>>>>>                             message
>>>>>               MCA notifier: parameter "notifier_command_timeout"
>>>>> (current value:<30>, data
>>>>>                             source: default value)
>>>>>                             Timeout (in seconds) of the command
>>>>>               MCA notifier: parameter "notifier_command_priority"
>>>>> (current value:<10>,
>>>>>                             data source: default value)
>>>>>                             Priority of this component
>>>>>               MCA notifier: parameter "notifier_syslog_priority"
>>>>> (current value:<0>, data
>>>>>                             source: default value)
>>>>>
>>>>> ====================================================================================================
>>>>>
>>>>> output of cat /proc/cpuinfo
>>>>> processor     : 0
>>>>> vendor_id     : AuthenticAMD
>>>>> cpu family    : 15
>>>>> model         : 75
>>>>> model name    : AMD Athlon(tm) 64 X2 Dual Core Processor 3800+
>>>>> stepping      : 2
>>>>> cpu MHz               : 1002.094
>>>>> cache size    : 512 KB
>>>>> physical id   : 0
>>>>> siblings      : 2
>>>>> core id               : 0
>>>>> cpu cores     : 2
>>>>> fpu           : yes
>>>>> fpu_exception : yes
>>>>> cpuid level   : 1
>>>>> wp            : yes
>>>>> flags         : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
>>>>> cmov
>>>>> pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext
>>>>> 3dnow pni cx16
>>>>> bogomips      : 2003.90
>>>>> TLB size      : 1088 4K pages
>>>>> clflush size  : 64
>>>>> cache_alignment       : 64
>>>>> address sizes : 40 bits physical, 48 bits virtual
>>>>> power management: ts fid vid ttp [4] [5]
>>>>>
>>>>> processor     : 1
>>>>> vendor_id     : AuthenticAMD
>>>>> cpu family    : 15
>>>>> model         : 75
>>>>> model name    : AMD Athlon(tm) 64 X2 Dual Core Processor 3800+
>>>>> stepping      : 2
>>>>> cpu MHz               : 1002.094
>>>>> cache size    : 512 KB
>>>>> physical id   : 0
>>>>> siblings      : 2
>>>>> core id               : 1
>>>>> cpu cores     : 2
>>>>> fpu           : yes
>>>>> fpu_exception : yes
>>>>> cpuid level   : 1
>>>>> wp            : yes
>>>>> flags         : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
>>>>> cmov
>>>>> pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext
>>>>> 3dnow pni cx16
>>>>> bogomips      : 2003.90
>>>>> TLB size      : 1088 4K pages
>>>>> clflush size  : 64
>>>>> cache_alignment       : 64
>>>>> address sizes : 40 bits physical, 48 bits virtual
>>>>> power management: ts fid vid ttp [4] [5]
>>>>>
>>>>>
>>>>> ====================================================================================================
>>>>> output of ifconfig -a from a compute node
>>>>> eth0      Link encap:Ethernet  HWaddr 00:18:F3:3F:84:A1
>>>>>             inet addr:192.168.0.2  Bcast:192.168.0.255  Mask:255.255.255.0
>>>>>             inet6 addr: fe80::218:f3ff:fe3f:84a1/64 Scope:Link
>>>>>             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>             RX packets:2006 errors:0 dropped:0 overruns:0 frame:0
>>>>>             TX packets:2064 errors:0 dropped:0 overruns:0 carrier:0
>>>>>             collisions:0 txqueuelen:1000
>>>>>             RX bytes:242685 (236.9 KiB)  TX bytes:0 (0.0 b)
>>>>>             Interrupt:11 Base address:0x8000
>>>>>
>>>>> lo        Link encap:Local Loopback
>>>>>             inet addr:127.0.0.1  Mask:255.0.0.0
>>>>>             inet6 addr: ::1/128 Scope:Host
>>>>>             UP LOOPBACK RUNNING  MTU:16436  Metric:1
>>>>>             RX packets:60 errors:0 dropped:0 overruns:0 frame:0
>>>>>             TX packets:60 errors:0 dropped:0 overruns:0 carrier:0
>>>>>             collisions:0 txqueuelen:0
>>>>>             RX bytes:4440 (4.3 KiB)  TX bytes:4440 (4.3 KiB)
>>>>>
>>>>> sit0      Link encap:IPv6-in-IPv4
>>>>>             NOARP  MTU:1480  Metric:1
>>>>>             RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>             TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>>             collisions:0 txqueuelen:0
>>>>>             RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
>>>>>
>>>>>
>>>>> ====================================================================================================
>>>>> output of ifconfig -a from the login node, where I run mpirun
>>>>>
>>>>> eth0      Link encap:Ethernet  HWaddr 00:18:F3:51:B3:6E
>>>>>             inet addr:192.168.0.1  Bcast:192.168.0.255  Mask:255.255.255.0
>>>>>             inet6 addr: fe80::218:f3ff:fe51:b36e/64 Scope:Link
>>>>>             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>             RX packets:7180758 errors:0 dropped:0 overruns:0 frame:0
>>>>>             TX packets:4989496 errors:0 dropped:0 overruns:0 carrier:0
>>>>>             collisions:0 txqueuelen:1000
>>>>>             RX bytes:6045614452 (5.6 GiB)  TX bytes:0 (0.0 b)
>>>>>             Interrupt:201 Base address:0xe000
>>>>>
>>>>> eth1      Link encap:Ethernet  HWaddr 00:01:02:13:AA:3C
>>>>>             inet addr:137.204.66.188  Bcast:137.204.66.255  
>>>>> Mask:255.255.255.0
>>>>>             inet6 addr: fe80::201:2ff:fe13:aa3c/64 Scope:Link
>>>>>             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>             RX packets:4750212 errors:0 dropped:0 overruns:0 frame:0
>>>>>             TX packets:405027 errors:0 dropped:0 overruns:0 carrier:0
>>>>>             collisions:0 txqueuelen:1000
>>>>>             RX bytes:629146679 (600.0 MiB)  TX bytes:332118265 (316.7 MiB)
>>>>>             Interrupt:177 Base address:0x9c00
>>>>>
>>>>> lo        Link encap:Local Loopback
>>>>>             inet addr:127.0.0.1  Mask:255.0.0.0
>>>>>             inet6 addr: ::1/128 Scope:Host
>>>>>             UP LOOPBACK RUNNING  MTU:16436  Metric:1
>>>>>             RX packets:288455 errors:0 dropped:0 overruns:0 frame:0
>>>>>             TX packets:288455 errors:0 dropped:0 overruns:0 carrier:0
>>>>>             collisions:0 txqueuelen:0
>>>>>             RX bytes:35908038 (34.2 MiB)  TX bytes:35908038 (34.2 MiB)
>>>>>
>>>>> sit0      Link encap:IPv6-in-IPv4
>>>>>             NOARP  MTU:1480  Metric:1
>>>>>             RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>             TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>>             collisions:0 txqueuelen:0
>>>>>             RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
>>>>>
>>>>> ====================================================================================================
>>>>> output of mpirun --bynode --hostfile my_hostfile.txt --tag-output
>>>>> ompi_info -v ompi full --parsable
>>>>>
>>>>> [1,0]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,0]<stdout>:ompi:version:full:1.6
>>>>> [1,0]<stdout>:ompi:version:svn:r26429
>>>>> [1,0]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,0]<stdout>:orte:version:full:1.6
>>>>> [1,0]<stdout>:orte:version:svn:r26429
>>>>> [1,0]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,0]<stdout>:opal:version:full:1.6
>>>>> [1,0]<stdout>:opal:version:svn:r26429
>>>>> [1,0]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,0]<stdout>:mpi-api:version:full:2.1
>>>>> [1,0]<stdout>:ident:1.6
>>>>> [1,6]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,6]<stdout>:ompi:version:full:1.6
>>>>> [1,6]<stdout>:ompi:version:svn:r26429
>>>>> [1,6]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,6]<stdout>:orte:version:full:1.6
>>>>> [1,6]<stdout>:orte:version:svn:r26429
>>>>> [1,6]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,6]<stdout>:opal:version:full:1.6
>>>>> [1,6]<stdout>:opal:version:svn:r26429
>>>>> [1,6]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,6]<stdout>:mpi-api:version:full:2.1
>>>>> [1,6]<stdout>:ident:1.6
>>>>> [1,9]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,10]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,3]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,3]<stdout>:ompi:version:full:1.6
>>>>> [1,3]<stdout>:ompi:version:svn:r26429
>>>>> [1,3]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,3]<stdout>:orte:version:full:1.6
>>>>> [1,3]<stdout>:orte:version:svn:r26429
>>>>> [1,3]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,3]<stdout>:opal:version:full:1.6
>>>>> [1,3]<stdout>:opal:version:svn:r26429
>>>>> [1,3]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,3]<stdout>:mpi-api:version:full:2.1
>>>>> [1,3]<stdout>:ident:1.6
>>>>> [1,4]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,4]<stdout>:ompi:version:full:1.6
>>>>> [1,4]<stdout>:ompi:version:svn:r26429
>>>>> [1,4]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,4]<stdout>:orte:version:full:1.6
>>>>> [1,4]<stdout>:orte:version:svn:r26429
>>>>> [1,4]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,4]<stdout>:opal:version:full:1.6
>>>>> [1,9]<stdout>:ompi:version:full:1.6
>>>>> [1,4]<stdout>:opal:version:svn:r26429
>>>>> [1,4]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,4]<stdout>:mpi-api:version:full:2.1
>>>>> [1,4]<stdout>:ident:1.6
>>>>> [1,9]<stdout>:ompi:version:svn:r26429
>>>>> [1,10]<stdout>:ompi:version:full:1.6
>>>>> [1,9]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,10]<stdout>:ompi:version:svn:r26429
>>>>> [1,9]<stdout>:orte:version:full:1.6
>>>>> [1,10]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,9]<stdout>:orte:version:svn:r26429
>>>>> [1,10]<stdout>:orte:version:full:1.6
>>>>> [1,10]<stdout>:orte:version:svn:r26429
>>>>> [1,9]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,10]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,9]<stdout>:opal:version:full:1.6
>>>>> [1,10]<stdout>:opal:version:full:1.6
>>>>> [1,9]<stdout>:opal:version:svn:r26429
>>>>> [1,10]<stdout>:opal:version:svn:r26429
>>>>> [1,9]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,10]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,9]<stdout>:mpi-api:version:full:2.1
>>>>> [1,9]<stdout>:ident:1.6
>>>>> [1,10]<stdout>:mpi-api:version:full:2.1
>>>>> [1,10]<stdout>:ident:1.6
>>>>> [1,2]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,2]<stdout>:ompi:version:full:1.6
>>>>> [1,2]<stdout>:ompi:version:svn:r26429
>>>>> [1,2]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,2]<stdout>:orte:version:full:1.6
>>>>> [1,2]<stdout>:orte:version:svn:r26429
>>>>> [1,2]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,2]<stdout>:opal:version:full:1.6
>>>>> [1,2]<stdout>:opal:version:svn:r26429
>>>>> [1,2]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,2]<stdout>:mpi-api:version:full:2.1
>>>>> [1,2]<stdout>:ident:1.6
>>>>> [1,8]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,8]<stdout>:ompi:version:full:1.6
>>>>> [1,8]<stdout>:ompi:version:svn:r26429
>>>>> [1,8]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,8]<stdout>:orte:version:full:1.6
>>>>> [1,8]<stdout>:orte:version:svn:r26429
>>>>> [1,8]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,8]<stdout>:opal:version:full:1.6
>>>>> [1,8]<stdout>:opal:version:svn:r26429
>>>>> [1,8]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,8]<stdout>:mpi-api:version:full:2.1
>>>>> [1,8]<stdout>:ident:1.6
>>>>> [1,11]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,11]<stdout>:ompi:version:full:1.6
>>>>> [1,11]<stdout>:ompi:version:svn:r26429
>>>>> [1,11]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,11]<stdout>:orte:version:full:1.6
>>>>> [1,11]<stdout>:orte:version:svn:r26429
>>>>> [1,11]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,11]<stdout>:opal:version:full:1.6
>>>>> [1,11]<stdout>:opal:version:svn:r26429
>>>>> [1,11]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,11]<stdout>:mpi-api:version:full:2.1
>>>>> [1,11]<stdout>:ident:1.6
>>>>> [1,5]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,5]<stdout>:ompi:version:full:1.6
>>>>> [1,5]<stdout>:ompi:version:svn:r26429
>>>>> [1,5]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,5]<stdout>:orte:version:full:1.6
>>>>> [1,5]<stdout>:orte:version:svn:r26429
>>>>> [1,5]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,5]<stdout>:opal:version:full:1.6
>>>>> [1,5]<stdout>:opal:version:svn:r26429
>>>>> [1,5]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,5]<stdout>:mpi-api:version:full:2.1
>>>>> [1,5]<stdout>:ident:1.6
>>>>> [1,1]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,7]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>> [1,7]<stdout>:ompi:version:full:1.6
>>>>> [1,7]<stdout>:ompi:version:svn:r26429
>>>>> [1,7]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,7]<stdout>:orte:version:full:1.6
>>>>> [1,7]<stdout>:orte:version:svn:r26429
>>>>> [1,7]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,7]<stdout>:opal:version:full:1.6
>>>>> [1,7]<stdout>:opal:version:svn:r26429
>>>>> [1,7]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,7]<stdout>:mpi-api:version:full:2.1
>>>>> [1,7]<stdout>:ident:1.6
>>>>> [1,1]<stdout>:ompi:version:full:1.6
>>>>> [1,1]<stdout>:ompi:version:svn:r26429
>>>>> [1,1]<stdout>:ompi:version:release_date:May 10, 2012
>>>>> [1,1]<stdout>:orte:version:full:1.6
>>>>> [1,1]<stdout>:orte:version:svn:r26429
>>>>> [1,1]<stdout>:orte:version:release_date:May 10, 2012
>>>>> [1,1]<stdout>:opal:version:full:1.6
>>>>> [1,1]<stdout>:opal:version:svn:r26429
>>>>> [1,1]<stdout>:opal:version:release_date:May 10, 2012
>>>>> [1,1]<stdout>:mpi-api:version:full:2.1
>>>>> [1,1]<stdout>:ident:1.6
>>>>> _______________________________________________
>>>>> users mailing list
>>>>> us...@open-mpi.org
>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> _______________________________________________
>>>> users mailing list
>>>> us...@open-mpi.org
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>
>>>> End of users Digest, Vol 2339, Issue 5
>>>> **************************************
>>>
>>> ------------------------------
>>>
>>> _______________________________________________
>>> users mailing list
>>> us...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>> End of users Digest, Vol 2340, Issue 1
>>> **************************************
>> _______________________________________________
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> ------------------------------
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> End of users Digest, Vol 2343, Issue 2
> **************************************

Reply via email to