This is something you probably need to work on with your sys admin - it sounds 
like there is something unreliable in your network, and that's usually a 
somewhat hard thing to diagnose.


On Sep 3, 2012, at 2:49 PM, Andrea Negri <negri.an...@gmail.com> wrote:

> In which ways can I check the failure of the ethernet connections?
> 
> 2012/9/3  <users-requ...@open-mpi.org>:
>> Send users mailing list submissions to
>>        us...@open-mpi.org
>> 
>> To subscribe or unsubscribe via the World Wide Web, visit
>>        http://www.open-mpi.org/mailman/listinfo.cgi/users
>> or, via email, send a message with subject or body 'help' to
>>        users-requ...@open-mpi.org
>> 
>> You can reach the person managing the list at
>>        users-ow...@open-mpi.org
>> 
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of users digest..."
>> 
>> 
>> Today's Topics:
>> 
>>   1. -hostfile ignored in 1.6.1 / SGE integration broken (Reuti)
>>   2. Re: some mpi processes "disappear" on a cluster of        servers
>>      (Ralph Castain)
>> 
>> 
>> ----------------------------------------------------------------------
>> 
>> Message: 1
>> Date: Mon, 3 Sep 2012 23:12:14 +0200
>> From: Reuti <re...@staff.uni-marburg.de>
>> Subject: [OMPI users] -hostfile ignored in 1.6.1 / SGE integration
>>        broken
>> To: Open MPI Users <us...@open-mpi.org>
>> Message-ID:
>>        <b8136f9f-da01-4f04-a9f2-0f72d2b7a...@staff.uni-marburg.de>
>> Content-Type: text/plain; charset=us-ascii
>> 
>> Hi all,
>> 
>> I just compiled Open MPI 1.6.1 and before digging any deeper: does anyone 
>> else notice, that the command:
>> 
>> $ mpiexec -n 4 -machinefile mymachines ./mpihello
>> 
>> will ignore the argument "-machinefile mymachines" and use the file 
>> "openmpi-default-hostfile" instead all the time?
>> 
>> ==
>> 
>> SGE issue
>> 
>> I usually don't install new versions instantly, so I only noticed right now, 
>> that in 1.4.5 I get a wrong allocation inside SGE (always one process less 
>> than requested with `qsub -pe orted N ...`. This I tried only, as with 1.6.1 
>> I get:
>> 
>> --------------------------------------------------------------------------
>> There are no nodes allocated to this job.
>> --------------------------------------------------------------------------
>> 
>> all the time.
>> 
>> ==
>> 
>> I configured with:
>> 
>> ./configure --prefix=$HOME/local/... --enable-static --disable-shared 
>> --with-sge
>> 
>> and adjusted my PATHs accordingly (at least: I hope so).
>> 
>> -- Reuti
>> 
>> 
>> ------------------------------
>> 
>> Message: 2
>> Date: Mon, 3 Sep 2012 14:32:48 -0700
>> From: Ralph Castain <r...@open-mpi.org>
>> Subject: Re: [OMPI users] some mpi processes "disappear" on a cluster
>>        of      servers
>> To: Open MPI Users <us...@open-mpi.org>
>> Message-ID: <c04139de-10b2-42b0-935d-40b104936...@open-mpi.org>
>> Content-Type: text/plain; charset=us-ascii
>> 
>> It looks to me like the network is losing connections - your error messages 
>> all state "no route to host", which implies that the network interface 
>> dropped out.
>> 
>> On Sep 3, 2012, at 1:39 PM, Andrea Negri <negri.an...@gmail.com> wrote:
>> 
>>> I have asked to my admin and he said that no log messages were present
>>> in /var/log, apart my login on the compute node.
>>> No killed processes, no full stack errors, the memory is ok, 1GB is
>>> used and 2GB are free.
>>> Actually I don't know what kind of problem coud be, does someone have
>>> ideas? Or at least a suspect?
>>> 
>>> Please, don't let me alone!
>>> 
>>> Sorry for the trouble with the mail
>>> 
>>> 2012/9/1  <users-requ...@open-mpi.org>:
>>>> Send users mailing list submissions to
>>>>       us...@open-mpi.org
>>>> 
>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>       http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>> or, via email, send a message with subject or body 'help' to
>>>>       users-requ...@open-mpi.org
>>>> 
>>>> You can reach the person managing the list at
>>>>       users-ow...@open-mpi.org
>>>> 
>>>> When replying, please edit your Subject line so it is more specific
>>>> than "Re: Contents of users digest..."
>>>> 
>>>> 
>>>> Today's Topics:
>>>> 
>>>>  1. Re: some mpi processes "disappear" on a cluster of        servers
>>>>     (John Hearns)
>>>>  2. Re: users Digest, Vol 2339, Issue 5 (Andrea Negri)
>>>> 
>>>> 
>>>> ----------------------------------------------------------------------
>>>> 
>>>> Message: 1
>>>> Date: Sat, 1 Sep 2012 08:48:56 +0100
>>>> From: John Hearns <hear...@googlemail.com>
>>>> Subject: Re: [OMPI users] some mpi processes "disappear" on a cluster
>>>>       of      servers
>>>> To: Open MPI Users <us...@open-mpi.org>
>>>> Message-ID:
>>>>       <capqne2wo3bgefjiyfums6yquoduhjj75zjoeeyjrdq60fmz...@mail.gmail.com>
>>>> Content-Type: text/plain; charset=ISO-8859-1
>>>> 
>>>> Apologies, I have not taken the time to read your comprehensive 
>>>> diagnostics!
>>>> 
>>>> As Gus says, this sounds like a memory problem.
>>>> My suspicion would be the kernel Out Of Memory (OOM) killer.
>>>> Log into those nodes (or ask your systems manager to do this). Look
>>>> closely at /var/log/messages where there will be notifications when
>>>> the OOM Killer kicks in and - well - kills large memory processes!
>>>> Grep for "killed process" in /var/log/messages
>>>> 
>>>> http://linux-mm.org/OOM_Killer
>>>> 
>>>> 
>>>> ------------------------------
>>>> 
>>>> Message: 2
>>>> Date: Sat, 1 Sep 2012 11:50:59 +0200
>>>> From: Andrea Negri <negri.an...@gmail.com>
>>>> Subject: Re: [OMPI users] users Digest, Vol 2339, Issue 5
>>>> To: us...@open-mpi.org
>>>> Message-ID:
>>>>       <CAPUxaiQ4RFqSK1kz7fM7U9XRxjQh8N+=98paqm2ikvr7bv-...@mail.gmail.com>
>>>> Content-Type: text/plain; charset=ISO-8859-1
>>>> 
>>>> Hi, Gus and John,
>>>> 
>>>> my code (zeusmp2) is a F77 code ported in F95, and the very first time
>>>> I have launched it, the processed disappears almost immediately.
>>>> I checked the code with valgrind and some unallocated arrays were
>>>> passed wrongly to some subroutines.
>>>> After having corrected this bug, the code runs for a while and after
>>>> occour all the stuff described in my first post.
>>>> However, the code still performs a lot of main temporal cycle before
>>>> "die" (I don't know if thies piece of information is useful).
>>>> 
>>>> Now I'm going to check the memory usage, (I also have a lot of unused
>>>> variables in this pretty large code, maybe I shoud comment them).
>>>> 
>>>> uname -a returns
>>>> Linux cloud 2.6.9-42.0.3.ELsmp #1 SMP Thu Oct 5 16:29:37 CDT 2006
>>>> x86_64 x86_64 x86_64 GNU/Linux
>>>> 
>>>> ulimit -a returns:
>>>> core file size                        (blocks, -c) 0
>>>> data seg size                       (kbytes, -d) unlimited
>>>> file size                               (blocks, -f) unlimited
>>>> pending signals                    (-i) 1024
>>>> max locked memory             (kbytes, -l) 32
>>>> max memory size                (kbytes, -m) unlimited
>>>> open files                           (-n) 1024
>>>> pipe size                            (512 bytes, -p) 8
>>>> POSIX message queues     (bytes, -q) 819200
>>>> stack size                           (kbytes, -s) 10240
>>>> cpu time                             (seconds, -t) unlimited
>>>> max user processes             (-u) 36864
>>>> virtual memory                   (kbytes, -v) unlimited
>>>> file locks                            (-x) unlimited
>>>> 
>>>> 
>>>> I can log on the logins nodes, but unfortunately the command ls
>>>> /var/log/messages return
>>>> acpid               cron.4              messages.3         secure.4
>>>> anaconda.log        cups                messages.4         spooler
>>>> anaconda.syslog     dmesg               mpi_uninstall.log  spooler.1
>>>> anaconda.xlog       gdm                 ppp                spooler.2
>>>> audit               httpd               prelink.log        spooler.3
>>>> boot.log            itac_uninstall.log  rpmpkgs            spooler.4
>>>> boot.log.1          lastlog             rpmpkgs.1          vbox
>>>> boot.log.2          mail                rpmpkgs.2          wtmp
>>>> boot.log.3          maillog             rpmpkgs.3          wtmp.1
>>>> boot.log.4          maillog.1           rpmpkgs.4          Xorg.0.log
>>>> cmkl_install.log    maillog.2           samba              Xorg.0.log.old
>>>> cmkl_uninstall.log  maillog.3           scrollkeeper.log   yum.log
>>>> cron                maillog.4           secure             yum.log.1
>>>> cron.1              messages            secure.1
>>>> cron.2              messages.1          secure.2
>>>> cron.3              messages.2          secure.3
>>>> 
>>>> so, the log should be in some of these files (I don't have read
>>>> permission on these files). I'll contact the admin for that.
>>>> 
>>>> 2012/9/1  <users-requ...@open-mpi.org>:
>>>>> Send users mailing list submissions to
>>>>>       us...@open-mpi.org
>>>>> 
>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>>       http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>> or, via email, send a message with subject or body 'help' to
>>>>>       users-requ...@open-mpi.org
>>>>> 
>>>>> You can reach the person managing the list at
>>>>>       users-ow...@open-mpi.org
>>>>> 
>>>>> When replying, please edit your Subject line so it is more specific
>>>>> than "Re: Contents of users digest..."
>>>>> 
>>>>> 
>>>>> Today's Topics:
>>>>> 
>>>>>  1. Re: some mpi processes "disappear" on a cluster of        servers
>>>>>     (Gus Correa)
>>>>> 
>>>>> 
>>>>> ----------------------------------------------------------------------
>>>>> 
>>>>> Message: 1
>>>>> Date: Fri, 31 Aug 2012 20:11:41 -0400
>>>>> From: Gus Correa <g...@ldeo.columbia.edu>
>>>>> Subject: Re: [OMPI users] some mpi processes "disappear" on a cluster
>>>>>       of      servers
>>>>> To: Open MPI Users <us...@open-mpi.org>
>>>>> Message-ID: <504152bd.3000...@ldeo.columbia.edu>
>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>> 
>>>>> Hi Andrea
>>>>> 
>>>>> I would guess this is a memory problem.
>>>>> Do you know how much memory each node has?
>>>>> Do you know the memory that
>>>>> each MPI process in the CFD code requires?
>>>>> If the program starts swapping/paging into disk, because of
>>>>> low memory, those interesting things that you described can happen.
>>>>> 
>>>>> I would login to the compute nodes and monitor the
>>>>> amount of memory being used with "top" right after the program
>>>>> starts to run.  If there is a pattern of which node tends to fail,
>>>>> track login to that fail-prone node and monitor it.
>>>>> 
>>>>> Alternatively, if you cluster is running Ganglia,
>>>>> you can see the memory use graphically,
>>>>> in the Ganglia web page in a web browser.
>>>>> 
>>>>> If your cluster
>>>>> doesn't allow direct user logins to compute nodes,
>>>>> you can ask the system administrator to do this for you.
>>>>> 
>>>>> It may well be that the code has a memory leak, or that
>>>>> it has a memory request spike, which may not be caught by
>>>>> OpenMPI.
>>>>> [Jeff and Ralph will probably correct me soon for
>>>>> saying this, and I know the OpenMPI safeguards against
>>>>> process misbehavior are great, but ...]
>>>>> 
>>>>> Anyway, we had several codes here that would make a node go south
>>>>> because of either type of memory problem, and subsequently the
>>>>> program would die, or the other processes in other nodes would
>>>>> continue "running" [i.e. mostly waiting for MPI calls to the
>>>>> dead node that would never return] as you described.
>>>>> 
>>>>> If the problem is benign, i.e., if it is just that the
>>>>> memory-per-processor is not large enough to run in 10 processors,
>>>>> you can get around it by running in, say, 20 processors.
>>>>> 
>>>>> Yet another issue that you may check is the stacksize in the
>>>>> compute nodes.  Many codes require a large stacksize, i.e.,
>>>>> they create large arrays in subroutines, etc, and
>>>>> the default stacksize in standard Linux distributions
>>>>> may not be as large as needed.
>>>>> We use ulimited stacksize in our compute nodes.
>>>>> 
>>>>> You can ask the system administrator to check this for you,
>>>>> and perhaps change it in /etc/security/limits.conf to make it
>>>>> unlimited or at least larger than the default.
>>>>> The Linux shell command "ulimit -a" [bash] or
>>>>> "limit" [tcsh] will tell what the limits are.
>>>>> 
>>>>> I hope this helps,
>>>>> Gus Correa
>>>>> 
>>>>> On 08/31/2012 07:15 PM, Andrea Negri wrote:
>>>>>> Hi, I have been in trouble for a year.
>>>>>> 
>>>>>> I run a pure MPI (no openMP) Fortran fluid dynamical code on a cluster
>>>>>> of server, and I obtain a strange behaviour by running the code on
>>>>>> multiple nodes.
>>>>>> The cluster is formed by 16 pc (1 pc is a node) with a dual core 
>>>>>> processor.
>>>>>> Basically, I'm able to run the code from the login node with the command:
>>>>>> mpirun  --mca btl_base_verbose 100 --mca backtrace_base_verbose 100
>>>>>> --mca memory_base_verbose 100 --mca sysinfo_base_verbose 100  -nolocal
>>>>>> -hostfile ./host_file -n 10  ./zeusmp2.x>>  zmp_errors 2>&1
>>>>>> by selecting one process per core (i.e. in this case I use 5 nodes)
>>>>>> 
>>>>>> The code starts, and it runs correctely for some time.
>>>>>> After that, an entire node (sometimes two) "disappears" and it cannot
>>>>>> be reached with the ssh command, which returns: No route to host.
>>>>>> Sometimes the node is still reachable, but the two processes that was
>>>>>> running on the node are disappears.
>>>>>> In addition, on the other nodes, the others processes are still running.
>>>>>> 
>>>>>> If I have a look on the output and error file of mpirun, the following
>>>>>> error is present: [btl_tcp_frag.c:215:mca_btl_tcp_frag_recv]
>>>>>> mca_btl_tcp_frag_recv: readv failed: No route to host (113)
>>>>>> 
>>>>>> PS: I'm not the admin of the cluster, I've installed the gcc and
>>>>>> openmpi on my own because the complier aviable on that machine are 8
>>>>>> years old.
>>>>>> 
>>>>>> 
>>>>>> I post here some information, if you want other info, you have only to
>>>>>> tell me which command I have to type on the bash and I will
>>>>>> immediately reply.
>>>>>> 
>>>>>> 
>>>>>> complier: gcc 4.7 (which was also used to compile openmpi)
>>>>>> openmpi version: 1.6
>>>>>> 
>>>>>> output of "ompi_info --all" from the node where I launch mpirun (which
>>>>>> is also the login node of the cluster)
>>>>>> 
>>>>>>  Package: Open MPI and...@cloud.bo.astro.it Distribution
>>>>>>                Open MPI: 1.6
>>>>>>   Open MPI SVN revision: r26429
>>>>>>   Open MPI release date: May 10, 2012
>>>>>>                Open RTE: 1.6
>>>>>>   Open RTE SVN revision: r26429
>>>>>>   Open RTE release date: May 10, 2012
>>>>>>                    OPAL: 1.6
>>>>>>       OPAL SVN revision: r26429
>>>>>>       OPAL release date: May 10, 2012
>>>>>>                 MPI API: 2.1
>>>>>>            Ident string: 1.6
>>>>>>           MCA backtrace: execinfo (MCA v2.0, API v2.0, Component v1.6)
>>>>>>              MCA memory: linux (MCA v2.0, API v2.0, Component v1.6)
>>>>>>           MCA paffinity: hwloc (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA carto: auto_detect (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA carto: file (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA shmem: mmap (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA shmem: posix (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA shmem: sysv (MCA v2.0, API v2.0, Component v1.6)
>>>>>>           MCA maffinity: first_use (MCA v2.0, API v2.0, Component v1.6)
>>>>>>           MCA maffinity: hwloc (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA timer: linux (MCA v2.0, API v2.0, Component v1.6)
>>>>>>         MCA installdirs: env (MCA v2.0, API v2.0, Component v1.6)
>>>>>>         MCA installdirs: config (MCA v2.0, API v2.0, Component v1.6)
>>>>>>             MCA sysinfo: linux (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA hwloc: hwloc132 (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA dpm: orte (MCA v2.0, API v2.0, Component v1.6)
>>>>>>              MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.6)
>>>>>>           MCA allocator: basic (MCA v2.0, API v2.0, Component v1.6)
>>>>>>           MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                MCA coll: basic (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                MCA coll: hierarch (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                MCA coll: inter (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                MCA coll: self (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                MCA coll: sm (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                MCA coll: sync (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                MCA coll: tuned (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                  MCA io: romio (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA mpool: fake (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA mpool: sm (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA pml: bfo (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA pml: csum (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA pml: ob1 (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA pml: v (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA bml: r2 (MCA v2.0, API v2.0, Component v1.6)
>>>>>>              MCA rcache: vma (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA btl: self (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA btl: sm (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA btl: tcp (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                MCA topo: unity (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA osc: pt2pt (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA osc: rdma (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA iof: hnp (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA iof: orted (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA iof: tool (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA oob: tcp (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                MCA odls: default (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA ras: cm (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA ras: loadleveler (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA ras: slurm (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA rmaps: load_balance (MCA v2.0, API v2.0, Component 
>>>>>> v1.6)
>>>>>>               MCA rmaps: rank_file (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA rmaps: resilient (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA rmaps: round_robin (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA rmaps: seq (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA rmaps: topo (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA rml: oob (MCA v2.0, API v2.0, Component v1.6)
>>>>>>              MCA routed: binomial (MCA v2.0, API v2.0, Component v1.6)
>>>>>>              MCA routed: cm (MCA v2.0, API v2.0, Component v1.6)
>>>>>>              MCA routed: direct (MCA v2.0, API v2.0, Component v1.6)
>>>>>>              MCA routed: linear (MCA v2.0, API v2.0, Component v1.6)
>>>>>>              MCA routed: radix (MCA v2.0, API v2.0, Component v1.6)
>>>>>>              MCA routed: slave (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA plm: rsh (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA plm: slurm (MCA v2.0, API v2.0, Component v1.6)
>>>>>>               MCA filem: rsh (MCA v2.0, API v2.0, Component v1.6)
>>>>>>              MCA errmgr: default (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA ess: env (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA ess: hnp (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA ess: singleton (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA ess: slave (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA ess: slurm (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA ess: slurmd (MCA v2.0, API v2.0, Component v1.6)
>>>>>>                 MCA ess: tool (MCA v2.0, API v2.0, Component v1.6)
>>>>>>             MCA grpcomm: bad (MCA v2.0, API v2.0, Component v1.6)
>>>>>>             MCA grpcomm: basic (MCA v2.0, API v2.0, Component v1.6)
>>>>>>             MCA grpcomm: hier (MCA v2.0, API v2.0, Component v1.6)
>>>>>>            MCA notifier: command (MCA v2.0, API v1.0, Component v1.6)
>>>>>>            MCA notifier: syslog (MCA v2.0, API v1.0, Component v1.6)
>>>>>>                  Prefix: /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7
>>>>>>             Exec_prefix: /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7
>>>>>>                  Bindir: 
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/bin
>>>>>>                 Sbindir: 
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/sbin
>>>>>>                  Libdir: 
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib
>>>>>>                  Incdir:
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/include
>>>>>>                  Mandir:
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/man
>>>>>>               Pkglibdir:
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib/openmpi
>>>>>>              Libexecdir:
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/libexec
>>>>>>             Datarootdir: 
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share
>>>>>>                 Datadir: 
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share
>>>>>>              Sysconfdir: 
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/etc
>>>>>>          Sharedstatedir: 
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/com
>>>>>>           Localstatedir: 
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/var
>>>>>>                 Infodir:
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/info
>>>>>>              Pkgdatadir:
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/openmpi
>>>>>>               Pkglibdir:
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib/openmpi
>>>>>>           Pkgincludedir:
>>>>>> /home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/include/openmpi
>>>>>> Configured architecture: x86_64-unknown-linux-gnu
>>>>>>          Configure host: cloud.bo.astro.it
>>>>>>           Configured by: andrea
>>>>>>           Configured on: Tue Jul 31 10:53:46 CEST 2012
>>>>>>          Configure host: cloud.bo.astro.it
>>>>>>                Built by: andrea
>>>>>>                Built on: Tue Jul 31 11:08:33 CEST 2012
>>>>>>              Built host: cloud.bo.astro.it
>>>>>>              C bindings: yes
>>>>>>            C++ bindings: yes
>>>>>>      Fortran77 bindings: yes (all)
>>>>>>      Fortran90 bindings: yes
>>>>>> Fortran90 bindings size: medium
>>>>>>              C compiler: /home/andrea/library/gcc/gcc-objects/bin/gcc
>>>>>>     C compiler absolute:
>>>>>>  C compiler family name: GNU
>>>>>>      C compiler version: 4.7.1
>>>>>>             C char size: 1
>>>>>>             C bool size: 1
>>>>>>            C short size: 2
>>>>>>              C int size: 4
>>>>>>             C long size: 8
>>>>>>            C float size: 4
>>>>>>           C double size: 8
>>>>>>          C pointer size: 8
>>>>>>            C char align: 1
>>>>>>            C bool align: 1
>>>>>>             C int align: 4
>>>>>>           C float align: 4
>>>>>>          C double align: 8
>>>>>>            C++ compiler: /home/andrea/library/gcc/gcc-objects/bin/g++
>>>>>>   C++ compiler absolute: none
>>>>>>      Fortran77 compiler: 
>>>>>> /home/andrea/library/gcc/gcc-objects/bin/gfortran
>>>>>>  Fortran77 compiler abs:
>>>>>>      Fortran90 compiler: 
>>>>>> /home/andrea/library/gcc/gcc-objects/bin/gfortran
>>>>>>  Fortran90 compiler abs:
>>>>>>       Fort integer size: 4
>>>>>>       Fort logical size: 4
>>>>>> Fort logical value true: 1
>>>>>>      Fort have integer1: yes
>>>>>>      Fort have integer2: yes
>>>>>>      Fort have integer4: yes
>>>>>>      Fort have integer8: yes
>>>>>>     Fort have integer16: no
>>>>>>         Fort have real4: yes
>>>>>>         Fort have real8: yes
>>>>>>        Fort have real16: no
>>>>>>      Fort have complex8: yes
>>>>>>     Fort have complex16: yes
>>>>>>     Fort have complex32: no
>>>>>>      Fort integer1 size: 1
>>>>>>      Fort integer2 size: 2
>>>>>>      Fort integer4 size: 4
>>>>>>      Fort integer8 size: 8
>>>>>>     Fort integer16 size: -1
>>>>>>          Fort real size: 4
>>>>>>         Fort real4 size: 4
>>>>>>         Fort real8 size: 8
>>>>>>        Fort real16 size: 16
>>>>>>      Fort dbl prec size: 8
>>>>>>          Fort cplx size: 8
>>>>>>      Fort dbl cplx size: 16
>>>>>>         Fort cplx8 size: 8
>>>>>>        Fort cplx16 size: 16
>>>>>>        Fort cplx32 size: 32
>>>>>>      Fort integer align: 4
>>>>>>     Fort integer1 align: 1
>>>>>>     Fort integer2 align: 2
>>>>>>     Fort integer4 align: 4
>>>>>>     Fort integer8 align: 8
>>>>>>    Fort integer16 align: -1
>>>>>>         Fort real align: 4
>>>>>>        Fort real4 align: 4
>>>>>>        Fort real8 align: 8
>>>>>>       Fort real16 align: 16
>>>>>>     Fort dbl prec align: 8
>>>>>>         Fort cplx align: 4
>>>>>>     Fort dbl cplx align: 8
>>>>>>        Fort cplx8 align: 4
>>>>>>       Fort cplx16 align: 8
>>>>>>       Fort cplx32 align: 16
>>>>>>             C profiling: yes
>>>>>>           C++ profiling: yes
>>>>>>     Fortran77 profiling: yes
>>>>>>     Fortran90 profiling: yes
>>>>>>          C++ exceptions: no
>>>>>>          Thread support: posix (MPI_THREAD_MULTIPLE: no, progress: no)
>>>>>>           Sparse Groups: no
>>>>>>            Build CFLAGS: -DNDEBUG -g -O2 -finline-functions
>>>>>> -fno-strict-aliasing
>>>>>>                          -pthread
>>>>>>          Build CXXFLAGS: -O3 -DNDEBUG -finline-functions -pthread
>>>>>>            Build FFLAGS:
>>>>>>           Build FCFLAGS:
>>>>>>           Build LDFLAGS: -Wl,--rpath
>>>>>> -Wl,/home/andrea/library/gcc/gcc-objects/lib64
>>>>>>              Build LIBS: -lrt -lnsl  -lutil -lm
>>>>>>    Wrapper extra CFLAGS: -pthread
>>>>>>  Wrapper extra CXXFLAGS: -pthread
>>>>>>    Wrapper extra FFLAGS: -pthread
>>>>>>   Wrapper extra FCFLAGS: -pthread
>>>>>>   Wrapper extra LDFLAGS:
>>>>>>      Wrapper extra LIBS: -ldl  -lm -lnuma -lrt -lnsl  -lutil -lm
>>>>>>  Internal debug support: no
>>>>>>  MPI interface warnings: yes
>>>>>>     MPI parameter check: runtime
>>>>>> Memory profiling support: no
>>>>>> Memory debugging support: no
>>>>>>         libltdl support: no
>>>>>>   Heterogeneous support: no
>>>>>> mpirun default --prefix: yes
>>>>>>         MPI I/O support: yes
>>>>>>       MPI_WTIME support: gettimeofday
>>>>>>     Symbol vis. support: yes
>>>>>>   Host topology support: yes
>>>>>>          MPI extensions: affinity example
>>>>>>   FT Checkpoint support: no (checkpoint thread: no)
>>>>>>     VampirTrace support: yes
>>>>>>  MPI_MAX_PROCESSOR_NAME: 256
>>>>>>    MPI_MAX_ERROR_STRING: 256
>>>>>>     MPI_MAX_OBJECT_NAME: 64
>>>>>>        MPI_MAX_INFO_KEY: 36
>>>>>>        MPI_MAX_INFO_VAL: 256
>>>>>>       MPI_MAX_PORT_NAME: 1024
>>>>>>  MPI_MAX_DATAREP_STRING: 128
>>>>>>                 MCA mca: parameter "mca_param_files" (current value:
>>>>>> 
>>>>>> </home/andrea/.openmpi/mca-params.conf:/home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/etc/openmpi-mca-params.conf>,
>>>>>>                          data source: default value)
>>>>>>                          Path for MCA configuration files containing
>>>>>> default parameter
>>>>>>                          values
>>>>>>                 MCA mca: parameter "mca_base_param_file_prefix"
>>>>>> (current value:<none>,
>>>>>>                          data source: default value)
>>>>>>                          Aggregate MCA parameter file sets
>>>>>>                 MCA mca: parameter "mca_base_param_file_path" (current 
>>>>>> value:
>>>>>> 
>>>>>> </home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/share/openmpi/amca-param-sets:/home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/bin>,
>>>>>>                          data source: default value)
>>>>>>                          Aggregate MCA parameter Search path
>>>>>>                 MCA mca: parameter "mca_base_param_file_path_force"
>>>>>> (current value:
>>>>>>                          <none>, data source: default value)
>>>>>>                          Forced Aggregate MCA parameter Search path
>>>>>>                 MCA mca: parameter "mca_component_path" (current value:
>>>>>> 
>>>>>> </home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/lib/openmpi:/home/andrea/.openmpi/components>,
>>>>>>                          data source: default value)
>>>>>>                          Path where to look for Open MPI and ORTE 
>>>>>> components
>>>>>>                 MCA mca: parameter "mca_component_show_load_errors"
>>>>>> (current value:<1>,
>>>>>>                          data source: default value)
>>>>>>                          Whether to show errors for components that
>>>>>> failed to load or
>>>>>>                          not
>>>>>>                 MCA mca: parameter "mca_component_disable_dlopen"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Whether to attempt to disable opening
>>>>>> dynamic components or not
>>>>>>                 MCA mca: parameter "mca_verbose" (current value:
>>>>>> <stderr>, data source:
>>>>>>                          default value)
>>>>>>                          Specifies where the default error output
>>>>>> stream goes (this is
>>>>>>                          separate from distinct help messages).  Accepts 
>>>>>> a
>>>>>>                          comma-delimited list of: stderr, stdout, syslog,
>>>>>>                          syslogpri:<notice|info|debug>,
>>>>>> syslogid:<str>  (where str is the
>>>>>>                          prefix string for all syslog notices),
>>>>>> file[:filename] (if
>>>>>>                          filename is not specified, a default
>>>>>> filename is used),
>>>>>>                          fileappend (if not specified, the file is 
>>>>>> opened for
>>>>>>                          truncation), level[:N] (if specified,
>>>>>> integer verbose level;
>>>>>>                          otherwise, 0 is implied)
>>>>>>                 MCA mpi: parameter "mpi_paffinity_alone" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value, synonym of:
>>>>>> opal_paffinity_alone)
>>>>>>                          If nonzero, assume that this job is the only 
>>>>>> (set of)
>>>>>>                          process(es) running on each node and bind 
>>>>>> processes to
>>>>>>                          processors, starting with processor ID 0
>>>>>>                 MCA mpi: parameter "mpi_param_check" (current value:
>>>>>> <1>, data source:
>>>>>>                          default value)
>>>>>>                          Whether you want MPI API parameters checked
>>>>>> at run-time or not.
>>>>>>                           Possible values are 0 (no checking) and 1
>>>>>> (perform checking at
>>>>>>                          run-time)
>>>>>>                 MCA mpi: parameter "mpi_yield_when_idle" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                          Yield the processor when waiting for MPI
>>>>>> communication (for MPI
>>>>>>                          processes, will default to 1 when
>>>>>> oversubscribing nodes)
>>>>>>                 MCA mpi: parameter "mpi_event_tick_rate" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                          How often to progress TCP communications (0
>>>>>> = never, otherwise
>>>>>>                          specified in microseconds)
>>>>>>                 MCA mpi: parameter "mpi_show_handle_leaks" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether MPI_FINALIZE shows all MPI handles
>>>>>> that were not freed
>>>>>>                          or not
>>>>>>                 MCA mpi: parameter "mpi_no_free_handles" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether to actually free MPI objects when
>>>>>> their handles are
>>>>>>                          freed
>>>>>>                 MCA mpi: parameter "mpi_show_mpi_alloc_mem_leaks"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          If>0, MPI_FINALIZE will show up to this
>>>>>> many instances of
>>>>>>                          memory allocated by MPI_ALLOC_MEM that was
>>>>>> not freed by
>>>>>>                          MPI_FREE_MEM
>>>>>>                 MCA mpi: parameter "mpi_show_mca_params" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          Whether to show all MCA parameter values
>>>>>> during MPI_INIT or not
>>>>>>                          (good for reproducability of MPI jobs for
>>>>>> debug purposes).
>>>>>>                          Accepted values are all, default, file, api,
>>>>>> and enviro - or a
>>>>>>                          comma delimited combination of them
>>>>>>                 MCA mpi: parameter "mpi_show_mca_params_file"
>>>>>> (current value:<none>,
>>>>>>                          data source: default value)
>>>>>>                          If mpi_show_mca_params is true, setting this
>>>>>> string to a valid
>>>>>>                          filename tells Open MPI to dump all the MCA
>>>>>> parameter values
>>>>>>                          into a file suitable for reading via the
>>>>>> mca_param_files
>>>>>>                          parameter (good for reproducability of MPI jobs)
>>>>>>                 MCA mpi: parameter "mpi_keep_peer_hostnames" (current
>>>>>> value:<1>, data
>>>>>>                          source: default value)
>>>>>>                          If nonzero, save the string hostnames of all
>>>>>> MPI peer processes
>>>>>>                          (mostly for error / debugging output
>>>>>> messages).  This can add
>>>>>>                          quite a bit of memory usage to each MPI process.
>>>>>>                 MCA mpi: parameter "mpi_abort_delay" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          If nonzero, print out an identifying message
>>>>>> when MPI_ABORT is
>>>>>>                          invoked (hostname, PID of the process that
>>>>>> called MPI_ABORT)
>>>>>>                          and delay for that many seconds before
>>>>>> exiting (a negative
>>>>>>                          delay value means to never abort).  This
>>>>>> allows attaching of a
>>>>>>                          debugger before quitting the job.
>>>>>>                 MCA mpi: parameter "mpi_abort_print_stack" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          If nonzero, print out a stack trace when
>>>>>> MPI_ABORT is invoked
>>>>>>                 MCA mpi: parameter "mpi_preconnect_mpi" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value, synonyms: 
>>>>>> mpi_preconnect_all)
>>>>>>                          Whether to force MPI processes to fully
>>>>>> wire-up the MPI
>>>>>>                          connections between MPI processes during
>>>>>> MPI_INIT (vs. making
>>>>>>                          connections lazily -- upon the first MPI
>>>>>> traffic between each
>>>>>>                          process peer pair)
>>>>>>                 MCA mpi: parameter "mpi_preconnect_all" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value, deprecated, synonym of:
>>>>>>                          mpi_preconnect_mpi)
>>>>>>                          Whether to force MPI processes to fully
>>>>>> wire-up the MPI
>>>>>>                          connections between MPI processes during
>>>>>> MPI_INIT (vs. making
>>>>>>                          connections lazily -- upon the first MPI
>>>>>> traffic between each
>>>>>>                          process peer pair)
>>>>>>                 MCA mpi: parameter "mpi_leave_pinned" (current value:
>>>>>> <-1>, data source:
>>>>>>                          default value)
>>>>>>                          Whether to use the "leave pinned" protocol
>>>>>> or not.  Enabling
>>>>>>                          this setting can help bandwidth performance
>>>>>> when repeatedly
>>>>>>                          sending and receiving large messages with
>>>>>> the same buffers over
>>>>>>                          RDMA-based networks (0 = do not use "leave
>>>>>> pinned" protocol, 1
>>>>>>                          = use "leave pinned" protocol, -1 = allow
>>>>>> network to choose at
>>>>>>                          runtime).
>>>>>>                 MCA mpi: parameter "mpi_leave_pinned_pipeline"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether to use the "leave pinned pipeline"
>>>>>> protocol or not.
>>>>>>                 MCA mpi: parameter "mpi_warn_on_fork" (current value:
>>>>>> <1>, data source:
>>>>>>                          default value)
>>>>>>                          If nonzero, issue a warning if program forks
>>>>>> under conditions
>>>>>>                          that could cause system errors
>>>>>>                 MCA mpi: information "mpi_have_sparse_group_storage"
>>>>>> (value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether this Open MPI installation supports
>>>>>> storing of data in
>>>>>>                          MPI groups in "sparse" formats (good for
>>>>>> extremely large
>>>>>>                          process count MPI jobs that create many
>>>>>> communicators/groups)
>>>>>>                 MCA mpi: parameter "mpi_use_sparse_group_storage"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Whether to use "sparse" storage formats for
>>>>>> MPI groups (only
>>>>>>                          relevant if mpi_have_sparse_group_storage is 1)
>>>>>>                 MCA mpi: parameter "mpi_notify_init_finalize"
>>>>>> (current value:<1>, data
>>>>>>                          source: default value)
>>>>>>                          If nonzero, send two notifications during
>>>>>> MPI_INIT: one near
>>>>>>                          when MPI_INIT starts, and another right
>>>>>> before MPI_INIT
>>>>>>                          finishes, and send 2 notifications during
>>>>>> MPI_FINALIZE: one
>>>>>>                          right when MPI_FINALIZE starts, and another 
>>>>>> near when
>>>>>>                          MPI_FINALIZE finishes.
>>>>>>                MCA orte: parameter "orte_base_help_aggregate"
>>>>>> (current value:<1>, data
>>>>>>                          source: default value)
>>>>>>                          If orte_base_help_aggregate is true,
>>>>>> duplicate help messages
>>>>>>                          will be aggregated rather than displayed
>>>>>> individually.  This
>>>>>>                          can be helpful for parallel jobs that
>>>>>> experience multiple
>>>>>>                          identical failures; rather than print out
>>>>>> the same help/failure
>>>>>>                          message N times, display it once with a
>>>>>> count of how many
>>>>>>                          processes sent the same message.
>>>>>>                MCA orte: parameter "orte_tmpdir_base" (current value:
>>>>>> <none>, data
>>>>>>                          source: default value)
>>>>>>                          Base of the session directory tree
>>>>>>                MCA orte: parameter "orte_no_session_dirs" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          Prohibited locations for session directories 
>>>>>> (multiple
>>>>>>                          locations separated by ',', default=NULL)
>>>>>>                MCA orte: parameter "orte_send_profile" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                          Send profile info in launch message
>>>>>>                MCA orte: parameter "orte_debug" (current value:<0>,
>>>>>> data source:
>>>>>>                          default value)
>>>>>>                          Top-level ORTE debug switch (default verbosity: 
>>>>>> 1)
>>>>>>                MCA orte: parameter "orte_debug_verbose" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for ORTE debug messages 
>>>>>> (default: 1)
>>>>>>                MCA orte: parameter "orte_debug_daemons" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether to debug the ORTE daemons or not
>>>>>>                MCA orte: parameter "orte_debug_daemons_file" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether want stdout/stderr of daemons to go
>>>>>> to a file or not
>>>>>>                MCA orte: parameter "orte_daemon_bootstrap" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Bootstrap the connection to the HNP
>>>>>>                MCA orte: parameter "orte_leave_session_attached"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Whether applications and/or daemons should
>>>>>> leave their sessions
>>>>>>                          attached so that any output can be received
>>>>>> - this allows X
>>>>>>                          forwarding without all the attendant debugging 
>>>>>> output
>>>>>>                MCA orte: parameter "orte_output_debugger_proctable"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Whether or not to output the debugger
>>>>>> proctable after launch
>>>>>>                          (default: false)
>>>>>>                MCA orte: parameter "orte_debugger_test_daemon"
>>>>>> (current value:<none>,
>>>>>>                          data source: default value)
>>>>>>                          Name of the executable to be used to
>>>>>> simulate a debugger
>>>>>>                          colaunch (relative or absolute path)
>>>>>>                MCA orte: parameter "orte_debugger_test_attach"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Test debugger colaunch after debugger attachment
>>>>>>                MCA orte: parameter "orte_debugger_check_rate"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Set rate (in secs) for auto-detect of
>>>>>> debugger attachment (0 =>
>>>>>>                          do not check)
>>>>>>                MCA orte: parameter "orte_do_not_launch" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Perform all necessary operations to prepare
>>>>>> to launch the
>>>>>>                          application, but do not actually launch it
>>>>>>                MCA orte: parameter "orte_daemon_spin" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Have any orteds spin until we can connect a
>>>>>> debugger to them
>>>>>>                MCA orte: parameter "orte_daemon_fail" (current value:
>>>>>> <-1>, data source:
>>>>>>                          default value)
>>>>>>                          Have the specified orted fail after init for
>>>>>> debugging purposes
>>>>>>                MCA orte: parameter "orte_daemon_fail_delay" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Have the specified orted fail after
>>>>>> specified number of seconds
>>>>>>                          (default: 0 =>  no delay)
>>>>>>                MCA orte: parameter "orte_heartbeat_rate" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Seconds between checks for daemon
>>>>>> state-of-health (default: 0
>>>>>>                          =>  do not check)
>>>>>>                MCA orte: parameter "orte_startup_timeout" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Milliseconds/daemon to wait for startup
>>>>>> before declaring
>>>>>>                          failed_to_start (default: 0 =>  do not check)
>>>>>>                MCA orte: parameter "orte_timing" (current value:<0>,
>>>>>> data source:
>>>>>>                          default value)
>>>>>>                          Request that critical timing loops be measured
>>>>>>                MCA orte: parameter "orte_timing_details" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Request that detailed timing data by reported
>>>>>>                MCA orte: parameter "orte_base_user_debugger" (current
>>>>>> value:<totalview
>>>>>>                          @mpirun@ -a @mpirun_args@ : ddt -n @np@
>>>>>> -start @executable@
>>>>>>                          @executable_argv@ @single_app@ : fxp @mpirun@ -a
>>>>>>                          @mpirun_args@>, data source: default value)
>>>>>>                          Sequence of user-level debuggers to search
>>>>>> for in orterun
>>>>>>                MCA orte: parameter "orte_abort_timeout" (current
>>>>>> value:<1>, data
>>>>>>                          source: default value)
>>>>>>                          Max time to wait [in secs] before aborting
>>>>>> an ORTE operation
>>>>>>                          (default: 1sec)
>>>>>>                MCA orte: parameter "orte_timeout_step" (current
>>>>>> value:<1000>, data
>>>>>>                          source: default value)
>>>>>>                          Time to wait [in usecs/proc] before aborting
>>>>>> an ORTE operation
>>>>>>                          (default: 1000 usec/proc)
>>>>>>                MCA orte: parameter "orte_default_hostfile" (current 
>>>>>> value:
>>>>>> 
>>>>>> </home/andrea/library/openmpi/openmpi-1.6-gnu-4.7/etc/openmpi-default-hostfile>,
>>>>>>                          data source: default value)
>>>>>>                          Name of the default hostfile (relative or
>>>>>> absolute path, "none"
>>>>>>                          to ignore environmental or default MCA param 
>>>>>> setting)
>>>>>>                MCA orte: parameter "orte_rankfile" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value, synonyms: rmaps_rank_file_path)
>>>>>>                          Name of the rankfile to be used for mapping
>>>>>> processes (relative
>>>>>>                          or absolute path)
>>>>>>                MCA orte: parameter "orte_keep_fqdn_hostnames"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether or not to keep FQDN hostnames [default: 
>>>>>> no]
>>>>>>                MCA orte: parameter "orte_use_regexp" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Whether or not to use regular expressions
>>>>>> for launch [default:
>>>>>>                          no]
>>>>>>                MCA orte: parameter "orte_tag_output" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Tag all output with [job,rank] (default: false)
>>>>>>                MCA orte: parameter "orte_xml_output" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Display all output in XML format (default: 
>>>>>> false)
>>>>>>                MCA orte: parameter "orte_xml_file" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          Provide all output in XML format to the 
>>>>>> specified file
>>>>>>                MCA orte: parameter "orte_timestamp_output" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Timestamp all application process output
>>>>>> (default: false)
>>>>>>                MCA orte: parameter "orte_output_filename" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          Redirect output from application processes
>>>>>> into filename.rank
>>>>>>                          [default: NULL]
>>>>>>                MCA orte: parameter "orte_show_resolved_nodenames"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Display any node names that are resolved to
>>>>>> a different name
>>>>>>                          (default: false)
>>>>>>                MCA orte: parameter "orte_hetero_apps" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Indicates that multiple app_contexts are
>>>>>> being provided that
>>>>>>                          are a mix of 32/64 bit binaries (default: false)
>>>>>>                MCA orte: parameter "orte_launch_agent" (current
>>>>>> value:<orted>, data
>>>>>>                          source: default value)
>>>>>>                          Command used to start processes on remote
>>>>>> nodes (default:
>>>>>>                          orted)
>>>>>>                MCA orte: parameter "orte_allocation_required"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether or not an allocation by a resource
>>>>>> manager is required
>>>>>>                          [default: no]
>>>>>>                MCA orte: parameter "orte_xterm" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          Create a new xterm window and display output
>>>>>> from the specified
>>>>>>                          ranks there [default: none]
>>>>>>                MCA orte: parameter "orte_forward_job_control"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Forward SIGTSTP (after converting to
>>>>>> SIGSTOP) and SIGCONT
>>>>>>                          signals to the application procs [default: no]
>>>>>>                MCA orte: parameter "orte_rsh_agent" (current value:
>>>>>> <ssh : rsh>, data
>>>>>>                          source: default value, synonyms:
>>>>>> pls_rsh_agent, plm_rsh_agent)
>>>>>>                          The command used to launch executables on 
>>>>>> remote nodes
>>>>>>                          (typically either "ssh" or "rsh")
>>>>>>                MCA orte: parameter "orte_assume_same_shell" (current
>>>>>> value:<1>, data
>>>>>>                          source: default value, synonyms:
>>>>>> plm_rsh_assume_same_shell)
>>>>>>                          If set to 1, assume that the shell on the
>>>>>> remote node is the
>>>>>>                          same as the shell on the local node.
>>>>>> Otherwise, probe for what
>>>>>>                          the remote shell [default: 1]
>>>>>>                MCA orte: parameter "orte_report_launch_progress"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Output a brief periodic report on launch
>>>>>> progress [default: no]
>>>>>>                MCA orte: parameter "orte_num_boards" (current value:
>>>>>> <1>, data source:
>>>>>>                          default value)
>>>>>>                          Number of processor boards/node (1-256) 
>>>>>> [default: 1]
>>>>>>                MCA orte: parameter "orte_num_sockets" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Number of sockets/board (1-256)
>>>>>>                MCA orte: parameter "orte_num_cores" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Number of cores/socket (1-256)
>>>>>>                MCA orte: parameter "orte_cpu_set" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          Comma-separated list of ranges specifying 
>>>>>> logical cpus
>>>>>>                          allocated to this job [default: none]
>>>>>>                MCA orte: parameter "orte_process_binding" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          Policy for binding processes [none | core |
>>>>>> socket | board]
>>>>>>                          (supported qualifier: if-avail)
>>>>>>                MCA orte: parameter "orte_report_bindings" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Report bindings
>>>>>>                MCA orte: parameter "orte_report_events" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          URI to which events are to be reported
>>>>>> (default: NULL)]
>>>>>>                MCA opal: parameter "opal_net_private_ipv4" (current 
>>>>>> value:
>>>>>> 
>>>>>> <10.0.0.0/8;172.16.0.0/12;192.168.0.0/16;169.254.0.0/16>, data
>>>>>>                          source: default value)
>>>>>>                          Semicolon-delimited list of CIDR notation
>>>>>> entries specifying
>>>>>>                          what networks are considered "private"
>>>>>> (default value based on
>>>>>>                          RFC1918 and RFC3330)
>>>>>>                MCA opal: parameter "opal_signal" (current value:
>>>>>> <6,7,8,11>, data
>>>>>>                          source: default value)
>>>>>>                          Comma-delimited list of integer signal
>>>>>> numbers to Open MPI to
>>>>>>                          attempt to intercept.  Upon receipt of the
>>>>>> intercepted signal,
>>>>>>                          Open MPI will display a stack trace and
>>>>>> abort.  Open MPI will
>>>>>>                          *not* replace signals if handlers are
>>>>>> already installed by the
>>>>>>                          time MPI_INIT is invoked.  Optionally append
>>>>>> ":complain" to any
>>>>>>                          signal number in the comma-delimited list to
>>>>>> make Open MPI
>>>>>>                          complain if it detects another signal
>>>>>> handler (and therefore
>>>>>>                          does not insert its own).
>>>>>>                MCA opal: parameter "opal_profile" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Set to non-zero to profile component selections
>>>>>>                MCA opal: parameter "opal_profile_file" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          Name of the file containing the cluster 
>>>>>> configuration
>>>>>>                          information
>>>>>>                MCA opal: parameter "opal_paffinity_alone" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value, synonyms: 
>>>>>> mpi_paffinity_alone)
>>>>>>                          If nonzero, assume that this job is the only 
>>>>>> (set of)
>>>>>>                          process(es) running on each node and bind 
>>>>>> processes to
>>>>>>                          processors, starting with processor ID 0
>>>>>>                MCA opal: parameter "opal_set_max_sys_limits" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Set to non-zero to automatically set any
>>>>>> system-imposed limits
>>>>>>                          to the maximum allowed
>>>>>>                MCA opal: parameter "opal_event_include" (current
>>>>>> value:<poll>, data
>>>>>>                          source: default value)
>>>>>>                          Comma-delimited list of libevent subsystems
>>>>>> to use (epoll,
>>>>>>                          poll, select -- available on your platform)
>>>>>>           MCA backtrace: parameter "backtrace" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          Default selection set of components for the
>>>>>> backtrace framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>           MCA backtrace: parameter "backtrace_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the backtrace framework
>>>>>> (0 = no verbosity)
>>>>>>           MCA backtrace: parameter "backtrace_execinfo_priority"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>          MCA memchecker: parameter "memchecker" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          Default selection set of components for the 
>>>>>> memchecker
>>>>>>                          framework (<none>  means use all components
>>>>>> that can be found)
>>>>>>              MCA memory: parameter "memory" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> memory framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>              MCA memory: parameter "memory_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the memory framework (0
>>>>>> = no verbosity)
>>>>>>              MCA memory: information
>>>>>> "memory_linux_ptmalloc2_available" (value:<1>,
>>>>>>                          data source: default value)
>>>>>>                          Whether ptmalloc2 support is included in
>>>>>> Open MPI or not (1 =
>>>>>>                          yes, 0 = no)
>>>>>>              MCA memory: information
>>>>>> "memory_linux_ummunotify_available" (value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Whether ummunotify support is included in
>>>>>> Open MPI or not (1 =
>>>>>>                          yes, 0 = no)
>>>>>>              MCA memory: parameter "memory_linux_ptmalloc2_enable"
>>>>>> (current value:<-1>,
>>>>>>                          data source: default value)
>>>>>>                          Whether to enable ptmalloc2 support or not
>>>>>> (negative = try to
>>>>>>                          enable, but continue even if support is not
>>>>>> available, 0 = do
>>>>>>                          not enable support, positive = try to enable
>>>>>> and fail if
>>>>>>                          support is not available)
>>>>>>              MCA memory: parameter "memory_linux_ummunotify_enable"
>>>>>> (current value:
>>>>>>                          <-1>, data source: default value)
>>>>>>                          Whether to enable ummunotify support or not
>>>>>> (negative = try to
>>>>>>                          enable, but continue even if support is not
>>>>>> available, 0 = do
>>>>>>                          not enable support, positive = try to enable
>>>>>> and fail if
>>>>>>                          support is not available)
>>>>>>              MCA memory: parameter "memory_linux_disable" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          If this MCA parameter is set to 1 **VIA
>>>>>> ENVIRONMENT VARIABLE
>>>>>>                          ONLY*** (this MCA parameter *CANNOT* be set
>>>>>> in a file or on the
>>>>>>                          mpirun command line!), this component will
>>>>>> be disabled and will
>>>>>>                          not attempt to use either ummunotify or
>>>>>> memory hook support
>>>>>>              MCA memory: parameter "memory_linux_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>           MCA paffinity: parameter "paffinity_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level of the paffinity framework
>>>>>>           MCA paffinity: parameter "paffinity" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          Default selection set of components for the
>>>>>> paffinity framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>           MCA paffinity: parameter "paffinity_hwloc_priority"
>>>>>> (current value:<40>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the hwloc paffinity component
>>>>>>               MCA carto: parameter "carto_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level of the carto framework
>>>>>>               MCA carto: parameter "carto" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> carto framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>               MCA carto: parameter "carto_auto_detect_priority"
>>>>>> (current value:<11>,
>>>>>>                          data source: default value)
>>>>>>                          Priority of the auto_detect carto component
>>>>>>               MCA carto: parameter "carto_file_path" (current value:
>>>>>> <none>, data
>>>>>>                          source: default value)
>>>>>>                          The path to the cartography file
>>>>>>               MCA carto: parameter "carto_file_priority" (current
>>>>>> value:<10>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the file carto component
>>>>>>               MCA shmem: parameter "shmem_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level of the shmem framework
>>>>>>               MCA shmem: parameter "shmem" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> shmem framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>               MCA shmem: parameter "shmem_mmap_enable_nfs_warning"
>>>>>> (current value:<1>,
>>>>>>                          data source: default value)
>>>>>>                          Enable the warning emitted when Open MPI
>>>>>> detects that its
>>>>>>                          shared memory backing file is located on a
>>>>>> network filesystem
>>>>>>                          (1 = enabled, 0 = disabled).
>>>>>>               MCA shmem: parameter "shmem_mmap_priority" (current
>>>>>> value:<50>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the mmap shmem component
>>>>>>               MCA shmem: parameter "shmem_mmap_relocate_backing_file"
>>>>>> (current value:
>>>>>>                          <0>, data source: default value)
>>>>>>                          Whether to change the default placement of
>>>>>> backing files or not
>>>>>>                          (Negative = try to relocate backing files to
>>>>>> an area rooted at
>>>>>>                          the path specified by
>>>>>> 
>>>>>> shmem_mmap_opal_shmem_mmap_backing_file_base_dir, but continue
>>>>>>                          with the default path if the relocation
>>>>>> fails, 0 = do not
>>>>>>                          relocate, Positive = same as the negative
>>>>>> option, but will fail
>>>>>>                          if the relocation fails.
>>>>>>               MCA shmem: parameter "shmem_mmap_backing_file_base_dir"
>>>>>> (current value:
>>>>>>                          </dev/shm>, data source: default value)
>>>>>>                          Specifies where backing files will be created 
>>>>>> when
>>>>>>                          shmem_mmap_relocate_backing_file is in use.
>>>>>>               MCA shmem: parameter "shmem_posix_priority" (current
>>>>>> value:<40>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the posix shmem component
>>>>>>               MCA shmem: parameter "shmem_sysv_priority" (current
>>>>>> value:<30>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the sysv shmem component
>>>>>>           MCA maffinity: parameter "maffinity_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level of the maffinity framework
>>>>>>           MCA maffinity: parameter "maffinity" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          Default selection set of components for the
>>>>>> maffinity framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>           MCA maffinity: parameter "maffinity_first_use_priority"
>>>>>> (current value:<10>,
>>>>>>                          data source: default value)
>>>>>>                          Priority of the first_use maffinity component
>>>>>>           MCA maffinity: parameter "maffinity_hwloc_priority"
>>>>>> (current value:<40>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the hwloc maffinity component
>>>>>>               MCA timer: parameter "timer" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> timer framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>               MCA timer: parameter "timer_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the timer framework (0 =
>>>>>> no verbosity)
>>>>>>               MCA timer: parameter "timer_linux_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>             MCA sysinfo: parameter "sysinfo" (current value:<none>,
>>>>>> data source:
>>>>>>                          default value)
>>>>>>                          Default selection set of components for the
>>>>>> sysinfo framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>             MCA sysinfo: parameter "sysinfo_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the sysinfo framework (0
>>>>>> = no verbosity)
>>>>>>             MCA sysinfo: parameter "sysinfo_linux_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>               MCA hwloc: parameter "hwloc_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level of the hwloc framework
>>>>>>               MCA hwloc: parameter "hwloc_base_mem_alloc_policy"
>>>>>> (current value:<none>,
>>>>>>                          data source: default value)
>>>>>>                          Policy that determines how general memory
>>>>>> allocations are bound
>>>>>>                          after MPI_INIT.  A value of "none" means
>>>>>> that no memory policy
>>>>>>                          is applied.  A value of "local_only" means
>>>>>> that all memory
>>>>>>                          allocations will be restricted to the local
>>>>>> NUMA node where
>>>>>>                          each process is placed.  Note that operating
>>>>>> system paging
>>>>>>                          policies are unaffected by this setting.
>>>>>> For example, if
>>>>>>                          "local_only" is used and local NUMA node
>>>>>> memory is exhausted, a
>>>>>>                          new memory allocation may cause paging.
>>>>>>               MCA hwloc: parameter
>>>>>> "hwloc_base_mem_bind_failure_action" (current value:
>>>>>>                          <error>, data source: default value)
>>>>>>                          What Open MPI will do if it explicitly tries
>>>>>> to bind memory to
>>>>>>                          a specific NUMA location, and fails.  Note
>>>>>> that this is a
>>>>>>                          different case than the general allocation
>>>>>> policy described by
>>>>>>                          hwloc_base_alloc_policy.  A value of "warn"
>>>>>> means that Open MPI
>>>>>>                          will warn the first time this happens, but
>>>>>> allow the job to
>>>>>>                          continue (possibly with degraded
>>>>>> performance).  A value of
>>>>>>                          "error" means that Open MPI will abort the
>>>>>> job if this happens.
>>>>>>               MCA hwloc: parameter "hwloc" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> hwloc framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>               MCA hwloc: parameter "hwloc_hwloc132_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA dpm: parameter "dpm" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> dpm framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA dpm: parameter "dpm_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the dpm framework (0 =
>>>>>> no verbosity)
>>>>>>                 MCA dpm: parameter "dpm_orte_priority" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>              MCA pubsub: parameter "pubsub" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> pubsub framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>              MCA pubsub: parameter "pubsub_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the pubsub framework (0
>>>>>> = no verbosity)
>>>>>>              MCA pubsub: parameter "pubsub_orte_priority" (current
>>>>>> value:<50>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the pubsub pmi component
>>>>>>           MCA allocator: parameter "allocator" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          Default selection set of components for the
>>>>>> allocator framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>           MCA allocator: parameter "allocator_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the allocator framework
>>>>>> (0 = no verbosity)
>>>>>>           MCA allocator: parameter "allocator_basic_priority"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>           MCA allocator: parameter "allocator_bucket_num_buckets"
>>>>>> (current value:<30>,
>>>>>>                          data source: default value)
>>>>>>           MCA allocator: parameter "allocator_bucket_priority"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>                MCA coll: parameter "coll" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> coll framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                MCA coll: parameter "coll_base_verbose" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the coll framework (0 =
>>>>>> no verbosity)
>>>>>>                MCA coll: parameter "coll_basic_priority" (current
>>>>>> value:<10>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the basic coll component
>>>>>>                MCA coll: parameter "coll_basic_crossover" (current
>>>>>> value:<4>, data
>>>>>>                          source: default value)
>>>>>>                          Minimum number of processes in a
>>>>>> communicator before using the
>>>>>>                          logarithmic algorithms
>>>>>>                MCA coll: parameter "coll_hierarch_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the hierarchical coll component
>>>>>>                MCA coll: parameter "coll_hierarch_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Turn verbose message of the hierarchical
>>>>>> coll component on/off
>>>>>>                MCA coll: parameter "coll_hierarch_use_rdma" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Switch from the send btl list used to detect
>>>>>> hierarchies to the
>>>>>>                          rdma btl list
>>>>>>                MCA coll: parameter "coll_hierarch_ignore_sm" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Ignore sm protocol when detecting
>>>>>> hierarchies. Required to
>>>>>>                          enable the usage of protocol specific
>>>>>> collective operations
>>>>>>                MCA coll: parameter "coll_hierarch_detection_alg"
>>>>>> (current value:<2>,
>>>>>>                          data source: default value)
>>>>>>                          Used to specify the algorithm for detecting
>>>>>> Hierarchy.Choose
>>>>>>                          between all or two levels of hierarchy
>>>>>>                MCA coll: parameter "coll_hierarch_bcast_alg" (current
>>>>>> value:<4>, data
>>>>>>                          source: default value)
>>>>>>                          Used to specify the algorithm used for bcast
>>>>>> operations.
>>>>>>                MCA coll: parameter "coll_hierarch_segment_size"
>>>>>> (current value:<32768>,
>>>>>>                          data source: default value)
>>>>>>                          Used to specify the segment size for
>>>>>> segmented algorithms.
>>>>>>                MCA coll: parameter "coll_inter_priority" (current
>>>>>> value:<40>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the inter coll component
>>>>>>                MCA coll: parameter "coll_inter_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Turn verbose message of the inter coll
>>>>>> component on/off
>>>>>>                MCA coll: parameter "coll_self_priority" (current
>>>>>> value:<75>, data
>>>>>>                          source: default value)
>>>>>>                MCA coll: parameter "coll_sm_priority" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Priority of the sm coll component
>>>>>>                MCA coll: parameter "coll_sm_control_size" (current
>>>>>> value:<4096>, data
>>>>>>                          source: default value)
>>>>>>                          Length of the control data -- should usually
>>>>>> be either the
>>>>>>                          length of a cache line on most SMPs, or the
>>>>>> size of a page on
>>>>>>                          machines that support direct memory affinity
>>>>>> page placement (in
>>>>>>                          bytes)
>>>>>>                MCA coll: parameter "coll_sm_fragment_size" (current
>>>>>> value:<8192>, data
>>>>>>                          source: default value)
>>>>>>                          Fragment size (in bytes) used for passing
>>>>>> data through shared
>>>>>>                          memory (will be rounded up to the nearest
>>>>>> control_size size)
>>>>>>                MCA coll: parameter "coll_sm_comm_in_use_flags"
>>>>>> (current value:<2>, data
>>>>>>                          source: default value)
>>>>>>                          Number of "in use" flags, used to mark a
>>>>>> message passing area
>>>>>>                          segment as currently being used or not (must
>>>>>> be>= 2 and<=
>>>>>>                          comm_num_segments)
>>>>>>                MCA coll: parameter "coll_sm_comm_num_segments"
>>>>>> (current value:<8>, data
>>>>>>                          source: default value)
>>>>>>                          Number of segments in each communicator's
>>>>>> shared memory message
>>>>>>                          passing area (must be>= 2, and must be a 
>>>>>> multiple of
>>>>>>                          comm_in_use_flags)
>>>>>>                MCA coll: parameter "coll_sm_tree_degree" (current
>>>>>> value:<4>, data
>>>>>>                          source: default value)
>>>>>>                          Degree of the tree for tree-based operations
>>>>>> (must be =>  1 and
>>>>>>                          <= min(control_size, 255))
>>>>>>                MCA coll: parameter "coll_sm_info_num_procs" (current
>>>>>> value:<4>, data
>>>>>>                          source: default value)
>>>>>>                          Number of processes to use for the calculation 
>>>>>> of the
>>>>>>                          shared_mem_size MCA information parameter
>>>>>> (must be =>  2)
>>>>>>                MCA coll: information "coll_sm_shared_mem_used_data"
>>>>>> (value:<548864>,
>>>>>>                          data source: default value)
>>>>>>                          Amount of shared memory used, per
>>>>>> communicator, in the shared
>>>>>>                          memory data area for info_num_procs
>>>>>> processes (in bytes)
>>>>>>                MCA coll: parameter "coll_sync_priority" (current
>>>>>> value:<50>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the sync coll component; only 
>>>>>> relevant if
>>>>>>                          barrier_before or barrier_after is>  0
>>>>>>                MCA coll: parameter "coll_sync_barrier_before"
>>>>>> (current value:<1000>,
>>>>>>                          data source: default value)
>>>>>>                          Do a synchronization before each Nth collective
>>>>>>                MCA coll: parameter "coll_sync_barrier_after" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Do a synchronization after each Nth collective
>>>>>>                MCA coll: parameter "coll_tuned_priority" (current
>>>>>> value:<30>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the tuned coll component
>>>>>>                MCA coll: parameter
>>>>>> "coll_tuned_pre_allocate_memory_comm_size_limit"
>>>>>>                          (current value:<32768>, data source: default 
>>>>>> value)
>>>>>>                          Size of communicator were we stop
>>>>>> pre-allocating memory for the
>>>>>>                          fixed internal buffer used for message
>>>>>> requests etc that is
>>>>>>                          hung off the communicator data segment. I.e.
>>>>>> if you have a
>>>>>>                          100'000 nodes you might not want to
>>>>>> pre-allocate 200'000
>>>>>>                          request handle slots per communicator instance!
>>>>>>                MCA coll: parameter "coll_tuned_init_tree_fanout"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                          Inital fanout used in the tree topologies for 
>>>>>> each
>>>>>>                          communicator. This is only an initial guess,
>>>>>> if a tuned
>>>>>>                          collective needs a different fanout for an
>>>>>> operation, it build
>>>>>>                          it dynamically. This parameter is only for
>>>>>> the first guess and
>>>>>>                          might save a little time
>>>>>>                MCA coll: parameter "coll_tuned_init_chain_fanout"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                          Inital fanout used in the chain (fanout
>>>>>> followed by pipeline)
>>>>>>                          topologies for each communicator. This is
>>>>>> only an initial
>>>>>>                          guess, if a tuned collective needs a
>>>>>> different fanout for an
>>>>>>                          operation, it build it dynamically. This
>>>>>> parameter is only for
>>>>>>                          the first guess and might save a little time
>>>>>>                MCA coll: parameter "coll_tuned_use_dynamic_rules"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Switch used to decide if we use static
>>>>>> (compiled/if statements)
>>>>>>                          or dynamic (built at runtime) decision function 
>>>>>> rules
>>>>>>                  MCA io: parameter "io_base_freelist_initial_size"
>>>>>> (current value:<16>,
>>>>>>                          data source: default value)
>>>>>>                          Initial MPI-2 IO request freelist size
>>>>>>                  MCA io: parameter "io_base_freelist_max_size"
>>>>>> (current value:<64>,
>>>>>>                          data source: default value)
>>>>>>                          Max size of the MPI-2 IO request freelist
>>>>>>                  MCA io: parameter "io_base_freelist_increment"
>>>>>> (current value:<16>,
>>>>>>                          data source: default value)
>>>>>>                          Increment size of the MPI-2 IO request freelist
>>>>>>                  MCA io: parameter "io" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> io framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                  MCA io: parameter "io_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the io framework (0 = no
>>>>>> verbosity)
>>>>>>                  MCA io: parameter "io_romio_priority" (current
>>>>>> value:<10>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the io romio component
>>>>>>                  MCA io: parameter "io_romio_delete_priority"
>>>>>> (current value:<10>, data
>>>>>>                          source: default value)
>>>>>>                          Delete priority of the io romio component
>>>>>>                  MCA io: information "io_romio_version" (value:<from
>>>>>> MPICH2 v1.3.1 with
>>>>>>                          an additional patch from
>>>>>> romio-ma...@mcs.anl.gov about an
>>>>>>                          attribute issue>, data source: default value)
>>>>>>                          Version of ROMIO
>>>>>>                  MCA io: information "io_romio_user_configure_params"
>>>>>> (value:<none>,
>>>>>>                          data source: default value)
>>>>>>                          User-specified command line parameters
>>>>>> passed to ROMIO's
>>>>>>                          configure script
>>>>>>                  MCA io: information
>>>>>> "io_romio_complete_configure_params" (value:<
>>>>>>                          CFLAGS='-DNDEBUG -g -O2 -finline-functions
>>>>>> -fno-strict-aliasing
>>>>>>                          -pthread' CPPFLAGS='
>>>>>> 
>>>>>> -I/home/andrea/library/openmpi/openmpi-1.6/opal/mca/hwloc/hwloc132/hwloc/include
>>>>>>                            -I/usr/include/infiniband 
>>>>>> -I/usr/include/infiniband'
>>>>>>                          FFLAGS='' LDFLAGS='-Wl,--rpath
>>>>>>                          -Wl,/home/andrea/library/gcc/gcc-objects/lib64 '
>>>>>>                          --enable-shared --enable-static
>>>>>> 
>>>>>> --prefix=/home/andrea/library/openmpi/openmpi-1.6-gnu-4.7
>>>>>>                          --with-mpi=open_mpi --disable-aio>, data
>>>>>> source: default value)
>>>>>>                          Complete set of command line parameters
>>>>>> passed to ROMIO's
>>>>>>                          configure script
>>>>>>               MCA mpool: parameter "mpool" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> mpool framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>               MCA mpool: parameter "mpool_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the mpool framework (0 =
>>>>>> no verbosity)
>>>>>>               MCA mpool: parameter "mpool_fake_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>               MCA mpool: parameter "mpool_rdma_rcache_name" (current
>>>>>> value:<vma>, data
>>>>>>                          source: default value)
>>>>>>                          The name of the registration cache the mpool
>>>>>> should use
>>>>>>               MCA mpool: parameter "mpool_rdma_rcache_size_limit"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          the maximum size of registration cache in
>>>>>> bytes. 0 is unlimited
>>>>>>                          (default 0)
>>>>>>               MCA mpool: parameter "mpool_rdma_print_stats" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          print pool usage statistics at the end of the 
>>>>>> run
>>>>>>               MCA mpool: parameter "mpool_rdma_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>               MCA mpool: parameter "mpool_sm_allocator" (current
>>>>>> value:<bucket>, data
>>>>>>                          source: default value)
>>>>>>                          Name of allocator component to use with sm mpool
>>>>>>               MCA mpool: parameter "mpool_sm_min_size" (current
>>>>>> value:<67108864>, data
>>>>>>                          source: default value)
>>>>>>                          Minimum size of the sm mpool shared memory file
>>>>>>               MCA mpool: parameter "mpool_sm_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Enable verbose output for mpool sm component
>>>>>>               MCA mpool: parameter "mpool_sm_priority" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA pml: parameter "pml_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level of the PML framework
>>>>>>                 MCA pml: parameter "pml" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> pml framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA pml: parameter "pml_bfo_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA pml: parameter "pml_bfo_free_list_num" (current
>>>>>> value:<4>, data
>>>>>>                          source: default value)
>>>>>>                 MCA pml: parameter "pml_bfo_free_list_max" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                 MCA pml: parameter "pml_bfo_free_list_inc" (current
>>>>>> value:<64>, data
>>>>>>                          source: default value)
>>>>>>                 MCA pml: parameter "pml_bfo_priority" (current value:
>>>>>> <5>, data source:
>>>>>>                          default value)
>>>>>>                 MCA pml: parameter "pml_bfo_send_pipeline_depth"
>>>>>> (current value:<3>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_bfo_recv_pipeline_depth"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_bfo_rdma_put_retries_limit"
>>>>>> (current value:<5>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_bfo_max_rdma_per_request"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_bfo_max_send_per_range"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_bfo_unexpected_limit"
>>>>>> (current value:<128>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_bfo_allocator" (current
>>>>>> value:<bucket>, data
>>>>>>                          source: default value)
>>>>>>                          Name of allocator component for unexpected 
>>>>>> messages
>>>>>>                 MCA pml: parameter "pml_cm_free_list_num" (current
>>>>>> value:<4>, data
>>>>>>                          source: default value)
>>>>>>                          Initial size of request free lists
>>>>>>                 MCA pml: parameter "pml_cm_free_list_max" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                          Maximum size of request free lists
>>>>>>                 MCA pml: parameter "pml_cm_free_list_inc" (current
>>>>>> value:<64>, data
>>>>>>                          source: default value)
>>>>>>                          Number of elements to add when growing
>>>>>> request free lists
>>>>>>                 MCA pml: parameter "pml_cm_priority" (current value:
>>>>>> <10>, data source:
>>>>>>                          default value)
>>>>>>                          CM PML selection priority
>>>>>>                 MCA pml: parameter "pml_csum_free_list_num" (current
>>>>>> value:<4>, data
>>>>>>                          source: default value)
>>>>>>                 MCA pml: parameter "pml_csum_free_list_max" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                 MCA pml: parameter "pml_csum_free_list_inc" (current
>>>>>> value:<64>, data
>>>>>>                          source: default value)
>>>>>>                 MCA pml: parameter "pml_csum_send_pipeline_depth"
>>>>>> (current value:<3>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_csum_recv_pipeline_depth"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_csum_rdma_put_retries_limit"
>>>>>> (current value:
>>>>>>                          <5>, data source: default value)
>>>>>>                 MCA pml: parameter "pml_csum_max_rdma_per_request"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_csum_max_send_per_range"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_csum_unexpected_limit"
>>>>>> (current value:<128>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_csum_allocator" (current
>>>>>> value:<bucket>, data
>>>>>>                          source: default value)
>>>>>>                          Name of allocator component for unexpected 
>>>>>> messages
>>>>>>                 MCA pml: parameter "pml_csum_priority" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA pml: parameter "pml_ob1_free_list_num" (current
>>>>>> value:<4>, data
>>>>>>                          source: default value)
>>>>>>                 MCA pml: parameter "pml_ob1_free_list_max" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                 MCA pml: parameter "pml_ob1_free_list_inc" (current
>>>>>> value:<64>, data
>>>>>>                          source: default value)
>>>>>>                 MCA pml: parameter "pml_ob1_priority" (current value:
>>>>>> <20>, data source:
>>>>>>                          default value)
>>>>>>                 MCA pml: parameter "pml_ob1_send_pipeline_depth"
>>>>>> (current value:<3>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_ob1_recv_pipeline_depth"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_ob1_rdma_put_retries_limit"
>>>>>> (current value:<5>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_ob1_max_rdma_per_request"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_ob1_max_send_per_range"
>>>>>> (current value:<4>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_ob1_unexpected_limit"
>>>>>> (current value:<128>,
>>>>>>                          data source: default value)
>>>>>>                 MCA pml: parameter "pml_ob1_allocator" (current
>>>>>> value:<bucket>, data
>>>>>>                          source: default value)
>>>>>>                          Name of allocator component for unexpected 
>>>>>> messages
>>>>>>                 MCA pml: parameter "pml_v_priority" (current value:
>>>>>> <-1>, data source:
>>>>>>                          default value)
>>>>>>                 MCA pml: parameter "pml_v_output" (current value:
>>>>>> <stderr>, data source:
>>>>>>                          default value)
>>>>>>                 MCA pml: parameter "pml_v_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA bml: parameter "bml" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> bml framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA bml: parameter "bml_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the bml framework (0 =
>>>>>> no verbosity)
>>>>>>                 MCA bml: parameter "bml_r2_show_unreach_errors"
>>>>>> (current value:<1>,
>>>>>>                          data source: default value)
>>>>>>                          Show error message when procs are unreachable
>>>>>>                 MCA bml: parameter "bml_r2_priority" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>              MCA rcache: parameter "rcache" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> rcache framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>              MCA rcache: parameter "rcache_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the rcache framework (0
>>>>>> = no verbosity)
>>>>>>              MCA rcache: parameter "rcache_vma_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level of the BTL framework
>>>>>>                 MCA btl: parameter "btl" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> btl framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA btl: parameter "btl_self_free_list_num" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Number of fragments by default
>>>>>>                 MCA btl: parameter "btl_self_free_list_max" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                          Maximum number of fragments
>>>>>>                 MCA btl: parameter "btl_self_free_list_inc" (current
>>>>>> value:<32>, data
>>>>>>                          source: default value)
>>>>>>                          Increment by this number of fragments
>>>>>>                 MCA btl: parameter "btl_self_exclusivity" (current
>>>>>> value:<65536>, data
>>>>>>                          source: default value)
>>>>>>                          BTL exclusivity (must be>= 0)
>>>>>>                 MCA btl: parameter "btl_self_flags" (current value:
>>>>>> <10>, data source:
>>>>>>                          default value)
>>>>>>                          BTL bit flags (general flags: SEND=1, PUT=2, 
>>>>>> GET=4,
>>>>>>                          SEND_INPLACE=8, RDMA_MATCHED=64,
>>>>>> HETEROGENEOUS_RDMA=256; flags
>>>>>>                          only used by the "dr" PML (ignored by others): 
>>>>>> ACK=16,
>>>>>>                          CHECKSUM=32, RDMA_COMPLETION=128; flags only
>>>>>> used by the "bfo"
>>>>>>                          PML (ignored by others): FAILOVER_SUPPORT=512)
>>>>>>                 MCA btl: parameter "btl_self_rndv_eager_limit"
>>>>>> (current value:<131072>,
>>>>>>                          data source: default value)
>>>>>>                          Size (in bytes) of "phase 1" fragment sent
>>>>>> for all large
>>>>>>                          messages (must be>= 0 and<= eager_limit)
>>>>>>                 MCA btl: parameter "btl_self_eager_limit" (current
>>>>>> value:<131072>, data
>>>>>>                          source: default value)
>>>>>>                          Maximum size (in bytes) of "short" messages
>>>>>> (must be>= 1).
>>>>>>                 MCA btl: parameter "btl_self_max_send_size" (current
>>>>>> value:<262144>,
>>>>>>                          data source: default value)
>>>>>>                          Maximum size (in bytes) of a single "phase
>>>>>> 2" fragment of a
>>>>>>                          long message when using the pipeline
>>>>>> protocol (must be>= 1)
>>>>>>                 MCA btl: parameter
>>>>>> "btl_self_rdma_pipeline_send_length" (current value:
>>>>>>                          <2147483647>, data source: default value)
>>>>>>                          Length of the "phase 2" portion of a large
>>>>>> message (in bytes)
>>>>>>                          when using the pipeline protocol.  This part
>>>>>> of the message
>>>>>>                          will be split into fragments of size
>>>>>> max_send_size and sent
>>>>>>                          using send/receive semantics (must be>= 0;
>>>>>> only relevant when
>>>>>>                          the PUT flag is set)
>>>>>>                 MCA btl: parameter "btl_self_rdma_pipeline_frag_size"
>>>>>> (current value:
>>>>>>                          <2147483647>, data source: default value)
>>>>>>                          Maximum size (in bytes) of a single "phase
>>>>>> 3" fragment from a
>>>>>>                          long message when using the pipeline
>>>>>> protocol.  These fragments
>>>>>>                          will be sent using RDMA semantics (must be
>>>>>>> = 1; only relevant
>>>>>>                          when the PUT flag is set)
>>>>>>                 MCA btl: parameter "btl_self_min_rdma_pipeline_size"
>>>>>> (current value:
>>>>>>                          <0>, data source: default value)
>>>>>>                          Messages smaller than this size (in bytes)
>>>>>> will not use the
>>>>>>                          RDMA pipeline protocol.  Instead, they will
>>>>>> be split into
>>>>>>                          fragments of max_send_size and sent using 
>>>>>> send/receive
>>>>>>                          semantics (must be>=0, and is automatically
>>>>>> adjusted up to at
>>>>>>                          least
>>>>>> (eager_limit+btl_rdma_pipeline_send_length); only
>>>>>>                          relevant when the PUT flag is set)
>>>>>>                 MCA btl: parameter "btl_self_bandwidth" (current
>>>>>> value:<100>, data
>>>>>>                          source: default value)
>>>>>>                          Approximate maximum bandwidth of
>>>>>> interconnect(0 = auto-detect
>>>>>>                          value at run-time [not supported in all BTL
>>>>>> modules],>= 1 =
>>>>>>                          bandwidth in Mbps)
>>>>>>                 MCA btl: parameter "btl_self_latency" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Approximate latency of interconnect (must be>= 
>>>>>> 0)
>>>>>>                 MCA btl: parameter "btl_self_priority" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA btl: information "btl_sm_have_knem_support"
>>>>>> (value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether this component supports the knem
>>>>>> Linux kernel module or
>>>>>>                          not
>>>>>>                 MCA btl: parameter "btl_sm_use_knem" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Whether knem support is desired or not
>>>>>> (negative = try to
>>>>>>                          enable knem support, but continue even if it
>>>>>> is not available,
>>>>>>                          0 = do not enable knem support, positive =
>>>>>> try to enable knem
>>>>>>                          support and fail if it is not available)
>>>>>>                 MCA btl: parameter "btl_sm_knem_dma_min" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Minimum message size (in bytes) to use the
>>>>>> knem DMA mode;
>>>>>>                          ignored if knem does not support DMA mode (0
>>>>>> = do not use the
>>>>>>                          knem DMA mode)
>>>>>>                 MCA btl: parameter "btl_sm_knem_max_simultaneous"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Max number of simultaneous ongoing knem
>>>>>> operations to support
>>>>>>                          (0 = do everything synchronously, which
>>>>>> probably gives the best
>>>>>>                          large message latency;>0 means to do all 
>>>>>> operations
>>>>>>                          asynchronously, which supports better
>>>>>> overlap for simultaneous
>>>>>>                          large message sends)
>>>>>>                 MCA btl: parameter "btl_sm_free_list_num" (current
>>>>>> value:<8>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_sm_free_list_max" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_sm_free_list_inc" (current
>>>>>> value:<64>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_sm_max_procs" (current value:
>>>>>> <-1>, data source:
>>>>>>                          default value)
>>>>>>                 MCA btl: parameter "btl_sm_mpool" (current value:
>>>>>> <sm>, data source:
>>>>>>                          default value)
>>>>>>                 MCA btl: parameter "btl_sm_fifo_size" (current value:
>>>>>> <4096>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_sm_num_fifos" (current value:
>>>>>> <1>, data source:
>>>>>>                          default value)
>>>>>>                 MCA btl: parameter "btl_sm_fifo_lazy_free" (current
>>>>>> value:<120>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_sm_sm_extra_procs" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_sm_exclusivity" (current
>>>>>> value:<65535>, data
>>>>>>                          source: default value)
>>>>>>                          BTL exclusivity (must be>= 0)
>>>>>>                 MCA btl: parameter "btl_sm_flags" (current value:
>>>>>> <1>, data source:
>>>>>>                          default value)
>>>>>>                          BTL bit flags (general flags: SEND=1, PUT=2, 
>>>>>> GET=4,
>>>>>>                          SEND_INPLACE=8, RDMA_MATCHED=64,
>>>>>> HETEROGENEOUS_RDMA=256; flags
>>>>>>                          only used by the "dr" PML (ignored by others): 
>>>>>> ACK=16,
>>>>>>                          CHECKSUM=32, RDMA_COMPLETION=128; flags only
>>>>>> used by the "bfo"
>>>>>>                          PML (ignored by others): FAILOVER_SUPPORT=512)
>>>>>>                 MCA btl: parameter "btl_sm_rndv_eager_limit" (current
>>>>>> value:<4096>,
>>>>>>                          data source: default value)
>>>>>>                          Size (in bytes) of "phase 1" fragment sent
>>>>>> for all large
>>>>>>                          messages (must be>= 0 and<= eager_limit)
>>>>>>                 MCA btl: parameter "btl_sm_eager_limit" (current
>>>>>> value:<4096>, data
>>>>>>                          source: default value)
>>>>>>                          Maximum size (in bytes) of "short" messages
>>>>>> (must be>= 1).
>>>>>>                 MCA btl: parameter "btl_sm_max_send_size" (current
>>>>>> value:<32768>, data
>>>>>>                          source: default value)
>>>>>>                          Maximum size (in bytes) of a single "phase
>>>>>> 2" fragment of a
>>>>>>                          long message when using the pipeline
>>>>>> protocol (must be>= 1)
>>>>>>                 MCA btl: parameter "btl_sm_bandwidth" (current value:
>>>>>> <9000>, data
>>>>>>                          source: default value)
>>>>>>                          Approximate maximum bandwidth of
>>>>>> interconnect(0 = auto-detect
>>>>>>                          value at run-time [not supported in all BTL
>>>>>> modules],>= 1 =
>>>>>>                          bandwidth in Mbps)
>>>>>>                 MCA btl: parameter "btl_sm_latency" (current value:
>>>>>> <1>, data source:
>>>>>>                          default value)
>>>>>>                          Approximate latency of interconnect (must be>= 
>>>>>> 0)
>>>>>>                 MCA btl: parameter "btl_sm_priority" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA btl: parameter "btl_tcp_links" (current value:
>>>>>> <1>, data source:
>>>>>>                          default value)
>>>>>>                 MCA btl: parameter "btl_tcp_if_include" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          Comma-delimited list of devices or CIDR
>>>>>> notation of networks to
>>>>>>                          use for MPI communication (e.g., "eth0,eth1" or
>>>>>>                          "192.168.0.0/16,10.1.4.0/24").  Mutually
>>>>>> exclusive with
>>>>>>                          btl_tcp_if_exclude.
>>>>>>                 MCA btl: parameter "btl_tcp_if_exclude" (current
>>>>>> value:<lo,sppp>, data
>>>>>>                          source: default value)
>>>>>>                          Comma-delimited list of devices or CIDR
>>>>>> notation of networks to
>>>>>>                          NOT use for MPI communication -- all devices
>>>>>> not matching these
>>>>>>                          specifications will be used (e.g., "eth0,eth1" 
>>>>>> or
>>>>>>                          "192.168.0.0/16,10.1.4.0/24").  Mutually
>>>>>> exclusive with
>>>>>>                          btl_tcp_if_include.
>>>>>>                 MCA btl: parameter "btl_tcp_free_list_num" (current
>>>>>> value:<8>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_tcp_free_list_max" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_tcp_free_list_inc" (current
>>>>>> value:<32>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_tcp_sndbuf" (current value:
>>>>>> <131072>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_tcp_rcvbuf" (current value:
>>>>>> <131072>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_tcp_endpoint_cache" (current
>>>>>> value:<30720>,
>>>>>>                          data source: default value)
>>>>>>                          The size of the internal cache for each TCP
>>>>>> connection. This
>>>>>>                          cache is used to reduce the number of
>>>>>> syscalls, by replacing
>>>>>>                          them with memcpy. Every read will read the
>>>>>> expected data plus
>>>>>>                          the amount of the endpoint_cache
>>>>>>                 MCA btl: parameter "btl_tcp_use_nagle" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                          Whether to use Nagle's algorithm or not (using 
>>>>>> Nagle's
>>>>>>                          algorithm may increase short message latency)
>>>>>>                 MCA btl: parameter "btl_tcp_port_min_v4" (current
>>>>>> value:<1024>, data
>>>>>>                          source: default value)
>>>>>>                          The minimum port where the TCP BTL will try
>>>>>> to bind (default
>>>>>>                          1024)
>>>>>>                 MCA btl: parameter "btl_tcp_port_range_v4" (current
>>>>>> value:<64511>, data
>>>>>>                          source: default value)
>>>>>>                          The number of ports where the TCP BTL will
>>>>>> try to bind (default
>>>>>>                          64511). This parameter together with the
>>>>>> port min, define a
>>>>>>                          range of ports where Open MPI will open sockets.
>>>>>>                 MCA btl: parameter "btl_tcp_exclusivity" (current
>>>>>> value:<100>, data
>>>>>>                          source: default value)
>>>>>>                          BTL exclusivity (must be>= 0)
>>>>>>                 MCA btl: parameter "btl_tcp_flags" (current value:
>>>>>> <314>, data source:
>>>>>>                          default value)
>>>>>>                          BTL bit flags (general flags: SEND=1, PUT=2, 
>>>>>> GET=4,
>>>>>>                          SEND_INPLACE=8, RDMA_MATCHED=64,
>>>>>> HETEROGENEOUS_RDMA=256; flags
>>>>>>                          only used by the "dr" PML (ignored by others): 
>>>>>> ACK=16,
>>>>>>                          CHECKSUM=32, RDMA_COMPLETION=128; flags only
>>>>>> used by the "bfo"
>>>>>>                          PML (ignored by others): FAILOVER_SUPPORT=512)
>>>>>>                 MCA btl: parameter "btl_tcp_rndv_eager_limit"
>>>>>> (current value:<65536>,
>>>>>>                          data source: default value)
>>>>>>                          Size (in bytes) of "phase 1" fragment sent
>>>>>> for all large
>>>>>>                          messages (must be>= 0 and<= eager_limit)
>>>>>>                 MCA btl: parameter "btl_tcp_eager_limit" (current
>>>>>> value:<65536>, data
>>>>>>                          source: default value)
>>>>>>                          Maximum size (in bytes) of "short" messages
>>>>>> (must be>= 1).
>>>>>>                 MCA btl: parameter "btl_tcp_max_send_size" (current
>>>>>> value:<131072>,
>>>>>>                          data source: default value)
>>>>>>                          Maximum size (in bytes) of a single "phase
>>>>>> 2" fragment of a
>>>>>>                          long message when using the pipeline
>>>>>> protocol (must be>= 1)
>>>>>>                 MCA btl: parameter
>>>>>> "btl_tcp_rdma_pipeline_send_length" (current value:
>>>>>>                          <131072>, data source: default value)
>>>>>>                          Length of the "phase 2" portion of a large
>>>>>> message (in bytes)
>>>>>>                          when using the pipeline protocol.  This part
>>>>>> of the message
>>>>>>                          will be split into fragments of size
>>>>>> max_send_size and sent
>>>>>>                          using send/receive semantics (must be>= 0;
>>>>>> only relevant when
>>>>>>                          the PUT flag is set)
>>>>>>                 MCA btl: parameter "btl_tcp_rdma_pipeline_frag_size"
>>>>>> (current value:
>>>>>>                          <2147483647>, data source: default value)
>>>>>>                          Maximum size (in bytes) of a single "phase
>>>>>> 3" fragment from a
>>>>>>                          long message when using the pipeline
>>>>>> protocol.  These fragments
>>>>>>                          will be sent using RDMA semantics (must be
>>>>>>> = 1; only relevant
>>>>>>                          when the PUT flag is set)
>>>>>>                 MCA btl: parameter "btl_tcp_min_rdma_pipeline_size"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Messages smaller than this size (in bytes)
>>>>>> will not use the
>>>>>>                          RDMA pipeline protocol.  Instead, they will
>>>>>> be split into
>>>>>>                          fragments of max_send_size and sent using 
>>>>>> send/receive
>>>>>>                          semantics (must be>=0, and is automatically
>>>>>> adjusted up to at
>>>>>>                          least
>>>>>> (eager_limit+btl_rdma_pipeline_send_length); only
>>>>>>                          relevant when the PUT flag is set)
>>>>>>                 MCA btl: parameter "btl_tcp_bandwidth" (current
>>>>>> value:<100>, data
>>>>>>                          source: default value)
>>>>>>                          Approximate maximum bandwidth of
>>>>>> interconnect(0 = auto-detect
>>>>>>                          value at run-time [not supported in all BTL
>>>>>> modules],>= 1 =
>>>>>>                          bandwidth in Mbps)
>>>>>>                 MCA btl: parameter "btl_tcp_latency" (current value:
>>>>>> <100>, data source:
>>>>>>                          default value)
>>>>>>                          Approximate latency of interconnect (must be>= 
>>>>>> 0)
>>>>>>                 MCA btl: parameter "btl_tcp_disable_family" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_tcp_if_seq" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          If specified, a comma-delimited list of TCP
>>>>>> interfaces.
>>>>>>                          Interfaces will be assigned, one to each MPI
>>>>>> process, in a
>>>>>>                          round-robin fashion on each server.  For
>>>>>> example, if the list
>>>>>>                          is "eth0,eth1" and four MPI processes are
>>>>>> run on a single
>>>>>>                          server, then local ranks 0 and 2 will use
>>>>>> eth0 and local ranks
>>>>>>                          1 and 3 will use eth1.
>>>>>>                 MCA btl: parameter "btl_tcp_priority" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA btl: parameter "btl_base_include" (current value:
>>>>>> <none>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_base_exclude" (current value:
>>>>>> <none>, data
>>>>>>                          source: default value)
>>>>>>                 MCA btl: parameter "btl_base_warn_component_unused"
>>>>>> (current value:<1>,
>>>>>>                          data source: default value)
>>>>>>                          This parameter is used to turn on warning
>>>>>> messages when certain
>>>>>>                          NICs are not used
>>>>>>                 MCA mtl: parameter "mtl" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> mtl framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA mtl: parameter "mtl_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the mtl framework (0 =
>>>>>> no verbosity)
>>>>>>                MCA topo: parameter "topo" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> topo framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                MCA topo: parameter "topo_base_verbose" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the topo framework (0 =
>>>>>> no verbosity)
>>>>>>                MCA topo: parameter "topo_unity_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA osc: parameter "osc" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> osc framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA osc: parameter "osc_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the osc framework (0 =
>>>>>> no verbosity)
>>>>>>                 MCA osc: parameter "osc_pt2pt_no_locks" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Enable optimizations available only if
>>>>>> MPI_LOCK is not used.
>>>>>>                 MCA osc: parameter "osc_pt2pt_eager_limit" (current
>>>>>> value:<16384>, data
>>>>>>                          source: default value)
>>>>>>                          Max size of eagerly sent data
>>>>>>                 MCA osc: parameter "osc_pt2pt_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA osc: parameter "osc_rdma_eager_send" (current
>>>>>> value:<1>, data
>>>>>>                          source: default value)
>>>>>>                          Attempt to start data movement during
>>>>>> communication call,
>>>>>>                          instead of at synchrnoization time.  Info
>>>>>> key of same name
>>>>>>                          overrides this value.
>>>>>>                 MCA osc: parameter "osc_rdma_use_buffers" (current
>>>>>> value:<1>, data
>>>>>>                          source: default value)
>>>>>>                          Coalesce messages during an epoch to reduce 
>>>>>> network
>>>>>>                          utilization.  Info key of same name
>>>>>> overrides this value.
>>>>>>                 MCA osc: parameter "osc_rdma_use_rdma" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                          Use real RDMA operations to transfer data.
>>>>>> Info key of same
>>>>>>                          name overrides this value.
>>>>>>                 MCA osc: parameter "osc_rdma_rdma_completion_wait"
>>>>>> (current value:<1>,
>>>>>>                          data source: default value)
>>>>>>                          Wait for all completion of rdma events before 
>>>>>> sending
>>>>>>                          acknowledgment.  Info key of same name
>>>>>> overrides this value.
>>>>>>                 MCA osc: parameter "osc_rdma_no_locks" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                          Enable optimizations available only if
>>>>>> MPI_LOCK is not used.
>>>>>>                          Info key of same name overrides this value.
>>>>>>                 MCA osc: parameter "osc_rdma_priority" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                  MCA op: parameter "op_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level of the op framework
>>>>>>                 MCA iof: parameter "iof" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> iof framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA iof: parameter "iof_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the iof framework (0 =
>>>>>> no verbosity)
>>>>>>                 MCA iof: parameter "iof_hnp_priority" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA iof: parameter "iof_orted_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA iof: parameter "iof_tool_priority" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA oob: parameter "oob" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> oob framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA oob: parameter "oob_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the oob framework (0 =
>>>>>> no verbosity)
>>>>>>                 MCA oob: parameter "oob_tcp_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbose level for the OOB tcp component
>>>>>>                 MCA oob: parameter "oob_tcp_peer_limit" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                          Maximum number of peer connections to
>>>>>> simultaneously maintain
>>>>>>                          (-1 = infinite)
>>>>>>                 MCA oob: parameter "oob_tcp_peer_retries" (current
>>>>>> value:<60>, data
>>>>>>                          source: default value)
>>>>>>                          Number of times to try shutting down a
>>>>>> connection before giving
>>>>>>                          up
>>>>>>                 MCA oob: parameter "oob_tcp_debug" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Enable (1) / disable (0) debugging output
>>>>>> for this component
>>>>>>                 MCA oob: parameter "oob_tcp_sndbuf" (current value:
>>>>>> <131072>, data
>>>>>>                          source: default value)
>>>>>>                          TCP socket send buffering size (in bytes)
>>>>>>                 MCA oob: parameter "oob_tcp_rcvbuf" (current value:
>>>>>> <131072>, data
>>>>>>                          source: default value)
>>>>>>                          TCP socket receive buffering size (in bytes)
>>>>>>                 MCA oob: parameter "oob_tcp_if_include" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          Comma-delimited list of TCP interfaces to use
>>>>>>                 MCA oob: parameter "oob_tcp_if_exclude" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          Comma-delimited list of TCP interfaces to 
>>>>>> exclude
>>>>>>                 MCA oob: parameter "oob_tcp_connect_sleep" (current
>>>>>> value:<1>, data
>>>>>>                          source: default value)
>>>>>>                          Enable (1) / disable (0) random sleep for
>>>>>> connection wireup.
>>>>>>                 MCA oob: parameter "oob_tcp_listen_mode" (current
>>>>>> value:<event>, data
>>>>>>                          source: default value)
>>>>>>                          Mode for HNP to accept incoming connections: 
>>>>>> event,
>>>>>>                          listen_thread.
>>>>>>                 MCA oob: parameter "oob_tcp_listen_thread_max_queue"
>>>>>> (current value:
>>>>>>                          <10>, data source: default value)
>>>>>>                          High water mark for queued accepted socket
>>>>>> list size.  Used
>>>>>>                          only when listen_mode is listen_thread.
>>>>>>                 MCA oob: parameter "oob_tcp_listen_thread_wait_time"
>>>>>> (current value:
>>>>>>                          <10>, data source: default value)
>>>>>>                          Time in milliseconds to wait before actively
>>>>>> checking for new
>>>>>>                          connections when listen_mode is listen_thread.
>>>>>>                 MCA oob: parameter "oob_tcp_static_ports" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          Static ports for daemons and procs (IPv4)
>>>>>>                 MCA oob: parameter "oob_tcp_dynamic_ports" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          Range of ports to be dynamically used by
>>>>>> daemons and procs
>>>>>>                          (IPv4)
>>>>>>                 MCA oob: parameter "oob_tcp_disable_family" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Disable IPv4 (4) or IPv6 (6)
>>>>>>                 MCA oob: parameter "oob_tcp_priority" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                MCA odls: parameter "odls_base_sigkill_timeout"
>>>>>> (current value:<1>, data
>>>>>>                          source: default value)
>>>>>>                          Time to wait for a process to die after
>>>>>> issuing a kill signal
>>>>>>                          to it
>>>>>>                MCA odls: parameter "odls" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> odls framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                MCA odls: parameter "odls_base_verbose" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the odls framework (0 =
>>>>>> no verbosity)
>>>>>>                MCA odls: parameter "odls_default_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA ras: parameter "ras_base_display_alloc" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether to display the allocation after it
>>>>>> is determined
>>>>>>                 MCA ras: parameter "ras_base_display_devel_alloc"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Whether to display a developer-detail
>>>>>> allocation after it is
>>>>>>                          determined
>>>>>>                 MCA ras: parameter "ras" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> ras framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA ras: parameter "ras_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the ras framework (0 =
>>>>>> no verbosity)
>>>>>>                 MCA ras: parameter "ras_cm_priority" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA ras: parameter "ras_loadleveler_priority"
>>>>>> (current value:<90>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the loadleveler ras component
>>>>>>                 MCA ras: parameter "ras_slurm_priority" (current
>>>>>> value:<75>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the slurm ras component
>>>>>>               MCA rmaps: parameter "rmaps_rank_file_path" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value, synonym of: 
>>>>>> orte_rankfile)
>>>>>>                          Name of the rankfile to be used for mapping
>>>>>> processes (relative
>>>>>>                          or absolute path)
>>>>>>               MCA rmaps: parameter "rmaps_base_schedule_policy"
>>>>>> (current value:<slot>,
>>>>>>                          data source: default value)
>>>>>>                          Scheduling Policy for RMAPS. [slot
>>>>>> (alias:core) | socket |
>>>>>>                          board | node]
>>>>>>               MCA rmaps: parameter "rmaps_base_pernode" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Launch one ppn as directed
>>>>>>               MCA rmaps: parameter "rmaps_base_n_pernode" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                          Launch n procs/node
>>>>>>               MCA rmaps: parameter "rmaps_base_n_perboard" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                          Launch n procs/board
>>>>>>               MCA rmaps: parameter "rmaps_base_n_persocket" (current
>>>>>> value:<-1>, data
>>>>>>                          source: default value)
>>>>>>                          Launch n procs/socket
>>>>>>               MCA rmaps: parameter "rmaps_base_loadbalance" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Balance total number of procs across all
>>>>>> allocated nodes
>>>>>>               MCA rmaps: parameter "rmaps_base_cpus_per_proc"
>>>>>> (current value:<1>, data
>>>>>>                          source: default value, synonyms:
>>>>>> rmaps_base_cpus_per_rank)
>>>>>>                          Number of cpus to use for each rank [1-2**15
>>>>>> (default=1)]
>>>>>>               MCA rmaps: parameter "rmaps_base_cpus_per_rank"
>>>>>> (current value:<1>, data
>>>>>>                          source: default value, synonym of:
>>>>>> rmaps_base_cpus_per_proc)
>>>>>>                          Number of cpus to use for each rank [1-2**15
>>>>>> (default=1)]
>>>>>>               MCA rmaps: parameter "rmaps_base_stride" (current
>>>>>> value:<1>, data source:
>>>>>>                          default value)
>>>>>>                          When binding multiple cores to a rank, the
>>>>>> step size to use
>>>>>>                          between cores [1-2**15 (default: 1)]
>>>>>>               MCA rmaps: parameter "rmaps_base_slot_list" (current
>>>>>> value:<none>, data
>>>>>>                          source: default value)
>>>>>>                          List of processor IDs to bind MPI processes
>>>>>> to (e.g., used in
>>>>>>                          conjunction with rank files) [default=NULL]
>>>>>>               MCA rmaps: parameter "rmaps_base_no_schedule_local"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          If false, allow scheduling MPI applications
>>>>>> on the same node as
>>>>>>                          mpirun (default).  If true, do not schedule any 
>>>>>> MPI
>>>>>>                          applications on the same node as mpirun
>>>>>>               MCA rmaps: parameter "rmaps_base_no_oversubscribe"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          If true, then do not allow oversubscription
>>>>>> of nodes - mpirun
>>>>>>                          will return an error if there aren't enough
>>>>>> nodes to launch all
>>>>>>                          processes without oversubscribing
>>>>>>               MCA rmaps: parameter "rmaps_base_display_map" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Whether to display the process map after it
>>>>>> is computed
>>>>>>               MCA rmaps: parameter "rmaps_base_display_devel_map"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>                          Whether to display a developer-detail
>>>>>> process map after it is
>>>>>>                          computed
>>>>>>               MCA rmaps: parameter "rmaps" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> rmaps framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>               MCA rmaps: parameter "rmaps_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the rmaps framework (0 =
>>>>>> no verbosity)
>>>>>>               MCA rmaps: parameter "rmaps_load_balance_priority"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>               MCA rmaps: parameter "rmaps_rank_file_priority"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>               MCA rmaps: parameter "rmaps_resilient_fault_grp_file"
>>>>>> (current value:
>>>>>>                          <none>, data source: default value)
>>>>>>                          Filename that contains a description of
>>>>>> fault groups for this
>>>>>>                          system
>>>>>>               MCA rmaps: parameter "rmaps_resilient_priority"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>               MCA rmaps: parameter "rmaps_round_robin_priority"
>>>>>> (current value:<0>,
>>>>>>                          data source: default value)
>>>>>>               MCA rmaps: parameter "rmaps_seq_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>               MCA rmaps: parameter "rmaps_topo_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA rml: parameter "rml_wrapper" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          Use a Wrapper component around the selected
>>>>>> RML component
>>>>>>                 MCA rml: parameter "rml" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> rml framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA rml: parameter "rml_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the rml framework (0 =
>>>>>> no verbosity)
>>>>>>                 MCA rml: parameter "rml_oob_priority" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>              MCA routed: parameter "routed" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> routed framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>              MCA routed: parameter "routed_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the routed framework (0
>>>>>> = no verbosity)
>>>>>>              MCA routed: parameter "routed_binomial_priority"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>              MCA routed: parameter "routed_cm_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>              MCA routed: parameter "routed_direct_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>              MCA routed: parameter "routed_linear_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>              MCA routed: parameter "routed_radix_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>              MCA routed: parameter "routed_slave_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA plm: parameter "plm_rsh_agent" (current value:
>>>>>> <ssh : rsh>, data
>>>>>>                          source: default value, deprecated, synonym
>>>>>> of: orte_rsh_agent)
>>>>>>                          The command used to launch executables on 
>>>>>> remote nodes
>>>>>>                          (typically either "ssh" or "rsh")
>>>>>>                 MCA plm: parameter "plm_rsh_assume_same_shell"
>>>>>> (current value:<1>, data
>>>>>>                          source: default value, deprecated, synonym of:
>>>>>>                          orte_assume_same_shell)
>>>>>>                          If set to 1, assume that the shell on the
>>>>>> remote node is the
>>>>>>                          same as the shell on the local node.
>>>>>> Otherwise, probe for what
>>>>>>                          the remote shell [default: 1]
>>>>>>                 MCA plm: parameter "plm" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> plm framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA plm: parameter "plm_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the plm framework (0 =
>>>>>> no verbosity)
>>>>>>                 MCA plm: parameter "plm_rsh_num_concurrent" (current
>>>>>> value:<128>, data
>>>>>>                          source: default value)
>>>>>>                          How many plm_rsh_agent instances to invoke
>>>>>> concurrently (must
>>>>>>                          be>  0)
>>>>>>                 MCA plm: parameter "plm_rsh_force_rsh" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                          Force the launcher to always use rsh
>>>>>>                 MCA plm: parameter "plm_rsh_disable_qrsh" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Disable the launcher to use qrsh when under
>>>>>> the SGE parallel
>>>>>>                          environment
>>>>>>                 MCA plm: parameter "plm_rsh_daemonize_qrsh" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Daemonize the orted under the SGE parallel 
>>>>>> environment
>>>>>>                 MCA plm: parameter "plm_rsh_disable_llspawn" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Disable the use of llspawn when under the 
>>>>>> LoadLeveler
>>>>>>                          environment
>>>>>>                 MCA plm: parameter "plm_rsh_daemonize_llspawn"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Daemonize the orted when under the
>>>>>> LoadLeveler environment
>>>>>>                 MCA plm: parameter "plm_rsh_priority" (current value:
>>>>>> <10>, data source:
>>>>>>                          default value)
>>>>>>                          Priority of the rsh plm component
>>>>>>                 MCA plm: parameter "plm_rsh_delay" (current value:
>>>>>> <1>, data source:
>>>>>>                          default value)
>>>>>>                          Delay (in seconds) between invocations of
>>>>>> the remote agent, but
>>>>>>                          only used when the "debug" MCA parameter is
>>>>>> true, or the
>>>>>>                          top-level MCA debugging is enabled
>>>>>> (otherwise this value is
>>>>>>                          ignored)
>>>>>>                 MCA plm: parameter "plm_rsh_tree_spawn" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          If set to 1, launch via a tree-based topology
>>>>>>                 MCA plm: parameter "plm_slurm_args" (current value:
>>>>>> <none>, data source:
>>>>>>                          default value)
>>>>>>                          Custom arguments to srun
>>>>>>                 MCA plm: parameter "plm_slurm_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>               MCA filem: parameter "filem" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Which Filem component to use (empty = 
>>>>>> auto-select)
>>>>>>               MCA filem: parameter "filem_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the filem framework (0 =
>>>>>> no verbosity)
>>>>>>               MCA filem: parameter "filem_rsh_priority" (current
>>>>>> value:<20>, data
>>>>>>                          source: default value)
>>>>>>                          Priority of the FILEM rsh component
>>>>>>               MCA filem: parameter "filem_rsh_verbose" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbose level for the FILEM rsh component
>>>>>>               MCA filem: parameter "filem_rsh_rcp" (current value:
>>>>>> <scp>, data source:
>>>>>>                          default value)
>>>>>>                          The rsh cp command for the FILEM rsh component
>>>>>>               MCA filem: parameter "filem_rsh_cp" (current value:
>>>>>> <cp>, data source:
>>>>>>                          default value)
>>>>>>                          The Unix cp command for the FILEM rsh component
>>>>>>               MCA filem: parameter "filem_rsh_rsh" (current value:
>>>>>> <ssh>, data source:
>>>>>>                          default value)
>>>>>>                          The remote shell command for the FILEM rsh 
>>>>>> component
>>>>>>               MCA filem: parameter "filem_rsh_max_incomming" (current
>>>>>> value:<10>, data
>>>>>>                          source: default value)
>>>>>>                          Maximum number of incomming connections (0 = 
>>>>>> any)
>>>>>>               MCA filem: parameter "filem_rsh_max_outgoing" (current
>>>>>> value:<10>, data
>>>>>>                          source: default value)
>>>>>>                          Maximum number of out going connections (0 = 
>>>>>> any)
>>>>>>              MCA errmgr: parameter "errmgr" (current value:<none>,
>>>>>> data source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> errmgr framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>              MCA errmgr: parameter "errmgr_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the errmgr framework (0
>>>>>> = no verbosity)
>>>>>>              MCA errmgr: parameter "errmgr_default_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA ess: parameter "ess" (current value:<none>, data
>>>>>> source: default
>>>>>>                          value)
>>>>>>                          Default selection set of components for the
>>>>>> ess framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>                 MCA ess: parameter "ess_base_verbose" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                          Verbosity level for the ess framework (0 =
>>>>>> no verbosity)
>>>>>>                 MCA ess: parameter "ess_env_priority" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA ess: parameter "ess_hnp_priority" (current value:
>>>>>> <0>, data source:
>>>>>>                          default value)
>>>>>>                 MCA ess: parameter "ess_singleton_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA ess: parameter "ess_slave_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA ess: parameter "ess_slurm_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA ess: parameter "ess_slurmd_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                 MCA ess: parameter "ess_tool_priority" (current
>>>>>> value:<0>, data source:
>>>>>>                          default value)
>>>>>>             MCA grpcomm: parameter "grpcomm" (current value:<none>,
>>>>>> data source:
>>>>>>                          default value)
>>>>>>                          Default selection set of components for the
>>>>>> grpcomm framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>             MCA grpcomm: parameter "grpcomm_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the grpcomm framework (0
>>>>>> = no verbosity)
>>>>>>             MCA grpcomm: parameter "grpcomm_bad_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>             MCA grpcomm: parameter "grpcomm_basic_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>             MCA grpcomm: parameter "grpcomm_hier_priority" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>            MCA notifier: parameter "notifier_threshold_severity"
>>>>>> (current value:
>>>>>>                          <critical>, data source: default value)
>>>>>>                          Report all events at or above this severity
>>>>>> [default: critical]
>>>>>>            MCA notifier: parameter "notifier" (current value:<none>,
>>>>>> data source:
>>>>>>                          default value)
>>>>>>                          Default selection set of components for the
>>>>>> notifier framework
>>>>>>                          (<none>  means use all components that can be 
>>>>>> found)
>>>>>>            MCA notifier: parameter "notifier_base_verbose" (current
>>>>>> value:<0>, data
>>>>>>                          source: default value)
>>>>>>                          Verbosity level for the notifier framework
>>>>>> (0 = no verbosity)
>>>>>>            MCA notifier: parameter "notifier_command_cmd" (current
>>>>>> value:</sbin/initlog
>>>>>>                          -f $s -n "Open MPI" -s "$S: $m (errorcode:
>>>>>> $e)">, data source:
>>>>>>                          default value)
>>>>>>                          Command to execute, with substitution.  $s =
>>>>>> integer severity;
>>>>>>                          $S = string severity; $e = integer error
>>>>>> code; $m = string
>>>>>>                          message
>>>>>>            MCA notifier: parameter "notifier_command_timeout"
>>>>>> (current value:<30>, data
>>>>>>                          source: default value)
>>>>>>                          Timeout (in seconds) of the command
>>>>>>            MCA notifier: parameter "notifier_command_priority"
>>>>>> (current value:<10>,
>>>>>>                          data source: default value)
>>>>>>                          Priority of this component
>>>>>>            MCA notifier: parameter "notifier_syslog_priority"
>>>>>> (current value:<0>, data
>>>>>>                          source: default value)
>>>>>> 
>>>>>> ====================================================================================================
>>>>>> 
>>>>>> output of cat /proc/cpuinfo
>>>>>> processor     : 0
>>>>>> vendor_id     : AuthenticAMD
>>>>>> cpu family    : 15
>>>>>> model         : 75
>>>>>> model name    : AMD Athlon(tm) 64 X2 Dual Core Processor 3800+
>>>>>> stepping      : 2
>>>>>> cpu MHz               : 1002.094
>>>>>> cache size    : 512 KB
>>>>>> physical id   : 0
>>>>>> siblings      : 2
>>>>>> core id               : 0
>>>>>> cpu cores     : 2
>>>>>> fpu           : yes
>>>>>> fpu_exception : yes
>>>>>> cpuid level   : 1
>>>>>> wp            : yes
>>>>>> flags         : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
>>>>>> cmov
>>>>>> pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext
>>>>>> 3dnow pni cx16
>>>>>> bogomips      : 2003.90
>>>>>> TLB size      : 1088 4K pages
>>>>>> clflush size  : 64
>>>>>> cache_alignment       : 64
>>>>>> address sizes : 40 bits physical, 48 bits virtual
>>>>>> power management: ts fid vid ttp [4] [5]
>>>>>> 
>>>>>> processor     : 1
>>>>>> vendor_id     : AuthenticAMD
>>>>>> cpu family    : 15
>>>>>> model         : 75
>>>>>> model name    : AMD Athlon(tm) 64 X2 Dual Core Processor 3800+
>>>>>> stepping      : 2
>>>>>> cpu MHz               : 1002.094
>>>>>> cache size    : 512 KB
>>>>>> physical id   : 0
>>>>>> siblings      : 2
>>>>>> core id               : 1
>>>>>> cpu cores     : 2
>>>>>> fpu           : yes
>>>>>> fpu_exception : yes
>>>>>> cpuid level   : 1
>>>>>> wp            : yes
>>>>>> flags         : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
>>>>>> cmov
>>>>>> pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext lm 3dnowext
>>>>>> 3dnow pni cx16
>>>>>> bogomips      : 2003.90
>>>>>> TLB size      : 1088 4K pages
>>>>>> clflush size  : 64
>>>>>> cache_alignment       : 64
>>>>>> address sizes : 40 bits physical, 48 bits virtual
>>>>>> power management: ts fid vid ttp [4] [5]
>>>>>> 
>>>>>> 
>>>>>> ====================================================================================================
>>>>>> output of ifconfig -a from a compute node
>>>>>> eth0      Link encap:Ethernet  HWaddr 00:18:F3:3F:84:A1
>>>>>>          inet addr:192.168.0.2  Bcast:192.168.0.255  Mask:255.255.255.0
>>>>>>          inet6 addr: fe80::218:f3ff:fe3f:84a1/64 Scope:Link
>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>          RX packets:2006 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:2064 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:1000
>>>>>>          RX bytes:242685 (236.9 KiB)  TX bytes:0 (0.0 b)
>>>>>>          Interrupt:11 Base address:0x8000
>>>>>> 
>>>>>> lo        Link encap:Local Loopback
>>>>>>          inet addr:127.0.0.1  Mask:255.0.0.0
>>>>>>          inet6 addr: ::1/128 Scope:Host
>>>>>>          UP LOOPBACK RUNNING  MTU:16436  Metric:1
>>>>>>          RX packets:60 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:60 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:0
>>>>>>          RX bytes:4440 (4.3 KiB)  TX bytes:4440 (4.3 KiB)
>>>>>> 
>>>>>> sit0      Link encap:IPv6-in-IPv4
>>>>>>          NOARP  MTU:1480  Metric:1
>>>>>>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:0
>>>>>>          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
>>>>>> 
>>>>>> 
>>>>>> ====================================================================================================
>>>>>> output of ifconfig -a from the login node, where I run mpirun
>>>>>> 
>>>>>> eth0      Link encap:Ethernet  HWaddr 00:18:F3:51:B3:6E
>>>>>>          inet addr:192.168.0.1  Bcast:192.168.0.255  Mask:255.255.255.0
>>>>>>          inet6 addr: fe80::218:f3ff:fe51:b36e/64 Scope:Link
>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>          RX packets:7180758 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:4989496 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:1000
>>>>>>          RX bytes:6045614452 (5.6 GiB)  TX bytes:0 (0.0 b)
>>>>>>          Interrupt:201 Base address:0xe000
>>>>>> 
>>>>>> eth1      Link encap:Ethernet  HWaddr 00:01:02:13:AA:3C
>>>>>>          inet addr:137.204.66.188  Bcast:137.204.66.255  
>>>>>> Mask:255.255.255.0
>>>>>>          inet6 addr: fe80::201:2ff:fe13:aa3c/64 Scope:Link
>>>>>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>>>>          RX packets:4750212 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:405027 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:1000
>>>>>>          RX bytes:629146679 (600.0 MiB)  TX bytes:332118265 (316.7 MiB)
>>>>>>          Interrupt:177 Base address:0x9c00
>>>>>> 
>>>>>> lo        Link encap:Local Loopback
>>>>>>          inet addr:127.0.0.1  Mask:255.0.0.0
>>>>>>          inet6 addr: ::1/128 Scope:Host
>>>>>>          UP LOOPBACK RUNNING  MTU:16436  Metric:1
>>>>>>          RX packets:288455 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:288455 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:0
>>>>>>          RX bytes:35908038 (34.2 MiB)  TX bytes:35908038 (34.2 MiB)
>>>>>> 
>>>>>> sit0      Link encap:IPv6-in-IPv4
>>>>>>          NOARP  MTU:1480  Metric:1
>>>>>>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>>>>          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>>>>          collisions:0 txqueuelen:0
>>>>>>          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
>>>>>> 
>>>>>> ====================================================================================================
>>>>>> output of mpirun --bynode --hostfile my_hostfile.txt --tag-output
>>>>>> ompi_info -v ompi full --parsable
>>>>>> 
>>>>>> [1,0]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,0]<stdout>:ompi:version:full:1.6
>>>>>> [1,0]<stdout>:ompi:version:svn:r26429
>>>>>> [1,0]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,0]<stdout>:orte:version:full:1.6
>>>>>> [1,0]<stdout>:orte:version:svn:r26429
>>>>>> [1,0]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,0]<stdout>:opal:version:full:1.6
>>>>>> [1,0]<stdout>:opal:version:svn:r26429
>>>>>> [1,0]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,0]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,0]<stdout>:ident:1.6
>>>>>> [1,6]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,6]<stdout>:ompi:version:full:1.6
>>>>>> [1,6]<stdout>:ompi:version:svn:r26429
>>>>>> [1,6]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,6]<stdout>:orte:version:full:1.6
>>>>>> [1,6]<stdout>:orte:version:svn:r26429
>>>>>> [1,6]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,6]<stdout>:opal:version:full:1.6
>>>>>> [1,6]<stdout>:opal:version:svn:r26429
>>>>>> [1,6]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,6]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,6]<stdout>:ident:1.6
>>>>>> [1,9]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,10]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,3]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,3]<stdout>:ompi:version:full:1.6
>>>>>> [1,3]<stdout>:ompi:version:svn:r26429
>>>>>> [1,3]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,3]<stdout>:orte:version:full:1.6
>>>>>> [1,3]<stdout>:orte:version:svn:r26429
>>>>>> [1,3]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,3]<stdout>:opal:version:full:1.6
>>>>>> [1,3]<stdout>:opal:version:svn:r26429
>>>>>> [1,3]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,3]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,3]<stdout>:ident:1.6
>>>>>> [1,4]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,4]<stdout>:ompi:version:full:1.6
>>>>>> [1,4]<stdout>:ompi:version:svn:r26429
>>>>>> [1,4]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,4]<stdout>:orte:version:full:1.6
>>>>>> [1,4]<stdout>:orte:version:svn:r26429
>>>>>> [1,4]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,4]<stdout>:opal:version:full:1.6
>>>>>> [1,9]<stdout>:ompi:version:full:1.6
>>>>>> [1,4]<stdout>:opal:version:svn:r26429
>>>>>> [1,4]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,4]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,4]<stdout>:ident:1.6
>>>>>> [1,9]<stdout>:ompi:version:svn:r26429
>>>>>> [1,10]<stdout>:ompi:version:full:1.6
>>>>>> [1,9]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,10]<stdout>:ompi:version:svn:r26429
>>>>>> [1,9]<stdout>:orte:version:full:1.6
>>>>>> [1,10]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,9]<stdout>:orte:version:svn:r26429
>>>>>> [1,10]<stdout>:orte:version:full:1.6
>>>>>> [1,10]<stdout>:orte:version:svn:r26429
>>>>>> [1,9]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,10]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,9]<stdout>:opal:version:full:1.6
>>>>>> [1,10]<stdout>:opal:version:full:1.6
>>>>>> [1,9]<stdout>:opal:version:svn:r26429
>>>>>> [1,10]<stdout>:opal:version:svn:r26429
>>>>>> [1,9]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,10]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,9]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,9]<stdout>:ident:1.6
>>>>>> [1,10]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,10]<stdout>:ident:1.6
>>>>>> [1,2]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,2]<stdout>:ompi:version:full:1.6
>>>>>> [1,2]<stdout>:ompi:version:svn:r26429
>>>>>> [1,2]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,2]<stdout>:orte:version:full:1.6
>>>>>> [1,2]<stdout>:orte:version:svn:r26429
>>>>>> [1,2]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,2]<stdout>:opal:version:full:1.6
>>>>>> [1,2]<stdout>:opal:version:svn:r26429
>>>>>> [1,2]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,2]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,2]<stdout>:ident:1.6
>>>>>> [1,8]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,8]<stdout>:ompi:version:full:1.6
>>>>>> [1,8]<stdout>:ompi:version:svn:r26429
>>>>>> [1,8]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,8]<stdout>:orte:version:full:1.6
>>>>>> [1,8]<stdout>:orte:version:svn:r26429
>>>>>> [1,8]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,8]<stdout>:opal:version:full:1.6
>>>>>> [1,8]<stdout>:opal:version:svn:r26429
>>>>>> [1,8]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,8]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,8]<stdout>:ident:1.6
>>>>>> [1,11]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,11]<stdout>:ompi:version:full:1.6
>>>>>> [1,11]<stdout>:ompi:version:svn:r26429
>>>>>> [1,11]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,11]<stdout>:orte:version:full:1.6
>>>>>> [1,11]<stdout>:orte:version:svn:r26429
>>>>>> [1,11]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,11]<stdout>:opal:version:full:1.6
>>>>>> [1,11]<stdout>:opal:version:svn:r26429
>>>>>> [1,11]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,11]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,11]<stdout>:ident:1.6
>>>>>> [1,5]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,5]<stdout>:ompi:version:full:1.6
>>>>>> [1,5]<stdout>:ompi:version:svn:r26429
>>>>>> [1,5]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,5]<stdout>:orte:version:full:1.6
>>>>>> [1,5]<stdout>:orte:version:svn:r26429
>>>>>> [1,5]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,5]<stdout>:opal:version:full:1.6
>>>>>> [1,5]<stdout>:opal:version:svn:r26429
>>>>>> [1,5]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,5]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,5]<stdout>:ident:1.6
>>>>>> [1,1]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,7]<stdout>:package:Open MPI and...@cloud.bo.astro.it Distribution
>>>>>> [1,7]<stdout>:ompi:version:full:1.6
>>>>>> [1,7]<stdout>:ompi:version:svn:r26429
>>>>>> [1,7]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,7]<stdout>:orte:version:full:1.6
>>>>>> [1,7]<stdout>:orte:version:svn:r26429
>>>>>> [1,7]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,7]<stdout>:opal:version:full:1.6
>>>>>> [1,7]<stdout>:opal:version:svn:r26429
>>>>>> [1,7]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,7]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,7]<stdout>:ident:1.6
>>>>>> [1,1]<stdout>:ompi:version:full:1.6
>>>>>> [1,1]<stdout>:ompi:version:svn:r26429
>>>>>> [1,1]<stdout>:ompi:version:release_date:May 10, 2012
>>>>>> [1,1]<stdout>:orte:version:full:1.6
>>>>>> [1,1]<stdout>:orte:version:svn:r26429
>>>>>> [1,1]<stdout>:orte:version:release_date:May 10, 2012
>>>>>> [1,1]<stdout>:opal:version:full:1.6
>>>>>> [1,1]<stdout>:opal:version:svn:r26429
>>>>>> [1,1]<stdout>:opal:version:release_date:May 10, 2012
>>>>>> [1,1]<stdout>:mpi-api:version:full:2.1
>>>>>> [1,1]<stdout>:ident:1.6
>>>>>> _______________________________________________
>>>>>> users mailing list
>>>>>> us...@open-mpi.org
>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>> 
>>>>> 
>>>>> 
>>>>> ------------------------------
>>>>> 
>>>>> _______________________________________________
>>>>> users mailing list
>>>>> us...@open-mpi.org
>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>> 
>>>>> End of users Digest, Vol 2339, Issue 5
>>>>> **************************************
>>>> 
>>>> 
>>>> ------------------------------
>>>> 
>>>> _______________________________________________
>>>> users mailing list
>>>> us...@open-mpi.org
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>> 
>>>> End of users Digest, Vol 2340, Issue 1
>>>> **************************************
>>> _______________________________________________
>>> users mailing list
>>> us...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>> 
>> 
>> 
>> 
>> ------------------------------
>> 
>> _______________________________________________
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>> 
>> End of users Digest, Vol 2342, Issue 3
>> **************************************
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users


Reply via email to