I 've got the same problem as described in:
http://www.open-mpi.org/community/lists/users/2006/07/1537.php

From: Chengwen Chen (chenchengwen_at_[hidden])
List-Post: users@lists.open-mpi.org
Date: 2006-07-04 03:53:26



The problem seems to occur randomly! It occurs more often if I use a
larger number of cpu, but always never if I use a small number of cpus.
So far my cure to the problem is to kill and restart my application
(mpirun ...) as often untill the error won't occur and mpirun will run.

So is the problem resolved. Can anybody give me an hint?

I am using a amd64 linux (suse10) cluster with infiniband conection and
openmpi-1.2a1r10111.

I attach the ompi_info --param all all output, hope it helps.

Regards Jens
                 MCA mca: parameter "mca_param_files" (current value: 
"/home/klosterm/.openmpi/mca-params.conf:/home/pub/OpenFOAM/OpenFOAM-1.3/src/openmpi-1.2a1r10111/platforms/linuxAMD64Gcc4DPOpt/etc/openmpi-mca-params.conf")
                          Path for MCA configuration files containing default 
parameter values
                 MCA mca: parameter "mca_component_path" (current value: 
"/home/pub/OpenFOAM/OpenFOAM-1.3/src/openmpi-1.2a1r10111/platforms/linuxAMD64Gcc4DPOpt/lib/openmpi:/home/klosterm/.openmpi/components")
                          Path where to look for Open MPI and ORTE components
                 MCA mca: parameter "mca_verbose" (current value: <none>)
                          Top-level verbosity parameter
                 MCA mca: parameter "mca_component_show_load_errors" (current 
value: "1")
                          Whether to show errors for components that failed to 
load or not
                 MCA mca: parameter "mca_component_disable_dlopen" (current 
value: "0")
                          Whether to attempt to disable opening dynamic 
components or not
                 MCA mpi: parameter "mpi_param_check" (current value: "1")
                          Whether you want MPI API parameters checked at 
run-time or not.  Possible values are 0 (no checking) and 1 (perform checking 
at run-time)
                 MCA mpi: parameter "mpi_yield_when_idle" (current value: "0")
                          Yield the processor when waiting for MPI 
communication (for MPI processes, will default to 1 when oversubscribing nodes)
                 MCA mpi: parameter "mpi_event_tick_rate" (current value: "-1")
                          How often to progress TCP communications (0 = never, 
otherwise specified in microseconds)
                 MCA mpi: parameter "mpi_show_handle_leaks" (current value: "0")
                          Whether MPI_FINALIZE shows all MPI handles that were 
not freed or not
                 MCA mpi: parameter "mpi_no_free_handles" (current value: "0")
                          Whether to actually free MPI objects when their 
handles are freed
                 MCA mpi: parameter "mpi_show_mca_params" (current value: "0")
                          Whether to show all MCA parameter value during 
MPI_INIT or not (good for reproducability of MPI jobs)
                 MCA mpi: parameter "mpi_show_mca_params_file" (current value: 
<none>)
                          If mpi_show_mca_params is true, setting this string 
to a valid filename tells Open MPI to dump all the MCA parameter values into a 
file suitable for reading via the mca_param_files parameter (good for 
reproducability of MPI jobs)
                 MCA mpi: parameter "mpi_paffinity_alone" (current value: "0")
                          If nonzero, assume that this job is the only (set of) 
process(es) running on each node and bind processes to processors, starting 
with processor ID 0
                 MCA mpi: parameter "mpi_keep_peer_hostnames" (current value: 
"1")
                          If nonzero, save the string hostnames of all MPI peer 
processes (mostly for error / debugging output messages).  This can add quite a 
bit of memory usage to each MPI process.
                 MCA mpi: parameter "mpi_abort_delay" (current value: "0")
                          If nonzero, print out an identifying message when 
MPI_ABORT is invoked (hostname, PID of the process that called MPI_ABORT) and 
delay for that many seconds before exiting (a negative delay value means to 
never abort).  This allows attaching of a debugger before quitting the job.
                 MCA mpi: parameter "mpi_abort_print_stack" (current value: "0")
                          If nonzero, print out a stack trace when MPI_ABORT is 
invoked
                 MCA mpi: parameter "mpi_leave_pinned" (current value: "0")
                          leave_pinned
                 MCA mpi: parameter "mpi_leave_pinned_pipeline" (current value: 
"0")
                          leave_pinned_pipeline
                MCA orte: parameter "orte_base_user_debugger" (current value: 
"totalview @mpirun@ -a @mpirun_args@ : fxp @mpirun@ -a @mpirun_args@")
                          Sequence of user-level debuggers to search for in 
orterun
                MCA orte: parameter "orte_debug" (current value: "0")
                          Whether or not to enable debugging output for all 
ORTE components (0 or 1)
                MCA opal: parameter "opal_signal" (current value: "6,7,8,11")
                          If a signal is received, display the stack trace frame
              MCA memory: parameter "memory" (current value: <none>)
                          Default selection set of components for the memory 
framework (<none> means "use all components that can be found")
              MCA memory: parameter "memory_base_verbose" (current value: "0")
                          Verbosity level for the memory framework (0 = no 
verbosity)
              MCA memory: parameter "memory_ptmalloc2_priority" (current value: 
"0")
           MCA paffinity: parameter "paffinity" (current value: <none>)
                          Default selection set of components for the paffinity 
framework (<none> means "use all components that can be found")
           MCA paffinity: parameter "paffinity_linux_priority" (current value: 
"10")
                          Priority of the linux paffinity component
           MCA paffinity: information "paffinity_linux_have_cpu_set_t" (value: 
"1")
                          Whether this component was compiled on a system with 
the type cpu_set_t or not (1 = yes, 0 = no)
           MCA paffinity: information "paffinity_linux_CPU_ZERO_ok" (value: "1")
                          Whether this component was compiled on a system where 
CPU_ZERO() is functional or broken (1 = functional, 0 = broken/not available)
           MCA paffinity: information 
"paffinity_linux_sched_setaffinity_num_params" (value: "3")
                          The number of parameters that sched_set_affinity() 
takes on the machine where this component was compiled
           MCA maffinity: parameter "maffinity" (current value: <none>)
                          Default selection set of components for the maffinity 
framework (<none> means "use all components that can be found")
           MCA maffinity: parameter "maffinity_first_use_priority" (current 
value: "10")
                          Priority of the first_use maffinity component
           MCA maffinity: parameter "maffinity_libnuma_priority" (current 
value: "25")
                          Priority of the libnuma maffinity component
               MCA timer: parameter "timer" (current value: <none>)
                          Default selection set of components for the timer 
framework (<none> means "use all components that can be found")
               MCA timer: parameter "timer_base_verbose" (current value: "0")
                          Verbosity level for the timer framework (0 = no 
verbosity)
               MCA timer: parameter "timer_linux_priority" (current value: "0")
           MCA allocator: parameter "allocator" (current value: <none>)
                          Default selection set of components for the allocator 
framework (<none> means "use all components that can be found")
           MCA allocator: parameter "allocator_base_verbose" (current value: 
"0")
                          Verbosity level for the allocator framework (0 = no 
verbosity)
           MCA allocator: parameter "allocator_basic_priority" (current value: 
"0")
           MCA allocator: parameter "allocator_bucket_num_buckets" (current 
value: "30")
           MCA allocator: parameter "allocator_bucket_priority" (current value: 
"0")
                MCA coll: parameter "coll" (current value: <none>)
                          Default selection set of components for the coll 
framework (<none> means "use all components that can be found")
                MCA coll: parameter "coll_base_verbose" (current value: "0")
                          Verbosity level for the coll framework (0 = no 
verbosity)
                MCA coll: parameter "coll_basic_priority" (current value: "10")
                          Priority of the basic coll component
                MCA coll: parameter "coll_basic_crossover" (current value: "4")
                          Minimum number of processes in a communicator before 
using the logarithmic algorithms
                MCA coll: parameter "coll_hierarch_priority" (current value: 
"0")
                          Priority of the hierarchical coll component
                MCA coll: parameter "coll_hierarch_verbose" (current value: "0")
                          Turn verbose message of the hierarchical coll 
component on/off
                MCA coll: parameter "coll_hierarch_use_rdma" (current value: 
"0")
                          Switch from the send btl list used to detect 
hierarchies to the rdma btl list
                MCA coll: parameter "coll_hierarch_ignore_sm" (current value: 
"0")
                          Ignore sm protocol when detecting hierarchies. 
Required to enable the usage of protocol specific collective operations
                MCA coll: parameter "coll_hierarch_symmetric" (current value: 
"0")
                          Assume symmetric configuration
                MCA coll: parameter "coll_self_priority" (current value: "75")
                MCA coll: parameter "coll_sm_priority" (current value: "0")
                          Priority of the sm coll component
                MCA coll: parameter "coll_sm_control_size" (current value: 
"4096")
                          Length of the control data -- should usually be 
either the length of a cache line on most SMPs, or the size of a page on 
machines that support direct memory affinity page placement (in bytes)
                MCA coll: parameter "coll_sm_bootstrap_filename" (current 
value: "coll-sm-bootstrap")
                          Filename (in the Open MPI session directory) of the 
coll sm component bootstrap rendezvous mmap file
                MCA coll: parameter "coll_sm_bootstrap_num_segments" (current 
value: "8")
                          Number of segments in the bootstrap file
                MCA coll: parameter "coll_sm_fragment_size" (current value: 
"8192")
                          Fragment size (in bytes) used for passing data 
through shared memory (will be rounded up to the nearest control_size size)
                MCA coll: parameter "coll_sm_mpool" (current value: "sm")
                          Name of the mpool component to use
                MCA coll: parameter "coll_sm_comm_in_use_flags" (current value: 
"2")
                          Number of "in use" flags, used to mark a message 
passing area segment as currently being used or not (must be >= 2 and <= 
comm_num_segments)
                MCA coll: parameter "coll_sm_comm_num_segments" (current value: 
"8")
                          Number of segments in each communicator's shared 
memory message passing area (must be >= 2, and must be a multiple of 
comm_in_use_flags)
                MCA coll: parameter "coll_sm_tree_degree" (current value: "4")
                          Degree of the tree for tree-based operations (must be 
=> 1 and <= min(control_size, 255))
                MCA coll: information "coll_sm_shared_mem_used_bootstrap" 
(value: "216")
                          Amount of shared memory used in the shared memory 
bootstrap area (in bytes)
                MCA coll: parameter "coll_sm_info_num_procs" (current value: 
"4")
                          Number of processes to use for the calculation of the 
shared_mem_size MCA information parameter (must be => 2)
                MCA coll: information "coll_sm_shared_mem_used_data" (value: 
"548864")
                          Amount of shared memory used in the shared memory 
data area for info_num_procs processes (in bytes)
                MCA coll: parameter "coll_tuned_priority" (current value: "30")
                          Priority of the tuned coll component
                MCA coll: parameter 
"coll_tuned_pre_allocate_memory_comm_size_limit" (current value: "32768")
                          Size of communicator were we stop pre-allocating 
memory for the fixed internal buffer used for message requests etc that is hung 
off the communicator data segment. I.e. if you have a 100'000 nodes you might 
not want to pre-allocate 200'000 request handle slots per communicator instance!
                MCA coll: parameter "coll_tuned_use_dynamic_rules" (current 
value: "0")
                          Switch used to decide if we use static (compiled/if 
statements) or dynamic (built at runtime) decision function rules
                MCA coll: parameter "coll_tuned_init_tree_fanout" (current 
value: "4")
                          Inital fanout used in the tree topologies for each 
communicator. This is only an initial guess, if a tuned collective needs a 
different fanout for an operation, it build it dynamically. This parameter is 
only for the first guess and might save a little time
                MCA coll: parameter "coll_tuned_init_chain_fanout" (current 
value: "4")
                          Inital fanout used in the chain (fanout followed by 
pipeline) topologies for each communicator. This is only an initial guess, if a 
tuned collective needs a different fanout for an operation, it build it 
dynamically. This parameter is only for the first guess and might save a little 
time
                  MCA io: parameter "io_base_freelist_initial_size" (current 
value: "16")
                          Initial MPI-2 IO request freelist size
                  MCA io: parameter "io_base_freelist_max_size" (current value: 
"64")
                          Max size of the MPI-2 IO request freelist
                  MCA io: parameter "io_base_freelist_increment" (current 
value: "16")
                          Increment size of the MPI-2 IO request freelist
                  MCA io: parameter "io" (current value: <none>)
                          Default selection set of components for the io 
framework (<none> means "use all components that can be found")
                  MCA io: parameter "io_base_verbose" (current value: "0")
                          Verbosity level for the io framework (0 = no 
verbosity)
               MCA mpool: parameter "mpool" (current value: <none>)
                          Default selection set of components for the mpool 
framework (<none> means "use all components that can be found")
               MCA mpool: parameter "mpool_base_verbose" (current value: "0")
                          Verbosity level for the mpool framework (0 = no 
verbosity)
               MCA mpool: parameter "mpool_mvapi_priority" (current value: "0")
               MCA mpool: parameter "mpool_sm_size" (current value: "536870912")
               MCA mpool: parameter "mpool_sm_allocator" (current value: 
"bucket")
               MCA mpool: parameter "mpool_sm_priority" (current value: "0")
               MCA mpool: parameter "mpool_base_use_mem_hooks" (current value: 
"0")
                          use memory hooks for deregistering freed memory
               MCA mpool: parameter "mpool_use_mem_hooks" (current value: "0")
                          (deprecated, use mpool_base_use_mem_hooks)
               MCA mpool: parameter "mpool_base_disable_sbrk" (current value: 
"0")
                          use mallopt to override calling sbrk (doesn't return 
memory to OS!)
               MCA mpool: parameter "mpool_disable_sbrk" (current value: "0")
                          (deprecated, use mca_mpool_base_disable_sbrk)
                 MCA pml: parameter "pml" (current value: "ob1")
                          Default selection set of components for the pml 
framework (<none> means "use all components that can be found")
                 MCA pml: parameter "pml_base_verbose" (current value: "0")
                          Verbosity level for the pml framework (0 = no 
verbosity)
                 MCA pml: parameter "pml_dr_free_list_num" (current value: "4")
                 MCA pml: parameter "pml_dr_free_list_max" (current value: "-1")
                 MCA pml: parameter "pml_dr_free_list_inc" (current value: "64")
                 MCA pml: parameter "pml_dr_priority" (current value: "1")
                 MCA pml: parameter "pml_dr_eager_limit" (current value: 
"131072")
                 MCA pml: parameter "pml_dr_send_pipeline_depth" (current 
value: "3")
                 MCA pml: parameter "pml_dr_wdog_timer_sec" (current value: "5")
                 MCA pml: parameter "pml_dr_wdog_timer_usec" (current value: 
"0")
                 MCA pml: parameter "pml_dr_wdog_timer_multiplier" (current 
value: "1")
                 MCA pml: parameter "pml_dr_wdog_retry_max" (current value: "1")
                 MCA pml: parameter "pml_dr_ack_timer_sec" (current value: "10")
                 MCA pml: parameter "pml_dr_ack_timer_usec" (current value: "0")
                 MCA pml: parameter "pml_dr_ack_timer_multiplier" (current 
value: "1")
                 MCA pml: parameter "pml_dr_ack_retry_max" (current value: "3")
                 MCA pml: parameter "pml_dr_enable_csum" (current value: "1")
                 MCA pml: parameter "pml_ob1_free_list_num" (current value: "4")
                 MCA pml: parameter "pml_ob1_free_list_max" (current value: 
"-1")
                 MCA pml: parameter "pml_ob1_free_list_inc" (current value: 
"64")
                 MCA pml: parameter "pml_ob1_priority" (current value: "1")
                 MCA pml: parameter "pml_ob1_eager_limit" (current value: 
"131072")
                 MCA pml: parameter "pml_ob1_send_pipeline_depth" (current 
value: "3")
                 MCA pml: parameter "pml_ob1_recv_pipeline_depth" (current 
value: "4")
                 MCA bml: parameter "bml" (current value: <none>)
                          Default selection set of components for the bml 
framework (<none> means "use all components that can be found")
                 MCA bml: parameter "bml_base_verbose" (current value: "0")
                          Verbosity level for the bml framework (0 = no 
verbosity)
                 MCA bml: parameter "bml_r2_priority" (current value: "0")
              MCA rcache: parameter "rcache" (current value: <none>)
                          Default selection set of components for the rcache 
framework (<none> means "use all components that can be found")
              MCA rcache: parameter "rcache_base_verbose" (current value: "0")
                          Verbosity level for the rcache framework (0 = no 
verbosity)
              MCA rcache: parameter "rcache_rb_priority" (current value: "0")
                 MCA btl: parameter "btl_base_debug" (current value: "0")
                          If btl_base_debug is 1 standard debug is output, if > 
1 verbose debug is output
                 MCA btl: parameter "btl" (current value: <none>)
                          Default selection set of components for the btl 
framework (<none> means "use all components that can be found")
                 MCA btl: parameter "btl_base_verbose" (current value: "0")
                          Verbosity level for the btl framework (0 = no 
verbosity)
                 MCA btl: parameter "btl_mvapi_max_btls" (current value: "4")
                          maximum number of HCAs/ports to use
                 MCA btl: parameter "btl_mvapi_free_list_num" (current value: 
"8")
                          intial size of free lists
                 MCA btl: parameter "btl_mvapi_free_list_max" (current value: 
"-1")
                          maximum size of free lists
                 MCA btl: parameter "btl_mvapi_free_list_inc" (current value: 
"32")
                          increment size of free lists
                 MCA btl: parameter "btl_mvapi_mpool" (current value: "mvapi")
                          name of the memory pool to be used
                 MCA btl: parameter "btl_mvapi_reg_mru_len" (current value: 
"16")
                          length of the registration cache most recently used 
list
                 MCA btl: parameter "btl_mvapi_use_srq" (current value: "0")
                          if 1 use the IB shared receive queue to post receive 
descriptors
                 MCA btl: parameter "btl_mvapi_ib_cq_size" (current value: 
"10000")
                          size of the IB completion queue
                 MCA btl: parameter "btl_mvapi_ib_sg_list_size" (current value: 
"1")
                          size of IB segment list
                 MCA btl: parameter "btl_mvapi_ib_pkey_ix" (current value: "0")
                          IB pkey index
                 MCA btl: parameter "btl_mvapi_ib_psn" (current value: "0")
                          IB Packet sequence starting number
                 MCA btl: parameter "btl_mvapi_ib_qp_ous_rd_atom" (current 
value: "4")
                          IB outstanding atomic reads
                 MCA btl: parameter "btl_mvapi_ib_mtu" (current value: "3")
                          IB MTU
                 MCA btl: parameter "btl_mvapi_ib_min_rnr_timer" (current 
value: "5")
                          IB min rnr timer
                 MCA btl: parameter "btl_mvapi_ib_timeout" (current value: "10")
                          IB transmit timeout
                 MCA btl: parameter "btl_mvapi_ib_retry_count" (current value: 
"7")
                          IB transmit retry count
                 MCA btl: parameter "btl_mvapi_ib_rnr_retry" (current value: 
"7")
                          IB rnr retry
                 MCA btl: parameter "btl_mvapi_ib_max_rdma_dst_ops" (current 
value: "4")
                          IB max rdma destination operations
                 MCA btl: parameter "btl_mvapi_ib_service_level" (current 
value: "0")
                          IB service level
                 MCA btl: parameter "btl_mvapi_ib_static_rate" (current value: 
"0")
                          IB static rate
                 MCA btl: parameter "btl_mvapi_ib_src_path_bits" (current 
value: "0")
                          IB source path bits
                 MCA btl: parameter "btl_mvapi_rd_num" (current value: "8")
                          number of receive descriptors to post to a QP
                 MCA btl: parameter "btl_mvapi_rd_low" (current value: "6")
                          low water mark before reposting occurs
                 MCA btl: parameter "btl_mvapi_rd_win" (current value: "4")
                          window size at which generate explicity credit message
                 MCA btl: parameter "btl_mvapi_srq_rd_max" (current value: 
"1000")
                          Maximum number of receive descriptors posted per SRQ.

                 MCA btl: parameter "btl_mvapi_srq_rd_per_peer" (current value: 
"16")
                          receive descriptors posted per peer, SRQ mode only
                 MCA btl: parameter "btl_mvapi_srq_sd_max" (current value: "8")
                          Maximum number of send descriptors posted per process
                 MCA btl: parameter "btl_mvapi_use_eager_rdma" (current value: 
"1")
                          user RDMA for eager messages
                 MCA btl: parameter "btl_mvapi_eager_rdma_threshold" (current 
value: "16")
                          Open rdma channel for eager messages after this 
number of messages received from peer (zero to disable)
                 MCA btl: parameter "btl_mvapi_max_eager_rdma" (current value: 
"16")
                          Maximum number of eager RDMA connections
                 MCA btl: parameter "btl_mvapi_eager_rdma_num" (current value: 
"16")
                          Number of RDMA buffers for eager messages
                 MCA btl: parameter "btl_mvapi_exclusivity" (current value: 
"1024")
                          BTL exclusivity
                 MCA btl: parameter "btl_mvapi_eager_limit" (current value: 
"12288")
                          eager send limit
                 MCA btl: parameter "btl_mvapi_min_send_size" (current value: 
"32768")
                          minimum send size
                 MCA btl: parameter "btl_mvapi_max_send_size" (current value: 
"65536")
                          maximum send size
                 MCA btl: parameter "btl_mvapi_min_rdma_size" (current value: 
"1048576")
                          minimum rdma size
                 MCA btl: parameter "btl_mvapi_max_rdma_size" (current value: 
"1048576")
                          maximium rdma size
                 MCA btl: parameter "btl_mvapi_flags" (current value: "6")
                          BTL flags, SEND=1, PUT=2, GET=4
                 MCA btl: parameter "btl_mvapi_bandwidth" (current value: "800")
                          Approximate maximum bandwidth of interconnect
                 MCA btl: parameter "btl_mvapi_priority" (current value: "0")
                 MCA btl: parameter "btl_self_free_list_num" (current value: 
"0")
                 MCA btl: parameter "btl_self_free_list_max" (current value: 
"-1")
                 MCA btl: parameter "btl_self_free_list_inc" (current value: 
"32")
                 MCA btl: parameter "btl_self_eager_limit" (current value: 
"131072")
                 MCA btl: parameter "btl_self_max_send_size" (current value: 
"262144")
                 MCA btl: parameter "btl_self_max_rdma_size" (current value: 
"2147483647")
                 MCA btl: parameter "btl_self_exclusivity" (current value: 
"65536")
                 MCA btl: parameter "btl_self_flags" (current value: "2")
                 MCA btl: parameter "btl_self_priority" (current value: "0")
                 MCA btl: parameter "btl_sm_free_list_num" (current value: "8")
                 MCA btl: parameter "btl_sm_free_list_max" (current value: "-1")
                 MCA btl: parameter "btl_sm_free_list_inc" (current value: 
"256")
                 MCA btl: parameter "btl_sm_exclusivity" (current value: 
"65535")
                 MCA btl: parameter "btl_sm_latency" (current value: "100")
                 MCA btl: parameter "btl_sm_max_procs" (current value: "-1")
                 MCA btl: parameter "btl_sm_sm_extra_procs" (current value: "2")
                 MCA btl: parameter "btl_sm_mpool" (current value: "sm")
                 MCA btl: parameter "btl_sm_eager_limit" (current value: "4096")
                 MCA btl: parameter "btl_sm_max_frag_size" (current value: 
"32768")
                 MCA btl: parameter "btl_sm_size_of_cb_queue" (current value: 
"128")
                 MCA btl: parameter "btl_sm_cb_lazy_free_freq" (current value: 
"120")
                 MCA btl: parameter "btl_sm_priority" (current value: "0")
                 MCA btl: parameter "btl_tcp_if_include" (current value: <none>)
                 MCA btl: parameter "btl_tcp_if_exclude" (current value: "lo")
                 MCA btl: parameter "btl_tcp_free_list_num" (current value: "8")
                 MCA btl: parameter "btl_tcp_free_list_max" (current value: 
"-1")
                 MCA btl: parameter "btl_tcp_free_list_inc" (current value: 
"32")
                 MCA btl: parameter "btl_tcp_sndbuf" (current value: "131072")
                 MCA btl: parameter "btl_tcp_rcvbuf" (current value: "131072")
                 MCA btl: parameter "btl_tcp_endpoint_cache" (current value: 
"30720")
                 MCA btl: parameter "btl_tcp_exclusivity" (current value: "0")
                 MCA btl: parameter "btl_tcp_eager_limit" (current value: 
"65536")
                 MCA btl: parameter "btl_tcp_min_send_size" (current value: 
"65536")
                 MCA btl: parameter "btl_tcp_max_send_size" (current value: 
"131072")
                 MCA btl: parameter "btl_tcp_min_rdma_size" (current value: 
"131072")
                 MCA btl: parameter "btl_tcp_max_rdma_size" (current value: 
"2147483647")
                 MCA btl: parameter "btl_tcp_flags" (current value: "10")
                 MCA btl: parameter "btl_tcp_priority" (current value: "0")
                 MCA btl: parameter "btl_base_include" (current value: <none>)
                 MCA btl: parameter "btl_base_exclude" (current value: <none>)
                MCA topo: parameter "topo" (current value: <none>)
                          Default selection set of components for the topo 
framework (<none> means "use all components that can be found")
                MCA topo: parameter "topo_base_verbose" (current value: "0")
                          Verbosity level for the topo framework (0 = no 
verbosity)
                 MCA osc: parameter "osc" (current value: <none>)
                          Default selection set of components for the osc 
framework (<none> means "use all components that can be found")
                 MCA osc: parameter "osc_base_verbose" (current value: "0")
                          Verbosity level for the osc framework (0 = no 
verbosity)
                 MCA osc: parameter "osc_pt2pt_fence_sync_method" (current 
value: "reduce_scatter")
                          How to synchronize fence: reduce_scatter, allreduce, 
alltoall
                 MCA osc: parameter "osc_pt2pt_eager_send" (current value: "0")
                          Attempt to start data movement during communication 
call, instead of at synchrnoization time.  Info key of same name overrides this 
value, if info key given.
                 MCA osc: parameter "osc_pt2pt_no_locks" (current value: "0")
                          Enable optimizations available only if MPI_LOCK is 
not used.
                 MCA osc: parameter "osc_pt2pt_priority" (current value: "0")
              MCA errmgr: parameter "errmgr" (current value: <none>)
                          Default selection set of components for the errmgr 
framework (<none> means "use all components that can be found")
                 MCA gpr: parameter "gpr_base_maxsize" (current value: 
"2147483647")
                 MCA gpr: parameter "gpr_base_blocksize" (current value: "512")
                 MCA gpr: parameter "gpr" (current value: <none>)
                          Default selection set of components for the gpr 
framework (<none> means "use all components that can be found")
                 MCA gpr: parameter "gpr_null_priority" (current value: "0")
                 MCA gpr: parameter "gpr_proxy_debug" (current value: "0")
                 MCA gpr: parameter "gpr_proxy_priority" (current value: "0")
                 MCA gpr: parameter "gpr_replica_debug" (current value: "0")
                 MCA gpr: parameter "gpr_replica_isolate" (current value: "0")
                 MCA gpr: parameter "gpr_replica_priority" (current value: "0")
                 MCA iof: parameter "iof_base_window_size" (current value: 
"4096")
                 MCA iof: parameter "iof_base_service" (current value: "0.0.0")
                 MCA iof: parameter "iof" (current value: <none>)
                          Default selection set of components for the iof 
framework (<none> means "use all components that can be found")
                 MCA iof: parameter "iof_proxy_debug" (current value: "1")
                 MCA iof: parameter "iof_proxy_priority" (current value: "0")
                 MCA iof: parameter "iof_svc_debug" (current value: "1")
                 MCA iof: parameter "iof_svc_priority" (current value: "0")
                  MCA ns: parameter "ns" (current value: <none>)
                          Default selection set of components for the ns 
framework (<none> means "use all components that can be found")
                  MCA ns: parameter "ns_proxy_debug" (current value: "0")
                  MCA ns: parameter "ns_proxy_maxsize" (current value: 
"2147483647")
                  MCA ns: parameter "ns_proxy_blocksize" (current value: "512")
                  MCA ns: parameter "ns_proxy_priority" (current value: "0")
                  MCA ns: parameter "ns_replica_debug" (current value: "0")
                  MCA ns: parameter "ns_replica_isolate" (current value: "0")
                  MCA ns: parameter "ns_replica_maxsize" (current value: 
"2147483647")
                  MCA ns: parameter "ns_replica_blocksize" (current value: 
"512")
                  MCA ns: parameter "ns_replica_priority" (current value: "0")
                 MCA oob: parameter "oob" (current value: <none>)
                          Default selection set of components for the oob 
framework (<none> means "use all components that can be found")
                 MCA oob: parameter "oob_base_verbose" (current value: "0")
                          Verbosity level for the oob framework (0 = no 
verbosity)
                 MCA oob: parameter "oob_tcp_peer_limit" (current value: "-1")
                 MCA oob: parameter "oob_tcp_peer_retries" (current value: "60")
                 MCA oob: parameter "oob_tcp_debug" (current value: "0")
                 MCA oob: parameter "oob_tcp_include" (current value: <none>)
                 MCA oob: parameter "oob_tcp_exclude" (current value: <none>)
                 MCA oob: parameter "oob_tcp_sndbuf" (current value: "131072")
                 MCA oob: parameter "oob_tcp_rcvbuf" (current value: "131072")
                 MCA oob: parameter "oob_tcp_priority" (current value: "0")
                 MCA ras: parameter "ras" (current value: <none>)
                          Default selection set of components for the ras 
framework (<none> means "use all components that can be found")
                 MCA ras: parameter "ras_dash_host_priority" (current value: 
"5")
                          Selection priority for the dash_host RAS component
                 MCA ras: parameter "ras_hostfile_priority" (current value: 
"10")
                          Selection priority for the hostfile RAS component
                 MCA ras: parameter "ras_localhost_priority" (current value: 
"0")
                          Selection priority for the localhost RAS component
                 MCA ras: parameter "ras_slurm_priority" (current value: "75")
                          Priority of the slurm ras component
                 MCA rds: parameter "rds" (current value: <none>)
                          Default selection set of components for the rds 
framework (<none> means "use all components that can be found")
                 MCA rds: parameter "rds_hostfile_debug" (current value: "0")
                          Toggle debug output for hostfile RDS component
                 MCA rds: parameter "rds_hostfile_path" (current value: 
"/home/pub/OpenFOAM/OpenFOAM-1.3/src/openmpi-1.2a1r10111/platforms/linuxAMD64Gcc4DPOpt/etc/openmpi-default-hostfile")
                          ORTE Host filename
                 MCA rds: parameter "rds_hostfile_priority" (current value: "0")
                 MCA rds: parameter "rds_resfile_debug" (current value: "0")
                          Toggle debug output for resfile RDS component
                 MCA rds: parameter "rds_resfile_name" (current value: <none>)
                          ORTE Resource filename
                 MCA rds: parameter "rds_resfile_priority" (current value: "0")
               MCA rmaps: parameter "rmaps" (current value: <none>)
                          Default selection set of components for the rmaps 
framework (<none> means "use all components that can be found")
               MCA rmaps: parameter "rmaps_round_robin_debug" (current value: 
"1")
                          Toggle debug output for Round Robin RMAPS component
               MCA rmaps: parameter "rmaps_round_robin_priority" (current 
value: "1")
                          Selection priority for Round Robin RMAPS component
                MCA rmgr: parameter "rmgr" (current value: <none>)
                          Default selection set of components for the rmgr 
framework (<none> means "use all components that can be found")
                MCA rmgr: parameter "rmgr_proxy_priority" (current value: "0")
                MCA rmgr: parameter "rmgr_urm_priority" (current value: "0")
                 MCA rml: parameter "rml" (current value: <none>)
                          Default selection set of components for the rml 
framework (<none> means "use all components that can be found")
                 MCA rml: parameter "rml_base_verbose" (current value: "0")
                          Verbosity level for the rml framework (0 = no 
verbosity)
                 MCA rml: parameter "rml_oob_priority" (current value: "0")
                 MCA pls: parameter "pls" (current value: <none>)
                          Default selection set of components for the pls 
framework (<none> means "use all components that can be found")
                 MCA pls: parameter "pls_fork_reap" (current value: "1")
                          Whether to wait to reap all children before 
finalizing or not
                 MCA pls: parameter "pls_fork_reap_timeout" (current value: "0")
                          When killing children processes, first send a 
SIGTERM, then wait at least this timeout (in seconds), then send a SIGKILL
                 MCA pls: parameter "pls_fork_priority" (current value: "1")
                          Priority of this component
                 MCA pls: parameter "pls_fork_debug" (current value: "0")
                          Whether to enable debugging output or not
                 MCA pls: parameter "pls_rsh_debug" (current value: "0")
                          Whether or not to enable debugging output for the rsh 
pls component (0 or 1)
                 MCA pls: parameter "pls_rsh_num_concurrent" (current value: 
"128")
                          How many pls_rsh_agent instances to invoke 
concurrently (must be > 0)
                 MCA pls: parameter "pls_rsh_orted" (current value: "orted")
                          The command name that the rsh pls component will 
invoke for the ORTE daemon
                 MCA pls: parameter "pls_rsh_priority" (current value: "10")
                          Priority of the rsh pls component
                 MCA pls: parameter "pls_rsh_delay" (current value: "1")
                          Delay (in seconds) between invocations of the remote 
agent, but only used when the "debug" MCA parameter is true, or the top-level 
MCA debugging is enabled (otherwise this value is ignored)
                 MCA pls: parameter "pls_rsh_reap" (current value: "1")
                          If set to 1, wait for all the processes to complete 
before exiting.  Otherwise, quit immediately -- without waiting for 
confirmation that all other processes in the job have completed.
                 MCA pls: parameter "pls_rsh_assume_same_shell" (current value: 
"1")
                          If set to 1, assume that the shell on the remote node 
is the same as the shell on the local node.  Otherwise, probe for what the 
remote shell.
                 MCA pls: parameter "pls_rsh_agent" (current value: "ssh : rsh")
                          The command used to launch executables on remote 
nodes (typically either "ssh" or "rsh")
                 MCA pls: parameter "pls_slurm_debug" (current value: "0")
                          Enable debugging of slurm pls
                 MCA pls: parameter "pls_slurm_priority" (current value: "75")
                          Default selection priority
                 MCA pls: parameter "pls_slurm_orted" (current value: "orted")
                          Command to use to start proxy orted
                 MCA pls: parameter "pls_slurm_args" (current value: <none>)
                          Custom arguments to srun
                 MCA sds: parameter "sds" (current value: <none>)
                          Default selection set of components for the sds 
framework (<none> means "use all components that can be found")
                 MCA sds: parameter "sds_base_verbose" (current value: "0")
                          Verbosity level for the sds framework (0 = no 
verbosity)
                 MCA sds: parameter "sds_env_priority" (current value: "0")
                 MCA sds: parameter "sds_pipe_priority" (current value: "0")
                 MCA sds: parameter "sds_seed_priority" (current value: "0")
                 MCA sds: parameter "sds_singleton_priority" (current value: 
"0")
                 MCA sds: parameter "sds_slurm_priority" (current value: "0")
                 MCA soh: parameter "soh" (current value: <none>)
                          Default selection set of components for the soh 
framework (<none> means "use all components that can be found")

Reply via email to