Done. I have compiled 1.10.0 and 1.10.rc1 with --enable-debug and executed
mpirun --mca rmaps_base_verbose 10 --hetero-nodes --report-bindings
--bind-to core -np 32 ./affinity
In case of 1.10.rc1 I have also added :overload-allowed - output in a
separate file. This option did not make much difference for 1.10.0, so I
did not attach it here.
First thing I noted for 1.10.0 are lines like
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],27] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],27] ON c1-26 IS
NOT BOUND
with an empty BITMAP.
The SLURM environment is
set | grep SLURM
SLURM_JOBID=12714491
SLURM_JOB_CPUS_PER_NODE='4,2,5(x2),4,7,5'
SLURM_JOB_ID=12714491
SLURM_JOB_NODELIST='c1-[2,4,8,13,16,23,26]'
SLURM_JOB_NUM_NODES=7
SLURM_JOB_PARTITION=normal
SLURM_MEM_PER_CPU=2048
SLURM_NNODES=7
SLURM_NODELIST='c1-[2,4,8,13,16,23,26]'
SLURM_NODE_ALIASES='(null)'
SLURM_NPROCS=32
SLURM_NTASKS=32
SLURM_SUBMIT_DIR=/cluster/home/marcink
SLURM_SUBMIT_HOST=login-0-1.local
SLURM_TASKS_PER_NODE='4,2,5(x2),4,7,5'
I have submitted an interactive job on screen for 120 hours now to work
with one example, and not change it for every post :)
If you need anything else, let me know. I could introduce some
patch/printfs and recompile, if you need it.
Marcin
On 10/03/2015 07:17 PM, Ralph Castain wrote:
Rats - just realized I have no way to test this as none of the
machines I can access are setup for cgroup-based multi-tenant. Is this
a debug version of OMPI? If not, can you rebuild OMPI with —enable-debug?
Then please run it with —mca rmaps_base_verbose 10 and pass along the
output.
Thanks
Ralph
On Oct 3, 2015, at 10:09 AM, Ralph Castain <r...@open-mpi.org
<mailto:r...@open-mpi.org>> wrote:
What version of slurm is this? I might try to debug it here. I’m not
sure where the problem lies just yet.
On Oct 3, 2015, at 8:59 AM, marcin.krotkiewski
<marcin.krotkiew...@gmail.com <mailto:marcin.krotkiew...@gmail.com>>
wrote:
Here is the output of lstopo. In short, (0,16) are core 0, (1,17) -
core 1 etc.
Machine (64GB)
NUMANode L#0 (P#0 32GB)
Socket L#0 + L3 L#0 (20MB)
L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0
PU L#0 (P#0)
PU L#1 (P#16)
L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1
PU L#2 (P#1)
PU L#3 (P#17)
L2 L#2 (256KB) + L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2
PU L#4 (P#2)
PU L#5 (P#18)
L2 L#3 (256KB) + L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3
PU L#6 (P#3)
PU L#7 (P#19)
L2 L#4 (256KB) + L1d L#4 (32KB) + L1i L#4 (32KB) + Core L#4
PU L#8 (P#4)
PU L#9 (P#20)
L2 L#5 (256KB) + L1d L#5 (32KB) + L1i L#5 (32KB) + Core L#5
PU L#10 (P#5)
PU L#11 (P#21)
L2 L#6 (256KB) + L1d L#6 (32KB) + L1i L#6 (32KB) + Core L#6
PU L#12 (P#6)
PU L#13 (P#22)
L2 L#7 (256KB) + L1d L#7 (32KB) + L1i L#7 (32KB) + Core L#7
PU L#14 (P#7)
PU L#15 (P#23)
HostBridge L#0
PCIBridge
PCI 8086:1521
Net L#0 "eth0"
PCI 8086:1521
Net L#1 "eth1"
PCIBridge
PCI 15b3:1003
Net L#2 "ib0"
OpenFabrics L#3 "mlx4_0"
PCIBridge
PCI 102b:0532
PCI 8086:1d02
Block L#4 "sda"
NUMANode L#1 (P#1 32GB) + Socket L#1 + L3 L#1 (20MB)
L2 L#8 (256KB) + L1d L#8 (32KB) + L1i L#8 (32KB) + Core L#8
PU L#16 (P#8)
PU L#17 (P#24)
L2 L#9 (256KB) + L1d L#9 (32KB) + L1i L#9 (32KB) + Core L#9
PU L#18 (P#9)
PU L#19 (P#25)
L2 L#10 (256KB) + L1d L#10 (32KB) + L1i L#10 (32KB) + Core L#10
PU L#20 (P#10)
PU L#21 (P#26)
L2 L#11 (256KB) + L1d L#11 (32KB) + L1i L#11 (32KB) + Core L#11
PU L#22 (P#11)
PU L#23 (P#27)
L2 L#12 (256KB) + L1d L#12 (32KB) + L1i L#12 (32KB) + Core L#12
PU L#24 (P#12)
PU L#25 (P#28)
L2 L#13 (256KB) + L1d L#13 (32KB) + L1i L#13 (32KB) + Core L#13
PU L#26 (P#13)
PU L#27 (P#29)
L2 L#14 (256KB) + L1d L#14 (32KB) + L1i L#14 (32KB) + Core L#14
PU L#28 (P#14)
PU L#29 (P#30)
L2 L#15 (256KB) + L1d L#15 (32KB) + L1i L#15 (32KB) + Core L#15
PU L#30 (P#15)
PU L#31 (P#31)
On 10/03/2015 05:46 PM, Ralph Castain wrote:
Maybe I’m just misreading your HT map - that slurm nodelist syntax
is a new one to me, but they tend to change things around. Could
you run lstopo on one of those compute nodes and send the output?
I’m just suspicious because I’m not seeing a clear pairing of HT
numbers in your output, but HT numbering is BIOS-specific and I may
just not be understanding your particular pattern. Our error
message is clearly indicating that we are seeing individual HTs
(and not complete cores) assigned, and I don’t know the source of
that confusion.
On Oct 3, 2015, at 8:28 AM, marcin.krotkiewski
<marcin.krotkiew...@gmail.com
<mailto:marcin.krotkiew...@gmail.com>> wrote:
On 10/03/2015 04:38 PM, Ralph Castain wrote:
If mpirun isn’t trying to do any binding, then you will of course
get the right mapping as we’ll just inherit whatever we received.
Yes. I meant that whatever you received (what SLURM gives) is a
correct cpu map and assigns _whole_ CPUs, not a single HT to MPI
processes. In the case mentioned earlier openmpi should start 6
tasks on c1-30. If HT would be treated as separate and independent
cores, sched_getaffinity of an MPI process started on c1-30 would
return a map with 6 entries only. In my case it returns a map with
12 entries - 2 for each core. So one process is in fact allocated
both HTs, not only one. Is what I'm saying correct?
Looking at your output, it’s pretty clear that you are getting
independent HTs assigned and not full cores.
How do you mean? Is the above understanding wrong? I would expect
that on c1-30 with --bind-to core openmpi should bind to logical
cores 0 and 16 (rank 0), 1 and 17 (rank 2) and so on. All those
logical cores are available in sched_getaffinity map, and there is
twice as many logical cores as there are MPI processes started on
the node.
My guess is that something in slurm has changed such that it
detects that HT has been enabled, and then begins treating the
HTs as completely independent cpus.
Try changing “-bind-to core” to “-bind-to hwthread
-use-hwthread-cpus” and see if that works
I have and the binding is wrong. For example, I got this output
rank 0 @ compute-1-30.local 0,
rank 1 @ compute-1-30.local 16,
Which means that two ranks have been bound to the same physical
core (logical cores 0 and 16 are two HTs of the same core). If I
use --bind-to core, I get the following correct binding
rank 0 @ compute-1-30.local 0, 16,
The problem is many other ranks get bad binding with 'rank XXX is
not bound (or bound to all available processors)' warning.
But I think I was not entirely correct saying that 1.10.1rc1 did
not fix things. It still might have improved something, but not
everything. Consider this job:
SLURM_JOB_CPUS_PER_NODE='5,4,6,5(x2),7,5,9,5,7,6'
SLURM_JOB_NODELIST='c8-[31,34],c9-[30-32,35-36],c10-[31-34]'
If I run 32 tasks as follows (with 1.10.1rc1)
mpirun --hetero-nodes --report-bindings --bind-to core -np 32
./affinity
I get the following error:
--------------------------------------------------------------------------
A request was made to bind to that would result in binding more
processes than cpus on a resource:
Bind to: CORE
Node: c9-31
#processes: 2
#cpus: 1
You can override this protection by adding the "overload-allowed"
option to your binding directive.
--------------------------------------------------------------------------
If I now use --bind-to core:overload-allowed, then openmpi starts
and _most_ of the threads are bound correctly (i.e., map contains
two logical cores in ALL cases), except this case that required
the overload flag:
rank 15 @ compute-9-31.local 1, 17,
rank 16 @ compute-9-31.local 11, 27,
rank 17 @ compute-9-31.local 2, 18,
rank 18 @ compute-9-31.local 12, 28,
rank 19 @ compute-9-31.local 1, 17,
Note pair 1,17 is used twice. The original SLURM delivered map (no
binding) on this node is
rank 15 @ compute-9-31.local 1, 2, 11, 12, 13, 17, 18, 27, 28, 29,
rank 16 @ compute-9-31.local 1, 2, 11, 12, 13, 17, 18, 27, 28, 29,
rank 17 @ compute-9-31.local 1, 2, 11, 12, 13, 17, 18, 27, 28, 29,
rank 18 @ compute-9-31.local 1, 2, 11, 12, 13, 17, 18, 27, 28, 29,
rank 19 @ compute-9-31.local 1, 2, 11, 12, 13, 17, 18, 27, 28, 29,
Why does openmpi use cores (1,17) twice instead of using core
(13,29)? Clearly, the original SLURM-delivered map has 5 CPUs
included, enough for 5 MPI processes.
Cheers,
Marcin
On Oct 3, 2015, at 7:12 AM, marcin.krotkiewski
<marcin.krotkiew...@gmail.com
<mailto:marcin.krotkiew...@gmail.com>> wrote:
On 10/03/2015 01:06 PM, Ralph Castain wrote:
Thanks Marcin. Looking at this, I’m guessing that Slurm may be
treating HTs as “cores” - i.e., as independent cpus. Any chance
that is true?
Not to the best of my knowledge, and at least not intentionally.
SLURM starts as many processes as there are physical cores, not
threads. To verify this, consider this test case:
SLURM_JOB_CPUS_PER_NODE='6,8(x2),10'
SLURM_JOB_NODELIST='c1-[30-31],c2-[32,34]'
If I now execute only one mpi process WITH NO BINDING, it will
go onto c1-30 and should have a map with 6 CPUs (12 hw threads).
I run
mpirun --bind-to none -np 1 ./affinity
rank 0 @ compute-1-30.local 0, 1, 3, 4, 5, 6, 16, 17, 19, 20,
21, 22,
I have attached the affinity.c program FYI. Clearly,
sched_getaffinity in my test code returns the correct map.
Now if I try to start all 32 processes in this example (still no
binding):
rank 0 @ compute-1-30.local 0, 1, 3, 4, 5, 6, 16, 17, 19, 20,
21, 22,
rank 1 @ compute-1-30.local 0, 1, 3, 4, 5, 6, 16, 17, 19, 20,
21, 22,
rank 10 @ compute-1-31.local 2, 3, 7, 11, 12, 13, 14, 15, 18,
19, 23, 27, 28, 29, 30, 31,
rank 11 @ compute-1-31.local 2, 3, 7, 11, 12, 13, 14, 15, 18,
19, 23, 27, 28, 29, 30, 31,
rank 12 @ compute-1-31.local 2, 3, 7, 11, 12, 13, 14, 15, 18,
19, 23, 27, 28, 29, 30, 31,
rank 13 @ compute-1-31.local 2, 3, 7, 11, 12, 13, 14, 15, 18,
19, 23, 27, 28, 29, 30, 31,
rank 6 @ compute-1-31.local 2, 3, 7, 11, 12, 13, 14, 15, 18,
19, 23, 27, 28, 29, 30, 31,
rank 2 @ compute-1-30.local 0, 1, 3, 4, 5, 6, 16, 17, 19, 20,
21, 22,
rank 7 @ compute-1-31.local 2, 3, 7, 11, 12, 13, 14, 15, 18,
19, 23, 27, 28, 29, 30, 31,
rank 8 @ compute-1-31.local 2, 3, 7, 11, 12, 13, 14, 15, 18,
19, 23, 27, 28, 29, 30, 31,
rank 3 @ compute-1-30.local 0, 1, 3, 4, 5, 6, 16, 17, 19, 20,
21, 22,
rank 14 @ compute-2-32.local 7, 8, 9, 10, 11, 12, 13, 14, 23,
24, 25, 26, 27, 28, 29, 30,
rank 4 @ compute-1-30.local 0, 1, 3, 4, 5, 6, 16, 17, 19, 20,
21, 22,
rank 15 @ compute-2-32.local 7, 8, 9, 10, 11, 12, 13, 14, 23,
24, 25, 26, 27, 28, 29, 30,
rank 9 @ compute-1-31.local 2, 3, 7, 11, 12, 13, 14, 15, 18,
19, 23, 27, 28, 29, 30, 31,
rank 5 @ compute-1-30.local 0, 1, 3, 4, 5, 6, 16, 17, 19, 20,
21, 22,
rank 16 @ compute-2-32.local 7, 8, 9, 10, 11, 12, 13, 14, 23,
24, 25, 26, 27, 28, 29, 30,
rank 17 @ compute-2-32.local 7, 8, 9, 10, 11, 12, 13, 14, 23,
24, 25, 26, 27, 28, 29, 30,
rank 29 @ compute-2-34.local 0, 1, 2, 3, 4, 5, 6, 7, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 30, 31,
rank 30 @ compute-2-34.local 0, 1, 2, 3, 4, 5, 6, 7, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 30, 31,
rank 18 @ compute-2-32.local 7, 8, 9, 10, 11, 12, 13, 14, 23,
24, 25, 26, 27, 28, 29, 30,
rank 19 @ compute-2-32.local 7, 8, 9, 10, 11, 12, 13, 14, 23,
24, 25, 26, 27, 28, 29, 30,
rank 31 @ compute-2-34.local 0, 1, 2, 3, 4, 5, 6, 7, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 30, 31,
rank 20 @ compute-2-32.local 7, 8, 9, 10, 11, 12, 13, 14, 23,
24, 25, 26, 27, 28, 29, 30,
rank 22 @ compute-2-34.local 0, 1, 2, 3, 4, 5, 6, 7, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 30, 31,
rank 21 @ compute-2-32.local 7, 8, 9, 10, 11, 12, 13, 14, 23,
24, 25, 26, 27, 28, 29, 30,
rank 23 @ compute-2-34.local 0, 1, 2, 3, 4, 5, 6, 7, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 30, 31,
rank 24 @ compute-2-34.local 0, 1, 2, 3, 4, 5, 6, 7, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 30, 31,
rank 25 @ compute-2-34.local 0, 1, 2, 3, 4, 5, 6, 7, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 30, 31,
rank 26 @ compute-2-34.local 0, 1, 2, 3, 4, 5, 6, 7, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 30, 31,
rank 27 @ compute-2-34.local 0, 1, 2, 3, 4, 5, 6, 7, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 30, 31,
rank 28 @ compute-2-34.local 0, 1, 2, 3, 4, 5, 6, 7, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 30, 31,
Still looks ok to me. If I now turn the binding on, openmpi fails:
--------------------------------------------------------------------------
A request was made to bind to that would result in binding more
processes than cpus on a resource:
Bind to: CORE
Node: c1-31
#processes: 2
#cpus: 1
You can override this protection by adding the "overload-allowed"
option to your binding directive.
--------------------------------------------------------------------------
The above tests were done with 1.10.1rc1, so it does not fix the
problem.
Marcin
I’m wondering because bind-to core will attempt to bind your
proc to both HTs on the core. For some reason, we thought that
8.24 were HTs on the same core, which is why we tried to bind
to that pair of HTs. We got an error because HT #24 was not
allocated to us on node c6, but HT #8 was.
On Oct 3, 2015, at 2:43 AM, marcin.krotkiewski
<marcin.krotkiew...@gmail.com> wrote:
Hi, Ralph,
I submit my slurm job as follows
salloc --ntasks=64 --mem-per-cpu=2G --time=1:0:0
Effectively, the allocated CPU cores are spread amount many
cluster nodes. SLURM uses cgroups to limit the CPU cores
available for mpi processes running on a given cluster node.
Compute nodes are 2-socket, 8-core E5-2670 systems with
HyperThreading on
node 0 cpus: 0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23
node 1 cpus: 8 9 10 11 12 13 14 15 24 25 26 27 28 29 30 31
node distances:
node 0 1
0: 10 21
1: 21 10
I run MPI program with command
mpirun --report-bindings --bind-to core -np 64 ./affinity
The program simply runs sched_getaffinity for each process and
prints out the result.
-----------
TEST RUN 1
-----------
For this particular job the problem is more severe: openmpi
fails to run at all with error
--------------------------------------------------------------------------
Open MPI tried to bind a new process, but something went
wrong. The
process was killed without launching the target application.
Your job
will now abort.
Local host: c6-6
Application name: ./affinity
Error message: hwloc_set_cpubind returned "Error" for
bitmap "8,24"
Location: odls_default_module.c:551
--------------------------------------------------------------------------
This is SLURM environment variables:
SLURM_JOBID=12712225
SLURM_JOB_CPUS_PER_NODE='3(x2),2,1(x3),2(x2),1,3(x3),5,1,4,1,3,2,3,7,1,5,6,1'
SLURM_JOB_ID=12712225
SLURM_JOB_NODELIST='c6-[3,6-8,12,14,17,22-23],c8-[4,7,9,17,20,28],c15-[5,10,18,20,22-24,28],c16-11'
SLURM_JOB_NUM_NODES=24
SLURM_JOB_PARTITION=normal
SLURM_MEM_PER_CPU=2048
SLURM_NNODES=24
SLURM_NODELIST='c6-[3,6-8,12,14,17,22-23],c8-[4,7,9,17,20,28],c15-[5,10,18,20,22-24,28],c16-11'
SLURM_NODE_ALIASES='(null)'
SLURM_NPROCS=64
SLURM_NTASKS=64
SLURM_SUBMIT_DIR=/cluster/home/marcink
SLURM_SUBMIT_HOST=login-0-2.local
SLURM_TASKS_PER_NODE='3(x2),2,1(x3),2(x2),1,3(x3),5,1,4,1,3,2,3,7,1,5,6,1'
There is also a lot of warnings like
[compute-6-6.local:20158] MCW rank 4 is not bound (or bound to
all available processors)
-----------
TEST RUN 2
-----------
In another allocation I got a different error
--------------------------------------------------------------------------
A request was made to bind to that would result in binding more
processes than cpus on a resource:
Bind to: CORE
Node: c6-19
#processes: 2
#cpus: 1
You can override this protection by adding the "overload-allowed"
option to your binding directive.
--------------------------------------------------------------------------
and the allocation was the following
SLURM_JOBID=12712250
SLURM_JOB_CPUS_PER_NODE='3(x2),2,1,15,1,3,16,2,1,3(x2),2,5,4'
SLURM_JOB_ID=12712250
SLURM_JOB_NODELIST='c6-[3,6-8,12,14,17,19,22-23],c8-[4,7,9,17,28]'
SLURM_JOB_NUM_NODES=15
SLURM_JOB_PARTITION=normal
SLURM_MEM_PER_CPU=2048
SLURM_NNODES=15
SLURM_NODELIST='c6-[3,6-8,12,14,17,19,22-23],c8-[4,7,9,17,28]'
SLURM_NODE_ALIASES='(null)'
SLURM_NPROCS=64
SLURM_NTASKS=64
SLURM_SUBMIT_DIR=/cluster/home/marcink
SLURM_SUBMIT_HOST=login-0-2.local
SLURM_TASKS_PER_NODE='3(x2),2,1,15,1,3,16,2,1,3(x2),2,5,4'
If in this case I run on only 32 cores
mpirun --report-bindings --bind-to core -np 32 ./affinity
the process starts, but I get the original binding problem:
[compute-6-8.local:31414] MCW rank 8 is not bound (or bound to
all available processors)
Running with --hetero-nodes yields exactly the same results
Hope the above is useful. The problem with binding under SLURM
with CPU cores spread over nodes seems to be very
reproducible. It is actually very often that OpenMPI dies with
some error like above. These tests were run with openmpi-1.8.8
and 1.10.0, both giving same results.
One more suggestion. The warning message (MCW rank 8 is not
bound...) is ONLY displayed when I use --report-bindings. It
is never shown if I leave out this option, and although the
binding is wrong the user is not notified. I think it would be
better to show this warning in all cases binding fails.
Let me know if you need more information. I can help to debug
this - it is a rather crucial issue.
Thanks!
Marcin
On 10/02/2015 11:49 PM, Ralph Castain wrote:
Can you please send me the allocation request you made (so I
can see what you specified on the cmd line), and the mpirun
cmd line?
Thanks
Ralph
On Oct 2, 2015, at 8:25 AM, Marcin Krotkiewski
<marcin.krotkiew...@gmail.com> wrote:
Hi,
I fail to make OpenMPI bind to cores correctly when running
from within SLURM-allocated CPU resources spread over a
range of compute nodes in an otherwise homogeneous cluster.
I have found this thread
http://www.open-mpi.org/community/lists/users/2014/06/24682.php
and did try to use what Ralph suggested there
(--hetero-nodes), but it does not work (v. 1.10.0). When
running with --report-bindings I get messages like
[compute-9-11.local:27571] MCW rank 10 is not bound (or
bound to all available processors)
for all ranks outside of my first physical compute node.
Moreover, everything works as expected if I ask SLURM to
assign entire compute nodes. So it does look like Ralph's
diagnose presented in that thread is correct, just the
--hetero-nodes switch does not work for me.
I have written a short code that uses sched_getaffinity to
print the effective bindings: all MPI ranks except of those
on the first node are bound to all CPU cores allocated by SLURM.
Do I have to do something except of --hetero-nodes, or is
this a problem that needs further investigation?
Thanks a lot!
Marcin
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription:http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this
post:http://www.open-mpi.org/community/lists/users/2015/10/27770.php
_______________________________________________
users mailing list
us...@open-mpi.org <mailto:us...@open-mpi.org>
Subscription:http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this
post:http://www.open-mpi.org/community/lists/users/2015/10/27774.php
_______________________________________________
users mailing list
us...@open-mpi.org <mailto:us...@open-mpi.org>
Subscription:http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this
post:http://www.open-mpi.org/community/lists/users/2015/10/27776.php
_______________________________________________
users mailing list
us...@open-mpi.org <mailto:us...@open-mpi.org>
Subscription:http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this
post:http://www.open-mpi.org/community/lists/users/2015/10/27778.php
<affinity.c>_______________________________________________
users mailing list
us...@open-mpi.org <mailto:us...@open-mpi.org>
Subscription:http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this
post:http://www.open-mpi.org/community/lists/users/2015/10/27781.php
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription:http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this
post:http://www.open-mpi.org/community/lists/users/2015/10/27782.php
_______________________________________________
users mailing list
us...@open-mpi.org <mailto:us...@open-mpi.org>
Subscription:http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this
post:http://www.open-mpi.org/community/lists/users/2015/10/27783.php
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription:http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this
post:http://www.open-mpi.org/community/lists/users/2015/10/27784.php
_______________________________________________
users mailing list
us...@open-mpi.org <mailto:us...@open-mpi.org>
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2015/10/27785.php
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2015/10/27787.php
[login-0-1.local:03102] mca: base: components_register: registering rmaps
components
[login-0-1.local:03102] mca: base: components_register: found loaded component
round_robin
[login-0-1.local:03102] mca: base: components_register: component round_robin
register function successful
[login-0-1.local:03102] mca: base: components_register: found loaded component
rank_file
[login-0-1.local:03102] mca: base: components_register: component rank_file
register function successful
[login-0-1.local:03102] mca: base: components_register: found loaded component
seq
[login-0-1.local:03102] mca: base: components_register: component seq register
function successful
[login-0-1.local:03102] mca: base: components_register: found loaded component
resilient
[login-0-1.local:03102] mca: base: components_register: component resilient
register function successful
[login-0-1.local:03102] mca: base: components_register: found loaded component
staged
[login-0-1.local:03102] mca: base: components_register: component staged has no
register or open function
[login-0-1.local:03102] mca: base: components_register: found loaded component
mindist
[login-0-1.local:03102] mca: base: components_register: component mindist
register function successful
[login-0-1.local:03102] mca: base: components_register: found loaded component
ppr
[login-0-1.local:03102] mca: base: components_register: component ppr register
function successful
[login-0-1.local:03102] [[38240,0],0] rmaps:base set policy with NULL
[login-0-1.local:03102] mca: base: components_open: opening rmaps components
[login-0-1.local:03102] mca: base: components_open: found loaded component
round_robin
[login-0-1.local:03102] mca: base: components_open: component round_robin open
function successful
[login-0-1.local:03102] mca: base: components_open: found loaded component
rank_file
[login-0-1.local:03102] mca: base: components_open: component rank_file open
function successful
[login-0-1.local:03102] mca: base: components_open: found loaded component seq
[login-0-1.local:03102] mca: base: components_open: component seq open function
successful
[login-0-1.local:03102] mca: base: components_open: found loaded component
resilient
[login-0-1.local:03102] mca: base: components_open: component resilient open
function successful
[login-0-1.local:03102] mca: base: components_open: found loaded component
staged
[login-0-1.local:03102] mca: base: components_open: component staged open
function successful
[login-0-1.local:03102] mca: base: components_open: found loaded component
mindist
[login-0-1.local:03102] mca: base: components_open: component mindist open
function successful
[login-0-1.local:03102] mca: base: components_open: found loaded component ppr
[login-0-1.local:03102] mca: base: components_open: component ppr open function
successful
[login-0-1.local:03102] mca:rmaps:select: checking available component
round_robin
[login-0-1.local:03102] mca:rmaps:select: Querying component [round_robin]
[login-0-1.local:03102] mca:rmaps:select: checking available component rank_file
[login-0-1.local:03102] mca:rmaps:select: Querying component [rank_file]
[login-0-1.local:03102] mca:rmaps:select: checking available component seq
[login-0-1.local:03102] mca:rmaps:select: Querying component [seq]
[login-0-1.local:03102] mca:rmaps:select: checking available component resilient
[login-0-1.local:03102] mca:rmaps:select: Querying component [resilient]
[login-0-1.local:03102] mca:rmaps:select: checking available component staged
[login-0-1.local:03102] mca:rmaps:select: Querying component [staged]
[login-0-1.local:03102] mca:rmaps:select: checking available component mindist
[login-0-1.local:03102] mca:rmaps:select: Querying component [mindist]
[login-0-1.local:03102] mca:rmaps:select: checking available component ppr
[login-0-1.local:03102] mca:rmaps:select: Querying component [ppr]
[login-0-1.local:03102] [[38240,0],0]: Final mapper priorities
[login-0-1.local:03102] Mapper: ppr Priority: 90
[login-0-1.local:03102] Mapper: seq Priority: 60
[login-0-1.local:03102] Mapper: resilient Priority: 40
[login-0-1.local:03102] Mapper: mindist Priority: 20
[login-0-1.local:03102] Mapper: round_robin Priority: 10
[login-0-1.local:03102] Mapper: staged Priority: 5
[login-0-1.local:03102] Mapper: rank_file Priority: 0
[login-0-1.local:03102] mca:rmaps: mapping job [38240,1]
[login-0-1.local:03102] mca:rmaps: creating new map for job [38240,1]
[login-0-1.local:03102] mca:rmaps: nprocs 32
[login-0-1.local:03102] mca:rmaps[139] mapping not given - using bysocket
[login-0-1.local:03102] mca:rmaps:ppr: job [38240,1] not using ppr mapper
[login-0-1.local:03102] [[38240,0],0] rmaps:seq called on job [38240,1]
[login-0-1.local:03102] mca:rmaps:seq: job [38240,1] not using seq mapper
[login-0-1.local:03102] mca:rmaps:resilient: cannot perform initial map of job
[38240,1] - no fault groups
[login-0-1.local:03102] mca:rmaps:mindist: job [38240,1] not using mindist
mapper
[login-0-1.local:03102] mca:rmaps:rr: mapping job [38240,1]
[login-0-1.local:03102] [[38240,0],0] Starting with 7 nodes in list
[login-0-1.local:03102] [[38240,0],0] Filtering thru apps
[login-0-1.local:03102] [[38240,0],0] Retained 7 nodes in list
[login-0-1.local:03102] [[38240,0],0] node c1-2 has 4 slots available
[login-0-1.local:03102] [[38240,0],0] node c1-4 has 2 slots available
[login-0-1.local:03102] [[38240,0],0] node c1-8 has 5 slots available
[login-0-1.local:03102] [[38240,0],0] node c1-13 has 5 slots available
[login-0-1.local:03102] [[38240,0],0] node c1-16 has 4 slots available
[login-0-1.local:03102] [[38240,0],0] node c1-23 has 7 slots available
[login-0-1.local:03102] [[38240,0],0] node c1-26 has 5 slots available
[login-0-1.local:03102] AVAILABLE NODES FOR MAPPING:
[login-0-1.local:03102] node: c1-2 daemon: 1
[login-0-1.local:03102] node: c1-4 daemon: 2
[login-0-1.local:03102] node: c1-8 daemon: 3
[login-0-1.local:03102] node: c1-13 daemon: 4
[login-0-1.local:03102] node: c1-16 daemon: 5
[login-0-1.local:03102] node: c1-23 daemon: 6
[login-0-1.local:03102] node: c1-26 daemon: 7
[login-0-1.local:03102] [[38240,0],0] Starting bookmark at node c1-2
[login-0-1.local:03102] [[38240,0],0] Starting at node c1-2
[login-0-1.local:03102] mca:rmaps:rr: mapping no-span by Socket for job
[38240,1] slots 32 num_procs 32
[login-0-1.local:03102] mca:rmaps:rr: found 1 Socket objects on node c1-2
[login-0-1.local:03102] mca:rmaps:rr: found 2 Socket objects on node c1-4
[login-0-1.local:03102] mca:rmaps:rr: found 2 Socket objects on node c1-8
[login-0-1.local:03102] mca:rmaps:rr: found 1 Socket objects on node c1-13
[login-0-1.local:03102] mca:rmaps:rr: found 1 Socket objects on node c1-16
[login-0-1.local:03102] mca:rmaps:rr: found 2 Socket objects on node c1-23
[login-0-1.local:03102] mca:rmaps:rr: found 2 Socket objects on node c1-26
[login-0-1.local:03102] mca:rmaps:base: computing vpids by slot for job
[38240,1]
[login-0-1.local:03102] mca:rmaps:base: assigning rank 0 to node c1-2
[login-0-1.local:03102] mca:rmaps:base: assigning rank 1 to node c1-2
[login-0-1.local:03102] mca:rmaps:base: assigning rank 2 to node c1-2
[login-0-1.local:03102] mca:rmaps:base: assigning rank 3 to node c1-2
[login-0-1.local:03102] mca:rmaps:base: assigning rank 4 to node c1-4
[login-0-1.local:03102] mca:rmaps:base: assigning rank 5 to node c1-4
[login-0-1.local:03102] mca:rmaps:base: assigning rank 6 to node c1-8
[login-0-1.local:03102] mca:rmaps:base: assigning rank 7 to node c1-8
[login-0-1.local:03102] mca:rmaps:base: assigning rank 8 to node c1-8
[login-0-1.local:03102] mca:rmaps:base: assigning rank 9 to node c1-8
[login-0-1.local:03102] mca:rmaps:base: assigning rank 10 to node c1-8
[login-0-1.local:03102] mca:rmaps:base: assigning rank 11 to node c1-13
[login-0-1.local:03102] mca:rmaps:base: assigning rank 12 to node c1-13
[login-0-1.local:03102] mca:rmaps:base: assigning rank 13 to node c1-13
[login-0-1.local:03102] mca:rmaps:base: assigning rank 14 to node c1-13
[login-0-1.local:03102] mca:rmaps:base: assigning rank 15 to node c1-13
[login-0-1.local:03102] mca:rmaps:base: assigning rank 16 to node c1-16
[login-0-1.local:03102] mca:rmaps:base: assigning rank 17 to node c1-16
[login-0-1.local:03102] mca:rmaps:base: assigning rank 18 to node c1-16
[login-0-1.local:03102] mca:rmaps:base: assigning rank 19 to node c1-16
[login-0-1.local:03102] mca:rmaps:base: assigning rank 20 to node c1-23
[login-0-1.local:03102] mca:rmaps:base: assigning rank 21 to node c1-23
[login-0-1.local:03102] mca:rmaps:base: assigning rank 22 to node c1-23
[login-0-1.local:03102] mca:rmaps:base: assigning rank 23 to node c1-23
[login-0-1.local:03102] mca:rmaps:base: assigning rank 24 to node c1-23
[login-0-1.local:03102] mca:rmaps:base: assigning rank 25 to node c1-23
[login-0-1.local:03102] mca:rmaps:base: assigning rank 26 to node c1-23
[login-0-1.local:03102] mca:rmaps:base: assigning rank 27 to node c1-26
[login-0-1.local:03102] mca:rmaps:base: assigning rank 28 to node c1-26
[login-0-1.local:03102] mca:rmaps:base: assigning rank 29 to node c1-26
[login-0-1.local:03102] mca:rmaps:base: assigning rank 30 to node c1-26
[login-0-1.local:03102] mca:rmaps:base: assigning rank 31 to node c1-26
[login-0-1.local:03102] [[38240,0],0] rmaps:base:compute_usage
[login-0-1.local:03102] mca:rmaps: compute bindings for job [38240,1] with
policy CORE:OVERLOAD-ALLOWED[6008]
[login-0-1.local:03102] [[38240,0],0] reset_usage: node c1-2 has 4 procs on it
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],0]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],1]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],2]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],3]
[login-0-1.local:03102] [[38240,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03102] mca:rmaps: bind downward for job [38240,1] with
bindings CORE:OVERLOAD-ALLOWED
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],0] BITMAP 1,17
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],0][c1-2] TO socket
0[core 1[hwt 0-1]]: [../BB/../../../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],1] BITMAP 2,18
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],1][c1-2] TO socket
0[core 2[hwt 0-1]]: [../../BB/../../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],2] BITMAP 3,19
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],2][c1-2] TO socket
0[core 3[hwt 0-1]]: [../../../BB/../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],3] BITMAP 4,20
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],3][c1-2] TO socket
0[core 4[hwt 0-1]]: [../../../../BB/../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] reset_usage: node c1-4 has 2 procs on it
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],4]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],5]
[login-0-1.local:03102] [[38240,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03102] mca:rmaps: bind downward for job [38240,1] with
bindings CORE:OVERLOAD-ALLOWED
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],4] BITMAP 1,17
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],4][c1-4] TO socket
0[core 1[hwt 0-1]]: [../BB/../../../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],5] BITMAP 15,31
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],5][c1-4] TO socket
1[core 15[hwt 0-1]]: [../../../../../../../..][../../../../../../../BB]
[login-0-1.local:03102] [[38240,0],0] reset_usage: node c1-8 has 5 procs on it
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],6]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],7]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],8]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],9]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],10]
[login-0-1.local:03102] [[38240,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03102] mca:rmaps: bind downward for job [38240,1] with
bindings CORE:OVERLOAD-ALLOWED
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],6] BITMAP 0,16
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],6][c1-8] TO socket
0[core 0[hwt 0-1]]: [BB/../../../../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],7] BITMAP 9,25
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],7][c1-8] TO socket
1[core 9[hwt 0-1]]: [../../../../../../../..][../BB/../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],8] BITMAP 5,21
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],8][c1-8] TO socket
0[core 5[hwt 0-1]]: [../../../../../BB/../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],9] BITMAP 13,29
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],9][c1-8] TO socket
1[core 13[hwt 0-1]]: [../../../../../../../..][../../../../../BB/../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],10] BITMAP 0,16
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],10][c1-8] TO socket
0[core 0[hwt 0-1]]: [BB/../../../../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] reset_usage: node c1-13 has 5 procs on it
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],11]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],12]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],13]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],14]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],15]
[login-0-1.local:03102] [[38240,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03102] mca:rmaps: bind downward for job [38240,1] with
bindings CORE:OVERLOAD-ALLOWED
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],11] BITMAP 3,19
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],11][c1-13] TO
socket 0[core 3[hwt 0-1]]: [../../../BB/../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],12] BITMAP 4,20
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],12][c1-13] TO
socket 0[core 4[hwt 0-1]]: [../../../../BB/../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],13] BITMAP 5,21
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],13][c1-13] TO
socket 0[core 5[hwt 0-1]]: [../../../../../BB/../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],14] BITMAP 6,22
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],14][c1-13] TO
socket 0[core 6[hwt 0-1]]: [../../../../../../BB/..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],15] BITMAP 7,23
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],15][c1-13] TO
socket 0[core 7[hwt 0-1]]: [../../../../../../../BB][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] reset_usage: node c1-16 has 4 procs on it
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],16]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],17]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],18]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],19]
[login-0-1.local:03102] [[38240,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03102] mca:rmaps: bind downward for job [38240,1] with
bindings CORE:OVERLOAD-ALLOWED
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],16] BITMAP 12,28
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],16][c1-16] TO
socket 1[core 12[hwt 0-1]]: [../../../../../../../..][../../../../BB/../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],17] BITMAP 13,29
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],17][c1-16] TO
socket 1[core 13[hwt 0-1]]: [../../../../../../../..][../../../../../BB/../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],18] BITMAP 14,30
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],18][c1-16] TO
socket 1[core 14[hwt 0-1]]: [../../../../../../../..][../../../../../../BB/..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],19] BITMAP 15,31
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],19][c1-16] TO
socket 1[core 15[hwt 0-1]]: [../../../../../../../..][../../../../../../../BB]
[login-0-1.local:03102] [[38240,0],0] reset_usage: node c1-23 has 7 procs on it
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],20]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],21]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],22]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],23]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],24]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],25]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],26]
[login-0-1.local:03102] [[38240,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03102] mca:rmaps: bind downward for job [38240,1] with
bindings CORE:OVERLOAD-ALLOWED
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],20] BITMAP 2,18
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],20][c1-23] TO
socket 0[core 2[hwt 0-1]]: [../../BB/../../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],21] BITMAP 8,24
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],21][c1-23] TO
socket 1[core 8[hwt 0-1]]: [../../../../../../../..][BB/../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],22] BITMAP 3,19
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],22][c1-23] TO
socket 0[core 3[hwt 0-1]]: [../../../BB/../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],23] BITMAP 13,29
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],23][c1-23] TO
socket 1[core 13[hwt 0-1]]: [../../../../../../../..][../../../../../BB/../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],24] BITMAP 4,20
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],24][c1-23] TO
socket 0[core 4[hwt 0-1]]: [../../../../BB/../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],25] BITMAP 14,30
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],25][c1-23] TO
socket 1[core 14[hwt 0-1]]: [../../../../../../../..][../../../../../../BB/..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],26] BITMAP 2,18
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],26][c1-23] TO
socket 0[core 2[hwt 0-1]]: [../../BB/../../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] reset_usage: node c1-26 has 5 procs on it
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],27]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],28]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],29]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],30]
[login-0-1.local:03102] [[38240,0],0] reset_usage: ignoring proc [[38240,1],31]
[login-0-1.local:03102] [[38240,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03102] mca:rmaps: bind downward for job [38240,1] with
bindings CORE:OVERLOAD-ALLOWED
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],27] BITMAP 1,17
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],27][c1-26] TO
socket 0[core 1[hwt 0-1]]: [../BB/../../../../../..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],28] BITMAP 11,27
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],28][c1-26] TO
socket 1[core 11[hwt 0-1]]: [../../../../../../../..][../../../BB/../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],29] BITMAP 6,22
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],29][c1-26] TO
socket 0[core 6[hwt 0-1]]: [../../../../../../BB/..][../../../../../../../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],30] BITMAP 13,29
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],30][c1-26] TO
socket 1[core 13[hwt 0-1]]: [../../../../../../../..][../../../../../BB/../..]
[login-0-1.local:03102] [[38240,0],0] GOT 1 CPUS
[login-0-1.local:03102] [[38240,0],0] PROC [[38240,1],31] BITMAP 1,17
[login-0-1.local:03102] [[38240,0],0] BOUND PROC [[38240,1],31][c1-26] TO
socket 0[core 1[hwt 0-1]]: [../BB/../../../../../..][../../../../../../../..]
[compute-1-4.local:28999] MCW rank 4 bound to socket 0[core 1[hwt 0-1]]:
[../BB/../../../../../..][../../../../../../../..]
[compute-1-4.local:28999] MCW rank 5 bound to socket 1[core 15[hwt 0-1]]:
[../../../../../../../..][../../../../../../../BB]
[compute-1-2.local:01197] MCW rank 1 bound to socket 0[core 2[hwt 0-1]]:
[../../BB/../../../../..][../../../../../../../..]
[compute-1-2.local:01197] MCW rank 2 bound to socket 0[core 3[hwt 0-1]]:
[../../../BB/../../../..][../../../../../../../..]
[compute-1-2.local:01197] MCW rank 3 bound to socket 0[core 4[hwt 0-1]]:
[../../../../BB/../../..][../../../../../../../..]
[compute-1-2.local:01197] MCW rank 0 bound to socket 0[core 1[hwt 0-1]]:
[../BB/../../../../../..][../../../../../../../..]
[compute-1-16.local:20892] MCW rank 16 bound to socket 1[core 12[hwt 0-1]]:
[../../../../../../../..][../../../../BB/../../..]
[compute-1-16.local:20892] MCW rank 17 bound to socket 1[core 13[hwt 0-1]]:
[../../../../../../../..][../../../../../BB/../..]
[compute-1-16.local:20892] MCW rank 18 bound to socket 1[core 14[hwt 0-1]]:
[../../../../../../../..][../../../../../../BB/..]
[compute-1-16.local:20892] MCW rank 19 bound to socket 1[core 15[hwt 0-1]]:
[../../../../../../../..][../../../../../../../BB]
[compute-1-13.local:32092] MCW rank 11 bound to socket 0[core 3[hwt 0-1]]:
[../../../BB/../../../..][../../../../../../../..]
[compute-1-13.local:32092] MCW rank 12 bound to socket 0[core 4[hwt 0-1]]:
[../../../../BB/../../..][../../../../../../../..]
[compute-1-13.local:32092] MCW rank 13 bound to socket 0[core 5[hwt 0-1]]:
[../../../../../BB/../..][../../../../../../../..]
[compute-1-13.local:32092] MCW rank 14 bound to socket 0[core 6[hwt 0-1]]:
[../../../../../../BB/..][../../../../../../../..]
[compute-1-13.local:32092] MCW rank 15 bound to socket 0[core 7[hwt 0-1]]:
[../../../../../../../BB][../../../../../../../..]
[compute-1-8.local:18760] MCW rank 8 bound to socket 0[core 5[hwt 0-1]]:
[../../../../../BB/../..][../../../../../../../..]
[compute-1-8.local:18760] MCW rank 9 bound to socket 1[core 13[hwt 0-1]]:
[../../../../../../../..][../../../../../BB/../..]
[compute-1-8.local:18760] MCW rank 10 bound to socket 0[core 0[hwt 0-1]]:
[BB/../../../../../../..][../../../../../../../..]
[compute-1-8.local:18760] MCW rank 6 bound to socket 0[core 0[hwt 0-1]]:
[BB/../../../../../../..][../../../../../../../..]
[compute-1-8.local:18760] MCW rank 7 bound to socket 1[core 9[hwt 0-1]]:
[../../../../../../../..][../BB/../../../../../..]
[compute-1-26.local:11478] MCW rank 31 bound to socket 0[core 1[hwt 0-1]]:
[../BB/../../../../../..][../../../../../../../..]
[compute-1-26.local:11478] MCW rank 27 bound to socket 0[core 1[hwt 0-1]]:
[../BB/../../../../../..][../../../../../../../..]
[compute-1-26.local:11478] MCW rank 28 bound to socket 1[core 11[hwt 0-1]]:
[../../../../../../../..][../../../BB/../../../..]
[compute-1-26.local:11478] MCW rank 29 bound to socket 0[core 6[hwt 0-1]]:
[../../../../../../BB/..][../../../../../../../..]
[compute-1-26.local:11478] MCW rank 30 bound to socket 1[core 13[hwt 0-1]]:
[../../../../../../../..][../../../../../BB/../..]
[compute-1-23.local:03591] MCW rank 20 bound to socket 0[core 2[hwt 0-1]]:
[../../BB/../../../../..][../../../../../../../..]
[compute-1-23.local:03591] MCW rank 21 bound to socket 1[core 8[hwt 0-1]]:
[../../../../../../../..][BB/../../../../../../..]
[compute-1-23.local:03591] MCW rank 22 bound to socket 0[core 3[hwt 0-1]]:
[../../../BB/../../../..][../../../../../../../..]
[compute-1-23.local:03591] MCW rank 23 bound to socket 1[core 13[hwt 0-1]]:
[../../../../../../../..][../../../../../BB/../..]
[compute-1-23.local:03591] MCW rank 24 bound to socket 0[core 4[hwt 0-1]]:
[../../../../BB/../../..][../../../../../../../..]
[compute-1-23.local:03591] MCW rank 25 bound to socket 1[core 14[hwt 0-1]]:
[../../../../../../../..][../../../../../../BB/..]
[compute-1-23.local:03591] MCW rank 26 bound to socket 0[core 2[hwt 0-1]]:
[../../BB/../../../../..][../../../../../../../..]
rank 0 @ compute-1-2.local 1, 17,
rank 1 @ compute-1-2.local 2, 18,
rank 4 @ compute-1-4.local 1, 17,
rank 5 @ compute-1-4.local 15, 31,
rank 2 @ compute-1-2.local 3, 19,
rank 9 @ compute-1-8.local 13, 29,
rank 13 @ compute-1-13.local 5, 21,
rank 3 @ compute-1-2.local 4, 20,
rank 10 @ compute-1-8.local 0, 16,
rank 14 @ compute-1-13.local 6, 22,
rank 6 @ compute-1-8.local 0, 16,
rank 7 @ compute-1-8.local 9, 25,
rank 15 @ compute-1-13.local 7, 23,
rank 11 @ compute-1-13.local 3, 19,
rank 8 @ compute-1-8.local 5, 21,
rank 12 @ compute-1-13.local 4, 20,
rank 16 @ compute-1-16.local 12, 28,
rank 17 @ compute-1-16.local 13, 29,
rank 23 @ compute-1-23.local 13, 29,
rank 27 @ compute-1-26.local 1, 17,
rank 18 @ compute-1-16.local 14, 30,
rank 24 @ compute-1-23.local 4, 20,
rank 28 @ compute-1-26.local 11, 27,
rank 19 @ compute-1-16.local 15, 31,
rank 25 @ compute-1-23.local 14, 30,
rank 29 @ compute-1-26.local 6, 22,
rank 26 @ compute-1-23.local 2, 18,
rank 30 @ compute-1-26.local 13, 29,
rank 20 @ compute-1-23.local 2, 18,
rank 31 @ compute-1-26.local 1, 17,
rank 21 @ compute-1-23.local 8, 24,
rank 22 @ compute-1-23.local 3, 19,
[login-0-1.local:03102] mca: base: close: component round_robin closed
[login-0-1.local:03102] mca: base: close: unloading component round_robin
[login-0-1.local:03102] mca: base: close: component rank_file closed
[login-0-1.local:03102] mca: base: close: unloading component rank_file
[login-0-1.local:03102] mca: base: close: component seq closed
[login-0-1.local:03102] mca: base: close: unloading component seq
[login-0-1.local:03102] mca: base: close: component resilient closed
[login-0-1.local:03102] mca: base: close: unloading component resilient
[login-0-1.local:03102] mca: base: close: component staged closed
[login-0-1.local:03102] mca: base: close: unloading component staged
[login-0-1.local:03102] mca: base: close: component mindist closed
[login-0-1.local:03102] mca: base: close: unloading component mindist
[login-0-1.local:03102] mca: base: close: component ppr closed
[login-0-1.local:03102] mca: base: close: unloading component ppr
[login-0-1.local:03004] mca: base: components_register: registering rmaps
components
[login-0-1.local:03004] mca: base: components_register: found loaded component
round_robin
[login-0-1.local:03004] mca: base: components_register: component round_robin
register function successful
[login-0-1.local:03004] mca: base: components_register: found loaded component
rank_file
[login-0-1.local:03004] mca: base: components_register: component rank_file
register function successful
[login-0-1.local:03004] mca: base: components_register: found loaded component
seq
[login-0-1.local:03004] mca: base: components_register: component seq register
function successful
[login-0-1.local:03004] mca: base: components_register: found loaded component
resilient
[login-0-1.local:03004] mca: base: components_register: component resilient
register function successful
[login-0-1.local:03004] mca: base: components_register: found loaded component
staged
[login-0-1.local:03004] mca: base: components_register: component staged has no
register or open function
[login-0-1.local:03004] mca: base: components_register: found loaded component
mindist
[login-0-1.local:03004] mca: base: components_register: component mindist
register function successful
[login-0-1.local:03004] mca: base: components_register: found loaded component
ppr
[login-0-1.local:03004] mca: base: components_register: component ppr register
function successful
[login-0-1.local:03004] [[37570,0],0] rmaps:base set policy with NULL
[login-0-1.local:03004] mca: base: components_open: opening rmaps components
[login-0-1.local:03004] mca: base: components_open: found loaded component
round_robin
[login-0-1.local:03004] mca: base: components_open: component round_robin open
function successful
[login-0-1.local:03004] mca: base: components_open: found loaded component
rank_file
[login-0-1.local:03004] mca: base: components_open: component rank_file open
function successful
[login-0-1.local:03004] mca: base: components_open: found loaded component seq
[login-0-1.local:03004] mca: base: components_open: component seq open function
successful
[login-0-1.local:03004] mca: base: components_open: found loaded component
resilient
[login-0-1.local:03004] mca: base: components_open: component resilient open
function successful
[login-0-1.local:03004] mca: base: components_open: found loaded component
staged
[login-0-1.local:03004] mca: base: components_open: component staged open
function successful
[login-0-1.local:03004] mca: base: components_open: found loaded component
mindist
[login-0-1.local:03004] mca: base: components_open: component mindist open
function successful
[login-0-1.local:03004] mca: base: components_open: found loaded component ppr
[login-0-1.local:03004] mca: base: components_open: component ppr open function
successful
[login-0-1.local:03004] mca:rmaps:select: checking available component
round_robin
[login-0-1.local:03004] mca:rmaps:select: Querying component [round_robin]
[login-0-1.local:03004] mca:rmaps:select: checking available component rank_file
[login-0-1.local:03004] mca:rmaps:select: Querying component [rank_file]
[login-0-1.local:03004] mca:rmaps:select: checking available component seq
[login-0-1.local:03004] mca:rmaps:select: Querying component [seq]
[login-0-1.local:03004] mca:rmaps:select: checking available component resilient
[login-0-1.local:03004] mca:rmaps:select: Querying component [resilient]
[login-0-1.local:03004] mca:rmaps:select: checking available component staged
[login-0-1.local:03004] mca:rmaps:select: Querying component [staged]
[login-0-1.local:03004] mca:rmaps:select: checking available component mindist
[login-0-1.local:03004] mca:rmaps:select: Querying component [mindist]
[login-0-1.local:03004] mca:rmaps:select: checking available component ppr
[login-0-1.local:03004] mca:rmaps:select: Querying component [ppr]
[login-0-1.local:03004] [[37570,0],0]: Final mapper priorities
[login-0-1.local:03004] Mapper: ppr Priority: 90
[login-0-1.local:03004] Mapper: seq Priority: 60
[login-0-1.local:03004] Mapper: resilient Priority: 40
[login-0-1.local:03004] Mapper: mindist Priority: 20
[login-0-1.local:03004] Mapper: round_robin Priority: 10
[login-0-1.local:03004] Mapper: staged Priority: 5
[login-0-1.local:03004] Mapper: rank_file Priority: 0
[login-0-1.local:03004] mca:rmaps: mapping job [37570,1]
[login-0-1.local:03004] mca:rmaps: creating new map for job [37570,1]
[login-0-1.local:03004] mca:rmaps: nprocs 32
[login-0-1.local:03004] mca:rmaps[139] mapping not given - using bysocket
[login-0-1.local:03004] mca:rmaps:ppr: job [37570,1] not using ppr mapper
[login-0-1.local:03004] [[37570,0],0] rmaps:seq called on job [37570,1]
[login-0-1.local:03004] mca:rmaps:seq: job [37570,1] not using seq mapper
[login-0-1.local:03004] mca:rmaps:resilient: cannot perform initial map of job
[37570,1] - no fault groups
[login-0-1.local:03004] mca:rmaps:mindist: job [37570,1] not using mindist
mapper
[login-0-1.local:03004] mca:rmaps:rr: mapping job [37570,1]
[login-0-1.local:03004] [[37570,0],0] Starting with 7 nodes in list
[login-0-1.local:03004] [[37570,0],0] Filtering thru apps
[login-0-1.local:03004] [[37570,0],0] Retained 7 nodes in list
[login-0-1.local:03004] [[37570,0],0] node c1-2 has 4 slots available
[login-0-1.local:03004] [[37570,0],0] node c1-4 has 2 slots available
[login-0-1.local:03004] [[37570,0],0] node c1-8 has 5 slots available
[login-0-1.local:03004] [[37570,0],0] node c1-13 has 5 slots available
[login-0-1.local:03004] [[37570,0],0] node c1-16 has 4 slots available
[login-0-1.local:03004] [[37570,0],0] node c1-23 has 7 slots available
[login-0-1.local:03004] [[37570,0],0] node c1-26 has 5 slots available
[login-0-1.local:03004] AVAILABLE NODES FOR MAPPING:
[login-0-1.local:03004] node: c1-2 daemon: 1
[login-0-1.local:03004] node: c1-4 daemon: 2
[login-0-1.local:03004] node: c1-8 daemon: 3
[login-0-1.local:03004] node: c1-13 daemon: 4
[login-0-1.local:03004] node: c1-16 daemon: 5
[login-0-1.local:03004] node: c1-23 daemon: 6
[login-0-1.local:03004] node: c1-26 daemon: 7
[login-0-1.local:03004] [[37570,0],0] Starting bookmark at node c1-2
[login-0-1.local:03004] [[37570,0],0] Starting at node c1-2
[login-0-1.local:03004] mca:rmaps:rr: mapping no-span by Socket for job
[37570,1] slots 32 num_procs 32
[login-0-1.local:03004] mca:rmaps:rr: found 1 Socket objects on node c1-2
[login-0-1.local:03004] mca:rmaps:rr: found 2 Socket objects on node c1-4
[login-0-1.local:03004] mca:rmaps:rr: found 2 Socket objects on node c1-8
[login-0-1.local:03004] mca:rmaps:rr: found 1 Socket objects on node c1-13
[login-0-1.local:03004] mca:rmaps:rr: found 1 Socket objects on node c1-16
[login-0-1.local:03004] mca:rmaps:rr: found 2 Socket objects on node c1-23
[login-0-1.local:03004] mca:rmaps:rr: found 2 Socket objects on node c1-26
[login-0-1.local:03004] mca:rmaps:base: computing vpids by slot for job
[37570,1]
[login-0-1.local:03004] mca:rmaps:base: assigning rank 0 to node c1-2
[login-0-1.local:03004] mca:rmaps:base: assigning rank 1 to node c1-2
[login-0-1.local:03004] mca:rmaps:base: assigning rank 2 to node c1-2
[login-0-1.local:03004] mca:rmaps:base: assigning rank 3 to node c1-2
[login-0-1.local:03004] mca:rmaps:base: assigning rank 4 to node c1-4
[login-0-1.local:03004] mca:rmaps:base: assigning rank 5 to node c1-4
[login-0-1.local:03004] mca:rmaps:base: assigning rank 6 to node c1-8
[login-0-1.local:03004] mca:rmaps:base: assigning rank 7 to node c1-8
[login-0-1.local:03004] mca:rmaps:base: assigning rank 8 to node c1-8
[login-0-1.local:03004] mca:rmaps:base: assigning rank 9 to node c1-8
[login-0-1.local:03004] mca:rmaps:base: assigning rank 10 to node c1-8
[login-0-1.local:03004] mca:rmaps:base: assigning rank 11 to node c1-13
[login-0-1.local:03004] mca:rmaps:base: assigning rank 12 to node c1-13
[login-0-1.local:03004] mca:rmaps:base: assigning rank 13 to node c1-13
[login-0-1.local:03004] mca:rmaps:base: assigning rank 14 to node c1-13
[login-0-1.local:03004] mca:rmaps:base: assigning rank 15 to node c1-13
[login-0-1.local:03004] mca:rmaps:base: assigning rank 16 to node c1-16
[login-0-1.local:03004] mca:rmaps:base: assigning rank 17 to node c1-16
[login-0-1.local:03004] mca:rmaps:base: assigning rank 18 to node c1-16
[login-0-1.local:03004] mca:rmaps:base: assigning rank 19 to node c1-16
[login-0-1.local:03004] mca:rmaps:base: assigning rank 20 to node c1-23
[login-0-1.local:03004] mca:rmaps:base: assigning rank 21 to node c1-23
[login-0-1.local:03004] mca:rmaps:base: assigning rank 22 to node c1-23
[login-0-1.local:03004] mca:rmaps:base: assigning rank 23 to node c1-23
[login-0-1.local:03004] mca:rmaps:base: assigning rank 24 to node c1-23
[login-0-1.local:03004] mca:rmaps:base: assigning rank 25 to node c1-23
[login-0-1.local:03004] mca:rmaps:base: assigning rank 26 to node c1-23
[login-0-1.local:03004] mca:rmaps:base: assigning rank 27 to node c1-26
[login-0-1.local:03004] mca:rmaps:base: assigning rank 28 to node c1-26
[login-0-1.local:03004] mca:rmaps:base: assigning rank 29 to node c1-26
[login-0-1.local:03004] mca:rmaps:base: assigning rank 30 to node c1-26
[login-0-1.local:03004] mca:rmaps:base: assigning rank 31 to node c1-26
[login-0-1.local:03004] [[37570,0],0] rmaps:base:compute_usage
[login-0-1.local:03004] mca:rmaps: compute bindings for job [37570,1] with
policy CORE[4008]
[login-0-1.local:03004] [[37570,0],0] reset_usage: node c1-2 has 4 procs on it
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],0]
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],1]
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],2]
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],3]
[login-0-1.local:03004] [[37570,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03004] mca:rmaps: bind downward for job [37570,1] with
bindings CORE
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
[login-0-1.local:03004] [[37570,0],0] PROC [[37570,1],0] BITMAP 1,17
[login-0-1.local:03004] [[37570,0],0] BOUND PROC [[37570,1],0][c1-2] TO socket
0[core 1[hwt 0-1]]: [../BB/../../../../../..][../../../../../../../..]
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
[login-0-1.local:03004] [[37570,0],0] PROC [[37570,1],1] BITMAP 2,18
[login-0-1.local:03004] [[37570,0],0] BOUND PROC [[37570,1],1][c1-2] TO socket
0[core 2[hwt 0-1]]: [../../BB/../../../../..][../../../../../../../..]
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
[login-0-1.local:03004] [[37570,0],0] PROC [[37570,1],2] BITMAP 3,19
[login-0-1.local:03004] [[37570,0],0] BOUND PROC [[37570,1],2][c1-2] TO socket
0[core 3[hwt 0-1]]: [../../../BB/../../../..][../../../../../../../..]
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
[login-0-1.local:03004] [[37570,0],0] PROC [[37570,1],3] BITMAP 4,20
[login-0-1.local:03004] [[37570,0],0] BOUND PROC [[37570,1],3][c1-2] TO socket
0[core 4[hwt 0-1]]: [../../../../BB/../../..][../../../../../../../..]
[login-0-1.local:03004] [[37570,0],0] reset_usage: node c1-4 has 2 procs on it
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],4]
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],5]
[login-0-1.local:03004] [[37570,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03004] mca:rmaps: bind downward for job [37570,1] with
bindings CORE
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
[login-0-1.local:03004] [[37570,0],0] PROC [[37570,1],4] BITMAP 1,17
[login-0-1.local:03004] [[37570,0],0] BOUND PROC [[37570,1],4][c1-4] TO socket
0[core 1[hwt 0-1]]: [../BB/../../../../../..][../../../../../../../..]
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
[login-0-1.local:03004] [[37570,0],0] PROC [[37570,1],5] BITMAP 15,31
[login-0-1.local:03004] [[37570,0],0] BOUND PROC [[37570,1],5][c1-4] TO socket
1[core 15[hwt 0-1]]: [../../../../../../../..][../../../../../../../BB]
[login-0-1.local:03004] [[37570,0],0] reset_usage: node c1-8 has 5 procs on it
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],6]
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],7]
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],8]
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],9]
[login-0-1.local:03004] [[37570,0],0] reset_usage: ignoring proc [[37570,1],10]
[login-0-1.local:03004] [[37570,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03004] mca:rmaps: bind downward for job [37570,1] with
bindings CORE
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
[login-0-1.local:03004] [[37570,0],0] PROC [[37570,1],6] BITMAP 0,16
[login-0-1.local:03004] [[37570,0],0] BOUND PROC [[37570,1],6][c1-8] TO socket
0[core 0[hwt 0-1]]: [BB/../../../../../../..][../../../../../../../..]
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
[login-0-1.local:03004] [[37570,0],0] PROC [[37570,1],7] BITMAP 9,25
[login-0-1.local:03004] [[37570,0],0] BOUND PROC [[37570,1],7][c1-8] TO socket
1[core 9[hwt 0-1]]: [../../../../../../../..][../BB/../../../../../..]
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
[login-0-1.local:03004] [[37570,0],0] PROC [[37570,1],8] BITMAP 5,21
[login-0-1.local:03004] [[37570,0],0] BOUND PROC [[37570,1],8][c1-8] TO socket
0[core 5[hwt 0-1]]: [../../../../../BB/../..][../../../../../../../..]
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
[login-0-1.local:03004] [[37570,0],0] PROC [[37570,1],9] BITMAP 13,29
[login-0-1.local:03004] [[37570,0],0] BOUND PROC [[37570,1],9][c1-8] TO socket
1[core 13[hwt 0-1]]: [../../../../../../../..][../../../../../BB/../..]
[login-0-1.local:03004] [[37570,0],0] GOT 1 CPUS
--------------------------------------------------------------------------
A request was made to bind to that would result in binding more
processes than cpus on a resource:
Bind to: CORE
Node: c1-8
#processes: 2
#cpus: 1
You can override this protection by adding the "overload-allowed"
option to your binding directive.
--------------------------------------------------------------------------
[login-0-1.local:03004] mca: base: close: component round_robin closed
[login-0-1.local:03004] mca: base: close: unloading component round_robin
[login-0-1.local:03004] mca: base: close: component rank_file closed
[login-0-1.local:03004] mca: base: close: unloading component rank_file
[login-0-1.local:03004] mca: base: close: component seq closed
[login-0-1.local:03004] mca: base: close: unloading component seq
[login-0-1.local:03004] mca: base: close: component resilient closed
[login-0-1.local:03004] mca: base: close: unloading component resilient
[login-0-1.local:03004] mca: base: close: component staged closed
[login-0-1.local:03004] mca: base: close: unloading component staged
[login-0-1.local:03004] mca: base: close: component mindist closed
[login-0-1.local:03004] mca: base: close: unloading component mindist
[login-0-1.local:03004] mca: base: close: component ppr closed
[login-0-1.local:03004] mca: base: close: unloading component ppr
[login-0-1.local:03399] mca: base: components_register: registering rmaps
components
[login-0-1.local:03399] mca: base: components_register: found loaded component
round_robin
[login-0-1.local:03399] mca: base: components_register: component round_robin
register function successful
[login-0-1.local:03399] mca: base: components_register: found loaded component
rank_file
[login-0-1.local:03399] mca: base: components_register: component rank_file
register function successful
[login-0-1.local:03399] mca: base: components_register: found loaded component
seq
[login-0-1.local:03399] mca: base: components_register: component seq register
function successful
[login-0-1.local:03399] mca: base: components_register: found loaded component
resilient
[login-0-1.local:03399] mca: base: components_register: component resilient
register function successful
[login-0-1.local:03399] mca: base: components_register: found loaded component
staged
[login-0-1.local:03399] mca: base: components_register: component staged has no
register or open function
[login-0-1.local:03399] mca: base: components_register: found loaded component
mindist
[login-0-1.local:03399] mca: base: components_register: component mindist
register function successful
[login-0-1.local:03399] mca: base: components_register: found loaded component
ppr
[login-0-1.local:03399] mca: base: components_register: component ppr register
function successful
[login-0-1.local:03399] [[37945,0],0] rmaps:base set policy with NULL
[login-0-1.local:03399] mca: base: components_open: opening rmaps components
[login-0-1.local:03399] mca: base: components_open: found loaded component
round_robin
[login-0-1.local:03399] mca: base: components_open: component round_robin open
function successful
[login-0-1.local:03399] mca: base: components_open: found loaded component
rank_file
[login-0-1.local:03399] mca: base: components_open: component rank_file open
function successful
[login-0-1.local:03399] mca: base: components_open: found loaded component seq
[login-0-1.local:03399] mca: base: components_open: component seq open function
successful
[login-0-1.local:03399] mca: base: components_open: found loaded component
resilient
[login-0-1.local:03399] mca: base: components_open: component resilient open
function successful
[login-0-1.local:03399] mca: base: components_open: found loaded component
staged
[login-0-1.local:03399] mca: base: components_open: component staged open
function successful
[login-0-1.local:03399] mca: base: components_open: found loaded component
mindist
[login-0-1.local:03399] mca: base: components_open: component mindist open
function successful
[login-0-1.local:03399] mca: base: components_open: found loaded component ppr
[login-0-1.local:03399] mca: base: components_open: component ppr open function
successful
[login-0-1.local:03399] mca:rmaps:select: checking available component
round_robin
[login-0-1.local:03399] mca:rmaps:select: Querying component [round_robin]
[login-0-1.local:03399] mca:rmaps:select: checking available component rank_file
[login-0-1.local:03399] mca:rmaps:select: Querying component [rank_file]
[login-0-1.local:03399] mca:rmaps:select: checking available component seq
[login-0-1.local:03399] mca:rmaps:select: Querying component [seq]
[login-0-1.local:03399] mca:rmaps:select: checking available component resilient
[login-0-1.local:03399] mca:rmaps:select: Querying component [resilient]
[login-0-1.local:03399] mca:rmaps:select: checking available component staged
[login-0-1.local:03399] mca:rmaps:select: Querying component [staged]
[login-0-1.local:03399] mca:rmaps:select: checking available component mindist
[login-0-1.local:03399] mca:rmaps:select: Querying component [mindist]
[login-0-1.local:03399] mca:rmaps:select: checking available component ppr
[login-0-1.local:03399] mca:rmaps:select: Querying component [ppr]
[login-0-1.local:03399] [[37945,0],0]: Final mapper priorities
[login-0-1.local:03399] Mapper: ppr Priority: 90
[login-0-1.local:03399] Mapper: seq Priority: 60
[login-0-1.local:03399] Mapper: resilient Priority: 40
[login-0-1.local:03399] Mapper: mindist Priority: 20
[login-0-1.local:03399] Mapper: round_robin Priority: 10
[login-0-1.local:03399] Mapper: staged Priority: 5
[login-0-1.local:03399] Mapper: rank_file Priority: 0
[login-0-1.local:03399] mca:rmaps: mapping job [37945,1]
[login-0-1.local:03399] mca:rmaps: creating new map for job [37945,1]
[login-0-1.local:03399] mca:rmaps: nprocs 32
[login-0-1.local:03399] mca:rmaps[139] mapping not given - using bysocket
[login-0-1.local:03399] mca:rmaps:ppr: job [37945,1] not using ppr mapper
[login-0-1.local:03399] [[37945,0],0] rmaps:seq called on job [37945,1]
[login-0-1.local:03399] mca:rmaps:seq: job [37945,1] not using seq mapper
[login-0-1.local:03399] mca:rmaps:resilient: cannot perform initial map of job
[37945,1] - no fault groups
[login-0-1.local:03399] mca:rmaps:mindist: job [37945,1] not using mindist
mapper
[login-0-1.local:03399] mca:rmaps:rr: mapping job [37945,1]
[login-0-1.local:03399] [[37945,0],0] Starting with 7 nodes in list
[login-0-1.local:03399] [[37945,0],0] Filtering thru apps
[login-0-1.local:03399] [[37945,0],0] Retained 7 nodes in list
[login-0-1.local:03399] [[37945,0],0] node c1-2 has 4 slots available
[login-0-1.local:03399] [[37945,0],0] node c1-4 has 2 slots available
[login-0-1.local:03399] [[37945,0],0] node c1-8 has 5 slots available
[login-0-1.local:03399] [[37945,0],0] node c1-13 has 5 slots available
[login-0-1.local:03399] [[37945,0],0] node c1-16 has 4 slots available
[login-0-1.local:03399] [[37945,0],0] node c1-23 has 7 slots available
[login-0-1.local:03399] [[37945,0],0] node c1-26 has 5 slots available
[login-0-1.local:03399] AVAILABLE NODES FOR MAPPING:
[login-0-1.local:03399] node: c1-2 daemon: 1
[login-0-1.local:03399] node: c1-4 daemon: 2
[login-0-1.local:03399] node: c1-8 daemon: 3
[login-0-1.local:03399] node: c1-13 daemon: 4
[login-0-1.local:03399] node: c1-16 daemon: 5
[login-0-1.local:03399] node: c1-23 daemon: 6
[login-0-1.local:03399] node: c1-26 daemon: 7
[login-0-1.local:03399] [[37945,0],0] Starting bookmark at node c1-2
[login-0-1.local:03399] [[37945,0],0] Starting at node c1-2
[login-0-1.local:03399] mca:rmaps:rr: mapping no-span by Socket for job
[37945,1] slots 32 num_procs 32
[login-0-1.local:03399] mca:rmaps:rr: found 1 Socket objects on node c1-2
[login-0-1.local:03399] mca:rmaps:rr: found 2 Socket objects on node c1-4
[login-0-1.local:03399] mca:rmaps:rr: found 2 Socket objects on node c1-8
[login-0-1.local:03399] mca:rmaps:rr: found 1 Socket objects on node c1-13
[login-0-1.local:03399] mca:rmaps:rr: found 1 Socket objects on node c1-16
[login-0-1.local:03399] mca:rmaps:rr: found 2 Socket objects on node c1-23
[login-0-1.local:03399] mca:rmaps:rr: found 2 Socket objects on node c1-26
[login-0-1.local:03399] mca:rmaps:base: computing vpids by slot for job
[37945,1]
[login-0-1.local:03399] mca:rmaps:base: assigning rank 0 to node c1-2
[login-0-1.local:03399] mca:rmaps:base: assigning rank 1 to node c1-2
[login-0-1.local:03399] mca:rmaps:base: assigning rank 2 to node c1-2
[login-0-1.local:03399] mca:rmaps:base: assigning rank 3 to node c1-2
[login-0-1.local:03399] mca:rmaps:base: assigning rank 4 to node c1-4
[login-0-1.local:03399] mca:rmaps:base: assigning rank 5 to node c1-4
[login-0-1.local:03399] mca:rmaps:base: assigning rank 6 to node c1-8
[login-0-1.local:03399] mca:rmaps:base: assigning rank 7 to node c1-8
[login-0-1.local:03399] mca:rmaps:base: assigning rank 8 to node c1-8
[login-0-1.local:03399] mca:rmaps:base: assigning rank 9 to node c1-8
[login-0-1.local:03399] mca:rmaps:base: assigning rank 10 to node c1-8
[login-0-1.local:03399] mca:rmaps:base: assigning rank 11 to node c1-13
[login-0-1.local:03399] mca:rmaps:base: assigning rank 12 to node c1-13
[login-0-1.local:03399] mca:rmaps:base: assigning rank 13 to node c1-13
[login-0-1.local:03399] mca:rmaps:base: assigning rank 14 to node c1-13
[login-0-1.local:03399] mca:rmaps:base: assigning rank 15 to node c1-13
[login-0-1.local:03399] mca:rmaps:base: assigning rank 16 to node c1-16
[login-0-1.local:03399] mca:rmaps:base: assigning rank 17 to node c1-16
[login-0-1.local:03399] mca:rmaps:base: assigning rank 18 to node c1-16
[login-0-1.local:03399] mca:rmaps:base: assigning rank 19 to node c1-16
[login-0-1.local:03399] mca:rmaps:base: assigning rank 20 to node c1-23
[login-0-1.local:03399] mca:rmaps:base: assigning rank 21 to node c1-23
[login-0-1.local:03399] mca:rmaps:base: assigning rank 22 to node c1-23
[login-0-1.local:03399] mca:rmaps:base: assigning rank 23 to node c1-23
[login-0-1.local:03399] mca:rmaps:base: assigning rank 24 to node c1-23
[login-0-1.local:03399] mca:rmaps:base: assigning rank 25 to node c1-23
[login-0-1.local:03399] mca:rmaps:base: assigning rank 26 to node c1-23
[login-0-1.local:03399] mca:rmaps:base: assigning rank 27 to node c1-26
[login-0-1.local:03399] mca:rmaps:base: assigning rank 28 to node c1-26
[login-0-1.local:03399] mca:rmaps:base: assigning rank 29 to node c1-26
[login-0-1.local:03399] mca:rmaps:base: assigning rank 30 to node c1-26
[login-0-1.local:03399] mca:rmaps:base: assigning rank 31 to node c1-26
[login-0-1.local:03399] [[37945,0],0] rmaps:base:compute_usage
[login-0-1.local:03399] mca:rmaps: compute bindings for job [37945,1] with
policy CORE[4008]
[login-0-1.local:03399] [[37945,0],0] reset_usage: node c1-2 has 4 procs on it
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],0]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],1]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],2]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],3]
[login-0-1.local:03399] [[37945,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03399] mca:rmaps: bind downward for job [37945,1] with
bindings CORE
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],0] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],0] ON c1-2 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],1] BITMAP 1,17
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],1][c1-2] TO socket
0[core 1[hwt 0-1]]: [../BB/../../../../../..][../../../../../../../..]
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],2] BITMAP 2,18
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],2][c1-2] TO socket
0[core 2[hwt 0-1]]: [../../BB/../../../../..][../../../../../../../..]
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],3] BITMAP 3,19
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],3][c1-2] TO socket
0[core 3[hwt 0-1]]: [../../../BB/../../../..][../../../../../../../..]
[login-0-1.local:03399] [[37945,0],0] reset_usage: node c1-4 has 2 procs on it
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],4]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],5]
[login-0-1.local:03399] [[37945,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03399] mca:rmaps: bind downward for job [37945,1] with
bindings CORE
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],4] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],4] ON c1-4 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],5] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],5] ON c1-4 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] reset_usage: node c1-8 has 5 procs on it
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],6]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],7]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],8]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],9]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],10]
[login-0-1.local:03399] [[37945,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03399] mca:rmaps: bind downward for job [37945,1] with
bindings CORE
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],6] BITMAP 0,16
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],6][c1-8] TO socket
0[core 0[hwt 0-1]]: [BB/../../../../../../..][../../../../../../../..]
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],7] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],7] ON c1-8 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],8] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],8] ON c1-8 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],9] BITMAP 9,25
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],9][c1-8] TO socket
1[core 9[hwt 0-1]]: [../../../../../../../..][../BB/../../../../../..]
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],10] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],10] ON c1-8 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] reset_usage: node c1-13 has 5 procs on it
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],11]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],12]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],13]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],14]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],15]
[login-0-1.local:03399] [[37945,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03399] mca:rmaps: bind downward for job [37945,1] with
bindings CORE
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],11] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],11] ON c1-13 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],12] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],12] ON c1-13 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],13] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],13] ON c1-13 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],14] BITMAP 3,19
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],14][c1-13] TO
socket 0[core 3[hwt 0-1]]: [../../../BB/../../../..][../../../../../../../..]
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],15] BITMAP 4,20
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],15][c1-13] TO
socket 0[core 4[hwt 0-1]]: [../../../../BB/../../..][../../../../../../../..]
[login-0-1.local:03399] [[37945,0],0] reset_usage: node c1-16 has 4 procs on it
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],16]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],17]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],18]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],19]
[login-0-1.local:03399] [[37945,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03399] mca:rmaps: bind downward for job [37945,1] with
bindings CORE
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],16] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],16] ON c1-16 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],17] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],17] ON c1-16 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],18] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],18] ON c1-16 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],19] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],19] ON c1-16 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] reset_usage: node c1-23 has 7 procs on it
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],20]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],21]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],22]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],23]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],24]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],25]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],26]
[login-0-1.local:03399] [[37945,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03399] mca:rmaps: bind downward for job [37945,1] with
bindings CORE
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],20] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],20] ON c1-23 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],21] BITMAP 8,24
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],21][c1-23] TO
socket 1[core 8[hwt 0-1]]: [../../../../../../../..][BB/../../../../../../..]
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],22] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],22] ON c1-23 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],23] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],23] ON c1-23 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],24] BITMAP 2,18
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],24][c1-23] TO
socket 0[core 2[hwt 0-1]]: [../../BB/../../../../..][../../../../../../../..]
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],25] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],25] ON c1-23 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],26] BITMAP 3,19
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],26][c1-23] TO
socket 0[core 3[hwt 0-1]]: [../../../BB/../../../..][../../../../../../../..]
[login-0-1.local:03399] [[37945,0],0] reset_usage: node c1-26 has 5 procs on it
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],27]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],28]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],29]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],30]
[login-0-1.local:03399] [[37945,0],0] reset_usage: ignoring proc [[37945,1],31]
[login-0-1.local:03399] [[37945,0],0] bind_depth: 6 map_depth 2
[login-0-1.local:03399] mca:rmaps: bind downward for job [37945,1] with
bindings CORE
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],27] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],27] ON c1-26 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],28] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],28] ON c1-26 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],29] BITMAP 1,17
[login-0-1.local:03399] [[37945,0],0] BOUND PROC [[37945,1],29][c1-26] TO
socket 0[core 1[hwt 0-1]]: [../BB/../../../../../..][../../../../../../../..]
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],30] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],30] ON c1-26 IS NOT BOUND
[login-0-1.local:03399] [[37945,0],0] GOT 1 CPUS
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],31] BITMAP
[login-0-1.local:03399] [[37945,0],0] PROC [[37945,1],31] ON c1-26 IS NOT BOUND
[compute-1-4.local:29344] MCW rank 4 is not bound (or bound to all available
processors)
[compute-1-4.local:29344] MCW rank 5 is not bound (or bound to all available
processors)
[compute-1-2.local:01645] MCW rank 0 is not bound (or bound to all available
processors)
[compute-1-2.local:01645] MCW rank 1 bound to socket 0[core 1[hwt 0-1]]:
[../BB/../../../../../..][../../../../../../../..]
[compute-1-2.local:01645] MCW rank 2 bound to socket 0[core 2[hwt 0-1]]:
[../../BB/../../../../..][../../../../../../../..]
[compute-1-2.local:01645] MCW rank 3 bound to socket 0[core 3[hwt 0-1]]:
[../../../BB/../../../..][../../../../../../../..]
[compute-1-16.local:21338] MCW rank 16 is not bound (or bound to all available
processors)
[compute-1-16.local:21338] MCW rank 17 is not bound (or bound to all available
processors)
[compute-1-16.local:21338] MCW rank 18 is not bound (or bound to all available
processors)
[compute-1-16.local:21338] MCW rank 19 is not bound (or bound to all available
processors)
[compute-1-13.local:32486] MCW rank 11 is not bound (or bound to all available
processors)
[compute-1-13.local:32486] MCW rank 12 is not bound (or bound to all available
processors)
[compute-1-13.local:32486] MCW rank 13 is not bound (or bound to all available
processors)
[compute-1-13.local:32486] MCW rank 14 bound to socket 0[core 3[hwt 0-1]]:
[../../../BB/../../../..][../../../../../../../..]
[compute-1-13.local:32486] MCW rank 15 bound to socket 0[core 4[hwt 0-1]]:
[../../../../BB/../../..][../../../../../../../..]
[compute-1-26.local:11970] MCW rank 31 is not bound (or bound to all available
processors)
[compute-1-26.local:11970] MCW rank 27 is not bound (or bound to all available
processors)
[compute-1-26.local:11970] MCW rank 28 is not bound (or bound to all available
processors)
[compute-1-26.local:11970] MCW rank 29 bound to socket 0[core 1[hwt 0-1]]:
[../BB/../../../../../..][../../../../../../../..]
[compute-1-26.local:11970] MCW rank 30 is not bound (or bound to all available
processors)
[compute-1-8.local:19221] MCW rank 7 is not bound (or bound to all available
processors)
[compute-1-8.local:19221] MCW rank 8 is not bound (or bound to all available
processors)
[compute-1-8.local:19221] MCW rank 9 bound to socket 1[core 9[hwt 0-1]]:
[../../../../../../../..][../BB/../../../../../..]
[compute-1-8.local:19221] MCW rank 10 is not bound (or bound to all available
processors)
[compute-1-8.local:19221] MCW rank 6 bound to socket 0[core 0[hwt 0-1]]:
[BB/../../../../../../..][../../../../../../../..]
[compute-1-23.local:04078] MCW rank 20 is not bound (or bound to all available
processors)
[compute-1-23.local:04078] MCW rank 21 bound to socket 1[core 8[hwt 0-1]]:
[../../../../../../../..][BB/../../../../../../..]
[compute-1-23.local:04078] MCW rank 22 is not bound (or bound to all available
processors)
[compute-1-23.local:04078] MCW rank 23 is not bound (or bound to all available
processors)
[compute-1-23.local:04078] MCW rank 24 bound to socket 0[core 2[hwt 0-1]]:
[../../BB/../../../../..][../../../../../../../..]
[compute-1-23.local:04078] MCW rank 25 is not bound (or bound to all available
processors)
[compute-1-23.local:04078] MCW rank 26 bound to socket 0[core 3[hwt 0-1]]:
[../../../BB/../../../..][../../../../../../../..]
rank 0 @ compute-1-2.local 1, 2, 3, 4, 17, 18, 19, 20,
rank 1 @ compute-1-2.local 1, 17,
rank 2 @ compute-1-2.local 2, 18,
rank 3 @ compute-1-2.local 3, 19,
rank 4 @ compute-1-4.local 1, 15, 17, 31,
rank 5 @ compute-1-4.local 1, 15, 17, 31,
rank 6 @ compute-1-8.local 0, 16,
rank 7 @ compute-1-8.local 0, 5, 9, 13, 14, 16, 21, 25, 29, 30,
rank 8 @ compute-1-8.local 0, 5, 9, 13, 14, 16, 21, 25, 29, 30,
rank 9 @ compute-1-8.local 9, 25,
rank 10 @ compute-1-8.local 0, 5, 9, 13, 14, 16, 21, 25, 29, 30,
rank 11 @ compute-1-13.local 3, 4, 5, 6, 7, 19, 20, 21, 22, 23,
rank 12 @ compute-1-13.local 3, 4, 5, 6, 7, 19, 20, 21, 22, 23,
rank 13 @ compute-1-13.local 3, 4, 5, 6, 7, 19, 20, 21, 22, 23,
rank 14 @ compute-1-13.local 3, 19,
rank 15 @ compute-1-13.local 4, 20,
rank 17 @ compute-1-16.local 12, 13, 14, 15, 28, 29, 30, 31,
rank 18 @ compute-1-16.local 12, 13, 14, 15, 28, 29, 30, 31,
rank 16 @ compute-1-16.local 12, 13, 14, 15, 28, 29, 30, 31,
rank 20 @ compute-1-23.local 2, 3, 4, 8, 13, 14, 15, 18, 19, 20, 24, 29, 30,
31,
rank 21 @ compute-1-23.local 8, 24,
rank 19 @ compute-1-16.local 12, 13, 14, 15, 28, 29, 30, 31,
rank 22 @ compute-1-23.local 2, 3, 4, 8, 13, 14, 15, 18, 19, 20, 24, 29, 30,
31,
rank 23 @ compute-1-23.local 2, 3, 4, 8, 13, 14, 15, 18, 19, 20, 24, 29, 30,
31,
rank 24 @ compute-1-23.local 2, 18,
rank 25 @ compute-1-23.local 2, 3, 4, 8, 13, 14, 15, 18, 19, 20, 24, 29, 30,
31,
rank 26 @ compute-1-23.local 3, 19,
rank 27 @ compute-1-26.local 1, 6, 11, 13, 15, 17, 22, 27, 29, 31,
rank 28 @ compute-1-26.local 1, 6, 11, 13, 15, 17, 22, 27, 29, 31,
rank 29 @ compute-1-26.local 1, 17,
rank 30 @ compute-1-26.local 1, 6, 11, 13, 15, 17, 22, 27, 29, 31,
rank 31 @ compute-1-26.local 1, 6, 11, 13, 15, 17, 22, 27, 29, 31,
[login-0-1.local:03399] mca: base: close: component round_robin closed
[login-0-1.local:03399] mca: base: close: unloading component round_robin
[login-0-1.local:03399] mca: base: close: component rank_file closed
[login-0-1.local:03399] mca: base: close: unloading component rank_file
[login-0-1.local:03399] mca: base: close: component seq closed
[login-0-1.local:03399] mca: base: close: unloading component seq
[login-0-1.local:03399] mca: base: close: component resilient closed
[login-0-1.local:03399] mca: base: close: unloading component resilient
[login-0-1.local:03399] mca: base: close: component staged closed
[login-0-1.local:03399] mca: base: close: unloading component staged
[login-0-1.local:03399] mca: base: close: component mindist closed
[login-0-1.local:03399] mca: base: close: unloading component mindist
[login-0-1.local:03399] mca: base: close: component ppr closed
[login-0-1.local:03399] mca: base: close: unloading component ppr
