Thanks for a quick answer, Ralph!
This does not work, because em4 is only defined on the frontend node.
Now I get errors from the computes:
[compute-1-4.local:12206] found interface lo
[compute-1-4.local:12206] found interface em1
[compute-1-4.local:12206] mca: base: components_open: component
Hi, everyone,
I ran some bandwidth tests on two different systems with Mellanox IB
(FDR and EDR). I compiled the three supported versions of openmpi
(1.10.6, 2.0.2, 2.1.0) and measured the time it takes to send/receive
4MB arrays of doubles betweentwo hosts connected to the same IB switch.
MP
-o $(LIBNAME64) alignmalloc64.o
On 05/04/17 12:27, marcin.krotkiewski wrote:
The resultsare puzzling: it seems that something changed starting
from version
2.x, and the FDR system performs much worse than with the prior
1.10.x release.
___
use
Dear All,
I would appreciate some general advice on how to efficiently implement
the following scenario.
I am looking into how to send a large amount of data over IB _once_, to
multiple receivers. The trick is, of course, that while the ping-pong
benchmark delivers great bandwidth, it does s
I have a 4-socket machine with two dual-port Infiniband cards (devices
mlx4_0 and mlx4_1). The cards are conneted to PCI slots of different
CPUs (I hope..), both ports are active on both cards, everything
connected to the same physical network.
I use openmpi-1.10.0 and run the IBM-MPI1 benchma
I was wondering if it is possible, or considered to make it possible to
change the various MCA parameters by individual ranks during runtime in
addition to the command line?
I tried to google a bit, but did not get any indication that such topic
has even been discussed. It would be a very usef
:56PM +0200, marcin.krotkiewski wrote:
I was wondering if it is possible, or considered to make it possible to
change the various MCA parameters by individual ranks during runtime in
addition to the command line?
I tried to google a bit, but did not get any indication that such topic has
even been
I have run into a freeze / potential bug when using MPI_Comm_accept in a
simple client / server implementation. I have attached two simplest
programs I could produce:
1. mpi-receiver.c opens a port using MPI_Open_port, saves the port
name to a file
2. mpi-receiver enters infinite loop and
:06, marcin.krotkiewski a écrit :
I have run into a freeze / potential bug when using MPI_Comm_accept
in a simple client / server implementation. I have attached two
simplest programs I could produce:
1. mpi-receiver.c opens a port using MPI_Open_port, saves the port
name to a file
2. mpi
the sender,
and confirm authenticity of all links contained within the message.
With openmpi-1.7.5, the sender segfaults.
Sorry, I cannot see the problem in the codes. Perhaps people out there may help.
Jalel
Le 16/09/2015 16:40, marcin.krotkiewski a ?crit :
I have removed the MPI_Barrier
Hello, everyone
I am struggling a bit with IB performance when sending data from a POSIX
shared memory region (/dev/shm). The memory is shared among many MPI
processes within the same compute node. Essentially, I see a bit hectic
performance, but it seems that my code it is roughly twice slowe
27, 2015, at 1:38 PM, marcin.krotkiewski
wrote:
Hello, everyone
I am struggling a bit with IB performance when sending data from a POSIX shared
memory region (/dev/shm). The memory is shared among many MPI processes within
the same compute node. Essentially, I see a bit hectic performance, but it
s
Thank you, and Jeff, for clarification.
Before I bother you all more without the need, I should probably say I
was hoping to use libfabric/OpenMPI on an InfiniBand cluster. Somehow
now I feel I have confused this altogether, so maybe I should go one
step back:
1. libfabric is hardware indep
you prefer you can apply this patch to
either a 2.x or a master tarball.
https://github.com/hjelmn/ompi/commit/8839dbfae85ba8f443b2857f9bbefdc36c4ebc1a.patch
Let me know if this resolves the performance issues.
-Nathan
On Tue, Sep 29, 2015 at 09:57:54PM +0200, marcin.krotkiewski wrote:
I&#x
claim no improvement over psm, then I guess it is nothing to look
forward to, is it?
Anyway, thanks a lot for clearing this up
Marcin
On 09/30/2015 08:13 PM, Howard Pritchard wrote:
Hi Marcin,
2015-09-30 9:19 GMT-06:00 marcin.krotkiewski
mailto:marcin.krotkiew...@gmail.com>>:
Thank yo
Hi, Ralph,
I submit my slurm job as follows
salloc --ntasks=64 --mem-per-cpu=2G --time=1:0:0
Effectively, the allocated CPU cores are spread amount many cluster
nodes. SLURM uses cgroups to limit the CPU cores available for mpi
processes running on a given cluster node. Compute nodes are 2-so
your proc to both HTs
on the core. For some reason, we thought that 8.24 were HTs on the same core,
which is why we tried to bind to that pair of HTs. We got an error because HT
#24 was not allocated to us on node c6, but HT #8 was.
On Oct 3, 2015, at 2:43 AM, marcin.krotkiewski
wrote:
Hi, Ra
pute-9-31.local 1, 2, 11, 12, 13, 17, 18, 27, 28, 29,
Why does openmpi use cores (1,17) twice instead of using core (13,29)?
Clearly, the original SLURM-delivered map has 5 CPUs included, enough
for 5 MPI processes.
Cheers,
Marcin
On Oct 3, 2015, at 7:12 AM, marcin.krotkiewski
mailto:marci
d I may
just not be understanding your particular pattern. Our error message
is clearly indicating that we are seeing individual HTs (and not
complete cores) assigned, and I don’t know the source of that confusion.
On Oct 3, 2015, at 8:28 AM, marcin.krotkiewski
mailto:marcin.krotkiew...@gmail.com&
to:r...@open-mpi.org>> wrote:
What version of slurm is this? I might try to debug it here. I’m not
sure where the problem lies just yet.
On Oct 3, 2015, at 8:59 AM, marcin.krotkiewski
mailto:marcin.krotkiew...@gmail.com>>
wrote:
Here is the output of lstopo. In short, (0,16) are
chance to
track this down.
Gilles or anyone else who might have time - feel free to take a
gander and see if something pops out at you.
Ralph
On Oct 3, 2015, at 11:05 AM, marcin.krotkiewski
>
wrote:
Done. I have compiled 1.10.0 and 1.10.rc1 with --enab
slurm script, you can do
srun -N $SLURM_NNODES -n $SLURM_NNODES --cpu_bind=none -l grep
Cpus_allowed_list /proc/self/status
before invoking mpirun
Cheers,
Gilles
On 10/4/2015 11:55 PM, marcin.krotkiewski wrote:
Hi, all,
I played a bit more and it seems that the problem results from
trg_obj = opal_hwloc
s on the two sockets. It would also fail on some later nodes,
just that because of the error we never got there.
Let me know if you need more.
Marcin
Cheers,
Gilles
On 10/4/2015 11:55 PM, marcin.krotkiewski wrote:
Hi, all,
I played a bit more and it seems that the problem results fro
Yet another question about cpu binding under SLURM environment..
Short version: will OpenMPI support SLURM_CPUS_PER_TASK for the purpose
of cpu binding?
Full version: When you allocate a job like, e.g., this
salloc --ntasks=2 --cpus-per-task=4
SLURM will allocate 8 cores in total, 4 for eac
so as there isn’t any reason to
make you set it twice (well, other than trying to track which envar slurm is
using now).
On Oct 5, 2015, at 12:38 PM, marcin.krotkiewski
wrote:
Yet another question about cpu binding under SLURM environment..
Short version: will OpenMPI support
them tomorrow hopefully
Cheers,
Gilles
On Tuesday, October 6, 2015, marcin.krotkiewski
mailto:marcin.krotkiew...@gmail.com>>
wrote:
Hi, Gilles
you mentionned you had one failure with 1.10.1rc1 and -bind-to core
could you please send the full details (script, allocati
t; as far as I remember.
Regards,
Tetsuya Mishima
2015/10/06 5:40:33、"users"さんは「Re: [OMPI users] Hybrid OpenMPI+OpenMP
tasks using SLURM」で書きました
Hmmm…okay, try -map-by socket:pe=4
We’ll still hit the asymmetric topology issue, but otherwise this should
work
On Oct 5, 2015, at 1:25 PM,
--cpu_bind option, and make sure
your slurm config does support that :
srun --ntasks=2 --cpus-per-task=4 --cpu_bind=core,verbose -l grep
Cpus_allowed_list /proc/self/status
Cheers,
Gilles
On 10/6/2015 4:38 AM, marcin.krotkiewski wrote:
Yet another question about cpu binding under SLURM
e.g., loss of dynamics
support) - may or may not be of concern to your usage.
On Oct 6, 2015, at 11:57 AM, marcin.krotkiewski
wrote:
Thanks, Gilles. This is a good suggestion and I will pursue this direction. The
problem is that currently SLURM does not support --cpu_bind on my system for
e
issues we discussed
> i will make sure it applies fine vs latest 1.10 tarball from
tomorrow
>
> Cheers,
>
> Gilles
>
>
> On 10/6/2015 7:22 PM, marcin.krotkiewski wrote:
>> Gilles,
>>
>> Yes, it seemed tha
Sorry, I think I confused one thing:
On 10/08/2015 09:15 PM, marcin.krotkiewski wrote:
For version 1.10.1rc1 and up the situation is a bit different: it
seems that in many cases all cores are present in the cpuset, just
that the binding does not take place in a lot of cases. Instead
Hi, all,
I'm reading in the changelog 3.0.0 that
- Use UCX multi-threaded API in the UCX PML. Requires UCX 1.0 or later.
Also, the changelog for 3.1.0 it says that
- UCX PML improvements: add multi-threading support.
Could anyone briefly explain, what those mean in terms of the
functionalit
Hi,
I'm running the below example from the OpenMPI documentation:
#include
#include
main()
{
static int bigd[100];
int *ptr;
int i;
shmem_init();
if (shmem_my_pe() == 0) {
/* initialize PE 1’s bigd array */
ptr = shmem_ptr(bigd, 1);
if(!ptr){
fprintf(stderr, "get e
Hi, all
I have gone through some sweat to compile OpenMPI against a custom
(non-native) glibc. The reason I need this is that GCC can use the
vectorized libm, which came in glibc 2.22. And of course no HPC OS ships
with v2.22 - they are all behind a few years!
While using a custom Glibc for
Hi,
I have some problems running R + Rmpi with OpenMPI 3.1.0 + PMIx 2.1.1. A
simple R script, which starts a few tasks, hangs at the end on
diconnect. Here is the script:
library(parallel)
numWorkers <- as.numeric(Sys.getenv("SLURM_NTASKS")) - 1
myCluster <- makeCluster(numWorkers, type = "MP
On 06/04/2018 03:16 PM, r...@open-mpi.org wrote:
Try running the attached example dynamic code - if that works, then it likely
is something to do with how R operates.
On Jun 4, 2018, at 3:43 AM, marcin.krotkiewski
wrote:
Hi,
I have some problems running R + Rmpi with OpenMPI 3.1.0 +
osity, but would using Rmpi and/or doMPI help in any way?
-- bennet
On Mon, Jun 4, 2018 at 10:00 AM, marcin.krotkiewski
wrote:
Thanks, Ralph!
Your code finishes normally, I guess then the reason might be lying in R.
Running the R code with -mca pmix_base_verbose 1 i see that each rank calls
ex
works for me now.
Cheers,
Ben
On 5 Jun 2018, at 6:28 am, r...@open-mpi.org <mailto:r...@open-mpi.org>
wrote:
Yes, that does sound like a bug - the #connects must equal the
#disconnects.
On Jun 4, 2018, at 1:17 PM, marcin.krotkiewski
mailto:marcin.krotkiew...@gmail.com>>
wrote:
38 matches
Mail list logo
