Hi,
i cannot comment for the openib specific part.
the coll/tuned collective module is very likely to split messages in
order to use a more efficient
algorithm. an other way to put it is you probably do not want to use
large messages.
but if this is really what you want, then one option is to disable
coll/tuned,
mpirun --mca coll ^tuned ...
an other option is to force the the algorithm used by coll/tuned
mpirun --mca coll_tuned_use_dynamic_rules 1 --mca
coll_tuned_allreduce_algorithm <algo> ...
where <algo> is a number between 1 and 6 (see ompi_info --all for the
definition)
Cheers,
Gilles
On 7/19/2017 6:10 AM, Juncheng Gu wrote:
Hi,
I am trying to setup openMPI over RDMA cross machines.
I call MPI_AllReduce() with a 240MB data buffer.
But, it seems openMPI chunks data into small fragments (1MB ~ 15MB),
and then sends them out through RDMA.
Which mca parameters can affect the message size in openMPI?
How to configure "mca" to let openMPI use large message size
(fragment) in data transmission?
For example, is there any way we can set the mini message size of openMPI?
I have followed the instructions from
https://www.open-mpi.org/faq/?category=openfabrics#ofed-and-ompi. But
there is no improvement in message size.
I am using " --mca btl self,sm,openib ".
Best,
Juncheng
_______________________________________________
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
_______________________________________________
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
