Re: [OMPI users] How to specify the use of RDMA?

2017-03-20 Thread Rodrigo Escobar
Thank you Gilles, I think that has made it clear. Regards, Rodrigo On Mon, Mar 20, 2017 at 7:13 PM, Gilles Gouaillardet wrote: > Rodrigo, > > > i do not understand what you mean by "deactivate my IB interfaces" > > > the hostfile is only used in the wire-up phase > > (to keep things simple, mp

Re: [OMPI users] tuning sm/vader for large messages

2017-03-20 Thread Gilles Gouaillardet
Joshua, George previously explained you are limited by the size of your level X cache. that means that you might get optimal performance for a given message size, let's say when everything fits in the L2 cache. when you increase the message size, L2 cache is too small, and you have to mo

Re: [OMPI users] How to specify the use of RDMA?

2017-03-20 Thread Gilles Gouaillardet
Rodrigo, i do not understand what you mean by "deactivate my IB interfaces" the hostfile is only used in the wire-up phase (to keep things simple, mpirun does ssh orted under the hood, and is coming from your hostfile. so bottom line mpirun --mca btl openib,self,sm -hostfile hosts_eth

Re: [OMPI users] tuning sm/vader for large messages

2017-03-20 Thread Joshua Mora
I don't want to push it up. I just want to sustain the same bandwidth sending at that optimal size. I'd like to see a constant bw from that size and above , not a significant drop when I cross a msg size. -- Original Message -- Received: 05:11 PM CDT, 03/20/2017 From: George Bosilca  To:

Re: [OMPI users] tuning sm/vader for large messages

2017-03-20 Thread George Bosilca
On Mon, Mar 20, 2017 at 12:45 PM, Joshua Mora wrote: > If at certain x msg size you achieve X performance (MB/s) and at 2x msg > size > or higher you achieve Y performance, being Y significantly lower than X, > is it > possible to have a parameter that chops messages internally to x size in > ord

Re: [OMPI users] How to specify the use of RDMA?

2017-03-20 Thread Rodrigo Escobar
Thanks Guilles for the quick reply. I think I am confused about what the openib BTL specifies. What am I doing when I run with the openib BTL but specify my eth interface (...and deactivate my IB interfaces)? Is not openib only for IB interfaces? Am I using RDMA here? These two commands give the s

Re: [OMPI users] tuning sm/vader for large messages

2017-03-20 Thread Joshua Mora
If at certain x msg size you achieve X performance (MB/s) and at 2x msg size or higher you achieve Y performance, being Y significantly lower than X, is it possible to have a parameter that chops messages internally to x size in order to sustain X performance rather than let it choke ? sort of flow

Re: [OMPI users] How to specify the use of RDMA?

2017-03-20 Thread Gilles Gouaillardet
You will get similar results with hosts_ib and hosts_eth If you want to use tcp over ethernet, you have to mpirun --mca btl tcp,self,sm --mca btl_tcp_if_include eth0 ... If you want to use tcp over ib, then mpirun --mca btl tcp,self,sm --mca btl_tcp_if_include ib0 ... Keep in mind that IMB calls