Re: [OMPI users] runtime performance tuning for Intel OMA interconnect

2016-08-10 Thread Cabral, Matias A
To remain in eager mode you need to increase the size of PSM2_MQ_RNDV_HFI_THRESH. PSM2_MQ_EAGER_SDMA_SZ is the threshold at which PSM changes from PIO (uses the CPU) to start setting SDMA engines. This summary may help: PIO Eager Mode: 0 bytes -> PSM2_MQ_EAGER_SDMA_SZ - 1 SDMA Eage

Re: [OMPI users] runtime performance tuning for Intel OMA interconnect

2016-08-10 Thread Xiaolong Cui
Hi Matias, Thanks a lot, that's very helpful! What I need indeed is to always use eager mode. But I didn't find any information about PSM2_MQ_EAGER_SDMA_SZ online. Would you please elaborate on "Just in case PSM2_MQ_EAGER_SDMA_SZ changes PIO to SDMA, always in eager mode." Thanks! Michael On We

Re: [OMPI users] runtime performance tuning for Intel OMA interconnect

2016-08-10 Thread Cabral, Matias A
Hi Michael, When Open MPI run on Omni-Path it will choose the PSM2 MTL by default, to use the libpsm2.so. Strictly speaking, it has compatibility to run using the openib BTL. However, the performance so significantly impacted that it is, not only discouraged, but no tuning would make sense. Reg