Le 01/12/2012 12:45, Leta Melkamu a écrit : > Hello there, > > I have some doubts on the use of knem with openmpi, everything works fine. > However, it is a bit not clear on the usage of knem flags while > running my open-mpi program. > Something like --mca btl_sm_knem_dma_min 4860 is enough or I have to > add more flag like --mca btl_sm_eager_limit 4276 in the same run? or > can you please suggest me a good documentation link about knem flag > usage, I have tried to look around but no good info regarding those > staff. otherwise I will end up to test each command with different > flag value for each run.
What are you trying to do? * Use knem for direct copy through kernel for medium/large messages: (common use case) "--mca btl_sm_use_knem 1" is enough. You can "cat /dev/knem" before and after a run to see that knem counters have increased, which means OMPI successfully passed some copy requests to knem. * Use knem for short messages: "--mca btl_sm_eager_limit 4276" may help. But I am not sure that's a good idea since knem was designed for large messages. * Offload knem copies to I/OAT hardware on Intel servers. That's what "btl_sm_knem_dma_min" is for. Not sure you really want to do that either, it's not much interesting on current Intel servers. Brice
