Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature

2012-11-02 Thread Hodge, Gary C
...@open-mpi.org] On Behalf Of Jeff Squyres Sent: Friday, November 02, 2012 9:32 AM To: Open MPI Users Subject: Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature What's the memory usage on your nodes -- are you invoking swap, perchance? Can you try one of the other shared memory mecha

Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature

2012-11-02 Thread Jeff Squyres
er 01, 2012 12:25 AM > To: Open MPI Users > Cc: Hodge, Gary C > Subject: Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature > > > On Oct 30, 2012, at 09:57 , Jeff Squyres wrote: > >> On Oct 30, 2012, at 9:51 AM, Hodge, Gary C wrote: >> >>> FY

Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature

2012-11-01 Thread Hodge, Gary C
Cc: Hodge, Gary C Subject: Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature On Oct 30, 2012, at 09:57 , Jeff Squyres wrote: > On Oct 30, 2012, at 9:51 AM, Hodge, Gary C wrote: > >> FYI, recently, I was tracking down the source of page faults in our >> application

Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature

2012-11-01 Thread George Bosilca
On Oct 30, 2012, at 09:57 , Jeff Squyres wrote: > On Oct 30, 2012, at 9:51 AM, Hodge, Gary C wrote: > >> FYI, recently, I was tracking down the source of page faults in our >> application that has real-time requirements. I found that disabling the sm >> component (--mca btl ^sm) eliminated m

Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature

2012-10-30 Thread Hodge, Gary C
Our measurements are not for the entire mpirun job, rather they are for the time it takes to process a message through our processing pipeline consisting of 11 processes distributed over 8 nodes. Taking an extra microsecond here and there is better for us than jumping from 3 to 15 ms because th

Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature

2012-10-30 Thread Jeff Squyres
On Oct 30, 2012, at 9:51 AM, Hodge, Gary C wrote: > FYI, recently, I was tracking down the source of page faults in our > application that has real-time requirements. I found that disabling the sm > component (--mca btl ^sm) eliminated many page faults I was seeing. Good point. This is like

Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature

2012-10-30 Thread Hodge, Gary C
FYI, recently, I was tracking down the source of page faults in our application that has real-time requirements. I found that disabling the sm component (--mca btl ^sm) eliminated many page faults I was seeing. I now have much better deterministic performance in that I no longer see outlier me