Re: [OMPI users] Memory manager

2008-05-21 Thread Brian W. Barrett
Terry - It tells us that I'm not as smart as I thought :). If you're willing to help track this down, I'd like to try some other things that will require a more involved patch (it'll take me a day or two to get the patch right). Let me know if you'd be wiling to look further (hopefully only a

Re: [OMPI users] Memory manager

2008-05-21 Thread Terry Frankcombe
Hi Brian I ran your experiment. Changing the MMAP threshold made no difference to the memory footprint (>8GB/process out of the box, an order of magnitude smaller with --with-memory-manager=none). What does that tell us? Ciao Terry On Tue, 2008-05-20 at 06:51 -0600, Brian Barrett wrote: > Te

Re: [OMPI users] Memory manager

2008-05-20 Thread Brian Barrett
Terry - Would you be willing to do an experiment with the memory allocator? There are two values we change to try to make IB run faster (at the cost of corner cases you're hitting). I'm not sure one is strictly necessary, and I'm concerned that it's the one causing problems. If you don

Re: [OMPI users] Memory manager

2008-05-20 Thread Gleb Natapov
On Tue, May 20, 2008 at 12:17:02PM +1000, Terry Frankcombe wrote: > To tell you all what noone wanted to tell me, yes, it does seem to be > the memory manager. Compiling everything with > --with-memory-manager=none returns the vmem use to the more reasonable > ~100MB per process (down from >8GB).

Re: [OMPI users] Memory manager

2008-05-19 Thread Terry Frankcombe
To tell you all what noone wanted to tell me, yes, it does seem to be the memory manager. Compiling everything with --with-memory-manager=none returns the vmem use to the more reasonable ~100MB per process (down from >8GB). I take it this may affect my peak bandwidth over infiniband. What's the

[OMPI users] Memory manager

2008-05-12 Thread Terry Frankcombe
Hi folks I'm trying to run an MPI app on an infiniband cluster with OpenMPI 1.2.6. When run on a single node, this app is grabbing large chunks of memory (total per process ~8.5GB, including strace showing a single 4GB grab) but not using it. The resident memory use is ~40MB per process. When t

Re: [OMPI users] Memory manager

2007-11-26 Thread Jeff Squyres
On Nov 20, 2007, at 6:52 AM, Terry Frankcombe wrote: I posted this to the devel list the other day, but it raised no responses. Maybe people will have more to say here. Sorry Terry; many of us were at the SC conference last week, and this week is short because of the US holiday. Some of th

[OMPI users] Memory manager

2007-11-20 Thread Terry Frankcombe
Hi folks I posted this to the devel list the other day, but it raised no responses. Maybe people will have more to say here. Questions: How much does using the MPI wrappers influence the memory management at runtime? What has changed in this regard from 1.2.3 to 1.2.4? The reason I ask is t