Re: [OMPI users] OpenIB problems

2007-11-27 Thread Jeff Squyres
BTW, Andrew is correct about the unit for btl_openib_ib_timeout and that the value is simply passed down to the verbs library when making an IB connection. Open MPI does nothing else with that value; it's an IBTA-defined value. The help message was wrong on the 1.2 branch for a while; I th

Re: [OMPI users] OpenIB problems

2007-11-27 Thread Jeff Squyres
Sorry for jumping in late; the holiday and other travel prevented me from getting to all my mail recently... :-\ Have you checked the counters on the subnet manager to see if any other errors are occurring? It might be good to clear all the counters, run the job, and see if the counters a

Re: [OMPI users] warning:regcache incompatible with malloc

2007-11-27 Thread de Almeida, Valmor F.
Thanks, -- Valmor > -Original Message- > From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On > Behalf Of Tim Prins > Sent: Tuesday, November 27, 2007 10:37 AM > To: Open MPI Users > Subject: Re: [OMPI users] warning:regcache incompatible with malloc > > Hi Valmor, >

[OMPI users] Fwd: [all-osl-users] OSL system outage

2007-11-27 Thread Jeff Squyres
FYI -- all of open-mpi.org (www, svn) will be down for a short period next Monday. Begin forwarded message: From: DongInn Kim <> Date: November 27, 2007 2:15:43 PM CST Subject: [all-osl-users] OSL system outage Hi, The OSL systems need to reboot to for the regular maintenance on Dec 3rd

Re: [OMPI users] OpenIB problems

2007-11-27 Thread Brock Palen
Ok i will open a case with cisco, Brock Palen Center for Advanced Computing bro...@umich.edu (734)936-1985 On Nov 27, 2007, at 4:19 PM, Andrew Friedley wrote: Brock Palen wrote: What would be a place to look? Should this just be default then for OMPI? ompi_info shows the default as 10

Re: [OMPI users] OpenIB problems

2007-11-27 Thread Andrew Friedley
Brock Palen wrote: What would be a place to look? Should this just be default then for OMPI? ompi_info shows the default as 10 seconds? Is that right 'seconds' ? The other IB guys can probably answer better than I can -- I'm not an expert in this part of IB (or really any part I guess :).

Re: [OMPI users] OpenIB problems

2007-11-27 Thread Brock Palen
What would be a place to look? Should this just be default then for OMPI? ompi_info shows the default as 10 seconds? Is that right 'seconds' ? The other IB guys can probably answer better than I can -- I'm not an expert in this part of IB (or really any part I guess :). Not sure why a

Re: [OMPI users] OpenIB problems

2007-11-27 Thread Andrew Friedley
Brock Palen wrote: On Nov 27, 2007, at 10:49 AM, Andrew Friedley wrote: Brock Palen wrote: On Nov 21, 2007, at 3:39 PM, Andrew Friedley wrote: If this is what I think it is, try using this MCA parameter: -mca btl_openib_ib_timeout 20 The user used this option and it allowed the run to com

Re: [OMPI users] OpenIB problems

2007-11-27 Thread Brock Palen
On Nov 27, 2007, at 10:49 AM, Andrew Friedley wrote: Brock Palen wrote: On Nov 21, 2007, at 3:39 PM, Andrew Friedley wrote: If this is what I think it is, try using this MCA parameter: -mca btl_openib_ib_timeout 20 The user used this option and it allowed the run to complete. You say its

Re: [OMPI users] Suggestions on multi-compiler/multi-mpi build?

2007-11-27 Thread Jim Kusznir
So another question on this topic: I'm running on a 64-bit cluster. Is it important/needed/useful to maintain 32-bit and 64-bit versions of openmpi and such? At the moment, I'm using Rocks' default openmpi, which includes both 32 and 64-bit versions of the libraries. Yet, so far, my attempt to

Re: [OMPI users] OpenIB problems

2007-11-27 Thread Andrew Friedley
Brock Palen wrote: On Nov 21, 2007, at 3:39 PM, Andrew Friedley wrote: If this is what I think it is, try using this MCA parameter: -mca btl_openib_ib_timeout 20 The user used this option and it allowed the run to complete. You say its a issue with the fabric ibshowerrors does not show any

Re: [OMPI users] warning:regcache incompatible with malloc

2007-11-27 Thread Tim Prins
Hi Valmor, I prefer to just set the environment variable in my .bashrc so I never have to think about it again. Also it might be slightly better since Open MPI tries to be network neutral, and linking the application against the Myrinet libraries violates that principle. But if you are only e

Re: [OMPI users] OpenIB problems

2007-11-27 Thread Brock Palen
On Nov 21, 2007, at 3:39 PM, Andrew Friedley wrote: If this is what I think it is, try using this MCA parameter: -mca btl_openib_ib_timeout 20 The user used this option and it allowed the run to complete. You say its a issue with the fabric ibshowerrors does not show any problems. Its to