Re: [OMPI users] failure of 1.8.5 on FreeBSD

2015-05-11 Thread Steve Kargl
On Mon, May 11, 2015 at 12:55:01PM -0700, Steve Kargl wrote: > I've configured and built openmpi-1.8.5 on FreeBSD-11 > with > > % ./configure --prefix /usr/local/openmpi-1.8.5 --disable-shared \ >--enable-static --enable-mpirun-prefix-by-default > Not necessarily a

[OMPI users] failure of 1.8.5 on FreeBSD

2015-05-11 Thread Steve Kargl
I've configured and built openmpi-1.8.5 on FreeBSD-11 with % ./configure --prefix /usr/local/openmpi-1.8.5 --disable-shared \ --enable-static --enable-mpirun-prefix-by-default This, of course, disables shared libraries and enables static libs. Now, when I go to build my program, I see % mak

Re: [OMPI users] fortran program with integer kind=8 using openmpi?

2012-07-03 Thread Steve Kargl
On Tue, Jul 03, 2012 at 04:03:51PM -0700, William Au wrote: > > Dear all, > > Thank you for all responses. There is another problem using > -fdefault-integer-8. I'll make the unsolicited suggestion that you really really really don't want to use the -fdefault-integer-8 option. It would be far b

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-14 Thread Steve Kargl
On Wed, Jul 13, 2011 at 08:27:13AM -0400, Jeff Squyres wrote: > On Jul 12, 2011, at 1:37 PM, Steve Kargl wrote: > > > (many lines removed) > > checking prefix for function in .type... @ > > checking if .size is needed... yes > > checking if .align directive

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-12 Thread Steve Kargl
On Tue, Jul 12, 2011 at 11:03:42AM -0700, Steve Kargl wrote: > On Tue, Jul 12, 2011 at 10:37:14AM -0700, Steve Kargl wrote: > > On Fri, Jul 08, 2011 at 07:03:13PM -0400, Jeff Squyres wrote: > > > Sorry -- I got distracted all afternoon... > > > > > > In additi

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-12 Thread Steve Kargl
On Tue, Jul 12, 2011 at 10:37:14AM -0700, Steve Kargl wrote: > On Fri, Jul 08, 2011 at 07:03:13PM -0400, Jeff Squyres wrote: > > Sorry -- I got distracted all afternoon... > > > > In addition to what Ralph said (i.e., I'm not sure if the CIDR > > notation stuff ma

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-12 Thread Steve Kargl
On Fri, Jul 08, 2011 at 07:03:13PM -0400, Jeff Squyres wrote: > Sorry -- I got distracted all afternoon... > > In addition to what Ralph said (i.e., I'm not sure if the CIDR > notation stuff made it over to the v1.5 branch or not, but it > is available from the nightly SVN trunk tarballs: > http:/

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-08 Thread Steve Kargl
On Fri, Jul 08, 2011 at 07:03:13PM -0400, Jeff Squyres wrote: > Sorry -- I got distracted all afternoon... No problem. We all have obligations that we prioritize. > In addition to what Ralph said (i.e., I'm not sure if the > CIDR notation stuff made it over to the v1.5 branch or not, > but it is

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-08 Thread Steve Kargl
On Fri, Jul 08, 2011 at 04:26:35PM -0400, Gus Correa wrote: > Steve Kargl wrote: > >On Fri, Jul 08, 2011 at 02:19:27PM -0400, Jeff Squyres wrote: > >>The easiest way to fix this is likely to use the btl_tcp_if_include > >>or btl_tcp_if_exclude MCA parameters -- i.e.,

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-08 Thread Steve Kargl
On Fri, Jul 08, 2011 at 12:09:09PM -0700, Steve Kargl wrote: > On Fri, Jul 08, 2011 at 02:19:27PM -0400, Jeff Squyres wrote: > > > > The easiest way to fix this is likely to use the btl_tcp_if_include > > or btl_tcp_if_exclude MCA parameters -- i.e., tell OMPI exactly > &

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-08 Thread Steve Kargl
On Fri, Jul 08, 2011 at 02:19:27PM -0400, Jeff Squyres wrote: > > The easiest way to fix this is likely to use the btl_tcp_if_include > or btl_tcp_if_exclude MCA parameters -- i.e., tell OMPI exactly > which interfaces to use: > > http://www.open-mpi.org/faq/?category=tcp#tcp-selection > Pe

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-08 Thread Steve Kargl
On Fri, Jul 08, 2011 at 02:19:27PM -0400, Jeff Squyres wrote: > On Jul 8, 2011, at 1:31 PM, Steve Kargl wrote: > > > It seems that openmpi-1.4.4 compiled code is trying to use the > > wrong nic. My /etc/hosts file has > > > > 10.208.78.111 hpc.apl.washi

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-08 Thread Steve Kargl
On Thu, Jul 07, 2011 at 08:38:56PM -0400, Jeff Squyres wrote: > On Jul 5, 2011, at 4:24 PM, Steve Kargl wrote: > > On Tue, Jul 05, 2011 at 01:14:06PM -0700, Steve Kargl wrote: > >> I have an application that appears to function as I expect > >> when compiled with openmpi

Re: [OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-05 Thread Steve Kargl
On Tue, Jul 05, 2011 at 01:14:06PM -0700, Steve Kargl wrote: > I have an application that appears to function as I expect > when compiled with openmpi-1.4.2 on FreeBSD 9.0. But, it > appears to hang during communication between nodes. What > follows is the long version. Argh I me

[OMPI users] tcp communication problems with 1.4.3 and 1.4.4 rc2 on FreeBSD

2011-07-05 Thread Steve Kargl
I have an application that appears to function as I expect when compiled with openmpi-1.4.2 on FreeBSD 9.0. But, it appears to hang during communication between nodes. What follows is the long version. I configure 1.4.2 with ./configure --prefix=/usr/local/openmpi-1.4.2 \ --enable-mpirun-prefi

Re: [OMPI users] MPI-3 Fortran feedback

2009-10-25 Thread Steve Kargl
On Fri, Oct 23, 2009 at 08:53:01AM -0400, Jeff Squyres wrote: > If you're a Fortran MPI developer, I have a question for you. > > In the MPI-3 Forum, we're working on revamping the Fortran bindings to > be "better" (for a variety of definitions of "better"). There's at > least one question th

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-10 Thread Steve Kargl
On Fri, Apr 10, 2009 at 06:13:43PM -0400, Jeff Squyres wrote: > On Apr 10, 2009, at 5:30 PM, Steve Kargl wrote: > > >Thanks for looking into this issue. As a side note, FreeBSD 7.1 > >and higher has the cpuset_getaffinity/cpuset_setaffinity system > >calls. I suspect th

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-10 Thread Steve Kargl
On Fri, Apr 10, 2009 at 05:10:29PM -0400, Jeff Squyres wrote: > On Apr 7, 2009, at 4:25 PM, Mostyn Lewis wrote: > > >Does OpenMPI know about the number of CPUS per node for FreeBSD? > > > > This is exactly the right question: apparently it does not. > > Specifically, it looks like we have a bad

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-10 Thread Steve Kargl
On Fri, Apr 10, 2009 at 05:10:29PM -0400, Jeff Squyres wrote: > On Apr 7, 2009, at 4:25 PM, Mostyn Lewis wrote: > > >Does OpenMPI know about the number of CPUS per node for FreeBSD? > > > > This is exactly the right question: apparently it does not. > > Specifically, it looks like we have a bad

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-07 Thread Steve Kargl
On Tue, Apr 07, 2009 at 02:23:45PM -0600, Ralph Castain wrote: > It isn't in a file - unless you specify it, OMPI will set it > automatically based on the number of procs on the node vs. what OMPI > thinks are the number of available processors. The question is: why > does OMPI not correctly

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-07 Thread Steve Kargl
On Tue, Apr 07, 2009 at 01:40:13PM -0600, Ralph Castain wrote: > I would really suggest looking at George's note first as I think you > are chasing your tail here. It sounds like the most likely problem is > that OMPI thinks you are oversubscribed and is setting sched_yield > accordingly. whi

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-07 Thread Steve Kargl
On Tue, Apr 07, 2009 at 03:18:31PM -0400, George Bosilca wrote: > Steve, > > I spotted a strange value for the mpi_yield_when_idle MCA parameter. 1 > means your processor is oversubscribed, and this trigger a call to > sched_yield after each check on the SM. Are you running the job > oversub

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-07 Thread Steve Kargl
On Tue, Apr 07, 2009 at 12:00:55PM -0700, Mostyn Lewis wrote: > Steve, > > Did you rebuild 1.2.9? As I see you have static libraries, maybe there's > a lurking phthread or something else that may have changed over time? > > DM Yes. I downloaded 1.2.9, 1.3, and 1.3.1, all within minutes of each

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-07 Thread Steve Kargl
On Tue, Apr 07, 2009 at 09:10:21AM -0700, Eugene Loh wrote: > Steve Kargl wrote: > > >I can rebuild 1.2.9 and 1.3.1. Is there any particular configure > >options that I should enable/disable? > > I hope someone else will chime in here, because I'm somewhat out of

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-07 Thread Steve Kargl
On Tue, Apr 07, 2009 at 08:39:20AM -0700, Eugene Loh wrote: > Iain Bason wrote: > > >But maybe Steve should try 1.3.2 instead? Does that have your > >improvements in it? > > 1.3.2 has the single-queue implementation and automatic sizing of the sm > mmap file, both intended to fix problems at

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-07 Thread Steve Kargl
On Tue, Apr 07, 2009 at 08:00:39AM -0700, Eugene Loh wrote: > Iain Bason wrote: > > >There are a bunch changes in the shared memory module between 1.2.9 > >and 1.3.1. One significant change is the introduction of the "sendi" > >internal interface. I believe George Bosilca did the initial >

Re: [OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-06 Thread Steve Kargl
On Mon, Apr 06, 2009 at 02:04:16PM -0700, Eugene Loh wrote: > Steve Kargl wrote: > > >I recently upgraded OpenMPI from 1.2.9 to 1.3 and then 1.3.1. > >One of my colleagues reported a dramatic drop in performance > >with one of his applications. My investigation shows a

[OMPI users] Factor of 10 loss in performance with 1.3.x

2009-04-06 Thread Steve Kargl
Hi, I recently upgraded OpenMPI from 1.2.9 to 1.3 and then 1.3.1. One of my colleagues reported a dramatic drop in performance with one of his applications. My investigation shows a factor of 10 drop in communication over the memory bus. I've placed a figure that iilustrates the problem at htt