Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-11-05 Thread Dave Love
Fabian Wein writes: >># hwloc-bind node1:1 hwloc-ps | grep hwloc >>13425 NUMANode:1 hwloc-ps > > I don't understand what you mean: > > opt/hwloc-1.11.1/bin/hwloc-bind > /opt/hwloc-1.11.1/bin/hwloc-bind: nothing to do! > > /opt/hwloc-1.11.1/bin/hwloc-bind node1:1 > /opt/hw

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-11-03 Thread Fabian Wein
On 11/03/2015 06:13 PM, Dave Love wrote: Fabian Wein writes: There is an old OpenFOAM installation which includes and old open-mpi, this might cause the trouble. OpenFOAM should definitely be built against the system MPI (and, in general, you should avoid bundled libraries wherever possible

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-11-03 Thread Dave Love
Fabian Wein writes: > There is an old OpenFOAM installation which includes and old open-mpi, > this might > cause the trouble. OpenFOAM should definitely be built against the system MPI (and, in general, you should avoid bundled libraries wherever possible IMHO). > I also suspect that sourcing

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-11-02 Thread Fabian Wein
> Am 02.11.2015 um 15:58 schrieb Dave Love : > > Fabian Wein writes: > >>> Am 30.10.2015 um 21:45 schrieb Jeff Squyres (jsquyres) : >>> >>> Oh, that's an interesting idea: perhaps the "bind to numa" is >>> failing -- but perhaps "bind to socket" would work. >>> >>> Can you try: >>> >>> /opt/

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-11-02 Thread Dave Love
Fabian Wein writes: >> Am 30.10.2015 um 21:45 schrieb Jeff Squyres (jsquyres) : >> >> Oh, that's an interesting idea: perhaps the "bind to numa" is >> failing -- but perhaps "bind to socket" would work. >> >> Can you try: >> >> /opt/openmpi-1.10.0-gcc/bin/mpiexec -bind-to numa -n 4 hostname >>

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-11-02 Thread Dave Love
Fabian Wein writes: > On 10/30/2015 02:48 PM, Dave Love wrote: >> Fabian Wein writes: >> >>> Is this a valid test? >>> >>> >>> /opt/openmpi-1.10.0-gcc/bin/mpiexec -n 4 hostname >>> leo >>> leo >>> leo >>> leo >> >> So, unless you turned off the default binding -- to socket? check the >> mpirun m

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-31 Thread Fabian Wein
> Am 30.10.2015 um 21:45 schrieb Jeff Squyres (jsquyres) : > > Oh, that's an interesting idea: perhaps the "bind to numa" is failing -- but > perhaps "bind to socket" would work. > > Can you try: > > /opt/openmpi-1.10.0-gcc/bin/mpiexec -bind-to numa -n 4 hostname > /opt/openmpi-1.10.0-gcc/bin/

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-30 Thread Jeff Squyres (jsquyres)
Oh, that's an interesting idea: perhaps the "bind to numa" is failing -- but perhaps "bind to socket" would work. Can you try: /opt/openmpi-1.10.0-gcc/bin/mpiexec -bind-to numa -n 4 hostname and /opt/openmpi-1.10.0-gcc/bin/mpiexec -bind-to socket -n 4 hostname > On Oct 30, 2015, at 12:02 P

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-30 Thread Jeff Squyres (jsquyres)
Brice -- I know this started on the hwloc list and then bounced over here, but we're running out of ideas. Got any clue what is happening here? From the OMPI config logs that Fabian sent, it looks like hwloc built with libnuma support properly...? > On Oct 30, 2015, at 4:34 PM, Ralph Castain

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-30 Thread Ralph Castain
I honestly have no ideas…best I can see, it looks like hwloc feels that it cannot perform that operation and returns an error. > On Oct 30, 2015, at 1:31 PM, Jeff Squyres (jsquyres) > wrote: > > He's building and running on a single server (leo). From the configure > output, all the numa li

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-30 Thread Jeff Squyres (jsquyres)
He's building and running on a single server (leo). From the configure output, all the numa libs and headers are available on this leo server. > On Oct 30, 2015, at 11:09 AM, Ralph Castain wrote: > > I think Dave has probably hit the problem - that node may well not have a > “numa” object. Y

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-30 Thread Fabian Wein
On 10/30/2015 04:09 PM, Ralph Castain wrote: I think Dave has probably hit the problem - that node may well not have a “numa” object. You also might check that node “leo” has libnuma on it I have a shared memory system, I run on the machine I build the stuff. Fabian

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-30 Thread Fabian Wein
On 10/30/2015 02:48 PM, Dave Love wrote: Fabian Wein writes: Is this a valid test? /opt/openmpi-1.10.0-gcc/bin/mpiexec -n 4 hostname leo leo leo leo So, unless you turned off the default binding -- to socket? check the mpirun man page -- it worked, but the "numa" level failed. I don't kno

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-30 Thread Ralph Castain
I think Dave has probably hit the problem - that node may well not have a “numa” object. You also might check that node “leo” has libnuma on it > On Oct 30, 2015, at 6:48 AM, Dave Love wrote: > > Fabian Wein writes: > >> Is this a valid test? >> >> >> /opt/openmpi-1.10.0-gcc/bin/mpiexec -n

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-30 Thread Dave Love
Fabian Wein writes: > Is this a valid test? > > > /opt/openmpi-1.10.0-gcc/bin/mpiexec -n 4 hostname > leo > leo > leo > leo So, unless you turned off the default binding -- to socket? check the mpirun man page -- it worked, but the "numa" level failed. I don't know if that level has to exist, a

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-30 Thread Jeff Squyres (jsquyres)
Yes, I think that test is valid. 1.10.0 is a good version to test. Ralph: it all looks good to me, but that message is coming from ORTE. Can you have a look? Sent from my phone. No type good. > On Oct 30, 2015, at 12:33 AM, Fabian Wein wrote: > >> On 10/30/2015 02:26 AM, Jeff Squyres (jsquy

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-30 Thread Fabian Wein
On 10/30/2015 02:26 AM, Jeff Squyres (jsquyres) wrote: Your Open MPI build looks good -- it seems to have found all the right libnuma stuff during configured, etc. The error message you're seeing indicates that the embedded hwloc is telling Open MPI that it doesn't have binding support, which

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Jeff Squyres (jsquyres)
Your Open MPI build looks good -- it seems to have found all the right libnuma stuff during configured, etc. The error message you're seeing indicates that the embedded hwloc is telling Open MPI that it doesn't have binding support, which would be unusual since you have both libnuma and libnuma

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Fabian Wein
On 10/29/2015 09:29 PM, Dave Love wrote: I guess the problem is not having hwloc/libnuma1 on the compute host, as opposed to the build host. Test like this on the host, though the component name may be different in 1.10: I Have shared-memory system (4 package opteron, 8 numa nodes, 48 cores),

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Fabian Wein
If your hwloc was built without libnuma support, you will get the following error: $ hwloc-bind --membind node:0 -- echo foo hwloc_set_membind 0x000f failed (errno 38 Function not implemented) Some days ago I installed hwloc and observed "libnuma memory support: no". Then I installed libn

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Dave Love
"Jeff Squyres (jsquyres)" writes: > +1 > > If you're compiling Open MPI from source, you need the -devel package > so that the libnuma header files are installed (and therefore Open MPI > [i.e., the hwloc embedded in Open MPI] can include those header files > and then compile support for libnuma)

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Brice Goglin
Le 29/10/2015 21:04, Fabian Wein a écrit : >> If you're compiling Open MPI from source, you need the -devel package >> so that the libnuma header files are installed (and therefore Open >> MPI [i.e., the hwloc embedded in Open MPI] can include those header >> files and then compile support for li

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Fabian Wein
If you're compiling Open MPI from source, you need the -devel package so that the libnuma header files are installed (and therefore Open MPI [i.e., the hwloc embedded in Open MPI] can include those header files and then compile support for libnuma). Can I also test the numa capability via a

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Fabian Wein
Is there any hint in the configure output? Yes. Please send all the info listed here: http://www.open-mpi.org/community/help/ I attached the files. I built it again, this time with gcc instead of intel This is how I call mpiexec to make sure I get the right one make MPIEXEC="/opt/o

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Jeff Squyres (jsquyres)
On Oct 29, 2015, at 9:26 AM, Fabian Wein wrote: > > Is there any hint in the configure output? Yes. Please send all the info listed here: http://www.open-mpi.org/community/help/ -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doin

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Fabian Wein
On 10/29/2015 03:01 PM, Ralph Castain wrote: You need to also install libnuma-devel, I’m afraid - just has to do with how they chose to split the library sudo apt-get install libnuma-devel ... E: Unable to locate package libnuma-devel sudo apt-get install libnuma-dev ... libnuma-dev is alread

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Jeff Squyres (jsquyres)
+1 If you're compiling Open MPI from source, you need the -devel package so that the libnuma header files are installed (and therefore Open MPI [i.e., the hwloc embedded in Open MPI] can include those header files and then compile support for libnuma). > On Oct 29, 2015, at 7:01 AM, Ralph Ca

Re: [OMPI users] Unable to compile for libnumactl and libnumactl-devel

2015-10-29 Thread Ralph Castain
You need to also install libnuma-devel, I’m afraid - just has to do with how they chose to split the library > On Oct 29, 2015, at 2:01 AM, Fabian Wein wrote: > > Hello, > > I'm new to the list, maybe the issue was somewhere else already solved. I > found a similar question in the mailing li